They whole journey from the initial idea for the solution till the final one (and the one I am using right now while writing this article) was quite interesting, way more than having found the good solution right away. Let’s think first of the hardware components you need to develop smart headphones that recognize when you have them on/off.
Sennheiser HD 429: Low-end Sennheiser headphones. I used them to attach the first sensors I tried. The sound quality is okish, but the materials, especially the cable, are quite fragile.
Magnat LZR 580: They have a nice audio quality for the price, but I found them a bit uncomfortable to use during long sessions. However, they proved to be a great fit for supporting force sensitive resistors.
ELEGIANT Q2: Cheap headphones, good for attaching sensors or even the microcontroller board to it.
The open source project Arduino was the best solution I found to interact with the many sensors that I needed to try. I purchased two of them, Arduino Uno and Arduino Nano. You can program it in C, and that’s enough for my project. You can download the Arduino IDE here.
I tried several sensors, including tilt sensors, accelerometers, IR proximity and force sensitive resistors. The use of every individual sensor while prototyping led to different conclusions and the discovery of new issues and blockers related to the nature of each sensor. All of them are explained next.
This was my first attempt. I initially thought that recognizing when their headphones are in a vertical position or horizontal position would suffice to trigger the play/stop on the pc.
The tilt sensor is composed by a little cylinder with two pins that do not touch. There is a little metal ball inside (mercury sensors are already banned) that when touching both pins, allows the voltage to pass thru. This provides a binary result (some voltage or none).
I placed the sensor on the top of the headphones, making the sensor to switch a led light on when vertical or off when horizontal. Placing the headphones on the desktop would immediately make the led to switch off, and taking them to put them on will make the led to switch on, even before you have actually placed the headphones on your head.
That was a nice user experience, since you want the music to be playing just a little bit before you get headphones on your head. Same behavior is desired when you take them off, you want them to stop, but not until you have taken them far off your head. This solution created a very positive user experience that works as long as your headphones use habits involve letting the headphones lay horizontally on your desktop. Well… that is not the case for me. There are two scenarios where the proposed solution failed, with no obvious solution if we are restricted to tilt sensors.
The first scenario is the people who uses headphone hangers. In that case, the headphones will still be in a vertical position when the user takes them off. The second scenario is when you take the headphones off and let them hang on your neck. In this case, due to the nature of the tilt sensor, the inclination of the headphones in that position still makes the ball inside the sensor to drop and make the connection between the two pins. Therefore, although very nice, that solution was not a working one. The findings on this experiment made me think about the next sensor.
Certainly, the accelerometer would recognize that the headphones are laying on the desktop, but also if they are placed on a hanger. They will fail to recognize them hanging on the neck, though. Let’s try.
As recap, the accelerometer sensor identifies the acceleration in 3 degrees of freedom of the object porting the sensor. For my project, I was not interested on the direction of movement, but rather recognizing that the headphones are moving. So I wrote the code running on the arduino to combine all the movement directions and set a threshold of movement, during some seconds. In this video you can see the detection of movement and the music play/pause.
This solution made it as expected, but unfortunately it had a very bad performance in the main use cases. The issue was on setting a threshold that quickly recognizes the headphones are steady, but that do not stop the music if the user is not moving the head so much ( while reading, studying etc ). As a result, I needed to set a threshold very much in favor of playing music.
That resulted, as you can imagine, on music seconds being lost while the headphones were steady on the desktop or hanger. It wasn’t a nice solution. Furthermore, there was a scenario that made this solution unusable. The user might want to listen to music while laying on bed or couch. Minimal head movement would be involved then, and therefore, the user would be annoyed by the unnecessary music pause. That is a bad user experience, but the user just needed to move the head a little bit to get his music back again. That was funny to try out but not feasible for a final solution. Let’s try a different sensor then.
Infrared proximity sensor
This sensor would definitely detect that you have the headphones on… Yeah! Done it. Thanks for reading and please stay tuned. Goodbye.
Well… not really. The IR sensor had a great performance, the best so far. I placed the sensor next to one of the headphones ear speakers. I calibrated the code to measure the distance between the sensor and the head and voilà!. The gadget worked in all scenarios I imagined before:
Headphones on, even when I’m laying on the couch? Detected. Headphones off? Sure, the proximity sensor detected a greater distance than the threshold. Headphones hanging on the neck? Yes, proximity sensor performed since the distance between the headphones and the neck was greater than the threshold.
Everything seemed to be good, as seen in this video. I connected the PC to the TV with a Chromecast so the whole solution looks even nicer.
I was satisfied until I asked a friend to try them out. She has long curly hair. When she started using the prototype, my face started to turn from satisfaction to “ouch!, I didn’t think about that”. The proximity sensor performed correctly in every scenario but one. When headphones hang on the neck, the hair was confusing the sensor and therefore the music was not stopping. Unfortunately, the solution was good but not perfect, so I needed to continue trying out.
Force Sensitive Resistor
I found a sensor that could detect tension on some surface, as long as there is a proper way to attach the sensor to the surface. That looked really promising. I just need one change in my approach – new headphones.
In order to use the force sensitive resistor, I needed to find headphones with movable parts, where two surfaces clash together with some tension when they are being used. If they are not being used, that tension should decrease dramatically. The more dramatic the drop of tension, the more accurate the prediction will be. Foldable headphones looked very interesting. I placed the sensor on the foldable section of the headphones, so when the tension ceases, the measured force would be less that when they are being used (and the headphones are really squeezing the sensor).
I found the Magnat LZR 580 to be a perfect fit. As you can see, they fold and the sensor can be placed between the fixed and the foldable part. This solution was great. It did solve the issues on all scenarios. It relied on the actual aperture of the headphones (no matter the orientation or movement) at any time. This is accurate since, even kids with small heads, will make the headphones to stretch and adapt to the head shape.
Furthermore, unlike the tilt sensor, the force sensor gives analog values. That means you can create user profiles. The value measured by the sensor while being used by different users will also be different. Thus, a calibration feature can be added to keep the accuracy to the maximum. Later, you can set the threshold to pause music to be, for example, a value 10% smaller than the “on” value. The next video (also included in the first post) shows the performance of this solution. In this case, the headphones control the playback of a video in Netflix.
Let’s continue by explaining the software solution: Part 3 – Software