Facebook have recently reassured users that they are not using the device microphone to snoop on conversations around our devices running the Facebook application. Although completely unrelated, a report surfaced today that a device's vibration motor could potentially be used to listen in on our conversations: it seems that there are more and more ways to use our personal electronics to listen in on us than we ever envisaged. Facebook are almost certainly not using our device vibration motors to listen in on conversations, but that it could be possible is interesting in itself.
Our story today concerns VibraPhone research from Romit Roy Choudhury and Nirupam Roy, a professor and PhD candidate at the Electrical and Computer Engineering school of the University of Illinois at Ubana-Champaign. This research is based on the premise that the vibration motor is essentially a tiny speaker, which means by design it is a microphone too. Indeed, all speakers are microphones and vice-versa, although components do need to be designed for each different role. Essentially, a speaker tends to make a poor microphone and vice-versa: speakers work by a magnetic surface changing its position as controlled by voltage, which means that a change in position caused by sound pressure can vary the voltage reported by the device. These two individuals have designed a system for using a smartphone's vibration motor to record sound, although because vibration motors are poor quality microphones there has been more than a little element of software development behind the hardware. I should also reassure readers that although the hardware has not been changed in the target devices, the vibration motor has been rewired, which although not a difficult job on the face of it, would require taking a device apart. However, in the words of an email send to the Via below from Romit, "It may be possible to exploit the power controller chip to collect the voltage fluctuation in the vibration motor, however, we did not investigate this possibility."
Once the necessary hardware hack has been performed, the vibration motor then shows changes in voltage associated with changes in sound pressure. The hardware is optimized for low frequency sound, which means that the resulting noise captured is also low frequency. The researchers used extrapolation to take a best guess as to what the original sound was and essentially to fill in the blanks. The results are surprising: four out of five people understand the original word spoken, whereas three out of five voice recognition applications understand the spoken words as recorded by the vibration motor. This is with relatively little software optimization. This sort of sound processing technology would be well suited to machine learning chips as over time this accuracy could surely be improved.
Thankfully, the VibraPhone team are not using this ability to convert our activity wristbands into eavesdropping devices (as most fitness or activity bands contain a vibration motor) but have instead looked into ways to make this technology more usable, such as with assistive hearing systems for individuals with hearing difficulties. Romit explained: "We are rather exploring if similar techniques can help us to recover speech from the subtle vibration of vocal cords, facial bones or skull. If possible, it can be useful to develop an assistive system for persons with speech impairment." There's more information in the embedded YouTube clip below and of course, the Source website.