Microsoft Turns Jazz Hands Into Gesture Commands Using Sound Waves
By Kevin Lee
Most gesture-based control systems we use today rely on either motion-capture cameras–like the Kinect–or a touchscreen device. But researchers from Microsoft Research and the University of Washington are developing a system that can detect object with sound waves, like how a bat does with echolocation.
With the SoundWave project, the researchers aim to bring gesture controls to any computer that has a set of speakers and microphone. The program uses the Doppler Effect of sound waves for to detect objects and recognize motion.
The Doppler Effect, as you might remember from so many years of science classes, is a principle of physics where the frequency of waves (sound waves, light waves) shifts depending on whether the source of that sound (or light, etc…) is moving toward or away from you. The most common example is a passing train, where the sound coming from the train seems to get deeper as it passes you.r.
SoundWave uses your computer’s speakers to emit an imperceptible, ultrasonic audio signal at 20-22 Kilohertz–and you can still play music while it’s doing this. If you wave your hand in the vicinity of your computer, the sound waves that would have traveled through the air bounce off of your hand instead.
The computers microphone can pick up the audio signal and the SoundWave program can interpret any changes in the frequency as a gesture command. So far the team has been able to detect five different variables involved with a change of frequency, including the speed, direction, proximity, size, and number of the objects, as well as the frequency of the motion.
From this, the Soundwave program can perform a number of the usual computer commands such as scrolling, clicking and double clicking, rotating objects, or automatically locking the computer while you’re away. The researchers say that SoundWave is 90 to100 percent accurate, even in a loud crowded room, and can even be used to play a round of Tetris.
The most significant advantage of the SoundWave system is that it could add gesture controls to computers and laptops that don’t have a touchscreen display or a webcam. That said, the practicality of flailing your hands around over your keyboard and trackpad to use your laptop is questionable.