Display Tech to Watch This Year: Multitouch Catches Fire
More gestures on the way
Multitouch system designers are anxious to expand on the repertoire of gestures pioneered by Apple. Synaptics, for instance, offers its Scrybe gesture suite, which lets system designers choose from a common library of gestures and create a few customized ones of their own.
"You could, for example, [designate] a gesture to take you right into Amazon.com to buy a product," Hsu says. Scrybe is currently marketed only for laptop touchpads, but Synaptics says the technology could be extended to touch screens.
See a quick demo of Scrybe gestures, currently only for laptop touchpads.
Swype offers a gesture suite for Android and other device platforms that lets users type by sliding their fingers over a virtual keyboard rather than tapping it. And GestureWorks' open-source gesture library offers more than 200 multitouch gestures to Flash and Flex developers.
Apple, too, is working to expand its gesture suite in its upcoming iOS 4.3, adding new swipe functions and support for up to five fingers -- although the company has made it clear that it doesn't intend to bring multitouch to vertical displays on desktops or laptops.
[Related: See the latest coverage of Apple's iPad 2.]
But these are not cross-platform standards, and HP's Bosley thinks the new gestures are less universal -- and less intuitive -- than the three fundamental gestures popularized by Apple. "I saw a [recent] Apple patent, and it looked like American Sign Language. Why would anyone want to learn a new language?" he asks. "People will use standard, intuitive gestures that make sense on the screen. I don't think they'll be willing to learn a whole vocabulary of them."
Mike McSherry, CEO of Swype, concurs. "It's not realistic to expect the average user to learn more than a couple-dozen gestures," he says. He thinks gestures will continue to be used mainly for launching apps, navigating and the like.
Moving into three dimensions
Capacitive touch-screen technology's proximity-sensing abilities mean that it can detect movements not just in the X and Y planes across the screen's surface but along the Z axis as well -- it can sense a finger as it approaches the touch surface. In the future, proximity sensing could carry touch-sensing screens into three dimensions -- if touch-screen makers can figure out how to apply it.
The technology has the potential to interpret not just the proximity of fingers but the gestures they make. For example, when a user flings his fingers outward, the touch screen might interpret that as a command to zoom in on an image on-screen.
"We can sense very, very small changes in capacitance, and because of this, we could sense swipes of a hand or opening and closure of a fist," says Trevor Davis, director of marketing for touch screens at Cypress Semiconductor. Interpreting such sophisticated gestures, however, is a challenge.
On the one hand, simple "hover sensing" applications are already in use, such as a smartphone display that switches off when you hold it close to your ear, or the LED buttons that light up on the touch-screen display of Dell's SP2009W monitor only when the user's hand approaches the screen. But detecting the presence of a finger above the display is the easy part.
"The biggest problem is trying to decipher user intent," says Hsu. Projected capacitive sensors don't know if the finger they detect hovering over the surface was placed there intentionally -- and, if so, what the user wants to do.
Even within an application context, the inability to clearly ascertain intent leads to usability challenges. "Once you dig through all of the interaction scenarios, proximity is really challenging," Hsu says. The only viable use of proximity sensing today, he says, is for a simple device wake-up function.
Nonetheless, there's quite a bit of R&D going on around the idea of using proximity sensors as a user interface, especially in automotive in-dash control systems, says Jérémie Bouchaud, an analyst at iSuppli. Developers are working on 3D gestures that will allow users to zoom in and out or move a map, and to "flick" content from one side of the display to the other by waving. The systems may also be able to discern whether the hand doing the waving belongs to the driver or the passenger, preventing the system from responding to driver gestures for safety reasons.
And for bigger touch screens that use optical rather than capacitive input, researchers are working on 3D optical touch technology that will detect motion within 50 cm of the screen. "These kinds of systems are expected within two years," Bouchaud says.
Proximity sensing has other benefits that go beyond gestures. For example, says Davis, smartphone manufacturers could use proximity sensing to determine whether a device is sitting on a table, in your hand or on your lap, and adjust radio emissions and fan activity accordingly.
Beyond traditional computing devices
However multitouch gestures develop, one thing is clear: Multitouch displays will continue to proliferate beyond smartphones and tablets and into a wide range of computing and consumer electronics products that people interact with every day.
Multitouch controls have already been integrated into high-end cameras, cars and even home appliances, says iSuppli analyst Rhoda Alexander, and will gradually expand into lower-end models as well. She says multitouch probably won't be embedded into large-screen televisions, but remote controls with electromechanical buttons will start to give way to multitouch controllers and multitouch apps running on smartphones, such as the L5 Remote for the iPhone.
"We're really just seeing the tip of it right now," Alexander says. "As we move forward, you're going to see all sorts of devices that were traditionally electromechanical using touch screens, and in particular multitouch, to drive the devices."
Check back next week for the next red-hot display technology to watch in 2011.