By adding a sensor layer to the backside of touch screens,
the Massachusetts Institute of Technology proposes that hand gestures can be
recognized in the air in front of the display.
Smartphones such as Apple's iPhone can recognize multitouch gestures today,
such as pinching to zoom, but only if your fingers are touching the screen. Now
MIT researchers claim to have enabled gestures to be recognized in the air in
front of a display by adding a sensor array on the backside.
This weekend at Siggraph Asia (Dec.
16-19, 2009, Yokohama, Japan),
MIT will show a prototype of its hand gesture recognition technology for liquid
crystal displays (LCDs). The experimental prototype will use an integrated
sensor array behind the LCD in place of a traditional camera, but MIT claims
that its prototype nevertheless proves the concept and will stimulate touch-screen
makers to start offering their LCDs with integrated sensor layers.
Just like in the movie "Minority Report" where Tom Cruise controls a
computer display by waving his hands in front of it, MIT's demonstration shows
that its technology will enable any device with an LCD to integrate gesture
control. Rival techniques have already been developed by Microsoft, for its SecondLight project, and by Canesta (Sunnyvale, Calif.),
for its 3D
vision chips. However, SecondLight requires a display too thick for
handheld devices, and Canesta's dedicated 3D vision chip requires an infrared
emitter to detect depth.
The MIT design, on the other hand, merely attaches a very thin sensor layer
onto the backside of a touch screen, which is then synchronized with the LCD.
Normally, the LCD will display the images you are accustomed to seeing, but at
regular intervals the touch screen will display an array of apertures so
quickly that the user does not notice. Each aperture is centered on an element
of the sensor array, thereby taking thousands of simultaneous snapshots of the
user's hands in front of the display. By calculating the parallax among the
otherwise identical images, a 3D model of the user's hands enables gesture
recognition.
The apertures must be made large enough to let in enough light for the sensor
array to quickly image the user's hands, but small enough that they act like an
array of optical lenses similar to pinholes. To remedy, the MIT team designed
an aperture template consisting of a checkerboard pattern using black-and-white
rectangles of various sizes. Software then decodes the 3D image of the user's
hands from the multiple 2D images created by the apertures.
MIT is currently courting LCD makers to add integrated sensor arrays so that OEMs
can start making gesture-recognizing displays for consumer products.
Watch a video here
of the MIT gesture-recognizing touch screen in action.

Good morning from Los Angeles! #ibmcloud
That's it from me! Over to North America.
The data processing of Roland Garros 2012 (#RG12) rests on IBM Private Cloud http://t.co/JUaY1ItM [French Press release]
IBM Accelerates Business from Supply to Demand with New #Cloud Offerings For Smarter Commerce http://t.co/OFxknOb0 [Press Release]
How IBM #SmartCloud Foundation technology powers cloud adoption?
IBM VP @SLHebner explains here http://t.co/sSzfa0O5 [VIDEO]
IBM's Fiona Cullen will present ‘The Power of #Cloud: Driving Business Model’ On May 24 @ Utrecht, Netherlands #cloudforum2012 #ibmcloud
Blog Post: Why service providers should not ignore cloud http://t.co/ZfQyue4r via @eMarcusNet #thoughtsoncloud
Have any #cloudmoment? Share your story with us via Twitter, Instagram, YouTube, Facebook and tag it. See other stories http://t.co/J4ntsaQ5
Sign up now for IBM #SmartCloud Enterprise! No charge for select VMs (only till May 28). More Details >> http://t.co/2LEzOUZC #ibmcloud
RT @HansMoen: See this video from @IBMCloud to learn how to cut costs in building innovation in your business http://t.co/XOyJoFn6 #clou ...