Paper from UIST 2007

Comments:
Comment #1
Comment #2
Comment #3

LucidTouch: A See-Through Mobile Device
Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, Chia Shen

Link to UIST 2007 paper and video.

Overview:
A potential drawback of touchscreen devices (on mobile phones) is that the user's fingers may obstruct his/her view of the screen or item they are selecting. This paper discusses a device (LucidTouch) that allows the user to control the application by touching the back of the device. This is done by overlaying an image of the user's hands onto the screen, making it appear that the device is semi-transparent. This allows users to easily select things on the screen that could otherwise potentially be obscured by their fingers or hands.

Device:
It combines a multi-touch input surface with a semi-transparent display that overlays an image of the user's hands onto the screen (illuminated semi-transparent bi-directional diffuser layer). It makes use of a camera mounted on the back of the device to capture video of the user's fingers and hands. Since there are eight possible contact points, something was needed to help users distinguish between fingers on the display. The LucidTouch makes use of red dots (hovering fingers) and blue dots (in contact with device).

Experiment Features:
The traditional QWERTY layout was used. Additionally, a modified version was also used that split the keyboard in half and reoriented it so that the user could maintain the usual "home row" while holding the device. While it might benefit touch typists, it could potentially be confusing for users who are unable to do this (look at keys when typing on a regular computer keyboard).

Since one finger can't span the entire screen space, coordinated actions between hands may be required (handing-off items from one to the other). LucidTouch expands small items as they get closer to the center of the screen to support easier hand-off between fingers.

Traversing a map was made easier by allowing it to take advantage of the multi-touch interface. The map-browsing application supported rotation, translation, and scaling, while remaining under the user’s fingers.

Experiment:
Map Browsing- users were presented a map of Cambridge, Massachusetts and asked to find the location of the lab that they were in. The task was presented four times: 1) Use only one thumb on the front of the device. 2) Use a thumb on the front and a finger on the back. 3) Make use of all fingers on the back of the device. 4) Overlaying image of the hands were removed and only the touch-cursors could be seen.

Text Entry- users were asked to type their name using both of the keyboard layouts. Each layout was used twice. First, they only used their thumbs on the front of the device. Then, they were allowed to use fingers on the back. When entering on the back of the device, first the semi-transparent image of the hands was used, and then it was removed.

Drag & Dock- users were asked to select an item and drag it to a specified location on the screen. They followed the same conditions used during the text entry part of the experiment.

Results:
For most tasks (all except for non-inverted QWERTY keyboard), participants found that using the multi-touch interface on the back was useful in accomplishing their tasks. Most users found that the semi-transparent overlay was useful. Without it, they found it difficult to determine which touch-cursor corresponded with which finger. They found that it would be useful to vary the pseudo-transparency, in order to minimize its intrusiveness.