User defined gestures for surface computing

1. Presented By: SUMMIYA FARHEEN
8th Sem CSE

3. INTRODUCTION
What is a gesture ?
Gestures are expressive , meaningful body motions with the intent to convey
information or to interact with the environment.
Body motions  physical movements of the fingers, hands, arms head, head,
face or body.
What is Surface computing ?
It is a computer that interacts with the user
through the surface of an ordinary object,
rather than a keyboard or monitor.
User interacts with touch sensitive screen.
People use their fingers to manipulate it.

4. A key challenge
Traditional input using the keyboard, mouse, and mouse-based widgets is no
longer preferable
No keyboard used
No mouse used
Human-computer interaction have been exploring interactive
TABLETOPS for use by individuals and groups, as part of
multi-display environments, and for fun and entertainment.

5. STRUCTURE OF TABLETOP
History
 This idea came up in 2001 by the Microsoft researchers “Steve bark son” &
“ Andy Wilson”.
 In 2003 they presented to bill gates for the approval.
 Later in 2007 this interactive table top device was built.
Tabletop Computer

6. WORKING OF MICROSOFT SURFACE
The Surface is not a touch-sensitive screen device…
It uses multiple infrared cameras beneath the screen to sense objects,
physical touch, etc. .
This information is processed and
displayed using “Rear Projection”.
(1)Screen Diffuser :
 ”Multi-touch" screen.
 Can process multiple inputs
and recognize objects by
their shapes or coded
"domino" tags.

7. Cont.…
(2) Infrared:
 The ”Machine Vision" is aimed at the screen.
 Once an object touches the tabletop ;
the light reflects back and is picked up by infrared cameras.
(3) CPU:
 Uses similar components as current desktop computers, Core 2 Duo processor,
2GB of RAM and a 256MB graphics card, Wireless communication,
Wi-Fi and Bluetooth antennas, Operating system.
(4) Projector:
 Uses a DLP (digital light processing) light engine.

8. CLASSIFICATION OF HUMAN GESTURE
“Microsoft Surface represents a fundamental change in the way we interact with
digital content.
With Surface, we can actually grab data with our hands, and move information
between objects with natural gestures and touch.”
1. Efron
 conducted studies of human gesture he categorized
it into five categories –
a) Physiographic
b) Kinetographic
c) Ideographic
d) Deictic and
e) Batons.

9. 2. Kendon
 Showed that gestures exist on a spectrum of formality and speech-dependency.
 From least to most formal, the spectrum was: language-like gestures, emblems,
and sign languages.
3. Tang
 Analyzed people collaborating around a large drawing surface.
 Gestures emerged as an important element for simulating operations,
indicating areas of interest, and referring to other group members.
 Tang noted actions and functions, i.e., behaviors and their effects,
which are like the signs and referents in our guessability methodology.

10. CLASSIFICATION OF SURFACE GESTURES
 Three authors independently designed
their own gestures for the 27
Referents.
 Among those 27 referents Four were
not assigned as gestures:
Insert
Maximize
Task switch
Close

11. Instead 
• Insert used by dragging
• Maximize used by enlarging
• Task switch by tapping an imaginary taskbar button.
• Close by tapping an imaginary button in the top-right corner of an open view.
 And they manually classified each gesture along four dimensions:
1. Form
2. Nature
3. Binding
4. Flow

12. 1. Form:
• The scope of this dimension is within one hand.
• It is applied separately to each hand in a 2-hand gesture.
• These are worth distinguishing because of their similarity to mouse actions.
2. Nature:
• In the nature dimension, symbolic gestures are visual depictions.
• Examples :
 Tracing a caret (“^”) to perform insert,
Or
 Forming the O.K. pose on the table (“ ”) for accept.

13. 3. Binding:
• In this dimension, object-centric gestures only require information about the object
they affect or produce.
• An example is pinching two fingers together on top of an object for shrink.
4. Flow:
• A gesture’s flow is discrete if the gesture is performed, delimited,
recognized, and responded to as an event.
• An example is tracing a question mark (“?”) to bring up help. .

14.  Although the authors are experts in human computer interaction, it was
hypothesized that the “wisdom of crowds” would generate a better set than the
authors.
 Each author individually came up with only 43.5% of the user-defined
gestures.
 Even combined, the authors only covered 60.9% of the users’ set.
 This suggests that three experts cannot generate the scope of
gestures that 20 participants can.

15.  Later the authors asked 20 people to participate to find different gestures based
on their ideas thinking it would make better gesture set.
 Among those 20 participants some used 1-hand gestures and some used 2-hand
gestures.
 Totally they made 1080 gestures set .
[27refrents X 20 participants X 2 = 1080]

16. THE USER-DEFINED
GESTURE SET

18. How is the Surface used ?
• Wireless! Transfer pictures from camera to
Surface and cell phone. “Drag and drop virtual
content to physical objects.”
• Digital interactive painting using paint brush
• At a phone store.
• At a restaurant.
• Play games and use the Internet.
• Jukebox! Browse music, make play lists.
• Maps and Billboard for advertising

19. FEATURES OF SURFACE COMPUTING
A form of computing that offers “a natural way of interacting with information,”
rather than traditional user interface:
• Direct Interaction: The ability to "grab" digital
information with hands - interacting with
touch/gesture, not with a mouse or keyboard.
• Multi–Touch: The ability to
recognize multiple points of
contact at the same time, not just
one (Ex. one finger, like with most
touch screens), but dozens.

20. • Multi–User: The Surface’s screen is horizontal,
allowing many people to come together around it
and experience a “collaborative , face–to–face
computing experience”.
• Object Recognition:
Physical objects can be placed on the
Surface’s screen to “trigger different
types of digital responses” .
(Ex. cell phones, cameras,
& glasses of wine).

21. WHO’S USING SURFACE TODAY ?
Financial Services
Healthcare
Leisure and
Entertainment
Retail
Casino
Hotels and Resorts
Disney Innovations
AT&T
 T-Mobile

22. MICROSOFT SURFACE PROBLEMS
• $$$$$$$ Surface machines very
expensive, cost between $5,000 and $10,000.
•Not portable .
•Poor accuracy as fat fingers are not as accurate
as mouse or stylus.
•Objects need to be tagged (domino tags)
• Must own devices such as a cell phone to upload photos into
or share with others.
• Have to be careful of the table surface not to damage it.

23. CONCLUSION
o The study of surface gestures leading to a user-defined gesture set based on
participants’ agreement over 1080 gestures.
o Beyond reflecting user behavior, the user-defined set has properties that
make it a good candidate for deployment in tabletop systems,
such as
 Ease of recognition
 Consistency
 Reversibility and versatility through aliasing.

24. o Also a taxonomy of surface gestures useful for analyzing and characterizing.
o This work represents a necessary step in bringing interactive surfaces closer
to the hands and minds of tabletop users.
o Its not just a touch screen but more of a touch -grab-move-slide-resize-and-
place-objects on top of the screen.
o Surface takes existing technology and presents it in a new way.
o Fundamentally changes the way we interacts with technology.
o So it opens up new possibilities that weren’t there before.

25. “A COMPUTER ON EVERY DESKTOP “
NOW WE SAY
“EVERY DESKTOP WILL BE A COMPUTER”
THANK YOU…… 