Interacting with computational devices was limited to using a keyboard and mouse for many years. More recently, touchscreens have become popular, especially in mobile computing. However, accurate three dimensional gestures are still difficult to recognize. One approach is the use of depth cameras, which are currently receiving much attention again in academic and in commercial fields due to the release of inexpensive consumer electronics like Microsoft’s Kinect in 2010. This presentation reflects depth camera technologies, and how gestures can be used for new interface designs. The first part establishes a framework to evaluate the suitability of depth cameras for gesture recognition. The second part focuses on interactions in three dimensional virtual spaces. I developed a gesture interface for an existing application to navigate in virtual spaces and to construct LEGO models. A user study was performed to compare the gesture input with traditional mouse and keyboard input. Finally, I discussed the influences of the findings for future research in gesture interfaces.

1. Evaluation of Gesture Interfaces
24 June 2013
Alexander Schulze
as2271@cam.ac.uk

6. hand gesture

7. depth image
sequence

8. hand gesture
position tracking
depth image sequence
gesture recognition
hand 1: U move
hand 2: show 5
gesture description

9. Agenda
Range-imaging technologies
How can a depth map be computed?
Experiment: Gestures based interfaces
Which gestures are suitable to control a computer?
Conclusion for gesture interfaces
Do gestures in"uence future interactions?

10. Range-imaging technologies

11. Range-imaging for gesture interfaces
§ What are suitable devices for gesture interfaces
to be widely used?

12. Range-imaging for gesture interfaces
§ What are suitable devices for gesture interfaces
to be widely used?
a#ordable

13. Range-imaging for gesture interfaces
§ What are suitable devices for gesture interfaces
to be widely used?
a#ordable
compact

14. Range-imaging for gesture interfaces
§ What are suitable devices for gesture interfaces
to be widely used?
a#ordable
compact
mobile

15. Range-imaging for gesture interfaces
§ What are suitable devices for gesture interfaces
to be widely used?
a#ordable
compact
mobile
markerless

16. Range-imaging technologies
Structured light
Time of "ight
Amplitude modulation

17. Range-imaging technologies
Structured light
Time of "ight
Amplitude modulation

18. Range-imaging technologies
Structured light
Time of "ight
Amplitude modulation

19. Triangulation Geometric distance computation

20. Structured Light Parallel light sources

21. Time of "ight Light distance di#erences

22. Amplitude modulation Sending light waves
coherent light source
signal
amplitude modulated light

23. Comparison Suitable depth cameras
Range
Amplitude
modulation
Accuracy
Time of
Structured
light
"ight
Image resolution
<
Time of
"ight
Amplitude
modulation
Structured
light
<
<
Time of
"ight
Amplitude
modulation
Structured
light

24. Comparison Suitable depth cameras
§
§
§
§
Depth resolution and range
Update rate and latency
Robustness
Spatial resolution of depth image

25. 0.01mm

26. Experiment: Gesture based interfaces

27. Semiotic gestures Navigation
Rotation
Zooming

28. Pseudo-ergotic gestures Manipulation
Selecting
Dragging
Dropping
Flipping

29. Results Times for traditional and gesture input
time [min] 35
30
25
20
15
10
5
0
Mouse
Gesture

30. Results Further results from questionnaires
§ Gesture input is more fun, but less e$cient and
accurate
§ Correlation between participants who never played
physical LEGO and long construction times for gesture
interface
§ Semiotic gestures are easy to use
§ Most participants prefer a combination of gesture input
together with traditional input

31. Conclusion for gesture interfaces

32. Navigation Accuracy and precision
§ Imprecise actions can be controlled by gestures
§ Gestures might support creativity

33. Manipulation Accuracy and precision
§ Gesture input cannot replace precise mouse and
keyboard input

34. Conclusion Gesture Interpretation
§ Most current gesture interfaces are built for controlling
the computer; however, gestures are a subconscious
expression of human feelings
§ Experience of haptic feedback in ergotic physical
gestures is helpful for simulating the haptic feedback in
related pseudo-ergotic gestures

35. Questions?

36. Photos
http://www.vriendinnen.be/user%les/images/maatschappij/
2011/april2011/interactiefwinkelraam.jpg
http://static.giantbomb.com/uploads/original/
8/86760/1804618-image_kinect_fun_lab_15890_2316_0007.jpg
https://www.leapmotion.com/media/leap_motion_media.zip
http://media.salon.com/2013/01/gadget-showintel.jpeg1-1280x960.jpg

37. Photos
http://www.misure.unisa.it/uploads/2814/scanner_struct.jpg
http://miac.unibas.ch/PMI/04-BasicsOfUltrasound-media/%gs/
time-of-"ight.png
https://www.leapmotion.com/media/leap_motion_media.zip
http://images.businessweek.com/ss/
06/11/1129_makingof_lego/image/intro.jpg