1. Sixth Sense
Technology

2. CONTENTS
•
•
•
•
•
•
INTRODUCTION
REQUIREMENTS
WORKING
APPLICATIONS
ADVANTAGES & LIMITATIONS
CONCLUSION

3. SIXTHSENSE TECHNOLOGY
• 'SixthSense' is a
wearable gestural
interface that
augments the physical
world around us with
digital information and
lets us use natural hand
gestures to interact with
that information.

4. INTRODUCTION
•This technology was developed by Pranav Mistry.
•Sixth sense is a wearable gestural interface device.
•It is named sixth sense as it recognizes objects around
us and give information about them.

5. Sixth Sense technology is the science of tomorrow with the aim of
connecting the digital world with the physical world seamlessly,
eliminating hardware devices
The 5 senses
Here comes the sixth sense

6. About Pranav Mistry
• Born in 14 May 1981 at
Palanpur, Gujarat, India
• Now is the head of Think
Tank Team(TTT) and
Vice president of
Research at Samsung
• He is best known for his
work on SixthSense and
Samsung Galaxy Gear.

7. REQUIREMENTS
•
•
•
•
•
Mirror
Wearable Camera
Pocket Projector
Internet enabled Smart phone
Colored markers

8. Wearable camera
•It captures an object in view and
tracks the user’s hand gestures.
•It acts like a digital eye,
connecting you to the world of
digital information.
Pocket projector
•It Projects visual information & enables
surfaces, walls and physical objects to be
used as interfaces.
•It displays data sent from smart phone on
any surface in view.

9. Smart phone
• As the smart phone is open sourced.
• The software searches the web and
interprets the hand gestures.
Color markers
• They help the camera to recognize the
hand gestures.
• The movements and arrangements of
these markers are interpreted and they act
as instructions for the projected
application interfaces.

10. Working
Capture
Images
Gestures
To process the video
sequence
Send info
Vision
Project image
Info on
surface

11. Vision based gesture
recognition
• Involves hand shape, position, and movement.
• Software's used in processing are Java,.net,SYMBIAN.
• It is done through following steps:
• Image capturing
• Finger tip detection
• Feature extraction
• tracking

12. Fingertip detection
Threshold image
Detecting finger tips

13. Feature extraction
1)Circles and ellipses
extracted from an image
of a hand.
2) Selected image features
corresponding to the palm,
the fingers and the finger
tips of a hand.
3) The selected image features
capture the essential structure
of a hand.

14. Tracking hand
Size variations
rotations
State change

15. Interaction based direct manipulation and
symbolic gestures
No of fingers
Operating mode
selector
Operating mode
location
Tracking
result
Application controller for
direct manipulation
Application
Gesture mode
selector
Size &
location of the
gesture
trajectory
Application controller for
symbolic gesture
Kind of gesture
Symbolic gesture
recognizer

16. APPLICATIONS

17. Call making
• You can use the
Sixth Sense to
project a keypad
onto your hand, then
use that virtual
keypad to make a
call.

18. Maps
• With the map
application we can
call up the map of
our choice and then
use thumbs and
index fingers to
navigate the map

19. Check the time
• Draw a circle on your
wrist to get a virtual
watch that gives you
the correct time

20. Enhanced news reading
• Sixth Sense can be
programmed to
project related
videos onto
newspaper articles
you are reading

21. Drawing application
• The drawing
application lets the
user draw on any
surface by tracking
the fingertip
movements of the
user’s index finger

22. Zooming features
• The user can zoom
in or zoom out using
hand movements

23. Get product information
• Sixth Sense uses
image recognition or
marker technology to
recognize products
we pick up, then
feeds us information
on those products

24. Get book information
• The system can
project Amazon
ratings on that book,
as well as reviews
and other relevant
information

25. Take pictures
If you fashion your index
fingers and thumbs into a
square ("framing" gesture), the
system will snap a photo.
After taking the desired number
of photos, we can project them
onto a surface, and use
gestures to sort through the
photos, and organize and
resize them.

26. Weather Report

27. Awards
TR35 2009 Young Inventor Under 35
Award by Technology Review
SixthSense wins 2009 INVENTION
AWARD by Popular Science

28. Advantages
•
•
•
•
•
•
Portable.
Multi touch and multi user interaction.
Time conserving.
connects the real world and digital world.
Mind map the idea anywhere.
Data access from machine directly in real time.
Limitations
 Image noise.
 High cost of individual components.

29. Future developments
• Gaming with out controllers.
• Invisible mouse.
• 3d interfaces.

30. Conclusion
• Using this technology:
“Finally one gets a taste of the world of Harry Potter.”
• Pranav had announced in Nov 2009 that the source
code will be released with an open source licence.

31. References
• www.Pranavmistry.com
• www.Freshcreation.com/entry/sixth_sense_technol
ogy
• www.6thsensetechnologies.com
• Boingboing.net/2009/11/12/sixth-sensetechnolo.html
• ted.com/talks/pattie_maes_demos_the_sixth_sense.
html

32. Thank you