Keynote on Empathic Computing given by Mark Billinghurst at AWE 2015 on June 10th 2015.

1. EMPATHIC COMPUTING
Mark Billinghurst
mark.billinghurst@unisa.edu.au
June 10th 2015

2. Hiroshi Ishii – AWE 2014

3. • daf

4. ARToolKit (1999)
First open source AR SDK
http://artoolkit.sourceforge.net/

5. MagicBook (2001)
• First AR story book
• Transitional AR to VR experience

6. AR Tennis (2005)
• First collaborative AR game on a mobile phone

7. Mobile AR Advertising (2007)
• First mobile AR ad campaign (Saatchi & Saatchi)

8. AR Business Today
• Around $600 Million USD in 2014 (>$2B 2015)
• > 80% Games and Marketing applications

9. “Do you want to sell sugar
water for the rest of your life
or do you want to come with
me and change the world?"
Steve Jobs to
John Sculley 1983

10. Mark’s Midlife Crisis
• Then
• Recently Married
• Finished sabbatical at Google
• Looking for new opportunities
• Now
• Resigned my job
• Moved to a new country
• Creating a new research group

11. Interaction Technology
Natural
Time
Punch
Card
Keyboard
Mouse
Speech
Gesture
Emotion
1950 1960 1980 1990 2000 2010
Thought

12. Physiological Sensing
Emotiv Empatica

13. Interaction Technology
Natural
Time
Punch
Card
Keyboard
Mouse
Speech
Gesture
Emotion
1950 1960 1980 1990 2000 2010
Thought
Implicit
Explicit

14. Content Capture
Realism
Time
Photo
Film
Live
Video
Panorama
360 Video
3D Space
1850 1900 1940 1990 2000 2010

15. 3D Image/Space Capture
Google Project TangoSamsung Project Beyond

16. Content Capture
Realism
Time
Photo
Film
Live
Video
Panorama
360 Video
3D Space
1850 1900 1940 1990 2000 2010
2D Static
Immersive
Live
Experience

17. Networking Speeds
Log (b/s)
Time
100 b/s
10 Kb/s
1 Mb/s
1980 1985 1990 1995 2000 2010
100 Mb/s
2005

18. Network Innovation
Universal Connectivity

19. Networking Speeds
Log (b/s)
Time
100 b/s
10 Kb/s
1 Mb/s
1980 1985 1990 1995 2000 2010
100 Mb/s
2005
Text
Audio
Natural
Video

20. Natural
Collaboration
Implicit
Understanding
Experience
Capture

21. Natural
Collaboration
Implicit
Understanding
Experience
Capture
Empathic
Computing

22. EMPATHIC COMPUTING
Systems that Create and Share Understanding

23. Empathy
“Seeing with the Eyes of another,
Listening with the Ears of another,
and Feeling with the Heart of another..”
Alfred Adler

24. Empathic Computing
1. Understanding: Systems that can
understand your feelings and emotions
2. Sharing: Systems that help you better
understand the feelings of others

25. Understanding: Affective Computing
• Ros Picard – MIT Media Lab
• Systems that recognize emotion

26. Appliances That Make You Happy
• Jun Rekimoto – University of Tokyo/Sony CSL
• Smile detection + smart appliances

27. Sharing
Can we develop systems
that allow us to share what
we are seeing, hearing and
feeling with others?

28. • sdfs

29. • sdfgs

31. • axcvxca

32. Movies are like a
machine that
generates Empathy
Roger Ebert

33. Wearable AR for Empathic Interfaces
• Wearable AR can:
• Be unobtrusive
• Capture emotion
• Share sights, and sounds
• Support remote collaboration
• Enhance interaction in the real world

34. Example: Google Glass
• Camera + Processing + Display + Connectivity
• Ego-Vision Collaboration (But with FixedView)

35. Social Panoramas (ISMAR 2014)
• Capture and share social spaces in real time
• Supports independent views into Panorama

36. Implementation
• Google Glass
• Capture live image panorama (compass + camera)
• Remote device (tablet)
• Immersive viewing, live annotation

37. User Interfaces
Glass View
Tablet View

38. Lessons Learned
• Good
• Communication easy and natural
• Users enjoy have view independence
• Very natural capturing panorama on Glass
• Sharing panorama enhances the shared experience
• Bad
• Difficult to support equal input
• Need to provide awareness cues

39. Example: CoSense (CHI 2015)
• Real time sharing - Emotion, video, and audio
• Wearable (sender) – Send emotion and view
• Desktop (receiver) - See remote view and emotion
Google Glasse-Health 2.0 board
+

40. Implementation
Data Capture
Feature Detection
Emotion Recognition
Emotion Representation
Empathic User Interface
Hardware
User
Interface

41. Wearable Interface
• Google Glass + e-Health + Spydroid + SSI
• Measure GSR, pulse oxygen, ECG, voice pitch
• Share video and audio remotely
• Representative emotions sent back to Glass user
!
!

42. Desktop Interface

43. Lessons Learned
• Good
• System was wearable
• Sender and receiver mirrored emotion
• Minimal cues provided best experience
• Bad
• System delays
• Need for good stimulus
• Difficult to represent emotion

44. Scaling Up
• Seeing actions of millions of users in the world
• Augmentation on city/country level

45. AR + Smart Sensors + Social Networks
• Track population at city scale (mobile networks)
• Match population data to external sensor data
• Mine data for applications

46. Example: MIT SENSEable City Lab
http://senseable.mit.edu/wikicity/rome/

47. Example: CSIRO WeFeel Tool
• Emotionally mining global Twitter feeds
• http://wefeel.csiro.au

48. Research Challenges
• How to capture emotion?
• How to measure empathy?
• Interface/interaction models?
• How to communicate emotion?
• How to create strong empathic bonds?
• How to scaling up to city/country scale?

49. Potential Applications
• Education
• Sports training
• Rich life logging
• Remote meeting support
• Psychological treatments
• Virtual Travel/Entertainment
• Surrogate Adventure Tourism
• First responders (stress, team cohesion)

50. Questions to Think About..
What’s your Vision?
How do you want to be
remembered?

51. “Do you want to sell sugar
water for the rest of your life or
do you want to come with me
and change the world?"

52. www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au