Presentation from June 21, 2015 to researchers and PhD students at the Center for Biological Cybernetics at the Max Planck Institute in Tübingen (Germany).
6. Virtual Reality Affects Firing Patterns
of Brain’s GPS Cells
Background: Selective spatial firing of neurons in the
hippocampus
Research Question: Do virtual environments activate a mental
map in the same way as the real world?
Findings: VR world stimulates response more like a
pedometer – the rat’s brains were tracking how many steps
they were taking, rather than incorporating multisensory cues
to integrate into a mental map
6
8. Presence: Definition
Presence is the “…subjective experience of
being transiently unaware of actions and
cognitions linked to the real environment while
concurrently perceiving oneself as situated in and
perceiving action possibilities associated with the
virtual environment (VE)” (Wirth et al., 2007)
8
9. High Vs. Low Presence Ratings 9
Baumgartner et al., 2008
10. Presence and The Brain
There are individual differences in how easily drawn into the virtual
environment a person feels (Ijsselsteijn and Riva, 2003; Wirth et
al., 2007)
Children and adults differ in the quality and speed with which
presence is evoked by VEs (Schaik et al., 2004)
Individual differences in susceptibility towards presence might be
modulated by the Prefrontal cortex, which is linked to executive
functions and inhibitory control over emotions and behavior (Aron
et al., 2004; Garavan et al., 1999; Koechlin et al., 2003;
Ridderinkhof et al., 2004)
10
15. Body Ownership Illusions and Agency
Body Ownership Illusions: Paired visual-tactile
stimulation (Rubber Hand Illusion) or paired visual-
motor stimulation
Tracked and rendered full-body motion in fully-
immersive virtual environments creates sense of
Agency
Bottom Line: Under specific multisensory conditions,
we can experience artificial body parts, fake bodies,
or virtual bodies as our own body parts or whole
body.
15
18. Avatar Effects Examples
Embodying a body of a different race reduces implicit racial biases
(Groom, Bailenson, and Nass, 2009)
Embodying a tall avatar increases self-confidence in negotiation tasks
(Yee and Bailenson, 2007)
Embodying an attractive avatar increases self-disclosure
(Yee and Bailenson, 2007)
Embodying a stressful posture in a virtual body can increase stress
even though physical body is not in that posture/position
(Bergström, Kilteni, and Slater, 2013)
Embodying a child avatar body causes subjects to overestimate
object sizes
(Banakou et al., 2013)
18
20. Ahn, Bailenson, & Park (2014) – Pro-
Environmental Self-Efficacy
Environmental locus of control is the perception that
one’s behaviors directly impacts the wellbeing of the
environment.
This is one of the most powerful drivers for environmental
behaviors.
20
21. Core Problem: Knowledge-to-Action
Gap
“the time-delayed, abstract, and often
statistical nature of the risks of global
warming does not evoke strong visceral
reactions.” – Weber (2006)
21
23. Benefits of Avatars
Controlled environment + Stimuli
Ability to have live interaction with your own and
another person’s avatar
Why use avatars when you can use real people?
23
25. Medial Prefrontal Cortex and Agency
(David et al., 2006)
Distinct neural substrates underlying
representations of the self and others
The medial prefrontal cortex is crucial for a
neural bases of the self and agency
Perspective taking and agency represent
independent constituents of self-
consciousness
25
27. Time-variant increase in activation in the left insular cortex during
observation of avatar hand
Imitation of avatar showed activation in angular gyrus and
extrastriate body area
Virtual hand avatar serves as disembodied training tools in
observation condition and as embodied “extensions” of subject’s
own body in the imitation condition
27FPP Hand Movement in VEs
(Adamovich et al., 2009)
29. “Think about it: there is no experience you have had
that you are not the absolute center of. The world
as you experience it is there in front of YOU or
behind YOU, to the left or right of YOU, on YOUR
TV or YOUR monitor. And so on. Other people’s
thoughts and feelings have to be communicated to
you somehow, but your own are so immediate,
urgent, real.”
- David Foster Wallace, “This is Water” (2005
Kenyon Commencement Speech)
29
30. Machine to Be Another: Live Stream
Perspective Swapping
30
32. THE MACHINE TO BE ANOTHER:
An Empathy Machine?
32
“Empathy is created the moment we try to put ourselves in
another person’s shoes.” – Aspen Baker, TED Radio Hour
“The machine to be another is designed to stimulate empathy
through embodied interaction between individuals.” (Bertrand et
al., 2014)
To what extent can we experience and understand what it is like
to be someone else?
36. How to Combine These Experiences?
Be Another Labs use live stream into the DK2
Benefit: Real-Time Interaction
Con: Low visual resolution + not stereo
Original plan: Live Stream in Stereo – Low visual
resolution + Delay
Used Stereo and Pre-Recorded Footage with
Binaural audio
36
46. Introducing Ex Nihilo
In The Republic, Plato asks, "Will we
say, of a painter, that he makes
something?" and answers, "Certainly
not, he merely imitates."
46
48. Subject Quote
“It almost feels like collectively discovering
something that was in her head all along. The
experience allowed this completely different way
of connecting to someone that doesn’t rely on
facial expressions or anything.”
48
50. Measuring Social Interaction in VR
How do you measure two people’s ability to
communicate effectively?
What does communicating “effectively” mean?
Suzanne Dikker – “shared semantic
understanding” and neural synchrony
50
52. Perceptual Crossing (Auvary et al.,2009)
Task: Participants recognize one another while avoiding
shadow and static objects
Mutual tactile interaction (trackball mouse and haptic
feedback)
Static object = too stable and predictable to be human
Shadow = too unstable since it moves but does not respond
Other subject’s avatar can react with communicative intent
52
53. Froese and Di Paolo (2011)
Hypothesized that the lack of personal recognition of the other in the
perceptual crossing paradigm was due to lack of a genuinely social
experimental task
The mark of the social is the co-regulation of mutual interaction
53
54. Primary Intersubjectivity in Infants
2-month old infants get distressed when shown a pre-recorded video of an
interaction with their mother, as opposed to a live streamed video.
Simple recognition of a pattern of movements is not sufficient, and instead
social action perception requires an understanding of how other's
movements are related to our own.
Infants display movement synchrony with adult speech (Condon & Sander,
1974)
Still face studies (Toda & Fogel. 1993; Tronick, Als, Adamson., Wise. &
Brazelton, 1979)
54
