In this talk we discuss insights from designing and studying immersive experiences aimed at improving early literacy outcomes through personalized learning, spanning virtual, augmented and mixed realities as well as non-immersive applications. Our serious games provide research evidence into how these varied media can enable adults (teachers, school leaders, families, and caregivers) to implement personalized literacy learning at the organizational and individual level. We will present lessons gained from designing experiences across immersive media such as 360 video, virtual environments with agents, mixed reality systems with human-in-the-loop characters (ex: Mursion https://mursion.com/), and augmented reality. We will also discuss approaches and takeaways for creating experiences intended to build empathy towards the unfamiliar (ex: our work on parents using VR to experience the world as young children with reading disabilities), experiences for detecting unconscious biases (ex: teachers educating a stimulated classroom of students in ways that may trigger innate biases), and experiences to contextually modify parental mindsets (ex: parents using augmented reality to alter their strategies for children’s literacy). Overall, we will present general lessons from building simulated authentic situations in which teachers and parents learn to overcome challenges in early literacy development. We will pause our talk/lecture occasionally for questions that enable brief small group interactions. Presented by the Serious Play Conference seriousplayconf.com at Montreal, Canada, Quebec, UNIVERSITÉ DU QUÉBEC À MONTRÉAL, UNIVERSITY OF QUEBEC IN MONTREAL, July 10-12, 2019

1. Designing Immersive Experiences that
Create Empathy, Reveal Biases, Alter Mindsets
Chris Dede and Iulian Radu
Harvard University, Graduate School of Education
Serious Play 2019, Montreal

2. Helping Adults Learn and Unlearn
Doing Better Things rather than Things Better
● Immersion in simulated authentic experiences
● Research about barriers to changes in behavior and identity: cognitive, affective, social
● Unlearning as a stage of learning

3. Authenticity through Virtual Environments
European Astronaut Centre, ESA

4. The power of direct experience
Virtual environments for
immersive simulation

5. PC virtual reality headsets
Technologies we are exploring
Smartphone-based
interactive 360 videos and
augmented reality
PC virtual simulations

6. Affordances for adult capacity-building
Authentic experiences for teacher / parent learning
Authentic assessments of individuals and groups
Reduced cost and risk
Technology-assisted data collection
Standardized authentic training and testing scenarios
Large distribution via web and smartphones

7. Themes in Current Work
■ Reducing implicit biases and increasing empathy
■ Classroom literacy practices
■ Home literacy practices
■ Apps for literacy and play
■ Developing learning communities: teachers, parents

8. Projects: Parent Empathy & Teacher Bias

9. Previous work: VR for Empathy
Virtual Human Interaction Lab, Stanford University
VARPU, University of Lapland
Fluid Interfaces Group, MIT

10. Virtual Human Interaction Lab, Stanford University
VARPU, University of Lapland
Fluid Interfaces Group, MIT
Previous work: VR for Empathy

11. Parent Empathy Building

15. Research Questions
Can immersive VR technology simulate the experience of living as a child
with reading difficulties ?
Can parental empathy be increased through VR technology ?
To what degree does the technology medium (VR - PC) impact empathy ?

17. Empathy: Design Elements
empathy
Somatic
CognitiveEmotional

18. Empathy: Design Elements
Engage multiple components of empathy
- Somatic: generate visceral reaction
- ex: heartbeats, small perspective
- Emotional: generate compassionate reaction
- ex: inability to achieve tasks, crying
- Cognitive: transfer understanding
- ex: information about other situations and parenting strategies
Balance non-interactive 360 videos vs. user agency
Follow semi-structured narrative
Allow interrupted initial setup, for participants unfamiliar with VR tech
empathy
Somatic
CognitiveEmotional

19. Findings
Participants feel empathy, anxiety, struggle, sadness for the child
Participants feel the struggle that parent is facing
Most participants feel the experience changed them to better understand
difficulties of parents-children with reading difficulties
"when I had to go back and try again with the
heartbeat, it was poignant because I think my
expectation was that I was going to find the
book, but I couldn’t. I felt mostly the anxiety that
a child could feel."
“I knew about dyslexia from a research
standpoint so it didn’t teach me much about
the problem, but it helped me understand
what it’s like as a child and as a parent”

20. Revealing Teacher Biases

21. Source: Wikipedia

22. Danaher, K. and Crandall, C.S., 2008. Stereotype threat in applied settings re‐examined. Journal of Applied Social Psychology, 38(6), pp.1639-1655.

23. Project: Detecting Teacher Biases
Real student
performance Perceived performance
Teacher Bias / Mindsets
ex: my teaching style works well
ex: I treat all equally performing
students in the same way
ex: high SES students are smarter
ex: low vocabulary is slow learning
Observed Lesson
1 2
3

24. Unconscious Biases / Mindsets in Teachers
overconfidence
skepticism of previous failures
identity threat
no transitional identity
fear of stress
lack of resources
lack of time
lack of community
...
Educational
reform
message
Perceived message

26. Research Questions
- Are teachers biased by knowledge of student SES ?
- Can immersive technology reveal such biases?
- Does the degree of experienced engagement relate to
the strength of bias revealed?

28. User talking to virtual classroom Hidden human puppeteer
controls virtual students

29. Study Design
Observed Lesson
1 2
3
Control group:
Similar progress
No SES information
Experimental group:
Similar progress
High and Low SES students

30. Study Design

31. Bias: Design Elements
Create immersion through socio-emotional engagement of “live” virtual students,
less focus on sensory immersion
Quickly reveal the “live” nature of the system, while training user
Controlled exposure to bias anchoring content
Balance between open-ended interaction vs. scripted research
Provide opportunities to reveal biases without lengthy interaction

32. Findings
- None of the teacher participants believed they were influenced by student
background or appearance
- Low SES students are consistently perceived as making more progress
- Disbelief that low-SES students could perform as well as they did
- High SES student often perceived as better than others overall
- Tension between perceptual bias and equity awareness
“Based on the background, at least Veronica
(high-SES) has so much exposure. Maybe
that is where her ability comes from. But
Betty's case does not make sense, because
Betty (low-SES) has no family support.”
“I was immersed as I was engaged with their
raising their hands, asking me questions.
The interactions felt real.“

33. Findings
- None of the teacher participants believed they were influenced by student
background or appearance
- Low SES students are consistently perceived as making more progress
- Disbelief that low-SES students could perform as well as they did
- High SES student often perceived as better than others overall
- Tension between perceptual bias and equity awareness
“Based on the background, at least Veronica
(high-SES) has so much exposure. Maybe
that is where her ability comes from. But
Betty's case does not make sense, because
Betty (low-SES) has no family support.”
“I was immersed as I was engaged with their
raising their hands, asking me questions.
The interactions felt real.“
“I need to take background information into
consideration. For example if one child is hungry and
doesn’t have money for lunch, I would want to know
that and take it into account."

34. General Guidelines
Interrupted Training: Users are unfamiliar with VR tech, will learn & adjust. Support usability
training within the experience; prepare for interrupted initial setup.
Personal Connection: Build strong connection and agency early in the activity; possibly by using
user’s own characteristics. Connection can be reduced later.
Engage the Senses: Use methods from cinematography and interactive narrative to increase
engagement and emotional affect.
Balance Agency vs. Pre-Recorded Content: Permit user agency to some degree, but carefully
consider what aspects need to be controlled.
Theoretical vs Embodied Information: Leverage immersion to design multi-layered
experiences with cognitive, emotional and visceral effects.

35. Lessons for Demos and Research Studies
- Initial setup may require people to take VR device on/off, causing reset
- Experience should work offline
- Useful to remotely view what participant sees
- Data logging (audio-video, and in-activity) very useful for offline analysis
- Collect quantitative (ex: std. survey) and qualitative data (ex: interviews)

36. Lessons for Design Process
- Don’t overestimate technology and users. Technology breaks down, and users don’t
always know what you expect.
- Lower-fidelity prototyping possible for testing 3D scenes (ex: Sims) and branching
narratives (ex: Muzzy Lane)
- 360 video post-production tools are scarce, plan to shoot in-camera

37. Open questions for audience:
Do you know of similar work (are you doing it) ?
What impact or guidelines have you found ?

38. Designing Immersive Experiences that Create
Empathy, Reveal Biases, Alter Mindsets
Chris Dede and Iulian Radu
Harvard University, Graduate School of Education
chris_dede@gse.harvard.edu , iulian_radu@gse.harvard.edu
Acknowledgements
Katie Leech, Rhonda Bondie,
Diana Feng, Michelle Chung,
Jennifer Wang, Guanhua Nie,
Karan Bhola, Madeleine Mortimore,
Merry Chin, Tara Nair,
Raouf Seyam, Mitch Scuzzarella