The document discusses how virtual reality can be used to replay and re-experience past events. It provides two case studies where VR was used to simulate an archaeological site and the historical city center of Sao Paulo in 1911. The authors conclude that VR has great potential as a medium to immerse users in multi-sensory simulations of past, present, and future environments and experiences in a way that feels like the user has become part of the environment.

1. Replay!
When past experiences are
reassembled through virtual reality
Prof. Eduardo Zilles Borba, Ph.D.* (ezb@lsi.usp.br)
Prof. Marcelo Knorich Zuffo, Ph.D (mkzuffo@lsi.usp.br)
* Fellow support by CNPq-Brazil
.............................................................................................................
Center of Interdisciplinary and Interactive Technologies (CITI-USP)
University of Sao Paulo, Brazil (USP)
IAMCR Conference – Leicester (UK), July 27-31, 2016
(Digital Divide Working Group)

2. IAMCR 2016 Conference - Leicester (UK) PG. 02/22
Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Topics
1) Virtual Reality (VR) as a multi-sensorial Media
2) Registering and preserving memories through VR
3) Replay! Re-living experiences with VR (cases and
discussion)
4) Conclusion

3. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
1) VR as a multi-sensorial Media
Nowadays much has been said about Virtual Reality (VR).
(Graft, 2014)
VR is an advanced user-computer interface which allows
the user to visualize, to interact and to manipulate digital
contents through a virtual experience that reproduces
aesthetics and activities from the physical world (or the
imaginary one). (Kirner and Tori, 2004)
popularization among the general public
immersion feeling
IAMCR 2016 Conference - Leicester (UK) PG. 03/22

4. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
multi-sensorial media experience
IAMCR 2016 Conference - Leicester (UK)
To create a simulation scene, more than a 3D image is
necessary. So, VR makes use of electronic devices such as
stereoscopic goggles, motion sensors and others technical
equipment responsible for stimulating sensory experiences
of the human body, specially: sight, hearing and touch.
(Zilles Borba, 2014)
When our body is stimulated to believe it really inhabits
other space, we have a feeling of IMMERSION. And, there
are 3 categories for immersion in VR:
PG. 04/22

5. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
Realism
Involvement Interaction
Zilles Borba and Zuffo (2015)
Immersion
IAMCR 2016 Conference - Leicester (UK) PG. 05/22

6. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
Realism
(computer graphics)
Realism + Involvement
(360º enviroment/scene)
Realism
+ Involvement
+ Interaction
(Natural to the User
Interactions, NUIs)
IAMCR 2016 Conference - Leicester (UK) PG. 06/22

7. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
Immersive VR vs. Non-immersive VR
(HMD, CAVEs, etc.) (TV, monitor screen, etc.)
Sense of presence
Virtual content/environment/scene
Media device
IAMCR 2016 Conference - Leicester (UK) PG. 07/22

8. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
2) Registering and preserving
memories through VR
It is already known that media have always been (and still
are) responsible for registering important facts of humanity
“to freeze moments”
“extensions of mankind”
photo,
video,
VR (?)
IAMCR 2016 Conference - Leicester (UK) PG. 08/22

9. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
But, in a multi-sensory way, we can say VR
After all, VR can reproduce the sense of space, objects,
activities and people relationship in a very realistic way
(forms, scales, proportions, depths, colors, lights and shadows).
photo (visual)
video (audiovisual)
i.e.
more than admiring a landscape,
through VR users can experience it!
IAMCR 2016 Conference - Leicester (UK) PG. 09/22

10. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
CAVE HMD
HMD + NUIs
IAMCR 2016 Conference - Leicester (UK) PG. 10/22

11. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
With VR, we can create simulations of spaces, scenes,
moments, activities to:
• Preserve feelings (sensations)
• Reconstruct situations (geographic, architectonic or
atmospheric notion of places)
• Explain emotions to other people (memories)
It might be related to ephemeral objects, people,
zones or even actions from the past/present to be
replayed by future people
IAMCR 2016 Conference - Leicester (UK) PG. 11/22

12. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Examples:
• User could feel like what it was to be in the middle of a
dense jungle during a war conflict that doesn’t exist
anymore because deforestation phenomenon;
• People could participate in musical gigs of an extint
band through a 360º video experience;
• A father/mother could interact with the hologram of
their children to remember family and childhood
memories.
» Those examples are all related to interesting topics on
cyberculture: telepresence, time-travel illusion, immersion...
IAMCR 2016 Conference - Leicester (UK) PG. 12/22

13. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
IAMCR 2016 Conference - Leicester (UK) PG. 13/22

14. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
3) Replay! Re-living experiences
with VR(cases and discussion)
» Research: the main objective of this work was to conduct
an initial (but relevant) discussion about VR as a media
that allows user to play past experiences in an immersive
and multi-sensorial way.
» Methodology: We decided to explore two cases about
past experiences reassembled using VR techniques created
by our research lab (CITI-USP):
a) Archaeological Site of Itapeva (2015)
b) Historial City Center of Sao Paulo (1911)
IAMCR 2016 Conference - Leicester (UK) PG. 14/22

15. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Case 1: Archaeological Site of Itapeva (2015)
“More than create a sensation of transporting the user to the
archaeological site, this work presents a great goal about how
VR can be particularly used to explore ephemeral spaces and
objects in a non-destructive way”, (Zilles Borba and Zuffo, 2016, p.7).
• Goal as a media:
1) To revive (or live for the first time) the feeling of exploring places
with really hard access (i.e. general public).
2) To observe and to perceive details of the landscape in a digital
model available anywhere and anytime (i.e. archaeologist)
3) To practice excavation or to view the digging process (i.e. students)
IAMCR 2016 Conference - Leicester (UK) PG. 15/22

16. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Case 1: Archaeological Site of Itapeva (2015)
• Oculus Rift (HMD): this VR device allows user to visualize the
space in a 360º view through the first-person perspective
(the user became the avatar; symbiosis between user and
avatar; the self-avatar).
• Razer Hydra (3D joystiks): provides a natural to the user
interaction (NUI), so it’s possible to navigate around the
archaeological space and to manipulate objects.
• Project web site: http://cavernadigital.org.br/ciberarch/index.html
IAMCR 2016 Conference - Leicester (UK) PG. 16/22

17. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
IAMCR 2016 Conference - Leicester (UK) PG. 17/22

18. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Case 2: Historical City Center of Sao Paulo (1911)
“The main objective was to create a time-travel illusion that
allows the user to fell the sensation of walk by a place that no
longer exists”, (Zilles Borba and Zuffo, 2016, p.8).
• Goal as a media: to reproduce more than an image of the
heritage urban space. It means, to reconstruct situations and
explore culture, architecture or habits of the brazilian people
in that period of History.
IAMCR 2016 Conference - Leicester (UK) PG. 18/22

19. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
» Case 2: Historical City Center of Sao Paulo (1911)
• CAVE: a multi-user experience inside a 3D projection room
(steoscopic goggles, real scale images, etc.). Also the audio
was mapped around the cubic room.
• Motion sensors: above the CAVE there were infrared
trackers to identify user position and to allow her/him to
navigate through the virtual environment. Also, a Wii remote
allows the user to manipulate objects (bluetooth technology).
IAMCR 2016 Conference - Leicester (UK) PG. 19/22

20. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
IAMCR 2016 Conference - Leicester (UK) PG. 20/22

21. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
Conclusion
» Self-avatar: it was concluded VR has a huge potential to create
multi-sensorial experiences to explore past, present and, even,
the future. It means, VR platforms will allow people to replay
experiences like the´re actually a part of the enviroment
» Huge potencial as media: it was also concluded that its
narratives and aesthetics gave a wide range of possibilities for the
Media field (journalism, advertising, marketing and PR actions).
» Immersive feelings: VR devices are bringing a new perspective to
our Media mass content consume, after all we finally are facing a
multi-sensorial experience with interfaces that really create the
feeling of being another or living/inhabiting virtual places.
IAMCR 2016 Conference - Leicester (UK) PG. 21/22

22. Replay! When past experiences are reassembled through virtual reality | Zilles Borba and Zuffo (2016)
Bibliography
» Graft, M. (2014). The 5 Trends That Defined the Game Industry in 2014. [Online]. Available:
<http://www.gamasutra.com/view/news/232257/The_5_trends_that_defized_the_game_indust
ry_in_2014.php> [24/jul. 2015].
» Kirner, C. and Tori, R. (2004). Introducao a Realidade Virtual, Misturada e Hiper-realidade. In:
Kirner, C. and Tori, R. (Eds.). Realidade Virtual. Sao Paulo: Ed. Senac, pp.3-20.
» Kerckhove, D. (1995). The Skin of Culture: Investigating the New Electronic Reality. London:
Kogan Page.
» McLuhan, M. (1964). Understanding Media: The Extensions of Man. New York: McGraw-Hill.
» Zilles Borba, E. and Zuffo, M. (2015). Natural to the Human Interactions with Digital
Interfaces: a new perspective to understand the virtual experiences. In: IAMCR Conference
Proceedings 2015. Montreal: Universite du Quebec a Montreal, pp. 312-317.
» Zilles Borba, E. (2014). Imersao Visual e Corporal: Paradigmas da Percepcao em Simuladores.
in: Soster, D. e Piccinin, F. (Orgs.). Narrativas Comunicacionais Complexificadas II – A Forma.
Santa Cruz do Sul: Edunisc, pp.239-258.
IAMCR 2016 Conference - Leicester (UK) PG. 22/22

23. thank
Prof. Eduardo Zilles Borba, Ph.D.* (ezb@lsi.usp.br) » Twitter: @ezillesborba
Prof. Marcelo Knorich Zuffo, Ph.D (mkzuffo@lsi.usp.br)
* Fellow supported by CNPq-Brazil
.............................................................................................................
Center of Interdisciplinary and Interactive Technologies (CITI-USP)
University of Sao Paulo, Brazil (USP)
you!
IAMCR Conference – Leicester (UK), July 27-31, 2016
(Digital Divide Working Group)