Presentation at iLRN 2019 - paper at http://hdl.handle.net/10400.2/8368 Complexity is core part of our lives. Aware or not, people need to understand and communicate complex ideas and perspectives. Understanding and communicating complexity can be facilitated through interactive simulations. Doing so in the physical world is often impractical, however. Users and developers are overloaded with information and ambiguity, costs are prohibitive, and unsupervised physical simulations raise safety concerns. Novel immersive technology might hold the key to transforming how we tackle understanding and communicating complexity. In this position paper, we propose empowering user agency and perception to take part in complex learning experiences and create their own, combining two factors: enhanced visual and spatial context provided by location-awareness, immersive environments, and somatic, embodied agency; and enhanced cultural and social context by leveraging as input methods the rich semantics of cultural-social gestures and rituals. To deem the feasibility of this argument, we propose developing two culture-aware prototypes, one for the Confederated Salish and Kootenai Tribes in Montana, United States, and another for a Western Europe cultural context.

1. Immersive Storytelling
for Understanding Complex Systems
June 26th, 2019
Luís Fernandes, Leonel Morgado, Hugo Paredes, António Coelho & Jonathon Richter

2. Complexity
“ cognitive wars ”

6. Outlets
Direct sunlight
FeetFeet

7. Bilbo Baggins
as drawn by the author,
J.R.R. Tolkien

8. Robert
J. Lee
1968
J.R.R.
Tolkien
1930…
?
Peter
Jackson
2012
Alan
Lee
1990…
?
Horus
Engels
1957

9. How old is Bilbo, in these
illustrations?

10. 20
30
40
50How do you look at the scene, thinking
about different ages for Bilbo?

11. John Conway,
1970:
the game “Life”

12. Communicate
Understand

13. Emergence
The behaviour of flocks/crowds
cannot be predicted from individual
motions of a single bird/person
Awareness (the
“feeling of self”)
is not
predictable from
the individual
behaviour of
each neuron.
Network behaviour
(computational,
social, political,
etc.) is not
predictable from
individual
behaviours of
citizens, laws or
devices.

14. Properties that lead to emergence
a) Multiple elements
b) Non-linearity & phase transitions
c) Connectivity
d) Self-organization & evolution

15. Understanding complexity:
“ Complex Learning ” strikes back

16. 𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝑠𝑜𝑚𝑒𝑡ℎ𝑖𝑛𝑔 𝑐𝑜𝑚𝑝𝑙𝑒𝑥 ֜ 𝐻𝑦𝑝𝑒𝑟𝑐𝑜𝑚𝑝𝑙𝑒𝑥𝑖𝑡𝑦

17. Learner exerts agency and perceives the outcome

19. Return of the culture anthropolojedi
(i.e. our proposal)

20. (…)
embracing the cultural
richness and depth of
human cultures it is
expectable (…) easier to
learn and remember
[how to use technology]
Leonel Morgado, "Cultural Awareness and
Personal Customization of Gestural Commands
Using a Shamanic Interface”, 2014

21. enhanced visual and spatial context enhanced cultural and social context
location-awareness, immersive environments,
and somatic, embodied agency
input: rich semantics of cultural-social
gestures and rituals

22. enhanced visual and spatial context
exploring the interconnected
concepts and data

23. enhanced cultural and social context
more powerful & diverse
agency semantics
deeper perception
better understanding

24. enhanced visual and spatial context enhanced cultural and social context
Communicate
Understand

25. create representations of
ideas that are explorable
dynamic experiences

26. How? 2 prototypes…

27. From the child book “Blue jay brings chinook wind”
Kicking Horse Reservoir — Flathead Indian
Reservation
From the child book “Luli’s Journey”
Geese V-shaped flight formation
Confederated Salish and Kootenai Tribes Case Study

28. Confederated Salish and Kootenai Tribes Case Study

29. Data found conclusive evidence
to support benefits for Learning
Rate and Capacity, Retention
and Memorization

31. Questions: aim & shoot!