Research seminar at the University of Brighton presenting the ARISE project. Here's the abstract: Augmented Reality (AR) has a range of affordances that resonate with learning theory. Reflecting the early stage of the technology however, much existing research into AR focuses on technical issues and is based on prototype systems not suitable for end user deployment. This presentation gives an insight into experiences gained in the European ARiSE (Augmented Reality in School Environments) project, which aims to develop a robust and affordable AR teaching platform suitable for deployment in schools. In order to evaluate the tabletop AR learning platform, three consecutive prototypes of educational applications were produced, each reflecting the evolving technological capabilities of the platform and addressing different pedagogical approaches. These ranged from process visualisation in a human biology setting based on behaviourist and cognitive approaches, through guided construction of chemical elements based on constructivist ideas, to knowledge creation through communication and negotiation of meaning around cultural heritage objects with peers from another country, based on theories of social learning. The first two applications have been evaluated in summer schools involving video observations and interviews with secondary school students from Romania, Latvia and Germany. Preliminary results indicate a high acceptance rate for the AR learning platform amongst students. The main advantages were seen in the 3D visualisation capabilities and the haptic user interface, which led to increased motivation, better concentration during learning activities, and faster and more accurate understanding of the learning content. Balancing these positive results were technical and usability issues that had a negative impact on the learning process.

1. Augmented Reality in School Environments CMIS Research Seminar, 19 th November 2008

3. Two minute overview - general idea Use Augmented Reality to improve teaching and learning: 1) Affordable Augmented Reality learning platform suitable for classroom environments 2) Content authoring by experts, teachers, students 3) e-learning standards compliance CMIS Research Seminar, 19 th November 2008

4. Fraunhofer Gesellschaft (IAIS), Germany University of Brighton, UK Siauliaui University, Lithuania Czech Technical University, Prague, Czech Republic National Institute for Research and Development in Informatics, Bucharest, Romania Siauliai City Juventa Basic School, Lithuania Rabanus-Maurus Gymnasium Mainz, Germany Two minute overview - project partners CMIS Research Seminar, 19 th November 2008

5. Overlay the real world with virtual objects Augmented Reality o Combines real and virtual objects in a real space o Registers objects in 3-D o Is interactive in real time Azuma et al (1997) Augmented Reality - definition CMIS Research Seminar, 19 th November 2008

7. Reality-Virtuality Continuum (Milgram and Kishino, 1994) Taxonomy for MR interfaces (Milgram and Kishino, 1994) • World Knowledge ("how much do we know about the world being displayed?") • Reproduction Fidelity ("how realistically are we able to display it?") • Presence Metaphor ("extent of the illusion of being present within that world?") Augmented Reality - classification CMIS Research Seminar, 19 th November 2008

8. Common uses: guidance in assembly, maintenance, training Augmented Reality – common uses 1 azuma97_printer_maintenance azuma97_engine_parts_labels azuma97_robot_path_planning azuma97_boeing_wire_bundle_assembly CMIS Research Seminar, 19 th November 2008

9. Common uses: medical Augmented Reality – common uses 2 azuma97_foetus inside womb of pregnant patient azuma97_biopsy 3-D graphics guide needle insertion … aviation and military applications azuma01_battlefield_AR_system CMIS Research Seminar, 19 th November 2008

10. Little research into educational uses of Augmented Reality MagicBook, SOLAR, Volcano Kiosk, etc. Billinghurst, Kato, Woods et al. (2002+) HITLab , New Zealand Studierstube, e.g. for Mathematics and geometry education Kaufmann & Schmalstieg (2003), Vienna Univ. of Technology Solar system, Plant mixed reality system Liu & Cheok (2007), Mixed Reality Lab, National Univ. of Singapore What are the specific learning affordances of AR? Affordances are “the perceived and actual properties of a thing, primarily those functional properties that determine just how the thing could possibly be used” (Salomon 1993). Augmented Reality – educational uses CMIS Research Seminar, 19 th November 2008

11. Presentation of objects in 3D o exploration of spatial problems that are difficult to grasp in 2D media e.g. Kaufmann & Schmalstieg (2003), Liu & Cheok (2007 o supports the development of spatial abilities , which are considered an important component of human intelligence (Gardner 1983) Learning Affordances – spacial presentation CMIS Research Seminar, 19 th November 2008 Kaufmann & Schmalstieg (2003) Woods et al (2004) SOLAR

12. Tangible user interfaces o No artificial seam between real world and digital task space (Ishii et al 1994) o May be more suitable for younger children (Billinghurst 2002) Learning materials and activities should involve the appropriate level of motor and mental operations for a child of given age (Piaget 1970). Learning Affordances – tangible user interfaces CMIS Research Seminar, 19 th November 2008 ARiSE 2007

13. Different views on the same object or situation o Facilitate extrapolation by prompting learners to go beyond the information given (Bruner 1973). o Cognitive development through adaptation by giving alternative views on already familiar objects or situations (Piaget 1970). Learning Affordances – dynamic visualisation CMIS Research Seminar, 19 th November 2008 Woods et al (2004) Volcano Kiosk MacIntyre et al (n.d.) Three angry men

14. Collaboration o Co-located collaboration using natural communication o Remote collaboration with shared task space Learning through communication, imitation and social interaction (Bandura, 1977; Vygotsky 1978) Feedback helps learners to reflect upon and conceptualise learning (Kolb 1984) Peer tutoring ( teachback ) a critical method of learning. (Pask 1975) Learning Affordances – collaboration CMIS Research Seminar, 19 th November 2008

15. Smooth transition from real to virtual Learning content should be interconnected and present information in different contexts (Spiro 1988) Knowledge is situated and needs to be presented in context (Lave and Wenger 1990) Supports a range of learning and teaching approaches by providing a bridge from instruction to construction. Learning Affordances – real to virtual transition CMIS Research Seminar, 19 th November 2008 Woods et al (2004) Whale bone club Billinghurst & Kato (2002) MagicBook

16. Engages and motivates “A great majority of the respondents referred to motivation and interest in the environment … as one of the most outstanding qualities” (Lamanauskas 2007) “ 3D augmentations and physical interaction tools are motivating for children and encourage playful interaction” (Hornecker and Dunser 2007) Motivation is a key concept in most theories of learning (e.g. Hull 1943, Bruner 1966, Piaget 1970) Lack of motivation and failure to make learning enjoyable are the chief impediments to learning (Csikszentmihalyi 1990) Learning Affordances – engages learners CMIS Research Seminar, 19 th November 2008

17. Aim: Leverage these learning affordances for ‘real world’ learning in school environments 1. Create an affordable, networked AR learning platform 2. Integrate the platform with learning management systems 3. Create and evaluate examples of educational applications Experiences in the ARiSE project CMIS Research Seminar, 19 th November 2008

18. Tabletop AR display Off-the-shelf hardware: - custom built frame - projector (120Hz) - mirror - projection surface - semi-transparent mirror - shutter glasses - Unibrain cameras - OptiTrack cameras - reasonably fast PC - network adapter Spinnstube - AR learning platform CMIS Research Seminar, 19 th November 2008

19. Open source software stack Spinnstube - AR learning platform CMIS Research Seminar, 19 th November 2008 OpenGL OpenSceneGraph Avango Python Linux Industry standard graphics API High performance 3D graphics toolkit Framework for networked / distributed interactive applications Avango scripting Free Suse Linux Operating System

20. Integration with Learning Management Systems 1. Conform to SCORM, a vendor and platform neutral standard for e-learning objects supported by most modern LMSs (e.g. Moodle, Blackboard) SCORM - Sharable Content Object Reference Model Share and reuse learning objects Find learning objects in online repositories Import learning objects into Learning Management Systems Export learning objects from LMSs and other authoring tools 2. Develop software that enables communication between SCORM Learning Object and Spinnstube AR Display Spinnstube – eLearning standards compliance CMIS Research Seminar, 19 th November 2008

21. Spinnstube – eLearning standards compliance LMS Load models, application Query results Content developer CMIS Research Seminar, 19 th November 2008 SCORM Course Online Repository

22. Spinnstube – eLearning standards compliance CMIS Research Seminar, 19 th November 2008

23. Examples of educational applications: 2006 - Process visualisation (cognitive learning theory) Human digestive system – instruction and knowledge consolidation 2007 - Guided construction (constructivist learning theory) Atomic structure and chemical bonds - active experimentation 2008 - Tele-Presence and communication (social learning theory) Cultural exchange – discussion and collaboration Educational applications CMIS Research Seminar, 19 th November 2008

24. Process visualisation - the human digestive system o Draws on cognitive learning theory o Learn through instruction and knowledge consolidation Model of open human torso augmented with animated images and audio explaining the digestive process. Students complete a number of assignments such as pointing out where specific food products are reduced in the digestive system. Educational applications - biology 1 CMIS Research Seminar, 19 th November 2008

25. Process visualisation - the human digestive system Educational applications - biology 2 CMIS Research Seminar, 19 th November 2008

26. Guided construction – atomic structure and chemical bonds o Draws on constructivist learning theory o Learn through active experimentation and knowledge construction Periodic table and foam rubber balls as graspable user interface. Touch periodic table with balls to turn them into the respective atoms. Construct chemical bonds by placing balls (atoms) next to each other. Audio cue guides students through experiments and tests. Educational applications - chemistry 1 CMIS Research Seminar, 19 th November 2008

27. Guided construction – atomic structure and chemical bonds Educational applications - chemistry 2 CMIS Research Seminar, 19 th November 2008

28. Tele-presence and communication – cultural exchange o Draws on social learning theory o Learn through observation, discussion, negotiation of meaning Students choose suitable topics from their local history and culture, and prepare meaningful 3D digital artefacts before the summer school. At summer school the artefacts are used to anchor and illustrate remote discussions between students about their local culture and customs. Three stages: 1. Preparation of artefacts 2. Introduction by video link 3. Spinnstube remote collaboration Educational applications – cultural exchange 1 CMIS Research Seminar, 19 th November 2008

29. Preparation of artefacts 1. Choose topic from local history / culture 2. Identify meaningful objects and collect material, photographs 3. Create 3D objects Educational applications – cultural exchange 2 Myslbek 3D sculpting tool Filipek 3D photo box CMIS Research Seminar, 19 th November 2008

30. Introduction by video link Before starting their Spinnstube session, students had the opportunity to get to know each other via a live video link using Skype. Very popular with students despite some language problems o talk to their counterparts (interests, favourites, grades,…) o organisational details of Spinnstube session (who talks first) o exchange contact details o arrange further Skype meetings outside summer school Educational applications – cultural exchange 3 CMIS Research Seminar, 19 th November 2008

31. Spinnstube remote collaboration Educational applications – cultural exchange 4 Siauliai, Lithuania St Augustin, Germany shared visual space live audio link CMIS Research Seminar, 19 th November 2008

32. All applications have been evaluated in summerschool projects with 13-15 year old students from Malta, Romania, Lithuania and Germany. Process visualisation - the human digestive system o Well received by students and teachers despite technical shortcomings and usability issues (Balog et al. 2007; Lamanauskas et al., 2006) Guided construction – atomic structure and chemical bonds o High acceptance rate among students and teachers, despite persisting technical and usability issues (Pribeanu et al., 2008) o Application helped students to understand complex issues more comprehensively and in a shorter time than conventional methods (Lamanauskas et al., 2008) Tele-presence and communication – cultural exchange o Two teams in each location: video, observation, questionnaires o Results are being analysed… Evaluation CMIS Research Seminar, 19 th November 2008

33. Many of the learning affordances of AR can be leveraged in a school context to create effective and enjoyable learning activities. o Diverse representations of related content as in the human biology application proved effective in conveying complex content. o Spatial 3D representations and haptic user interface in the chemistry application led to a deeper understanding of subject among students. o AR display dramatically improved students’ motivation and engagement , confirming similar reports in the literature. Remote collaboration results pending … Conclusions CMIS Research Seminar, 19 th November 2008

34. Azuma, R.T. (1997). A Survey of Augmented Reality. In Presence: Teleoperators and Virtual Environments , 6(4), pp. 355-385. Azuma, R.T. et al. (2001). Recent Advances in Augmented Reality. In IEEE Computer Graphics and Applications, 21(6), pp. 34-47. Balog, A., Pribeanu, C. & Iordache, D. (2007). Augmented Reality in Schools: Evaluation Results from a Summer School. Proceedings of the World Academy of Science, Engineering and Technology, October 12-14, 2007, Nice, France . pp 114-117. Bandura, A. (1977). Social Learning Theory . New York: General Learning Press. Billinghurst, M. (2002). Augmented Reality in Education. New Horizons for Learning . [online] http://it.civil.aau.dk/it/education/reports/ar_edu.pdf. [retrieved Nov 2008] Bruner, J. (1966). Toward a Theory of Instruction . Cambridge, MA: Harvard University Press Bruner, J. (1973). Going Beyond the Information Given . New York: Norton. Csikszentmihalyi, M. (1990). Literacy and Intrinsic Motivation. Deadalus , 119(2), p. 115. Gardner, H. (1983). Frames of Mind: The Theory of Multiple Intelligences . London: William Heinemann. Gibson, J. J. (1979). The ecological approach to visual perception . Boston, MA: Houghton Miflin. Hornecker, E. & Dunser, A. (2007). Supporting Early Literacy with Augmented Books - Experiences with an Exploratory Study. Proceedings of the German Society of Informatics Annual conference (GI-Jahrestagung) 2007 . Bonn: Köllen Verlag. Hull, C. (1943). Principles of Behavior . New York: Appleton-Century-Crofts. Ishii, H., Kobayashi, M. & Arita, K. (1994). Iterative Design of Seamless Collaboration Media. Communications of the ACM , 37(8), pp. 83-97. Kaufmann, H. and Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27 (2003), pp. 339–345 References CMIS Research Seminar, 19 th November 2008

35. Kolb, D. A. (1984). Experiential Learning: Experience as the source of learning and development . Englewood Cliffs, New Jersey: Prentice Hall. Lamanauskas, V., Vilkonis, R. & Klangaustas, A. (2006). Evaluation of the AR Teaching/Learning Platform. Internal Report on Task 5.3 in the ARISE project. Šiauliai University, Lithuania. (Not published) Lamanauskas, V., Vilkonis, R. & Bilbokaite, R. (2008). Pedagogical Evaluation of the Augmented Reality Platform. Internal Report on Task 5.3 in the ARISE project. Šiauliai University, Lithuania. (Not published) Lave, J., and Wenger, E. (1990). Situated Learning: Legitimate Periperal Participation. Cambridge, UK: Cambridge University Press. Liu, W and Cheok, D. (2007) Mixed Reality Classroom - Learning from Entertainment. Proceedings of the 2nd international conference on Digital Interactive Media in Entertainment and Arts, Perth, Australia, pp. 65-72 MacIntyre, B. et al (2002). Three Angry Men: Dramatizing Point-of-View using Augmented Reality. In SIGGRAPH 2002 Technical Sketches, San Antonio, TX, p. 268. Milgram, P. and Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems (Special Issue on Networked Reality), vol.E77-D, no.12, pp. 1321-1329 Pask, G. (1975). Conversation, Cognition, and Learning . New York: Elsevier. Piaget, J. (1952). The Origins of Intelligence in Children . New York: International Universities Press. Paiget, J. (1970). The Science of Education and the Psychology of the Child . New York: Grossman. References CMIS Research Seminar, 19 th November 2008

36. Pribeanu, C., Balog, A. & Iordache, D. (2008). Usability Evaluation – Summer School 2007. Internal Report on Task 5.1 in the ARISE project. National Institute for Research and Development in Informatics, Bucharest, Romania. (Not published) Salomon, G. (ed.) (1993). Distributed Cognitions: Psychological and educational considerations . Cambridge: Cambridge University Press. Spiro, R.J., Coulson, R.L., Feltovich, P.J. & DK Anderson (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In V. Patel (ed.), Proceedings of the 10th Annual Conference of the Cognitive Science Society . Hillsdale, NJ: Erlbaum. Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press. Woods, E., Billingshurst, M., Looser, J., Aldridge, G., Brown, D., Garrie, B. & C. Nelles. (2004). Augmenting the Science Centre and Museum Experience. Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and SouthEast Asia (Graphite 2004) , 15-18th June, Singapore, 2004, pp. 230-236. References CMIS Research Seminar, 19 th November 2008

37. Hands-on demonstration Augmented Reality System Room 622 More… Contact: Marcus Winter [email_address] -or- Telephone 01273 642491 Interactive Technologies Reasearch Group, CMIS 617 Watts Building, University of Brighton CMIS Research Seminar, 19 th November 2008