ISS2022 Keynote
â˘
2 likesâ˘1,375 views
Keynote speech given by Mark Billinghurst at the ISS 2022 conference. Presented on November 22nd, 2022. This keynote outlines some research opportunities in the Metaverse.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to ISS2022 Keynote
Similar to ISS2022 Keynote (20)
More from Mark Billinghurst
More from Mark Billinghurst (8)
Recently uploaded
Recently uploaded (20)
ISS2022 Keynote
- 1. METAVERSE Mark Billinghurst mark.billinghurst@auckland.ac.nz November 22nd 2022 Opportunities for Research Schmetaverse!!
- 2. The Metaverse is Hot!
- 3. Google Searches on the Metaverse ⢠5 year search volume to October 2022 ⢠Metaverse (blue) vs. Virtual Reality (red)
- 4. Publications about the Metaverse ⢠Going vertical !
- 6. A Definition ⢠Real-time, 3D, Interactive, Social, Persistent AWE 2022 John Riccitiello CEO, Unity Technologies
- 9. Back to 1994: Young Me in Seattle
- 10. Virtual Reality was HOT! .. In 1995..
- 11. My First VR Experience 1500 Polygons! $250,000
- 12. My First VR Experience ⌠https://www.youtube.com/watch?v=pAC5SeNH8jw
- 13. In Five to Ten years .. ⢠VR becoming commonplace ⢠VR devices less than $1000 ⢠Large scale networked worlds ⢠Billion-dollar VR industry
- 15. ⢠April 2007 Computer World ⢠VRVoted 7th on list of 21 biggest technology flops
- 16. What Happened After That ⢠Growth of internet ⢠Natural user interfaces ⢠Increase in graphics performance ⢠Explosion in mobile phones ⢠Pokemon Go!
- 17. Increase in VR Research ⢠Papers grew over fivefold..
- 19. Social VR Today Rec Room - 30 million users Fortnite - 24 million DAU - 28 million @ travis scott
- 21. A Better Definition ⢠Neal Stephensonâs âSnow Crashâ (1992) ⢠The Metaverse is the convergence of: ⢠1) virtually enhanced physical reality ⢠2) physically persistent virtual space A persistent virtual environment around The Street 15 million avatars on The Street at any one time
- 22. Metaverse Taxonomy ⢠Four Key Components ⢠Virtual Worlds ⢠Augmented Reality ⢠Mirror Worlds ⢠Lifelogging ⢠Metaverse Roadmap ⢠http://metaverseroadmap.org/
- 23. Mirror Worlds ⢠Simulations of external space/content ⢠Capturing and sharing surroundings ⢠Photorealistic content ⢠Digital twins Matterport Deep Mirror Google Street View Soul Machines
- 24. Lifelogging ⢠Measuring userâs internal state ⢠Capturing physiological cues ⢠Recording everyday life ⢠Augmenting humans Apple Fitbit Shimmer OpenBCI
- 28. Possible Research Directions ⢠Lifelogging to VR ⢠Bringing real world actions into VR, VR to experience lifelogging data ⢠AR to Lifelogging ⢠Using AR to view lifelogging data in everyday life, Sharing physiological data ⢠Mirror Worlds to VR ⢠VR copy of the real world, Mirroring real world collaboration in VR ⢠AR to Mirror Worlds ⢠Visualizing the past in place, Asymmetric collaboration ⢠And more..
- 29. Vision-Based Research (Hiroshi Ishii)
- 30. Communication Trends ⢠1. Experience Capture ⢠Move from sharing faces to sharing places ⢠2. Natural Collaboration ⢠Faster networks support more natural collaboration ⢠3. Implicit Understanding ⢠Systems that recognize behaviour and emotion
- 33. Empathic Computing Vision Can we develop systems that allow us to share what we are seeing, hearing and feeling with others?
- 34. Example: Sharing Communication Cues ⢠Measuring non-verbal cues ⢠Gaze, face expression, heart rate ⢠Sharing in Augmented Reality ⢠Collaborative AR experiences
- 35. Empathy Glasses ⢠Combine together eye-tracking, display, face expression ⢠Implicit cues â eye gaze, face expression + + Pupil Labs Epson BT-200 AffectiveWear Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
- 36. Remote Collaboration ⢠Eye gaze pointer and remote pointing ⢠Face expression display ⢠Implicit cues for remote collaboration
- 38. Example: AR/VR Collaboration ⢠Augmented Reality ⢠Bringing remote people into your real space ⢠Virtual Reality ⢠Bringing elements of the real world into VR ⢠AR/VR for sharing communication cues ⢠Sharing non-verbal communication cues
- 39. Sharing Virtual Communication Cues ⢠AR/VR displays ⢠Gesture input (Leap Motion) ⢠Room scale tracking ⢠Conditions ⢠Baseline, FoV, Head-gaze, Eye-gaze
- 41. Technology Trends ⢠Advanced displays ⢠Wide FOV, high resolution ⢠Real time space capture ⢠3D scanning, stitching, segmentation ⢠Natural gesture interaction ⢠Hand tracking, pose recognition ⢠Robust eye-tracking ⢠Gaze points, focus depth ⢠Emotion sensing/sharing ⢠Physiological sensing, emotion mapping
- 42. Sensor Enhanced HMDs Eye tracking, heart rate, pupillometry, and face camera HP Omnicept Project Galea EEG, EMG, EDA, PPG, EOG, eye gaze, etc.
- 43. ⢠Advanced displays ⢠Real time space capture ⢠Natural gesture interaction ⢠Robust eye-tracking ⢠Emotion sensing/sharing Empathic Tele-Existence
- 44. Empathic Tele-Existence ⢠Based on Empathic Computing ⢠Creating shared understanding ⢠Covering the entire Metaverse ⢠AR, VR, Lifelogging, Mirror Worlds ⢠Transforming collaboration ⢠Observer to participant ⢠Feeling of doing things together ⢠Supporting Implicit collaboration
- 45. NeuralDrum ⢠Using brain synchronicity to increase connection ⢠Collaborative VR drumming experience ⢠Measure brain activity using 3 EEG electrodes ⢠Use PLV to calculate synchronization ⢠More synchronization increases graphics effects/immersion Pai, Y. S., Hajika, R., Gupta, K., Sasikumar, P., & Billinghurst, M. (2020). NeuralDrum: Perceiving Brain Synchronicity in XR Drumming. In SIGGRAPH Asia 2020 Technical Communications (pp. 1-4).
- 46. Set Up ⢠HTC Vive HMD ⢠OpenBCI ⢠3 EEG electrodes
- 48. Results "Itâs quite interesting, I actually felt like my body was exchanged with my partner." Poor Player Good Player
- 49. Conclusions ⢠The Metaverse is overhyped ⢠Long-term impact could be more significant than imagined ⢠We need a broader Metaverse definition and taxonomy ⢠Many research opportunities in crossing boundaries ⢠Need to focus on high-level visions ⢠Empathic Computing encompasses the Metaverse
- 50. Delivering the Entire Metaverse