Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

ISS2022 Keynote

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Chargement dans…3
×

Consultez-les par la suite

1 sur 51 Publicité

Plus De Contenu Connexe

Plus par Mark Billinghurst (20)

Plus récents (20)

Publicité

ISS2022 Keynote

  1. 1. METAVERSE Mark Billinghurst mark.billinghurst@auckland.ac.nz November 22nd 2022 Opportunities for Research Schmetaverse!!
  2. 2. The Metaverse is Hot!
  3. 3. Google Searches on the Metaverse • 5 year search volume to October 2022 • Metaverse (blue) vs. Virtual Reality (red)
  4. 4. Publications about the Metaverse • Going vertical !
  5. 5. A Definition • Real-time, 3D, Interactive, Social, Persistent AWE 2022 John Riccitiello CEO, Unity Technologies
  6. 6. Meta’s Metaverse
  7. 7. Ready Player One
  8. 8. Back to 1994: Young Me in Seattle
  9. 9. Virtual Reality was HOT! .. In 1995..
  10. 10. My First VR Experience 1500 Polygons! $250,000
  11. 11. My First VR Experience … https://www.youtube.com/watch?v=pAC5SeNH8jw
  12. 12. In Five to Ten years .. • VR becoming commonplace • VR devices less than $1000 • Large scale networked worlds • Billion-dollar VR industry
  13. 13. • April 2007 Computer World • VRVoted 7th on list of 21 biggest technology flops
  14. 14. What Happened After That • Growth of internet • Natural user interfaces • Increase in graphics performance • Explosion in mobile phones • Pokemon Go!
  15. 15. Increase in VR Research • Papers grew over fivefold..
  16. 16. Social VR Today Rec Room - 30 million users Fortnite - 24 million DAU - 28 million @ travis scott
  17. 17. A Better Definition • Neal Stephenson’s “Snow Crash” (1992) • The Metaverse is the convergence of: • 1) virtually enhanced physical reality • 2) physically persistent virtual space A persistent virtual environment around The Street 15 million avatars on The Street at any one time
  18. 18. Metaverse Taxonomy • Four Key Components • Virtual Worlds • Augmented Reality • Mirror Worlds • Lifelogging • Metaverse Roadmap • http://metaverseroadmap.org/
  19. 19. Mirror Worlds • Simulations of external space/content • Capturing and sharing surroundings • Photorealistic content • Digital twins Matterport Deep Mirror Google Street View Soul Machines
  20. 20. Lifelogging • Measuring user’s internal state • Capturing physiological cues • Recording everyday life • Augmenting humans Apple Fitbit Shimmer OpenBCI
  21. 21. Sensing Immersing Augmenting Capturing
  22. 22. Metaverse Activities
  23. 23. Expanded Research Opportunities
  24. 24. Possible Research Directions • Lifelogging to VR • Bringing real world actions into VR, VR to experience lifelogging data • AR to Lifelogging • Using AR to view lifelogging data in everyday life, Sharing physiological data • Mirror Worlds to VR • VR copy of the real world, Mirroring real world collaboration in VR • AR to Mirror Worlds • Visualizing the past in place, Asymmetric collaboration • And more..
  25. 25. Vision-Based Research (Hiroshi Ishii)
  26. 26. Communication Trends • 1. Experience Capture • Move from sharing faces to sharing places • 2. Natural Collaboration • Faster networks support more natural collaboration • 3. Implicit Understanding • Systems that recognize behaviour and emotion
  27. 27. Natural Collaboration Implicit Understanding Experience Capture
  28. 28. Natural Collaboration Implicit Understanding Experience Capture Empathic Computing
  29. 29. Empathic Computing Vision Can we develop systems that allow us to share what we are seeing, hearing and feeling with others?
  30. 30. Example: Sharing Communication Cues • Measuring non-verbal cues • Gaze, face expression, heart rate • Sharing in Augmented Reality • Collaborative AR experiences
  31. 31. Empathy Glasses • Combine together eye-tracking, display, face expression • Implicit cues – eye gaze, face expression + + Pupil Labs Epson BT-200 AffectiveWear Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
  32. 32. Remote Collaboration • Eye gaze pointer and remote pointing • Face expression display • Implicit cues for remote collaboration
  33. 33. https://youtu.be/xwU7M1xaFJ8
  34. 34. Example: AR/VR Collaboration • Augmented Reality • Bringing remote people into your real space • Virtual Reality • Bringing elements of the real world into VR • AR/VR for sharing communication cues • Sharing non-verbal communication cues
  35. 35. Sharing Virtual Communication Cues • AR/VR displays • Gesture input (Leap Motion) • Room scale tracking • Conditions • Baseline, FoV, Head-gaze, Eye-gaze
  36. 36. https://youtu.be/j8GdnSS3nAY
  37. 37. Technology Trends • Advanced displays • Wide FOV, high resolution • Real time space capture • 3D scanning, stitching, segmentation • Natural gesture interaction • Hand tracking, pose recognition • Robust eye-tracking • Gaze points, focus depth • Emotion sensing/sharing • Physiological sensing, emotion mapping
  38. 38. Sensor Enhanced HMDs Eye tracking, heart rate, pupillometry, and face camera HP Omnicept Project Galea EEG, EMG, EDA, PPG, EOG, eye gaze, etc.
  39. 39. • Advanced displays • Real time space capture • Natural gesture interaction • Robust eye-tracking • Emotion sensing/sharing Empathic Tele-Existence
  40. 40. Empathic Tele-Existence • Based on Empathic Computing • Creating shared understanding • Covering the entire Metaverse • AR, VR, Lifelogging, Mirror Worlds • Transforming collaboration • Observer to participant • Feeling of doing things together • Supporting Implicit collaboration
  41. 41. NeuralDrum • Using brain synchronicity to increase connection • Collaborative VR drumming experience • Measure brain activity using 3 EEG electrodes • Use PLV to calculate synchronization • More synchronization increases graphics effects/immersion Pai, Y. S., Hajika, R., Gupta, K., Sasikumar, P., & Billinghurst, M. (2020). NeuralDrum: Perceiving Brain Synchronicity in XR Drumming. In SIGGRAPH Asia 2020 Technical Communications (pp. 1-4).
  42. 42. Set Up • HTC Vive HMD • OpenBCI • 3 EEG electrodes
  43. 43. https://www.youtube.com/watch?v=aG261GfiR90
  44. 44. Results "It’s quite interesting, I actually felt like my body was exchanged with my partner." Poor Player Good Player
  45. 45. Conclusions • The Metaverse is overhyped • Long-term impact could be more significant than imagined • We need a broader Metaverse definition and taxonomy • Many research opportunities in crossing boundaries • Need to focus on high-level visions • Empathic Computing encompasses the Metaverse
  46. 46. Delivering the Entire Metaverse
  47. 47. www.empathiccomputing.org @marknb00 mark.billinghurst@auckland.ac.nz

×