Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Metaverse_SGAsia2022.pdf

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Prochain SlideShare
Metaverse2021 woo
Metaverse2021 woo
Chargement dans…3
×

Consultez-les par la suite

1 sur 41 Publicité

Plus De Contenu Connexe

Plus par Woontack Woo (20)

Plus récents (20)

Publicité

Metaverse_SGAsia2022.pdf

  1. 1. Metaverse is Coming… Is the Metaverse Dying Again? BTS through Metaverse! [Invited Talk @ BrainLink2022, 12/4/22] Woontack Woo (禹雲澤), Ph.D. G-School of CT/ UVR Lab., Head KI-ITC AR Research Center, Director KAIST, Daejeon 34141, Korea
  2. 2. 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Y. Lee, S. Oh, C. Shin, W. Woo, "Recent Trends in Ubiquitous VR," in Proc. of International Symposium on Ubiquitous VR, pp. 33-36, 2008. Y. Lee, S. Oh, C. Shin, W. Woo, “Ubiquitous Virtual Reality and Its Key Dimension,” In Proc. of IWUVR, pp. 5-8., 2009 Vision-driven UVR Research Since 2001 사회관계망 Real World Seamless Augmentation btw dual worlds Simulate … 3D Link … Augment CoI 3D Link U-Content Virtualized World Metaverse ? … SPACE (3rd skin) Intimate/Private Social General/public Avatar lUbiquitous VR, HHC BTS with CaMAR! [Woo01] •*CaMAR: Context-aware Mediated & Augmented Reality §3D Link (with IoE) between twin (real & virtual) worlds §CoI (Context-of-Interest) Augmentation with information/knowledge, not just 5 senses (sight, sound, haptics, smell, taste, etc.). §Bidirectional Interaction/Collaboration for O2O/H2H Communications lUVR vs. Metaverse
  3. 3. lUbiquitous VR, HHC BTS with CaMAR! [Woo01] §From ‘Smart' of AR to ‘Smile' of Augmented Society! R&D Foci of UVR Lab. 2001-2022 © Woo, UVR Lab., Korea Augmented Reality • WWB • UVR • DT • Metaverse Augmented Human • Body+ • Brain+ • Social+ Ubiquitous Computing • IoT, IoE • Context-aware UI Wearable Robotics • Cyborg • Bionics, • Implantable, Virtual Augmentation Physical Augmentation Humans Environment
  4. 4. lToward DT-based Metaverse [UVR Lab’22] UVR3.0 (21-25) • Object-aware SLAM for Real- scale Realistic Digital Twin TranSpace3.0 (21~27) • Avatar-mediated Augmented Collaboration WISE UI/X (19-26) • Augmented Human with Wise Organic UI A-Metamuse (21-23) • Authoring for Augmented Meta-Gallery/Museum Meta-X (23?-) • Authoring for Augmented Lib., Concert, Tour & Game R&D Projects of UVR Lab. 2001-2022 © Woo, UVR Lab., Korea
  5. 5. WISE UI/UX in 2022 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  6. 6. Content 2001-2022 © Woo, UVR Lab., Korea Metaverse, Dying Again with XR? What is Metaverse? Is it Sustainable? Metaverse 3.0, Does it work for Our Daily Life? Metaverse 3.0, What Should We Do? Q&A
  7. 7. lWhat is Metaverse? §My short definition: UVR, an extended virtual world in smart space! §Why are we interested in Metaverse? How to make the most of it? Metaverse is Coming Again? 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea What do we do in Metaverse? VS: Virtual Space DS: Meta Space DT: Virtualized Space We are in Physical World!
  8. 8. lOmniverse by Jensen Huang [Nvidia 2020] Metaverse is Coming (Again)! 2001-2022 © Woo, UVR Lab., Korea
  9. 9. lA Honeypot Trap by John Carmack [Oculus 2021] Metaverse is Bullshit! 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  10. 10. Content 2001-2022 © Woo, UVR Lab., Korea Metaverse, Dying Again with XR? What is Metaverse? Is it Sustainable? Metaverse 3.0, Does it work for Our Daily Life? Metaverse 3.0, What Should We Do? Q&A
  11. 11. lSnow Crash [Neal Stephenson 92] §Second Life by Linden Lab [03] §Open Source Metaverse Project [04-08] •A future internet, a shared and open standards-based virtual space §Metaverse Roadmap [ASF ’07] •The nexus of ‘virtually-enhanced physical reality’ and ‘physically persistent virtual space’ •Nexus of • Environment • Humans • Simulation • Augmentation lWhat’s Next? §Multiverse §Pataverse What’s Metaverse? [ASF’07] 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Augmented Reality Virtual World (Digital Twin) Mirror World Life-Logging Augmented Society
  12. 12. lGame? §Roblox? Fortnite? lVR/AR/XR? §Second Life? Meta Horizon? lWeb3.0? §From 2D screen to 3D space? DAO? lAvatar World? §Zepeto? vs. D-Human? lVirtual Assets? Whatelse? §Weverse? OpenSea? What’s Metaverse? 2001-2022 © Woo, UVR Lab., Korea https://cmte.ieee.org/futuredirect ions/2017/09/27/can-we-have-a- digital-twin/
  13. 13. lCore Properties of Metaverse [Woo21] §Metaverse: next-generation internet (3D internet with web3)? •UVR, an Extended Virtual World: avatar-mediated, co-present, persistent, interoperable extended virtual world (open social platform) with trusted economic systems through real-virtual nexus §From <Hyperlink with HTML> to <Hyper-jump with USD> [NVidia’22] What’s Metaverse? 2012-2021 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Real-Virtual Nexus • Convergence of persistent virt ual world with the augmented world Avatar-mediated World • Avatar-mediated access, expl oration, utilization and experi ence in the virtual world Authoring and Possession • Connect, communicate, collab orate to engage in various dail y activities without time and s pace limitations • Create and Own Trusted Economic System • A reliable convergence econo mic system that enables econ omic compensation, ownershi p, and exchange of the results of various activities Interoperable Platform • Interoperable social platform between various metaverses t o expand the production, own ership, and exchange of econo mic values such as assets and experiences Remaining Challenges: •R-V nexus: 5G -> 6G/Edge/Cloud •Browser: DT/MV+ Avatar •In-situ Authoring with NUI •HTML -> USD & Web2.0 -> Web3 •Context-aware D-SLAM •Virtual Assets •Carriable -> Wearable UI/UX
  14. 14. Metaverse is a Bubble! Metaverse is Nowhere! Metaverse is Now Here! Then, is it Sustainable? Is Metaverse Dying Again? 2001-2022 © Woo, UVR Lab., Korea
  15. 15. lThe New D-Gold Rush in Metaverse? Is Metaverse Dying Again? 2001-2022 © Woo, UVR Lab., Korea China 2 0 0 3 2 0 2 1 2 0 0 8 2 0 0 9 2 0 1 4 USA Korea
  16. 16. lIs it just Buzzword? §on Silicon Velley or Wall Street? in Korea or China? lThe Metaverse is Bullshit because §It already exists, and it's called the internet §Tech Moguls missed the part where cyberpunk is dystopian §Its promised cross-compatibility doesn't actually work §No one can actually explain why it's better •[Wes Felon@ C Gamer, Oct. 29, 2021] https://outline.com/DUdr8Y lFrom FB to Meta! §Then, What will Meta do? §10K researchers with $10B/Y §+10K & $10B/Y for next 5 years Is Metaverse Dying Again? 2001-2022 © Woo, UVR Lab., Korea
  17. 17. lMetaverse3.0: Meta’s Vision (2021) D-Gold Rush in Metaverse3.0! 2001-2022 © Woo, UVR Lab., Korea
  18. 18. lMetaverse3.0 with XR Glasses [Woo’21] §Avatar + Network + R-V World/Society + Economy + what else? Metaverse (Novel, Web1.0 & Internet 1.0, 1990s-) Snow Crash (1992) Metaverse 1.0 with Internet (Web2.0, 2000s-) 2nd Life (2003) Metaverse 2.0 with Smart phone (2010s-) MV Returns (2020) Metaverse 3.0 with XR Glasses? (Web3.0, 2020s-) A-Metaverse (2030?) D-Gold Rush in Metaverse3.0! 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Every 15~20 years or so a new major platform emerges? [07] Metaverse [14] [19] [03] 2nd Life, HoloLens1 HoloLens2 Digital Twin [01] Ubiquitous VR Prosumer Login with id/pwd Creasumer Login with SNS id CreaOwner Login with Wallet
  19. 19. lEvolution(進化) of Personal Media §Powerful, cheaper, lighter, smaller, smarter, more intimate •Desktop(Type & Click) vs. Mobile (Point & Touch) §So, What’s next? XR, DT, Metaverse? •Speech, Gaze & Gesture for Wearable? http://goo.gl/K6xAIA D-Gold Rush in Metaverse3.0! 2001-2022 © Woo, UVR Lab., Korea
  20. 20. lEvolution(進化) of Immersive Content §UVR Content beyond TS [Woo’06] D-Gold Rush in Metaverse3.0! 2001-2022 © Woo, UVR Lab., Korea Perception (5+ senses) Interactive Experience Audio-visual+Haptic+Senses Intelligent Interaction Interaction Response +Dangerous +Expensive +Impossible
  21. 21. lSo, Is Metaverse Dying Again? lOr Is Metaverse Coming Again? D-Gold Rush in Metaverse3.0! 2001-2022 © Woo, UVR Lab., Korea “The best way to predict the future is to create it!”
  22. 22. Content 2001-2022 © Woo, UVR Lab., Korea Metaverse, Dying Again with XR? What is Metaverse? Is it Sustainable? Metaverse 3.0, Does it work for Our Daily Life? Metaverse 3.0, What Should We Do? Q&A
  23. 23. Play, culture and art Work, communication, collaboration Learning (Education), Health, Welfare, Consumption Energy/ Environment (water, air, etc.) Mobility Virtualized Twin City Real City @Magic-verse by Magic Leap lMetaverse Eco-System [Woo20] §Interworking with heterogeneous XR experience platforms §O2O Bidirectional Economic Activities §Data right (sovereignty, privacy, security) Augmented Society in Metaverse3.0 2012-2021 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Infra: IoT/IoE, 5G/6G, Edge/Cloud Platform: Digital Twin, VW, Simulation, AI Avatar: Realistic /CG Smart Human Personal UI/UX: Wearable Interaction Authoring Tool: Creating Content, Assets Economy System: NFT+, P2E Interoper able Standard: Avatar, Asset, Platform N P U D C E S
  24. 24. lAugmented Citizen? §AR3.0 over 5G/Edge/Cloud will augment on-device processing §Augmented humans will expand citizen’s body/brain/social abilities! Augmented Society in Metaverse3.0 2001-2022 © Woo, UVR Lab., Korea 5G/6G • High speed Networks • -> tracking data • <- retrieved/ rendered data Edge Cloud • Real-time response • Partial rendering • analysis • retrieving Cloud • computati onally intensive task • Less latency sensitive content Power-efficient, latency insensitive on-device rendering and tracking Round-Trip Delay (or RTT) of 5G for VR/AR • Handheld VR/AR: ~200ms • Head-mounted VR (CG,AI, HCI): ~ 50ms (SKT?) • Wearable AR (CV, CG, AI, HCI): ~ 10ms (KT?)
  25. 25. Augmented Society in Metaverse3.0 2001-2022 © Woo, UVR Lab., Korea lPossible Applications §Expand human abilities: body, brain, social relation §Transform the way we live and interact with the world §Revolutionize industries: productivity, efficiency, safety DT + Metaverse + XR Game performance exhibition Tourism, Theme park Advertisem ent Dramas, Movies Publication (Webton, Books, etc.) Sports Education, Training Medical care Manufacturing Constructions, architectures Cities And more…
  26. 26. Content 2001-2022 © Woo, UVR Lab., Korea Metaverse, Dying Again with XR? What is Metaverse? Is it Sustainable? Metaverse 3.0, Does it work for Our Daily Life? Metaverse 3.0, What Should We Do? Q&A
  27. 27. Infra: IoT/IoE, 5G/6G, Edge/Cloud Platform: Digital Twin, VW, Simulation, AI Avatar: Realistic CG Smart Human Personal UI/UX: Wearable Interaction Authoring Tool: Creating Content, Assets Economy System: NFT+, P2E Interoper able Standard: Avatar, Asset, Platform B+ B- C C C D C lWhere are We Now? What Should We Do? [Woo20] §Equal opportunity, §Fair competition, §Sustainable ecosystem, etc. UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  28. 28. lDT/MV+XR for A-Learning [KAIST+KISTI‘21] §Problem-solving, self-directed learning without time-space limitations (borders/schools/grades/classes) UVR Experience in Metaverse3.0 Era 2012-2021 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea <Issues: Asymmetries> •Physical environment •Experience platform •Available information (shared space, view, object size & representation, etc.) •Role/experience <Challenges> •Avatar reconstruction and representation •Avatar-based collaboration •Authenticity of Virtual Restoration: Objective, Structural, Existential •Minimizing asymmetry and maximizing social presence Object-aware 3D Scene Graph (Host) Science Museum IoT, Pad, XR Glasses Experience, Interaction, Collaboration in Metaverse IoT, Pad, XR Glasses (Guest) Class Room DT Generation and Management Building Room Object V-Camera
  29. 29. l3D Modeling: From SLAM to Spatial AI [CVPR20, ICL] §Interaction with environment §Spatial <AI vs. IA> §Cost-effective solutions? UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Level Task Technology Papers Outcome Level 5 [Woo] Understanding context of user’s activities in dynamic environment Space merge & object/activity recognition (in-situ modeling for unknown) 3D DSG by MIT? Augmented collaboration Level 4 Understanding relationships of surrounding objects Object-level dynamic mapping MID-Fusion Advanced interaction Level 3 Understanding what objects are in static environment Semantic understanding SemanticFusion Basic interaction Level 2 Recognizing spaces and walls Dense mapping with loop-closure ElasticFusion Collision avoidance / motion planning Level 1 Robust localizing 3D position Sparse Visual SLAM (+IMU) OKVIS/ OVKIS2 Position control https://www.slamcore.com/ Andrew Davison
  30. 30. lSemantic SLAM: Remaining Challenges §3D Data Acquisition (input point clouds): •Noisy, unsegmented, occlusion issues, etc. §3D Reconstruction: scene- vs. object-level §Dynamic Scene Management •Diversity of objects: cluttered, occluded, deform, move, etc. • Cues: geometric primitives, joints, repeats, … •Active 3D Scene Graph: objects, attributes, relationships, … lStill, Long Way to go… §To know (unknown) objects of interest (OoI) in dynamic environment §To understand multimodal interaction with both hands §To understand H-2-H communication/collaboration, etc. §To aware context of scene, object, recourse, user, etc. §And many more issues… UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Fully Human- designed algorithm End to end learned algorithm
  31. 31. l3D Avatar UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Immature Mature Realistic Unrealistic
  32. 32. l3D Avatar for Metaverse, Some Issues? §CG vs. Realistic 3D Video Avatar: •CG: Kawaii engineering? •Video-based: 3D face/body, gesture, clothes, hair/fur, etc. • Parameter estimation of face/body/hands? (SMPL-X model?) • Bare-face/body reconstruction with XR glasses? • Bare-hands interaction: (Self-)occlusion or grabbing objects • Facial expression? Dynamic gesture? Changing clothes, hairs? • Collaboration? (Sign) language interpreter? • And many more issues…. §Possible Solutions? Remaining issues? •Ill-posed 3D from 2D image(s): ground-truth for learning? •Controlling avatar with a sense of unity! • movement, expression, interaction, etc. •Social co-existence with heterogeneous personal platforms? •Intelligent (adaptive) CG/Video Avatar? • Using avatar-space/object information (e.g. collision, contact, etc.) UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea body shape capture Shape modeling Skeletal model ing Texture (cloth es) modeling Posture/expre ssion/motion estimation Transmission Video Avatar
  33. 33. lWearable XR Experiences! UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea Sensing (Video, IMU, IoT/E, etc.) Object/Space Recognition/ Tracking (CV) Localization of XR Glasses (CV) Augmentation (CG & InfoViz) Multimodal Interaction (HCI) Story/Flow Experience (UX) AI is everywhere! Edge/Cloud (DT+AI & XR Assets) XR Content Rendering DT+XR Modeling/ Authoring
  34. 34. lWhy is In-situ XR Authoring in Metaverse Important? §Limitations in authoring (due to a lack of resources to utilize CAD data), §Challenges creating net-new XR content without 3D files, §In addition, Bad UX that make content creation laborious lCapabilities of Authoring Tools §To render interactive content §To build the executable services §To provide spatial experiences lXR Authoring for Metaverse §Simpler, easier, faster, cheaper to create! •Use-case-driven design, lightweight, hybrid, UX §How hybrid solutions might be able to bridge the gap •Hybrid: CAD-inspired (Vuforia Studio) vs. lightweight options •Making AR content in AR: Vuforia Instruct, Microsoft’s Dynamics 365 Guides §Improvements to the (UX) UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  35. 35. lAuthoring Tool Landscape UVR Experience in Metaverse3.0 Era 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea ARToolKit Vuforia ARKit ARCore DART WebXR Amazon Sumerian Unity MARS Unreal Autodesk Adobe Substance 3D Vuforia Instruct MS MRTK Clay PPT Paper ***** Level of Functionality/Fidelity in XR (Programing) Skill/Experience Required
  36. 36. UVR=Metaverse3.0, What should We do? • Avatar-mediated interaction with virtual objects in real world • In-situ 3D modeling/manipulation of avatar & objects-aware Env. • Intuitive authoring and capturing augmented UX • Evaluation of metaverse (platform and UX), what else? UVR R&D Direction to prepare for Metaverse3.0 • 3D Dynamic Scene Graphs with Semantic/Context-aware SLAM • Authoring/capturing UX of space-telling in metaverse • Wise UI with HQS for wearable XR Glasses So, Still Remaining Challenges? • To connect context-aware SLAM, instant NeRF and USD? • To augment interaction/collaboration in a distance w/ co-presense? • To embed personalized wise agent with HQS Vision-driven UVR R&D 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  37. 37. lMetaverse3.0, Toward UVR! §Content is King! •The fruit is sweet, but… §Eco-system is Queen! •Plan, create, distribute, utilize, share •Content + Platform + Network + Device • IoT/IoE, Big Data, 5G, DT/MV, AI, AR/VR §User/UX is Emperor! •Economic value creation through creative activities! •Context-aware UX/services §R&E is God! •KAIST CT-AR Program in 2020 •KAIST Metaverse Program in 2022! Metaverse3.0, What Should We Do? 2001-2022 © Woo, UVR Lab., Korea
  38. 38. Summary 2001-2022 © Woo, UVR Lab., Korea Metaverse, Dying Again with XR? What is Metaverse? Is it Sustainable? Metaverse 3.0, Does it work for Our Daily Life? Metaverse 3.0, What Should We Do? Q&A
  39. 39. Metaverse Era, What is Our Choice? lUtopia or Dystopia? XR or eXtinct? "Do you want to sell sugared water for the rest of your life? Or do you want to come with me and change the world?“ - Steve Jobs “실수도 실패도 좌절도 있을 것입니다. 그래도 포기하지 않고 도전하는 사람이 세상을 바꿉니다!” - 우운택 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  40. 40. lARRC/UVR Lab. Members §Director: Woontack Woo §ARRC: 4 Ph.D. & 1 MS Researchers §UVR Lab.: 12 Ph.D. & 12 MS Students §1 undergraduate students (URP/Intern) §1 visiting scholars §2 administration staff lInternational Collaborators §Adjunct Professors @ GSCT (May 1, 2021 ~) • Mark Billinghurst (UniSA, AU), • Vincent Lepetit (ENPC ParisTech, FR), • Hideo Saito (Keio Univ. JP) §More Collaborators • A. Argyros, K. Kunze, T. Starner, etc. KAIST ARRC/UVR Lab. 2022 The Ubiquitous VR research aims at the development of new computing paradigms for ”DigiLog Life in Smart Spaces” 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea
  41. 41. lMore Information §Woontack Woo, Ph.D. • wwoo@kaist.ac.kr • http://uvrlab.org • http://arrc.kaist.ac.kr lSupported by §㈜PostMedia, ㈜VirNect, ㈜AST, ㈜MAXST ㈜ExTriple, , ㈜Victoria Production §과기정통부 (MIST) IITP, NRF, KISTI/NST §문화부 (MCST) GIST KCTRI/KOCCA §산업부 (MOTIE) KEA/KIAT §대전시 DICIA, KAIST lRelated Events §KAIST CT-AR since Sep. 2020 and Metaverse from 2022! “The future is already here. It is just not uniformly distributed” by William Gibson (SF writer) Welcome to UVR Era! 2012-2022 © Woo, KAIST ARRC & UVR Lab., Daejeon 34141, Korea

×