Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

20171018_Closing_the_Content_Gp_for_AR_andVR_Prof_Aljosa_Smolic

36 vues

Publié le

• Background on Free Viewpoint Video (FVV), VR, AR
• V-SENSE overview
• The Content Gap for VR and AR
• V-SENSE research on FVV for VR/AR content creation
• Creative Experiment: Virtual Play, after Samuel Beckett

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

20171018_Closing_the_Content_Gp_for_AR_andVR_Prof_Aljosa_Smolic

  1. 1. 18.10.2017 1 Closing the Content Gap for AR and VR Professor Aljosa Smolic SFI Research Professor of Creative Technologies Trinity College Dublin, The University of Dublin Outline • Background on Free Viewpoint Video (FVV), VR, AR • V-SENSE overview • The Content Gap for VR and AR • V-SENSE research on FVV for VR/AR content creation • Creative Experiment: Virtual Play, after Samuel Beckett
  2. 2. 18.10.2017 2 Trinity College Dublin, The University of Dublin Personal Background 2009 – 2016: 2001 - 2009 Trinity College Dublin, The University of Dublin Free Viewpoint Video ▪ Same functionality as CG objects ▪ free navigation, can be viewed from any viewpoint/direction ▪ Integration into complete scenes (virtual/augmented/real) ▪ But: depict appearance, motion, deformation of real world objects
  3. 3. 18.10.2017 3 Trinity College Dublin, The University of Dublin FVV Acquisition Trinity College Dublin, The University of Dublin 3D reconstruction of real world scenes and objects Interactive viewpoint selection Mixing real and virtual elements Free Viewpoint Video Free viewpoint video extraction, representation, coding, and rendering A Smolic, K Mueller, P Merkle, T Rein, M Kautzner, P Eisert, T Wiegand ICIP'04. 2004
  4. 4. 18.10.2017 4 Trinity College Dublin, The University of Dublin Free Viewpoint Video Trinity College Dublin, The University of Dublin 8i.com https://techcrunch.com/2017/02/13/time-warner-leads-27m- investment-in-celebrity-hologram-company-8i/
  5. 5. 18.10.2017 5 Trinity College Dublin, The University of Dublin Free Viewpoint Video Or just use a lot of cameras No 3D reconstruction https://www.youtube.com/watch?v=bKEcElcTUMk Trinity College Dublin, The University of Dublin X Virtual Camera Path Eye Vision
  6. 6. 18.10.2017 6 Trinity College Dublin, The University of Dublin ▪ Without any 3D geometry reconstruction => a lot of cameras ▪ Live Broadcast Event: Super Bowl 2001, 30 cameras ▪ Courtesy of Prof. Takeyo Kanade, Carnegie Mellon Univ. Eye Vision Trinity College Dublin, The University of Dublin Free-viewpoint Video Geometry based Image based Hybrid approach
  7. 7. 18.10.2017 7 Trinity College Dublin, The University of Dublin Image-based Geometry-based Ray-space Light-field 3D mesh model Lumigraph Layered Depth video View-dependent video texture mapping 3D point samples Video fragments Video plus depth Multi-view video plus depth 3D Scene Representations Trinity College Dublin, The University of Dublin Viewport Adaptive VR Video Streaming Patches Representation, coding and interactive rendering of high-resolution panoramic images and video using MPEG-4 S Heymann, A Smolic, K Mueller, Y Guo, J Rurainsky, P Eisert, T Wiegand, Proc. Panoramic Photogrammetry Workshop (PPW), 2005
  8. 8. 18.10.2017 8 Trinity College Dublin, The University of Dublin Augmented Reality S. Heymann, A. Smolić, K. Müller, and B. Froehlich, "Illumination Reconstruction from Real-time Video for Interactive Augmented Reality", Proc. WIAMIS 2005, 6th European Workshop on Image Analysis for Multimedia Interactive Services, Lisbon, Portugal, Montreux, Switzerland, April 13-15, 2005. S. Heymann, A. Smolić, K. Müller, T. Wiegand, and B. Froehlich, "Real-Time Video Capture for Illumina-tion Reconstruction in Augmented Reality Applications", Poster at I3D 2005, ACM SIGGRAPH 2005 Symposium on Interactive 3D Graphics and Games, Washington DC, USA, April 3-6, 2005. Trinity College Dublin, The University of Dublin Augmented Reality S. Heymann, A. Smolić, K. Müller, and B. Froehlich, "Illumination Reconstruction from Real-time Video for Interactive Augmented Reality", Proc. WIAMIS 2005, 6th European Workshop on Image Analysis for Multimedia Interactive Services, Lisbon, Portugal, Montreux, Switzerland, April 13-15, 2005. S. Heymann, A. Smolić, K. Müller, T. Wiegand, and B. Froehlich, "Real-Time Video Capture for Illumina-tion Reconstruction in Augmented Reality Applications", Poster at I3D 2005, ACM SIGGRAPH 2005 Symposium on Interactive 3D Graphics and Games, Washington DC, USA, April 3-6, 2005.
  9. 9. 18.10.2017 9 V-SENSE – Extending Visual Sensation through Image-Based Visual Computing Professor Aljosa Smolic SFI Research Professor of Creative Technologies Trinity College Dublin, The University of Dublin SFI Research Professor • Major funding from Science Foundation of Ireland • 5 years research program for 20+ researchers • Strategic investment • Significant excellence • Significant relevance • Significant opportunity, potential impact
  10. 10. 18.10.2017 10 Trinity College Dublin, The University of Dublin V-SENSE • Extending Visual Sensation through Image-Based Visual Computing • Visual computing at the intersection of • Computer graphics • Computer vision • Media technology • Algorithms on pixels from capture to display • Make visual media indistinguishable from reality and beyond https://v-sense.scss.tcd.ie/ Trinity College Dublin, The University of Dublin V-SENSE Team so far Project Administrator Gail Weadick Research Fellows (Postdocs) Mikael Le Pendue INRIA, Technicolor, Uni Rennes Cagri Ozcinar Uni Surrey, ParisTech Sebastian Knorr, senior TU Berlin, incube Jan Ondrej, senior INSA Rennes, Trinity College, Disney Research Konstantinos Amplianitis HU Berlin, Siemens Martin Alain INRIA, Technicolor, Uni Rennes Rafael Pages UP Madrid Mairead Grogan Trinity College Matis Hudon INRIA, Technicolor, Uni Rennes Neill O‘Dwyer Trinity College PI Aljosa Smolic DRZ, ETH, FhG-HHI, TUB PhD Students Sebastian Lutz Uni Tuebingen Yang Chen Xi'an Jiaotong University, Imperial College Tejo Chalasani IIIT Hyderabad, SUNY Stony Brook, Daqri Koustav Ghosal IIIT Hyderabad Rafael Monroy ETH Zurich, Fraunhofer IGD Simone Croci ETH Zurich, Disney Research David Hardman Edinborough Uni, Intel Samir Mahmalat Uni Dortmund, Disney Research, external
  11. 11. 18.10.2017 11 Trinity College Dublin, The University of Dublin Deep Learning Trinity College Dublin, The University of Dublin V-SENSE Research Areas • Free Viewpoint Video for VR/AR content creation • 360/VR Video • Light-fields • Visual Effects and Animation • Deep Learning
  12. 12. 18.10.2017 12 The Content Gap for AR and VR Professor Aljosa Smolic SFI Research Professor of Creative Technologies Trinity College Dublin, The University of Dublin Augmented Reality • While VR continues to make waves, AR (augmented reality) will prove to be the bigger market in time, projected to reach approximately $100 billion in total market worth by 2020, according to ABI Research. The report shows AR smart glasses are forecast to ship 21 million total units in 2020. Total revenues for the AR market will follow a similar trend, with an estimated CAGR of 73% from 2015 to 2020, and encompassing both dedicated AR hardware sales and mobile, as well as dedicated AR content and software revenues. ABI expects revenues to primarily favour the healthcare and industrial verticals, owning approximately 54% of the market.
  13. 13. 18.10.2017 13 Trinity College Dublin, The University of Dublin AR technology is mature and interesting applications exist (?) Trinity College Dublin, The University of Dublin Virtual Reality VR = • HMD without see-through • Can be rendered content • Or captured (panorama) Google Samsung Sony Facebook/Oculus Oldschool
  14. 14. 18.10.2017 14 Trinity College Dublin, The University of Dublin Today VR Technology is Mature (?) Trinity College Dublin, The University of Dublin Real-time Rendering Game engines are powerful enough to render interactive virtual environments in high quality and level of detail
  15. 15. 18.10.2017 15 Trinity College Dublin, The University of Dublin VR Content: Games Trinity College Dublin, The University of Dublin VR Content: Visualizations/Experiences (rendered)
  16. 16. 18.10.2017 16 Trinity College Dublin, The University of Dublin Omnidirectional Video Capturing Stitching 3840 2160 Trinity College Dublin, The University of Dublin Integrated VR Camera Systems https://www.google.com/get/cardboard/jump/ http://gopro.com/odyssey http://www.video-stitch.com/camera-360/
  17. 17. 18.10.2017 17 Trinity College Dublin, The University of Dublin http://www.jauntvr.com/#about http://www.panocam3d.com/ http://lucidcam.com/ Integrated VR Camera Systems Trinity College Dublin, The University of Dublin Tourism, concert, event, action, ... ‘Put me into the scene’ VR Content: Experiences (live action)
  18. 18. 18.10.2017 18 Trinity College Dublin, The University of Dublin VR Content: News New York Times http://www.nytimes.com/newsgraphics/2015/nytvr/ https://www.youtube.com/watch?v=o3yvjEMFbnM 1:35 artifact Trinity College Dublin, The University of Dublin VR Content: Documentary https://www.jauntvr.com/title/be663b520e
  19. 19. 18.10.2017 19 Trinity College Dublin, The University of Dublin VR Content: Live Action Cinematic VR http://www.jauntvr.com/#about https://www.jauntvr.com/title/ab5640cd90 consumer headsets 2H 2017 standalone ARstandalone
  20. 20. 18.10.2017 20 mobile AR 500 million devices today, +250m in 2018 Apple ARKit 2bn users today of which 1.75bn are on mobile Facebook 100 million devices today, +1bn in 2018 Google ARCore
  21. 21. 18.10.2017 21 And it’s even less competitive for AR FVV for AR and VR Content Creation Professor Aljosa Smolic SFI Research Professor of Creative Technologies
  22. 22. 18.10.2017 22 Trinity College Dublin, The University of Dublin FVV and AR/VR Content Creation Trinity College Dublin, The University of Dublin FVV – Static background reconstruction 3D Point Cloud 3D Mesh Textured Mesh
  23. 23. 18.10.2017 23 Trinity College Dublin, The University of Dublin FVV – Static background reconstruction 3D Point Cloud 3D Mesh Textured Mesh Trinity College Dublin, The University of Dublin FVV – AR/VR content creation
  24. 24. 18.10.2017 24 Trinity College Dublin, The University of Dublin FVV – AR/VR content creation Trinity College Dublin, The University of Dublin FVV
  25. 25. 18.10.2017 25 Trinity College Dublin, The University of Dublin FVV – View Synthesis Trinity College Dublin, The University of Dublin FVV – Dynamic scene reconstruction
  26. 26. 18.10.2017 26 Trinity College Dublin, The University of Dublin FVV – AR/VR content creation Trinity College Dublin, The University of Dublin FVV – View Synthesis
  27. 27. 18.10.2017 27 Trinity College Dublin, The University of Dublin FVV – AR/VR content creation Trinity College Dublin, The University of Dublin FVV – AR/VR content creation
  28. 28. 18.10.2017 28 Virtual Play, after Samuel Beckett Professor Aljosa Smolic SFI Research Professor of Creative Technologies Trinity College Dublin, The University of Dublin Virtual Play
  29. 29. 18.10.2017 29 Trinity College Dublin, The University of Dublin Creative Experiments • Engage with creative community • Showcase, prototype and test technology in real creative productions • Visibility, impact, proof-of-concept, feedback • Budget available increasing over next 3.5 years Trinity College Dublin, The University of Dublin Beckett in Virtual Reality Project Concept 1
  30. 30. 18.10.2017 30 Trinity College Dublin, The University of Dublin Beckett in Virtual Reality Project Concept 2 Trinity College Dublin, The University of Dublin Beckett in Virtual Reality Scenography: combination in Unity
  31. 31. 18.10.2017 31 Trinity College Dublin, The University of Dublin Free-Viewpoint Video From Idea… Trinity College Dublin, The University of Dublin Free-Viewpoint Video …to Implementation (Camera Setup)
  32. 32. 18.10.2017 32 Trinity College Dublin, The University of Dublin Free-Viewpoint Video Different camera views Trinity College Dublin, The University of Dublin Free-Viewpoint Video Character Segmentation
  33. 33. 18.10.2017 33 Trinity College Dublin, The University of Dublin Free-Viewpoint Video Reconstructed 3D Mesh + Texturing Reconstructed 3D Mesh Photorealistic Model Trinity College Dublin, The University of Dublin Free-Viewpoint Video Immersive Environment Environment in VR
  34. 34. 18.10.2017 34 Trinity College Dublin, The University of Dublin Summary & Conclusions • VR and AR will have tremendous impact on society and business over the coming years • There is a lot of interest, opportunity and investment • Lack of compelling content may become a limiting factor, esp. for real world content • Visual Computing technology as developed in V-SENSE can be a slolution to some of these challenges • Compelling examples like Virtual Play, sports, Trinity Long Room, etc. have been demonstrated Many thanks! smolica@scss.tcd.ie Professor Aljosa Smolic SFI Research Professor of Creative Technologies https://v-sense.scss.tcd.ie/

×