Immersive Virtual Reality

1. Immersive Virtual Reality Sébastien Kuntz http://cb.nowan.net/blog Sebastien.Kuntz@nowan.net Last update November 2012Friday, November 16, 12

2. License ”Immersive Virtual Reality Introduction” by Sebastien Kuntz is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/ • With the understanding that: You are free: • Waiver — Any of the above conditions can be • to Share — to copy, distribute and transmit the waived if you get permission from the copyright work holder. • Public Domain — Where the work or any of its • to Remix — to adapt the work elements is in the public domain under applicable • to make commercial use of the work law, that status is in no way affected by the license. Under the following conditions: • Other Rights — In no way are any of the following • Attribution — You must attribute the work in the rights affected by the license: manner speciﬁed by the author or licensor (but not in ◦ Your fair dealing or fair use rights, or other any way that suggests that they endorse you or your use applicable copyright exceptions and limitations; of the work). ◦ The authors moral rights; • You must attribute “Immersive VR introduction” to ◦ Rights other persons may have either in the (with this link): work itself or in how the work is used, such as • Sébastien Kuntz - http://cb.nowan.net/blog publicity or privacy rights. • Notice — For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page.Friday, November 16, 12

3. Plan Deﬁnitions VR Systems Applications TechniquesFriday, November 16, 12

4. WHO AM I ? • Founder editor of • http://www.imin-vr.com • AFRV - French VR association: Board member - http://www.af-rv.fr • VR Geeks association: President - http://www.vrgeeks.org • Blog: http://cb.nowan.net/blog • Twitter: @Cb_VRGeek • 10 years in VR • Virtools / Dassault Systèmes • SNCF - French railwaysFriday, November 16, 12

5. What is VR for you ?Friday, November 16, 12

6. Immersive Virtual Reality Presence in a virtual world (c) Barco (c) eMaginFriday, November 16, 12

7. Immersive VR = iVR iVR is the science and technology required for a user to feel present, via perceptive, cognitive and functional immersion and interaction, in a (computer) generated environment.Friday, November 16, 12

8. Presence Cognitive presence Perceptive presence Results Natural interactions Natural reactionsFriday, November 16, 12

9. What is VR ? Realtime 3D != VR Stereoscopy != VR 3D Tracking != VR But everything Haptics != VR combined can be VR 3D Sound != VR 3D Interactions != VRFriday, November 16, 12

10. Ridiculous If it’s real VR, you probably look ridiculousFriday, November 16, 12

11. Why now ? Already changing world in industry and research Democratization Are we in the future ? No ! We’re still in prehistory ! WE can make a difference, invent systems, applications, businesses, interactionsFriday, November 16, 12

12. Measure presence ?Friday, November 16, 12

13. Measure presence ? Can’t objectively measure presence (qualia) What you can measure is if people respond to this Virtual Environment as if it is real, which is a result of presence Compare real-life behavior vs virtual If avatar: brain zone activates when user accepts the avatar as realFriday, November 16, 12

14. Immersion parameters Graphics framerate Extent of tracking Tracking latency (measure?) Quality of the images Field of view Plausibility of behaviors Range of sensory modalities ...Friday, November 16, 12

15. Paul MilgramFriday, November 16, 12

16. VR SystemsFriday, November 16, 12

17. Projector based VR SystemsFriday, November 16, 12

18. CAVE http://www.jamstec.go.jp/esc/research/Perception/vr.en.htmlFriday, November 16, 12

19. Cave (c) Sébastien KuntzFriday, November 16, 12

20. http://www.youtube.com/watch?v=Hb29bLHEjUg Clarte - SAS CubeFriday, November 16, 12

21. Immersia 3 (c) Sébastien KuntzFriday, November 16, 12

22. Immersia 3 - IRISA Rennes http://www.youtube.com/watch?v=cTd5DLe8KD0Friday, November 16, 12

23. Calit2 - StarCave http://www.scivee.tv/node/3648 http://www.calit2.net/newsroom/release.php?id=1383Friday, November 16, 12

24. Projector based Reality center - HoloStage (c) Arnaud MutheletFriday, November 16, 12

25. Projector based - Workbench http://liris.cnrs.fr/~sbrandel/en/research/VR/ LSIIT-IGGFriday, November 16, 12

26. Projector based Workbench © PSA Peugeot Citroën Mediatheque.Friday, November 16, 12

27. Clarte R-Screen http://www.youtube.com/watch?v=AcV5ma5nEVgFriday, November 16, 12

28. Panoscope 360° (c) MatsushitaFriday, November 16, 12

29. La GéodeFriday, November 16, 12

30. jDome http://www.youtube.com/watch?v=rNsFSVhZi9wFriday, November 16, 12

31. Toob Immersive dome http://youtu.be/N6LiLaNOHHUFriday, November 16, 12

32. Barco - RP-360 • 13 projectors • Diamètre: 3.4mFriday, November 16, 12

33. Barco - RP-360 http://www.youtube.com/watch?v=3EbrQwpOx24Friday, November 16, 12

34. 6 users: stereo + tracking Weimar University (DE) http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.htmlFriday, November 16, 12

35. 6 users stereoscopy + tracking • video Weimar University (DE) http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.htmlFriday, November 16, 12

36. Screen based Full-scale NexCAVE developed and deployed by Calit2 for Saudi Arabias King Abdullah University of Science and Technology - http://www.calit2.net/newsroom/release.php?id=1743Friday, November 16, 12

37. Ultra-Short throw projectors • Sanyo PDG-DWL2500 • 1280x800 • 2500 lumens • Canon LV-8235 USTFriday, November 16, 12

38. Ultra-Short throw projectors http://www.youtube.com/watch?v=VSblSrTRJh0Friday, November 16, 12

39. TV-3D - 154” (c) Sébastien KuntzFriday, November 16, 12

40. Projector based VR Systems (-) (Very) expensive ! Lots of space ! ( front and behind, AC... ) Not 360° Cluster hard to program ! (+) +/- Collaborative Not much to wear High resolution, big FOVFriday, November 16, 12

41. HMD Head mounted displaysFriday, November 16, 12

42. Vuzix •640x480 •32° FOV •852x480 •32° FOVFriday, November 16, 12

43. eMagin Z800 800x600 40° FOVFriday, November 16, 12

44. Sony HMZ-T1 1280x1024 45°Friday, November 16, 12

45. Sensics piSight Multiple screens zSight 1280x1024 70° FOVFriday, November 16, 12

46. NVIS nVisor SX60 1280x1024 60° FOV nVisor SX111 1280x1024 111° FOVFriday, November 16, 12

47. Fakespace Labs Wide5 150° horizontal FOV 1600x1200 ProgressiveFriday, November 16, 12

48. OCULUS RIFT • Developer kit • 90° horizontal FOV • 1280x800 • 640x800 per eyeFriday, November 16, 12

49. HMD (-) Low resolution, low FOV Must wear, adjust No peripheral vision (+) Lightweight Compact Easier to program 360° trackingFriday, November 16, 12

50. Summary Projector/screen based Cave, holostage, workbench Collaborative, high FOV Expensive, Complex HMD 360° “Cheap” Low Fov/resolutionFriday, November 16, 12

51. Depth perceptionFriday, November 16, 12

52. Depth perception How do you perceive depth ?Friday, November 16, 12

53. Stereoscopy ? http://www.youtube.com/watch?v=Jd3-eiid-Uw Johnny Lee - Head Tracking with a wiimoteFriday, November 16, 12

54. Stereoscopy ? http://www.youtube.com/watch?v=nRYybo2WNwU Esiea Virtual ShowcaseFriday, November 16, 12

55. Stereoscopy You perceived depth without stereoscopy !Friday, November 16, 12

56. Depth perception Monoscopic cues : Stereoscopic cues : perspective eye accomodation atmospheric artifacts (fog) relative sizes eye convergence angle occlusion parallax disparity between left shadows and right images Parallax can be more important for depth perception! Stereoscopy is important for size and distance evaluationFriday, November 16, 12

57. StereoscopyFriday, November 16, 12

58. Friday, November 16, 12

59. Stereoscopy • Useful for • Relative distance perception • Detection of objects • Discrimination between two objects • Localisation • But... • Fusing problems • Headaches • Visual fatigue • Accommodation / Vergence dissociationFriday, November 16, 12

60. Different stereoscopic separation schemes Active shutter glasses Polarized glasses Anaglyph Inﬁtec Autostereoscopic HologramsFriday, November 16, 12

61. Active stereoscopyFriday, November 16, 12

62. Passive Stereoscopy Linear Circular Polarization PolarizationFriday, November 16, 12

63. AnaglyphFriday, November 16, 12

64. Inﬁtec - Eye separation by color spectrum -Friday, November 16, 12

65. Light Efﬁciency http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdfFriday, November 16, 12

66. AutostereoscopicFriday, November 16, 12

67. Holograms Stores ALL light interactions through an entire “window”Friday, November 16, 12

68. HologramsFriday, November 16, 12

69. Stereoscopy for multiple users ?Friday, November 16, 12

70. Stereoscopy for multiple users ? Active + polarized SequentialFriday, November 16, 12

71. 6 users: stereo + tracking Weimar University (DE) http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.htmlFriday, November 16, 12

72. 6 users stereoscopy + tracking • video Weimar University (DE) http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.htmlFriday, November 16, 12

73. Multi-displayFriday, November 16, 12

74. What system can we create with only one graphics card?Friday, November 16, 12

75. What can we do with one graphics card?Friday, November 16, 12

76. Nvidia SLI - ATI Crossﬁre Use two graphics card for output on only one graphics cards Alternate frame rendering Split frame renderingFriday, November 16, 12

77. Scalable graphics Image decomposition ( ~= SLI ) Useful if application is fill-rate limited frame rate decreases with increasing number of pixels (i.e. when the image resolution increases)Friday, November 16, 12

78. Scalable graphics Database decomposition Useful if application is CPU limited frame rate decreases with increasing size of 3D scenesFriday, November 16, 12

79. Scalable graphics StereoscopyFriday, November 16, 12

80. How many displays with one computer ?Friday, November 16, 12

81. Why multi-display ? Multiple display surfaces Cave Increase display size Increase brightness CPU/GPU Bottlenecks ClusterFriday, November 16, 12

82. How ? Matrox DualHead2Go Matrox TripleHead2Go Multiple graphics card in a single PC Often max 2 graphics cards CyvizFriday, November 16, 12

83. Multiple graphics cards in one PC Multi-pipe ~= Multi-GPU One rendering context per GPU Problems of resources sharing, threading Need speciﬁc graphics engine adaptations But best performancesFriday, November 16, 12

84. NVidia Quadro Plex•Up to : •4 GPU •4 outputs •16gb memory•Can combine two Quadro PlexFriday, November 16, 12

85. Quadro Mosaic Mode Mosaic mode hides multi-gpu Only shows one gpu to system Can have 4 outputs with 1 plex without needing multi-pipe adaptation Good performancesFriday, November 16, 12

86. ATI EyeFinityFriday, November 16, 12

87. Increase size or brightness Mersive autocalibrationFriday, November 16, 12

88. What if need more displays ? 96 projectors ?Friday, November 16, 12

89. Cluster !Friday, November 16, 12

90. Cluster Synchronisation Scene synchronisation (Scenelock) Swap-Buffer synchronisation (Swaplock) Eyes synchronisation (Genlock)Friday, November 16, 12

91. Cluster + Scalable Each node can be cheap Easy node replacement - Complex to manage/programFriday, November 16, 12

92. TrackingFriday, November 16, 12

93. Tracking Mechanical Fakespace’s Boom + Precise Fast Help Lift weight - Big, discontinuedFriday, November 16, 12

94. Tracking Magnetic Polhemus Ascension + Precise, fast - Wires Sensitive to metalFriday, November 16, 12

95. Tracking Inertial SpacePoint Fusion XSens Mocap Accelerometers, Gyroscopes, magnetometers + Very fast Light - Drift Orientation onlyFriday, November 16, 12

96. Tracking Optical ART ioTracker OptiTrack TrackIR + Wireless No electronics - Install cameras Line of sightFriday, November 16, 12

97. Leap motion https://www.youtube.com/watch?v=N6hCwjwzUHgFriday, November 16, 12

98. Tracking Mixing Optical Intersense IS-900 Inertial + Ultrasound IS-1200 Inertial + AR Patterns Wiimote Playstation MoveFriday, November 16, 12

99. Optical Body Tracking 4D Views http://www.4dviews.com/Friday, November 16, 12

101. Kinect fusion http://www.youtube.com/watch?v=quGhaggn3cQFriday, November 16, 12

102. GlovesFriday, November 16, 12

103. HapticsFriday, November 16, 12

104. Haptics Force feedbackFriday, November 16, 12

107. TN Games - Gaming VestFriday, November 16, 12

108. LocomotionFriday, November 16, 12

109. Locomotion CirculaFloor • http://www.youtube.com/watch?v=rYsvB2y2Ero Virtusphere • http://www.youtube.com/watch?v=qTnnJR-hS7k OmniDirectionnal Treadmill • http://www.youtube.com/watch?v=BQw1tsgrJOsFriday, November 16, 12

110. Kuka robot http://www.cyberneum.de/f1/Friday, November 16, 12

111. Force Dynamics http://www.force-dynamics.com/Friday, November 16, 12

112. Wizdish http://www.wizdish.comFriday, November 16, 12

113. ApplicationsFriday, November 16, 12

114. IVR Applications Design, prototyping Marketing Training Medical Research Geoscience, ﬁnance, archeologie, urbanism, sport, art...Friday, November 16, 12

115. iVR Applications Design, prototyping Car, aircraft, boat, tractors, house appliances Collaborative design review (c) Peugeot PSA Experience from user’s point of view Easy and fast to prototype More iterations, better products (c) MieleFriday, November 16, 12

116. PSA Peugeot-Citroen - http://www.youtube.com/watch?v=WhMh191RltQFriday, November 16, 12

117. FORD http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/Friday, November 16, 12

118. VR4D http://vr4d.ingenierium.com/Friday, November 16, 12

119. Ergo-Wide http://www.clarte.asso.fr/realite-virtuelle.php/en/ERGOWIDE%C2%B3/ https://www.youtube.com/watch?v=aczLiNtVyDIFriday, November 16, 12

120. Lockheed Martin http://www.youtube.com/watch?v=itOxIpj68MwFriday, November 16, 12

121. Mockup-builder http://www.youtube.com/watch?v=ZxJD9DXDB1EFriday, November 16, 12

122. iVR Applications Marketing Product experience (c) University of Gronigen Shopping experience Procter&Gamble Kimberley Clarke (c) VirtoolsFriday, November 16, 12

123. iVR Applications Training Repeatability Controlled environment (c) SNCF Can practice rare / dangerous situations Can analyze / quantify performance No health risk for patients Easy accessibility Reduced costs Adaptable difﬁculty and assistance Multiple users trained at same time (c) University of Florida AttractiveFriday, November 16, 12

124. SNCF - FiacreFriday, November 16, 12

125. SNCF - SimuratFriday, November 16, 12

126. Saint-Gobain & Clarte Use less material Environment friendly No water, nothing to clean Faster to setup http://www.youtube.com/watch?v=FEnalzHVmdIFriday, November 16, 12

127. RE-lion http://vimeo.com/25032061Friday, November 16, 12

128. ESCIN - Sivic - Crime scene investigation http://www.youtube.com/watch?v=FC0gHv-SnmwFriday, November 16, 12

129. iVR Applications Research Heal Exposure therapy PTSD, Phobias Rehabilitation : motor, addictions Create Empathy http://www.youtube.com/watch?v=SinlgK5Q4v4 Pain Distraction Crescent - AcrophobiaFriday, November 16, 12

130. Burn pain - Snow World http://www.youtube.com/watch?v=jNIqyyypojgFriday, November 16, 12

131. Fear of ﬂight Arte - “A chacun sa réalité”Friday, November 16, 12

132. Summary IVR strengths Controlled / adaptable / safe environment Natural interactions / reactions AnalyzeFriday, November 16, 12

133. iVR Games ? Cave Quake 3 (c) Paul Rajlich Cave Unreal Tournament (c) Jeffrey JacobsonFriday, November 16, 12

134. https://www.youtube.com/watch?v=r580mzSxSp8Friday, November 16, 12

135. Immersive rail shooter http://vimeo.com/9549613Friday, November 16, 12

136. VR Angry-bot http://www.youtube.com/watch?v=QSXxjr1SnqAFriday, November 16, 12

137. Battleﬁeld 3 simulator www.youtube.com/watch?v=nQR49JGySTMFriday, November 16, 12

138. 3D User InteractionsFriday, November 16, 12

139. VR Techniques 3D User Interactions Bowman classiﬁcation for generic interactions Selection Specifying one or more objects from a set Manipulation Modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.) Navigation Travel Wayﬁnding System Control Menus Issuing command Changing mode Choosing tool Symbolic Input Other interactions Measuring …? http://www.3dui.orgFriday, November 16, 12

140. Navidget http://www.youtube.com/watch?v=wpaKjLZll80Friday, November 16, 12

142. http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/Friday, November 16, 12

143. http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/Friday, November 16, 12

146. World builder http://www.youtube.com/watch?v=VzFpg271sm8Friday, November 16, 12

147. Perceptive illusionsFriday, November 16, 12

148. Body representation in VR Proteus effect : virtual appearance can transform our behaviors towards other people ( Jeremy Bailenson ) It seems possible to transfer our body in virtual bodiesFriday, November 16, 12

149. Rubber hand illusion http://www.youtube.com/watch?v=TCQbygjG0RUFriday, November 16, 12

150. Body transfer http://www.youtube.com/watch?v=8oF8sQvnTlMFriday, November 16, 12

151. Barbie doll illusion http://www.youtube.com/watch?v=_z8pSTMfGSoFriday, November 16, 12

152. First person VR illusion http://www.youtube.com/watch?v=DlbSVnVRLTUFriday, November 16, 12

153. Perceptive Ilusions Pseudo haptics http://kaji-lab.jp/ja/index.php?research http://www.irisa.fr/tactilesFriday, November 16, 12

154. Perceptive Ilusions Pseudo hapticsFriday, November 16, 12

155. Perceptive Illusions Play with perception’s limits Our perception is dominated by visual sense Balance Redirected Walking : (c) Franck SteinickeFriday, November 16, 12

156. Perceptive illusions Change blindness : (c) Evan Suma - USC/ICTFriday, November 16, 12

157. Institute of creative technologies - change blindess https://www.youtube.com/watch?v=FTWVjJ4um4AFriday, November 16, 12

159. Perceptive Illusions Passive haptics (c) Luv KohliFriday, November 16, 12

160. Passive haptics Ford PVMFriday, November 16, 12

161. Perceptive illusions Learn about human perception through its limitations Improve VR experience by taking advantage of those limitationsFriday, November 16, 12

162. Conclusion It is possible to transfer your body in VR Reality is not what you think VR is just another realityFriday, November 16, 12

163. http://www.af-rv.frFriday, November 16, 12

164. AFRV http://www.youtube.com/watch?v=lHZltjDZBJo&feature=plcpFriday, November 16, 12

165. Friday, November 16, 12 www.VRGeeks.org

166. http://www.vrgeeks.org/vr-experiences/vrescapeFriday, November 16, 12

167. http://www.vrgeeks.org/vr-experiences/vrsadisticFriday, November 16, 12

168. http://www.vrgeeks.org/vr-experiences/insnaketionFriday, November 16, 12

170. VR Challenges What we must solve VR is mainly a SOFTWARE problem “3DUI is a hard and unexplored software engineering (and ergonomic) problem” Markerless tracking Minimum latency HMDs Lightweight Wide FOV Wireless Full body haptics Multi-sensorial experience Locomotion Avatars VR For AllFriday, November 16, 12

171. VR “Experiences” Your goal : Create presence And don’t break it !!! Verdun 1916-Time machine http://www.youtube.com/watch?v=ZBfYIefenp0Friday, November 16, 12

172. ConclusionFriday, November 16, 12

173. Thank you ! Questions ? Sebastien.Kuntz@nowan.netFriday, November 16, 12