Bhubaneswar🌹Call Girls Bhubaneswar ❤Komal 9777949614 💟 Full Trusted CALL GIRL...
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
1. Computational Near-eye Displays with Focus
Cues
Gordon Wetzstein
Stanford University
SID Display Week 2017
www.computationalimaging.org
2.
3. Personal Computer
e.g. Commodore PET 1983
Laptop
e.g. Apple MacBook
Smartphone
e.g. Google Pixel
AR/VR
e.g. Microsoft Hololens
???
4. A Brief History of Virtual Reality
1838 1968 2012-2017
Stereoscopes
Wheatstone, Brewster, …
VR & AR
Ivan Sutherland
VR explosion
Oculus, Sony, HTC, MS, …
Nintendo
Virtual Boy
1995
VR 2.0
5. Ivan Sutherland’s HMD
• optical see-through AR, including:
• displays (2x 1” CRTs)
• rendering
• head tracking
• interaction
• model generation
• computer graphics
• human-computer interaction
I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968
13. 4D / 25cm Optical Infinity
Normal vision
Nearsighted/myopic
Farsighted/Hyperopic
Presbyopic
Focal range (range of clear vision)
Modified from Pamplona et al, Proc. of SIGGRAPH 2010
Nearsightedness & Farsightedness
37. Adaptive Focus - History
• M. Heilig “Sensorama”, 1962 (US Patent #3,050,870)
• P. Mills, H. Fuchs, S. Pizer “High-Speed Interaction On A Vibrating-Mirror 3D Display”, SPIE 0507 1984
• S. Shiwa, K. Omura, F. Kishino “Proposal for a 3-D display with accommodative compensation: 3DDAC”, JSID 1996
• S. McQuaide, E. Seibel, J. Kelly, B. Schowengerdt, T. Furness “A retinal scanning display system that produces multiple focal planes with
a deformable membrane mirror”, Displays 2003
• S. Liu, D. Cheng, H. Hua “An optical see-through head mounted display with addressable focal planes”, Proc. ISMAR 2008
manual focus adjustment
Heilig 1962
automatic focus adjustment
Mills 1984
deformabe mirrors & lenses
McQuaide 2003, Liu 2008
68. Monovision VR
Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
69. Monovision VR
Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
• monovision did not drive accommodation
more than conventional
• visually comfortable for most; particularly
uncomfortable for some users
71. Multiplane VR Displays
• Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39
• Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH)
• Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express
• Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID
• Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR
• Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express
• … many more ...
idea introduced
Rolland et al. 2000
benchtop prototype
Akeley 2004
near-eye display prototype
Liu 2008, Love 2009
72. Multiplane VR Displays
• Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39
• Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH)
• Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express
• Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID
• Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR
• Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express
• … many more ...
idea introduced
Rolland et al. 2000
benchtop prototype
Akeley 2004
near-eye display prototype
Liu 2008, Love 2009
90. Maxwellian-type Near-eye Displays
• eyebox of display is a pinhole very large depth of field (no
retinal blur cue)
• exit pupil size of ≤0.5 mm accommodation in open loop
• pinholes are dim and reduce eyebox severely! (not practical)
98. Other Upcoming Work at SIGGRAPH 2017
“Focal Surface Displays”, Matsuda et al.
“Accommodation and Comfort in Head Mounted Displays”, Koulieris et al.
“Holographic Near-eye Displays for Virtual and
Augmented Reality”, Maimone et al.
99. Summary
• focus cues in VR/AR are challenging
• adaptive focus can correct for refractive errors (myopia, hyperopia)
• gaze-contingent focus gives natural focus cues for non-presbyopes, but
require eyes tracking
• presbyopes require fixed focal plane with correction
• multiplane displays require very high speed microdisplays
• monovision has not demonstrated significant improvements
• Maxwellian-type displays can be interesting, but provide small eyebox
• light field displays may be the “ultimate” display need to solve “diffraction
problem”
106. Oculus DK2 (115 deg)
FOV (diag) and FOV location in some of the current products
Sony Morpheus (90 deg)
Sony HMZ-T3 (51deg)
Lumus DK-32 (40 deg)
Zeiss cinemizer (35 deg)
Optinvent ORA (24 deg)
Epson Moverio
(23 deg)
Occlusion
display
See
through
display
Google Glass
(15deg)
Vuzix M-100
(16deg)
Recon Jet
(16deg)
slidebyBernardKress
107. FOV and angular resolution for current productsAngularresolution(pixels/deg)
FOV (diag, deg)
Eye resolution limit (1.2 arc min or around 50 pixels/deg)
50
40
30
20
10
10 20 30 40 50 60 70 80 90 100 110
Glass Epson
Moverio
Vuzix
m100
Oculus
DK1
Sony HMZ
T3
Optinvent
Lumus
Sony
Morpheus
Resolution measured in dpd rather than dpi
Oculus
DK2
slidebyBernardKress
108. There are two main optical HMD architectures
slidebyBernardKress
109. Opaque Bird bath Free space
Waveguide
EPE
Lightguide
There are 5 main head-worn optical platform architectures
developed in industry today
See through architectures
slidebyBernardKress
112. Oculus DK2
Sony HMZ
Vuzix
Vuzix M100
MyVu
…
Occlusion
Huge FOV
Large eyebox
Partially occluded
Small FOV
Medium eye box
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
113. Most of the early occlusion HMDs are based on the combination of a
transmission lens and a 45 degrees mirror (either TIR or coated) , in either
monocular or binocular version.
MicroOptical Corp. –
MyVu Corp
Recon Instruments Corp.
slidebyBernardKress
114. “Bird bath” optical architectures
Google Glass
RockChip Ltd
ITRI
Good see through
Small FOV
Medium eye box
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
116. Laster Sarl
ODA Labs
…
Bug eye optics
Large FOV
Medium eyebox
Temple projector
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
117. The notion of « Bug Eye » in HMD combiner opticsslidebyBernardKress
118. This is what we mean with the « bug-eye » concept
slidebyBernardKress
119. ORA Associates (US)
Reflective “bug eye” combiner
Binocular version
Laster Sarl (Fr)
Reflective combiner (pupil forming
architecture)
slidebyBernardKress
120. Canon Ltd
Motorola HC1
Kopin Golden
eye
Distorted see-through
(or opaque)
Medium FOV
Medium eyebox
Without complement piece
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
121. Canon Ltd
Motorola HC1
Kopin Golden
eye
E-Magin Optics
Fraunhofer
Distorted see-through
(or opaque)
Medium FOV
Medium eyebox
Good see through
Medium FOV
Medium eye box
Without complement piece With complement piece
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
122. TIR / free-form surfaces combiner (including eye tracking)
Example with
complement piece
for see through
slidebyBernardKress
[Hua et al. 2013]
123. Epson Ltd
Moverio 1 & 2
…
OK see through
Medium FOV
Medium eyebox
Curve coated reflector combiner
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
124. Epson Corp, Moverio (first and second generations)
3 collimation lenses, prism injector, 6 TIR bounces and curved 50/50 mirror
Conventional optics (linear mirror in first gen and curved mirror in second gen)
slidebyBernardKress
125. Optinvent Sarl See through OK
Medium FOV
Large eyebox
Injection molded
Lumus Ltd See through OK
Medium FOV
Large eyebox
All glass
various coatings
Micro-prism combiner Cascaded coated mirrors combiner
Opaque Bird bath Free space
slidebyBernardKress
Waveguide
EPE
Lightguide
126. Word side
6 mirrorsWorld side
Lumus Ltd, DK32 (40deg FOV, 1.7mm)
and DK40 (25 deg FOV. 2.0mm)
(cascaded mirrors extractors)
Uses side collimator and prism coupler
slidebyBernardKress
127. Optinvent Sarl See through OK
Medium FOV
Large eyebox
Injection molded
Lumus Ltd See through OK
Medium FOV
Large eyebox
All glass
various coatings
Sony Ltd
Vuzix /Nokia
(M2000AR)
BAE Q-sight
Good see through
Medium FOV
Large eyebox
Photopolymer
Micro-prism combiner Cascaded coated mirrors combiner Volume holographic combiner
Diffractive combiner
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBernardKress
128. H. Mukawa et al. A full color eyewear
display using holographic planar
waveguides. Proc. SID 2008.
129. (Some) Technology Challenges
• Vergence-accommodation conflict (VAC)
• Vestibular-visual conflict (motion sickness)
• AR • occlusions
• aesthetics / form factor
• battery life
• heat
• wireless operation
• low-power computer vision
• registration of physical /
virtual world and eyes
• consistent lighting
• scanning real world
• VAC more important
• display contrast &
brightness
• fast, embedded GPUs
• …