Compressive Light Field Displays
•Télécharger en tant que PPTX, PDF•
7 j'aime•5,811 vues
Overview of a new generation of glasses-free 3D (light field) displays.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (20)
Similaire à Compressive Light Field Displays
Similaire à Compressive Light Field Displays (20)
Dernier
Dernier (20)
Compressive Light Field Displays
- 1. Compressive This slide has a 16:9 media window Light Field Displays Gordon Wetzstein - MIT Media Lab Collaborators: Doug Lanman, Matt Hirsch, Ramesh Raskar, Wolfgang Heidrich
- 2. This slide has a 16:9 media window
- 4. viewer moves right This slide has a 16:9 media window viewer moves down 4D Light Field
- 5. Compressive Displays Computational Display Optics Processing
- 6. This slide has a 16:9 media window Display-adaptive Compression Computed Tomography Nonnegative Tensor 4D Light Field Factorization Compressive Optics Uniform or Directional Backlight Stacked Layers (LCDs or Transparencies)
- 7. Prototype – Layered 3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
- 8. Video clip
- 9. Prototype – Tensor Display, SIGGRAPH 2012
- 10. Video clip
- 11. What do we mean by “glasses-free 3D”? binocular disparity convergence motion parallax accommodation/blur current glasses-based (stereoscopic) displays near-term glasses-free (light field) displays longer-term holographic displays
- 12. Is glasses-free 3D Technology ready? Nintendo 3DS MasterImage 3D Asus Eee Pad MeMO 3D LG Optimus 3D E3 2010 Computex 2011 Computex 2011 Mobile World Congress 2011 Toshiba 3DTV Prototype Sony 3DTV Prototype LG 3DTV Prototype CES 2011 CES 2011 CES 2011
- 14. Parallax Barriers – Ives 1903 barrier 2D display Low resolution & very dim Switchable 2D/3D with LCDs
- 15. Parallax Barriers – Ives 1903 barrier Nintendo 3DS 2D display Low resolution & very dim Switchable 2D/3D with LCDs
- 16. lenslets Integral Imaging – Lippmann 1908 2D display Brighter than parallax barriers Always low resolution, even for 2D
- 17. lenslets Integral Imaging – Lippmann 1908 2D display Alioscopy 3DHD 42'' (1920x1200, 1x8 views) Brighter than parallax barriers Always low resolution, even for 2D
- 18. Directional Backlighting – 3M & MS Wedge 3M Directional Backlight Film Nelson and Brott, 2010 US Patent 7,847,869 LED thin light guide LED Requires 120 Hz for stereo Not practical for multiview Microsoft Wedge
- 19. Directional Backlighting Lenslet array
- 20. Glasses-Free 3D Display LightSpace Sony Jones et al. 2007 Zebra Imaging MIT Holovideo Holograms Displays Volumetric 3Ddepth cues inside enclosure all objects only opticallymechanically moving parts mostly & computationally expensive 3D objects outside enclosure inexpensive off-the-shelf parts Compressive LF Displays no moving parts efficient computationally
- 21. From Conventional to Compressive 3D Displays mask 2 mask 1 Conventional Parallax Barriers t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 22. From Conventional to Compressive 3D Displays mask 2 mask 1 Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 23. From Conventional to Compressive 3D Displays Perceptual Integration time Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 24. From Conventional to Compressive 3D Displays mask 2 mask 1 High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 25. From Conventional to Compressive 3D Displays Perceptual Integration time High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 26. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 27. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 28. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 29. From Conventional to Compressive 3D Displays Perceptual Integration … … … time Tensor Displays Compression in Time & Pixels –Tensor Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 30. From Conventional to Compressive 3D Displays Perceptual Integration … … … time Tensor Displays – Multilayer & Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 31. From Conventional to Compressive 3D Displays Perceptual Integration thin! time Tensor Displays – Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 33. Prototype – Layered 3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
- 35. Computed Tomography (CT) x-ray sensor source: wikipedia 3D Reconstruction x-ray source Reconstructed 2D Slices 35
- 36. Tomographic Light Field Synthesis Image Formation Virtual Planes - ò c m (r )dr x L(x, q ) = e Attenuation Volume log L x, (r )dr c Backlight Tomographic Synthesis 2D Light Field log( L ) P 2 argmin log( L) P 2 0 x 36
- 37. CT vs. Layered 3D Computed Tomography Layered 3D reconstruct physical volume thin stack of optimized layers sensor noise no noise 37
- 38. Multi-Layer Decomposition viewer moves right viewer moves down Input 4D Light Field 1 2 3 1 2 4 3 5 4 5 Photographs of Prototype Optimized Attenuation Layers
- 39. Depth of Field for 3D Displays Integral Imaging Parallax Barriers Cutoff (cycles/cm) Maximum Resolution Display Thickness Zwicker et al. 2006 Antialiasing + Display Prefilter Distance of Virtual Plane from Middle of Display (cm)
- 40. How Do Layers Increase Depth of Field? Integral Imaging Parallax Barriers Layered 3D Cutoff (cycles/cm) ? Maximum Resolution Display Thickness Distance of Virtual Plane from Middle of Display (cm)
- 41. Optimization: Number of Layers Two Layers Three Layers Five Layers
- 42. Optimization: Display Thickness Average Reconstruction PSNR for All Scenes PSNR in dB Number of Layers
- 43. Application to HDR Display “Square Root” Layers
- 44. Application to HDR Display “Square Root” Layers
- 45. Application to HDR Display Optimized Layers
- 46. Video clip
- 47. Limitations: Field of View FOV 10º FOV 20º FOV 45º
- 48. Personal Glasses-Free 3D Display Challenges for dynamic display: Real-time computation Engineering issues, moiré
- 50. Multi-Layer LCD – SIGGRAPH ASIA 2011
- 51. Barco E-2320 PA Grayscale IPS LCD 1600x1200 @ 60 Hz
- 52. Four Stacked Liquid Crystal Panels Two Crossed Polarizers
- 53. Overview of LCDs I vertical polarizer color filter array liquid crystal cells horizontal polarizer I0 backlight Malus’ Law Intensity Modulation with Liquid Crystal Cells I = I0 sin2 (q )
- 54. Extending Layered 3D to Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Sequentially-crossed design LCD x 2 LCD 1 backlight 2D Light Field x
- 55. Extending Layered 3D to Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Use sequentially-crossed LCD x • Exploit field-sequential color 2 0.33 = 2.7% brightness LCD 1 backlight 2D Light Field x
- 56. Polarization Field Displays Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Use sequentially-crossed LCD x • Exploit field-sequential color 2 0.33 = 2.7% brightness LCD • Further optimize polarizers 1 Minimum is a crossed pair backlight 2D Light Field x
- 57. Modeling and Synthesizing Polarization Fields Virtual Planes LCD f3 3 Image Formation f2 K LCD x Q(x, q ) = åfk (x, q ) 2 k=1 f1 L(x, q ) = sin 2 (Q(x,q )) LCD 1 backlight Tomographic Synthesis Q(x,q ) = ±sin-1 ( ) L(x, q ) mod p 2D Light Field Q = Pf argmin Q - Pf 2 2 fmin £f £fmax x
- 58. Decompositions & Reconstructions viewer moves right viewer moves down Stacked Polarization Input 4D Light Field Rotating Layers 90° Optimized Rotation Angles for Each Layer 0°
- 59. Decompositions & Reconstructions viewer moves right viewer moves down Input 4D Light Field Reconstruction Results
- 60. Multi-layer LCD Attenuation Layers Polarization-Rotating Layers
- 61. Video clip
- 62. Tomographic Image Synthesis Target Light Field Projection Matrix LCD Pixel Values = * b=Ax
- 63. SART - Simultaneous Algebraic Reconstruction Technique b=Ax pre-compute some weights initial guess ATv Ax update clamp
- 64. Implementing Ax as Multiview Rendering
- 65. Implementing ATv as Projective Texture Mapping
- 68. Light Field “Slice” Representation Light Field moving to the right
- 69. Light Field “Slice” Representation Light Field Light Field Slice moving to the left
- 70. Light Field “Slice” Representation Light Field View from Above Light Field Slice moving to the left
- 71. Light Field “Slice” Representation Light Field Multilayer Light Field Display Light Field Slice moving to the left Front Layer Middle Layer Rear Layer Backlight
- 72. Light Field “Slice” Representation Light Field Multilayer Light Field Display Light Field Slice L( moving to the left Front Layer fm(3)( Middle Layer fm(2)( L( Rear Layer fm(1)( Backlight
- 73. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
- 74. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
- 75. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 76. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 77. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 78. Light Field Tensor Decomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
- 79. Light Field Tensor Decomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
- 80. Light Field Tensor Decomposition Nonlinear (Multilinear) Optimization Problem Iterative Update Rules (see paper for details) Efficient GPU Implementation Forward Projection (Multiview Rendering) Back Projection (Projective Texture Mapping)
- 82. Design Tradespace: Layers vs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames
- 83. Design Tradespace: Layers vs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames 2 Layers, Layers,Directional–Backlight (Tensor Display) 3 Layers, 1 Frame Frames 3D – SIGGRAPH 2010) 1 Layer, 3 3 3 Frames(Layered (Tensor Display) 2011) Frames, 3 (HR3D SIGGRAPH Asia Original Original 2 Layers, 3 Frames 3 Layers, 1 Frame 3 Layers, 3 Frames 1 L, 3 F, Directional BL
- 84. Tensor Display Prototypes Reconfigurable Directional Backlight Three Layer
- 85. Tensor Display Prototypes 3 Layer LCD Directional Backlight Hardware
- 86. Three Layer Prototype Video clip
- 87. Video clip
- 88. Directional Backlight Prototype Video clip
- 89. LCD + Directional BL View from above LCD with Directional Backlight, Rank 6 Directional BL
- 90. LCD with Directional Backlight, Rank 6 (as seen by obserer) Video clip
- 91. LCD with Directional Backlight, Rank 6 (as seen by obserer) Video clip
- 92. Filmed with High-speed Camera Directional Backlight Video clip Front LCD
- 93. Video clip Lenslets only LCD+DBL Rank 1 LCD+DBL Rank 6
- 94. Limitation: Compressible Light Fields “Natural” 4D Light Field Random 4D Light Field
- 95. Next-generation Technology What about Content? Computational Photography Consumer Light Field Cameras Rendered Footage Computational Displays Camera Rigs
- 96. SIGGRAPH 2012 Course on Computational Displays Code & Datasets online Use Layered 3D in your class! media.mit.edu/~gordonw cameraculture.media.mit.edu
Notes de l'éditeur
- [Gordon resumes at this slide.]
- Multi-layer 3D display is a computed tomography problem. (Multi-Layer is 3D)Natural light fields are compressible, so a thin display with few layers can achieve the illusion of a thick virtual scene. (thin display creates thick scenes)So this leads to a new type of computational/compressive display.[probably same slide as personal device]
- Basically, the tomographic light field synthesis then boils down to solving a linear equation system of the form Ax=b. b is the target light field, A the projection matrix, and the unknowns x are the LCD pixel values
- This equation system can be solved with SART – a technique developed, tested, and refined over decades in the medical imaging community.Algorithm is very simple:Initialize some data and find an initial guess of the pixel valuesIteratively update and clamp the solutionUpdate rules require two important operations: a function computing the matrix-vector multiplication Ax, and a function computing the transpose matrix-vector multiplication ATv
- Interpreting AX – is basically a multiview rendering step. In OpenGL just set the projection matrix to all different views of the light field and render the LCD layers textured with the given patterns.
- Interpreting AX – is basically a multiview rendering step. In OpenGL just set the projection matrix to all different views of the light field and render the LCD layers textured with the given patterns.
- Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as wellPossibly use static framesEmphasize