1. Edit this text to create aa 16:9 media
This slide has Heading
Computational Light Field window
Displays
 This subtitle is 20 points
 Bullets are blue
 They have 110% line spacing, 2 points before & after
 Longer bullets in the form of a paragraph are harder to
read if there is insufficient line spacing. This is the
maximum recommended number of lines per slide
(seven).
Douglas Lanman Matthew Hirsch
 Sub bulletsMIT Media Lab
look like this MIT Media Lab

2. Is “glasses-free 3D” ready?
Nintendo 3DS MasterImage 3D Asus Eee Pad MeMO 3D LG Optimus 3D
E3 2010 Computex 2011 Computex 2011 Mobile World Congress 2011
Toshiba 3DTV Prototype Sony 3DTV Prototype LG 3DTV Prototype
CES 2011 CES 2011 CES 2011

3. Taxonomy of Direct 3D Displays:
Glasses-bound vs. Unencumbered Designs
Immersive
(blocks direct-viewing of real world)
Head-mounted
(eyepiece-objective and microdisplay)
See-through
(superimposes synthetic images onto real world)
Glasses-bound
Stereoscopic
Spatially-multiplexed (field-concurrent)
(color filters, polarizers, autostereograms, etc.)
Multiplexed
(stereo pair with same display surface)
Temporally-multiplexed (field-sequential)
(LCD shutter glasses)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
Unencumbered (time-sequential projection onto swept surfaces)
Volumetric
(directly illuminate points within a volume) Transparent Substrates
Automultiscopic (intersecting laser beams, fog layers, etc.)
Static
(holographic films)
Holographic
(reconstructs wavefront using 2D element) Dynamic
(holovideo)
Taxonomy adapted from Hong Hua

4. Taxonomy of Direct 3D Displays:
Parallax Barriers
NewSight MV-42AD3 42''
(1920x1080, 1x8 views)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements)
Unencumbered Volumetric
(directly illuminate points within a volume)
Automultiscopic
Holographic
(reconstructs wavefront using 2D element)

5. Taxonomy of Direct 3D Displays:
Integral Imaging
Alioscopy 3DHD 42''
(1920x1200, 1x8 views)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Unencumbered Volumetric
(directly illuminate points within a volume)
Automultiscopic
Holographic
(reconstructs wavefront using 2D element)

6. Directional Backlighting
Nelson and Brott, 2010
US Patent 7,847,869
 Currently promoted by 3M
 Requires a high-speed (120 Hz) LCD panel, an additional double-sided prism film, and a pair of LEDs
 Allows multi-view display, but requires higher-speed LCD and additional light sources for each view

7. Taxonomy of Direct 3D Displays:
Multi-planar Volumetric Displays
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
Unencumbered (time-sequential projection onto swept surfaces)
Volumetric
(directly illuminate points within a volume)
Automultiscopic
Holographic
(reconstructs wavefront using 2D element)

8. Taxonomy of Direct 3D Displays:
Transparent-substrate Volumetric Displays
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
Unencumbered (time-sequential projection onto swept surfaces)
Volumetric
(directly illuminate points within a volume) Transparent Substrates
Automultiscopic (intersecting laser beams, fog layers, etc.)
Holographic
(reconstructs wavefront using 2D element)

9. Taxonomy of Direct 3D Displays:
Static Holograms
capture reconstruction
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
Unencumbered (time-sequential projection onto swept surfaces)
Volumetric
(directly illuminate points within a volume) Transparent Substrates
Automultiscopic (intersecting laser beams, fog layers, etc.)
Static
(holographic films)
Holographic
(reconstructs wavefront using 2D element)

10. Taxonomy of Direct 3D Displays:
Dynamic Holograms (Holovideo)
Tay et al. MIT Media Lab Spatial Imaging Group
[Nature, 2008] [Holovideo, 1989 – present]
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Parallax-based
(2D display with light-directing elements) Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
Unencumbered (time-sequential projection onto swept surfaces)
Volumetric
(directly illuminate points within a volume) Transparent Substrates
Automultiscopic (intersecting laser beams, fog layers, etc.)
Static
(holographic films)
Holographic
(reconstructs wavefront using 2D element) Dynamic
(holovideo)

11. What is meant by “glasses-free 3D”?
binocular disparity convergence motion parallax accommodation/blur
current glasses-based (stereoscopic) displays
near-term glasses-free (automultiscopic) displays
longer-term volumetric and holographic displays

12. Design Trade-offs
Integral Imaging Parallax Barriers Directional Backlighting
Integral Imaging Parallax Barriers Directional Backlighting
Spatial Resolution low low high
Brightness high low moderate
Cost low low – moderate moderate – high
Full-resolution 2D no yes (dual-layer LCD) yes
Motion Parallax yes yes no

13. Generalizing Parallax Barriers
mask K
…
mask 3
mask 2 mask 2 mask 2
mask 1 mask 1 mask 1
light box light box light box
Conventional Parallax Barrier High-Rank 3D (HR3D) Layered 3D and Polarization Fields
 Parallax barriers use heuristic design: front mask with slits/pinholes, rear mask with interlaced views
 High-Rank 3D (HR3D) considers dual-layer design with arbitrary opacity and temporal multiplexing
 Layered 3D and Polarization Fields considers multi-layer design without temporal multiplexing

14. Outline
 Automultiscopic Displays
 Multi-Layer Displays
 Layered 3D
- Polarization Fields
 Dual-Layer Displays
- High-Rank 3D (HR3D)

15. Layered 3D: Multi-Layer Displays
mask K
…
mask 3
mask 2
mask 1
light box
Layered 3D

16. Tomographic Light Field Synthesis
virtual plane
q attenuator
x
backlight
q
x
2D Light Field

17. Tomographic Light Field Synthesis
virtual plane
attenuator
x
backlight
q
x
2D Light Field

18. Tomographic Light Field Synthesis
virtual plane
attenuator
x
backlight
q
x
2D Light Field

19. Tomographic Light Field Synthesis
virtual plane
Image formation model:
ò
- m (r )dr
attenuator
L(x, q ) = I 0 e C
æ L(x, q ) ö
L(x, q ) = ln ç ÷ = - ò m (r)dr
è I0 ø C
backlight l = -Pa
Tomographic synthesis:
2
arg min l + Pa , for a ³ 0
a
2D Light Field

20. Tomographic Light Field Synthesis
virtual plane
Image formation model:
ò
- m (r )dr
attenuator
L(x, q ) = I 0 e C
æ L(x, q ) ö
L(x, q ) = ln ç ÷ = - ò m (r)dr
è I0 ø C
backlight l = -Pa
Tomographic synthesis:
2
arg min l + Pa , for a ³ 0
a
2D Light Field

21. Multi-Layer Light Field Decomposition
Reconstructed Views
Target 4D Light Field
Multi-Layer Decomposition

22. Prototype Layered 3D Display
Transparency stack with acrylic spacers Prototype in front of LCD (backlight source)

24. Outline
 Automultiscopic Displays
 Multi-Layer Displays
- Layered 3D
 Polarization Fields
 Dual-Layer Displays
- High-Rank 3D (HR3D)

25. Barco E-2320
PA
Grayscale IPS LCD
1600x1200 @ 60 Hz

26. Four Stacked Liquid Crystal Panels
Two Crossed Polarizers

27. Overview of LCDs
I
vertical polarizer
color filter array
liquid crystal cells
horizontal polarizer
I0
backlight
Malus’ Law Intensity Modulation with Liquid Crystal Cells
I = I0 sin2 (q )

28. Extending Layered 3D to Multi-Layer LCDs
Virtual Planes
Design Optimization
LCD 3 • Eliminate redundant polarizers
 Sequentially-crossed design
q
LCD 2 x
LCD 1
backlight
2D Light Field
q
x

29. Extending Layered 3D to Multi-Layer LCDs
Virtual Planes
Design Optimization
LCD 3 • Eliminate redundant polarizers
 Use sequentially-crossed
q
LCD 2 x • Exploit field-sequential color
 0.33 = 2.7% brightness
LCD 1
backlight
2D Light Field
q
x

30. Polarization Field Displays
Virtual Planes
Design Optimization
LCD 3 • Eliminate redundant polarizers
 Use sequentially-crossed
q
LCD 2 x • Exploit field-sequential color
 0.33 = 2.7% brightness
LCD 1 • Further optimize polarizers
 Minimum is a crossed pair
backlight
2D Light Field
q
x

31. Polarization Field Displays
Virtual Planes
LCD 3
f3
Image Formation
q f2 K
LCD 2 x Q(x, q ) = åfk (x, q )
k=1
f1 L(x, q ) = sin 2 (Q(x,q ))
LCD 1
backlight Tomographic Synthesis
2D Light Field
Q(x,q ) = ±sin-1 ( )
L(x, q ) mod p
Q = Pf
q argmin Q - Pf
2
2
fmin £f £fmax
x

32. Tomographic Image Synthesis
Projection Matrix
Target Light Field LCD Pixel Values
= *
b=Ax

33. SART [Simultaneous Algebraic Reconstruction Technique]
b=Ax
pre-compute some weights
initial guess
ATv Ax update
clamp

34. Efficient GPU Implementation
Ax ATv
Forward Projection (Multiview Rendering) Back Projection (Projective Texture Mapping)

35. Polarization Field Displaysviewer moves right
viewer moves down
Stacked Polarization
Input 4D Light Field Rotating Layers
90°
0°
Optimized Rotation Angles for Each Layer

36. Polarization Field Displaysviewer moves right
viewer moves down
Input 4D Light Field
Reconstruction Results

38. Outline
 Automultiscopic Displays
 Multi-Layer Displays
- Layered 3D
- Polarization Fields
 Dual-Layer Displays
 High-Rank 3D (HR3D)

39. Input 4D Light Field

40. Parallax Barrier: Front Layer

41. Parallax Barrier: Rear Layer

42. Analysis of Parallax Barriers
k
L[i,k]
i
k
g[k]
f[i]
i
L[i,k]
`
light box
L[i, k] = f [i]× g[k] L[i, k ]  f [i]  g[k ]

43. Analysis of Parallax Barriers
L[i,k]
k
g[k]
i
f[i] `
light box
T
Ken Perlin et al. An Autosteroscopic Display. 2000. L[i, k] = å ft [i] Ä gt [k]
Yunhee Kim et al. Electrically Movable Pinhole Arrays. 2007.
t=1

44. Content-Adaptive Parallax Barriers
L[i,k]
G
k
g[k]
i ~`
f[i] F L
light box
~
L  FG

45. Content-Adaptive Parallax Barriers
G
~`
L = F
arg min L - FG W , for F, G ³ 0
2
F,G

46. Content-Adaptive Parallax Barrier: Front Layer

47. Content-Adaptive Parallax Barrier: Rear Layer

48. Simulation Results

49. Prototype High-Rank 3D (HR3D) Display
http://cameraculture.media.mit.edu/byo3d
Matthew Hirsch and Douglas Lanman. Build Your Own 3D Display. SIGGRAPH 2010, SIGGRAPH Asia 2010, SIGGRAPH 2011.

50. Experimental Results
Time-Multiplexed Parallax Barrier High-Rank 3D (HR3D)

51. Outline
 Automultiscopic Displays
 Multi-Layer Displays
- Layered 3D
- Polarization Fields
 Dual-Layer Displays
- High-Rank 3D (HR3D)

52. High-Rank 3D (HR3D) Layered 3D Polarization Fields
www.hr3d.info www.layered3d.info tinyurl.com/polarization-fields
BiDi Screen Tensor Displays
www.bidiscreen.com tinyurl.com/tensordisplays