1. ➢Introduction: History and Physiology
 Display Taxonomy
 Multi-view Rendering using OpenGL/GLSL
 Designing Content for Glasses-free 3D Displays
 Emerging Technology
Stereo and 3D Displays

2. Monocular Depth Cues Supported by Conventional Displays
 relative and familiar size
 perspective and occlusion
 texture gradient, shading and lighting, atmospheric effects
Limitations of Conventional Displays

3. Limitations of Conventional DisplaysLimitations of Conventional Displays
Monocular Depth Cues with Conventional Displays
 relative and familiar size
 perspective and occlusion
 texture gradient, shading and lighting, atmospheric effects

4. Additional Monocular Depth Cues
 motion parallax [Hermann von Helmholtz, 1866]
 accommodation
What is missing?

5. Binocular Depth Cues
 retinal disparity [Charles Wheatstone, 1838]
 convergence
“It being thus established that the mind perceives an object of three dimensions by means of the
two dissimilar pictures projected by it on the two retinae, the following question occurs: What
would be the visual effect of simultaneously presenting to each eye, instead of the object itself, its
projection on a plane surface as it appears to that eye?”
Binocular Depth Cues

6. American Civil War-era stereoscopic photos
• Available from the US library of congress
•http://www.loc.gov/pictures/search - Search for “stereographs civil war prints”
• Lincoln in 3D
• Selection of stereographs converted to red-
cyan anaglyph images
•John J. Richter: ISBN 978-0811872317
Interesting Historical Example

7. l
The HVS can ignore conflicting or missing depth cues
l
Understand depth in 2D (monocular) video
l
Perceive shape in “noise”
Ponzo Illusion: © Walt Anthony 2006magiceye.com
Conflicting Cues

8. Stereo and 3D Displays
 Introduction: History and Physiology
➢Display Taxonomy
 Multi-view Rendering using OpenGL/GLSL
 Designing Content for Glasses-free 3D Displays
 Emerging Technology

9. Taxonomy of 3D Displays:
Glasses-bound vs. Unencumbered Designs
Glasses-bound
Stereoscopic
Immersive
(blocks direct-viewing of real world)
See-through
(superimposes synthetic images onto real world)
Head-mounted
(eyepiece-objective and microdisplay)
Multiplexed
(stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)
(color filters, polarizers, autostereograms, etc.)
Temporally-multiplexed (field-sequential)
(LCD shutter glasses)
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
(time-sequential projection onto swept surfaces)
Transparent Substrates
(intersecting laser beams, fog layers, etc.)
Static
(holographic films)
Dynamic
(holovideo)
Taxonomy adapted from Hong Hua

10. Taxonomy of 3D Displays:
Immersive Head-mounted Displays (HMDs)
Glasses-bound
Stereoscopic
Immersive
(blocks direct-viewing of real world)
Head-mounted
(eyepiece-objective and microdisplay)
Multiplexed
(stereo pair with same display surface)

11. Taxonomy of 3D Displays:
See-through Head-mounted Displays (HMDs)
Glasses-bound
Stereoscopic
Immersive
(blocks direct-viewing of real world)
See-through
(superimposes synthetic images onto real world)
Head-mounted
(eyepiece-objective and microdisplay)
Multiplexed
(stereo pair with same display surface)

12. Taxonomy of 3D Displays:
Spatial Multiplexing (e.g., Anaglyphs)
Glasses-bound
Stereoscopic
Immersive
(blocks direct-viewing of real world)
See-through
(superimposes synthetic images onto real world)
Head-mounted
(eyepiece-objective and microdisplay)
Multiplexed
(stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)
(color filters, polarizers, etc.)

13. Taxonomy of 3D Displays:
Temporal Multiplexing (e.g., Shutter Glasses)
Glasses-bound
Stereoscopic
Immersive
(blocks direct-viewing of real world)
See-through
(superimposes synthetic images onto real world)
Head-mounted
(eyepiece-objective and microdisplay)
Multiplexed
(stereo pair with same display surface)
Spatially-multiplexed (field-concurrent)
(color filters, polarizers, autostereograms, etc.)
Temporally-multiplexed (field-sequential)
(LCD shutter glasses)

14. Taxonomy of 3D Displays:
Parallax Barriers
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
NewSight MV-42AD3 42''
(1920x1080, 1x8 views)

15. Taxonomy of 3D Displays:
Integral Imaging
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Alioscopy 3DHD 42''
(1920x1200, 1x8 views)

16. Taxonomy of 3D Displays:
Multi-planar Volumetric Displays
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
(time-sequential projection onto swept surfaces)

17. Taxonomy of 3D Displays:
Transparent-substrate Volumetric Displays
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
(time-sequential projection onto swept surfaces)
Transparent Substrates
(intersecting laser beams, fog layers, etc.)

18. Taxonomy of 3D Displays:
Static Holograms
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
(time-sequential projection onto swept surfaces)
Transparent Substrates
(intersecting laser beams, fog layers, etc.)
Static
(holographic films)
capture reconstruction

19. Taxonomy of 3D Displays:
Dynamic Holograms (Holovideo)
Unencumbered
Automultiscopic
Parallax-based
(2D display with light-directing elements)
Volumetric
(directly illuminate points within a volume)
Holographic
(reconstructs wavefront using 2D element)
Parallax Barriers
(uniform array of 1D slits or 2D pinhole arrays)
Integral Imaging
(lenticular sheets or fly’s eye lenslet arrays)
Multi-planar
(time-sequential projection onto swept surfaces)
Transparent Substrates
(intersecting laser beams, fog layers, etc.)
Static
(holographic films)
Dynamic
(holovideo)
Tay et al.
[Nature, 2008]
MIT Media Lab Spatial Imaging Group
[Holovideo, 1989 – present]

20. Stereo and 3D Displays
 Introduction: History and Physiology
 Display Taxonomy
➢Multi-view Rendering using OpenGL/GLSL
 Designing Content for Glasses-free 3D Displays
 Emerging Technology

21. Overview:
GLSL: Programmable Pipeline
Fixed Function Pipeline
Simple 1-Slide Explanation!
Drawing APIDrawing API
Process VerticesProcess Vertices
Process PixelsProcess Pixels
FramebufferFramebuffer
Programmable Pipeline
Vertex ProgramVertex Program
Fragment ProgramFragment Program

22. l
Some graphics cards have support for stereo 3D (Not on mobile)
l
Double buffered stereo = Quad buffered
void
display(void)
{
glDrawBuffer(GL_BACK_LEFT);
<Draw left eye here>
glDrawBuffer(GL_BACK_RIGHT);
<Draw right eye here>
glutSwapBuffers();
}
int
main(int argc, char **argv)
{
glutInit(&argc, argv);
glutInitDisplayMode(
GLUT_DOUBLE | GLUT_RGB | GLUT_STEREO);
glutCreateWindow("stereo example");
glutDisplayFunc(display);
glutMainLoop();
return 0;
}
Anaglyphic Model Viewer:
Stereo 3D in OpenGL

23. Overview:
Multi-View Rendering in OpenGL
OpenGL
Draw Calls
Render
Standard Pipeline
Output
Multi-View Pipeline
Loop Over
Views
Backbuffer
Framebuffer
Object Array
Render View
Change
Camera
Screen:Memory:

24. Overview:
Multi-View Interlacing using GLSL Shaders
Framebuffer
Object
Array
Framebuffer
Object
Array
View 1
View 2
View 3
GLSL Program
Translate views
appropriately for
output device
Translate views
appropriately for
output device
BackbufferBackbuffer
Anaglyph
Glasses
Anaglyph
Glasses
LenticularLenticular
Shown in this course…
The model can apply to
many others

25. Multi-View Rendering in OpenGL:
Off-Axis Perspective Projection with glFrustum()
Output

26. Anaglyphic Model Viewer:
Anaglyph Compositing Algorithms
LL RR
3x3 Color Transform Matrix Pair3x3 Color Transform Matrix Pair
Full Color
Half Color
Optimized
L= R=
1 0 0
0 0 0
0 0 0
0 0 0
0 1 0
0 0 1
L= R=
0.299
0 0
0.587
0 0
0.114
0 0
0 0 0
0 1 0
0 0 1
L= R=
0 0 0
0.7 0 0
0.3 0 0
0 0 0
0 1 0
0 0 1
=
Source: http://3dtv.at/Knowhow/AnaglyphComparison_en.aspx

27. % read in images
ImL = imread('l.png');
ImR = imread('r.png');
% define "half color" matrices (see slides)
L = [.299 0 0
.587 0 0
.114 0 0];
R = [0 0 0
0 1 0
0 0 1];
% create a pixel x color array
ImL1d = double(reshape(ImL,prod(size(ImL(:,:,1))),3));
ImR1d = double(reshape(ImR,prod(size(ImR(:,:,1))),3));
% perform per pixel color permutation
ImL1d = ImL1d*L;
ImR1d = ImR1d*R;
Anaglyphic Model Viewer:
Making an Anaglyph Image in MATLAB
% convert back to 2d x color image
ImL = uint8(reshape(ImL1d,size(ImL)));
ImR = uint8(reshape(ImR1d,size(ImR)));
% create output
Iout = ImL + ImR;
anaglyph.m

28. Anaglyphic Model Viewer:
Demonstration

29. Stereo and 3D Displays
 Introduction: History and Physiology
 Display Taxonomy
 Multi-view Rendering using OpenGL/GLSL
➢Designing Content for Glasses-free 3D Displays
 Emerging Technology

30. • Stereo cameras (commercial and
improvised) are common
Source Data
Stereo Cameras

31. • Many researchers/hobbyists have built their own solutions to
capture light fields
• The PointGrey ProFusion is one of the few commercially available
PointGrey ProFusion
Stanford
Source Data
Light Field Cameras
MIT

32. Focal Plane
Example in Anaglyph Viewer
Screen
Virtual
Object
Placing objects farm from the
plane of focus is uncomfortable
Displays with limited DOF: objects further from screen plane are blurred
Rendering Tips
Accommodation & Object Placement

33. Focal Plane
Screen
Kirshnan, V. V., Stark, L. A heuristic model for the human vergence eye movement system, IEEE Trans. BioMed, 1977.
Limit distance of virtual object to viewer
Limit rate of change in scene distance
<1 m/s for
distant objects
Rendering Tips
Comfortable Vergence

34. Off-axis parallel projection Rotate and translate – Toe-in
• Puts ‘infinity’ at axis of rotation
•Requires user to focus beyond infinity
•Some find diverged eyes uncomfortable
Disadvantages of toe-in • Distortion between views
•Camera distance to most objects change
•Off axis objects will have different perspective
projection
Rendering Tips
Camera Model Choice

35. Avoid cases that cause a view to differ greatly from its neighbor
Left Right
Pillar pointing at viewer
Left Right
Clipped by edge of screen
Also watch out for
• Far objects clipped by near object
• Edges of hallways, tunnels, tubes,
etc
Also watch out for
• Don’t exit in front of screen plane
• More comfortable behind screen
• Avatar does a good job with this
Rendering Tips
Clipping and Degenerate Cases

36. Warzone 2100: GL Game Conversion

37. Stereo and 3D Displays
 Introduction: History and Physiology
 Display Taxonomy
 Multi-view Rendering using OpenGL/GLSL
 Designing Content for Glasses-free 3D Displays
➢Emerging Technology

38. Tensor Display

39. Tensor Display

40. Stereo and 3D Displays Resources
SIGGRAPH 2010/2011 Course: BYO3D
http://web.media.mit.edu/~mhirsch/byo3d/index.html
 Long-form slides
 Code and examples
display
blocks
Display Blocks blog
http://displayblocks.org
 Tutorials
 Building blocks explained
Gordon Wetzstein
gordonw@media.mit.edu
Matt Hirsch
mhirsch@media.mit.edu