7. A decade ago...
A Theory of Space by Space Carving
[ Kutulakos and Seitz, 1999 ]
Reconstructions w/ and w/o color
8. Volumetric graph cuts
by Vogiatzis, Torr, and Cipolla
CVPR 2005
1. This is called Haniwa.
2. This is Roberto Cipolla’s Haniwa.
This will be a final quiz!
13. ComputerVision in Industry
• Amazon
• Apple
• Facebook
• Google
• Microsoft
• Nokia
Mountain View
Seattle
Zurich
NYC
LA
…
14. ComputerVision in Industry
• Amazon
• Apple
• Facebook
• Google
• Microsoft
• Nokia
Mountain View
Seattle
Zurich
NYC
LA
…
15. Google Seattle - 3DVision Team
• Bundling streetview data (by Sameer Agarwal)
• Photo Tours (by the team)
• Picasa face movie (by Ira Kemelmacher-
Shlizerman and Rahul Garg)
• Lens Blur (by Carlos Hernandez)
Steve Seitz, Sameer Agarwal, Carlos Hernandez,
David Gallup, Changchang Wu, Li Zhang
47. Standard Depthmap
Markov Random Field (MRF)
A Comparative Study of Energy Minimization Methods for Markov Random Fields with
Smoothness-Based Priors [Szeliski et al., PAMI 2008]
48. A Comparative Study of Energy Minimization Methods for Markov Random Fields with
Smoothness-Based Priors [Szeliski et al., PAMI 2008]
Markov Random Field (MRF)
62. Piecewise planarity from MRF
1. Advanced MRF and optimization
• Global Stereo Reconstruction under Second Order Smoothness Priors [Woodford et al., CVPR
2008] Best Paper Award
2. Integrate with top-down (primitive) approach
• Manhattan World Stereo [Furukawa et al., CVPR 2009]
• Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
• Piecewise Planar and Non-Planar Stereo for urban Scene Reconstruction [Gallup et al., CVPR
2010]
63. Advanced MRF for
Depthmap Estimation
Global Stereo Reconstruction under Second Order Smoothness Priors
[Woodford et al., CVPR 2008] Best Paper Award
Reference image
Ground truth
79. How to enforce piecewise planar
1. Advanced MRF and optimization
• Global Stereo Reconstruction under Second Order Smoothness Priors [Woodford et al., CVPR
2008] Best Paper Award
2. Integrate with top-down (primitive) approach
• Manhattan World Stereo [Furukawa et al., CVPR 2009]
• Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
• Piecewise Planar and Non-Planar Stereo for urban Scene Reconstruction [Gallup et al., CVPR
2010]
96. How to enforce piecewise planar
1. Advanced MRF and optimization
• Global Stereo Reconstruction under Second Order Smoothness Priors [Woodford et al., CVPR
2008] Best Paper Award
2. Integrate with top-down (primitive) approach
• Manhattan World Stereo [Furukawa et al., CVPR 2009]
• Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
• Piecewise Planar and Non-Planar Stereo for urban Scene Reconstruction [Gallup et al., CVPR
2010]
98. How to enforce piecewise planar
1. Advanced MRF and optimization
• Global Stereo Reconstruction under Second Order Smoothness Priors [Woodford et al., CVPR
2008] Best Paper Award
2. Integrate with top-down (primitive) approach
• Manhattan World Stereo [Furukawa et al., CVPR 2009]
• Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
• Piecewise Planar and Non-Planar Stereo for urban Scene Reconstruction [Gallup et al., CVPR
2010]
99. Relaxing Mahnattan
Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Use sparse lines + sparse points to detect planes
• MRF + Graph-cuts
102. Enable Curved Surfaces
• Building reconstruction from a top down view
Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
104. How to enforce piecewise planar
1. Advanced MRF and optimization
• Global Stereo Reconstruction under Second Order Smoothness Priors [Woodford et al., CVPR
2008] Best Paper Award
2. Integrate with top-down (primitive) approach
• Manhattan World Stereo [Furukawa et al., CVPR 2009]
• Piecewise Planar Stereo for Image-based rendering [Sinha et al., ICCV 2009]
• Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial
Imagery [Zebedin et al., ECCV 2008]
• Piecewise Planar and Non-Planar Stereo for urban Scene Reconstruction [Gallup et al., CVPR
2010]
105. Piecewise
Planar
and
Non-‐Planar
Stereo
for
Urban
Scene
Reconstruction
David
Gallup
Jan-‐Michael
Frahm
Marc
Pollefeys
University
of
North
Carolina
ETH
Zurich
115. Summary: Shape priors
in Depthmap MVS
MRF with a triple term
Manhattan planemap Planemap
Planemap w/ curved surfaces
Mix of planemap
and depthmap
116. Outline
• Why priors?
• Prior enforcement through MRF
• Prior enforcement through primitives
• Prior enforcement through shortest-path
117. Priors through primitives for
large-scale indoor modeling
“Reconstructing the World’s Museums” [Xiao and Furukawa, 2012]
(Best Student Paper Award)
157. {# of museums} <<
{# of restaurants} +
{# of grocery stores} +
{# of clinic} + ...
158. Millions of small/medium-
scale businesses
• Image-based for scalable deployment
(especially for emerging markets)
• Compact model
(for browser on low-end PCs)
159. Millions of small/medium-
scale businesses
• Image-based for scalable deployment
(especially for emerging markets)
• Compact mesh model
(for browser on low-end PCs)
177. Features
• Regularize on {# of line segments}
• Piecewise planarity enforcement
• With Dynamic Programming,
exactly control the number of vertices
191. A. Who owns the Haniwa in the volumetric graph-cuts paper?
1. Yasutaka Furukawa.
2. Roberto Cipolla.
3. British Museum.
B. What is Uncanny valley for 3D reconstruction?
1. As the number of authors in a paper goes up, the model quality goes down.
2. As the face realism goes up, the model looks creepier.
3. As the model accuracy goes up, the rendering quality goes down.
C. How can one enforce piecewise planarity with a standard MRF formulation easily?
1. Use super-pixel segmentation.
2. Use primitive detection.
3. Use object recognition.