1. Robert Collins Robert Collins
CSE486, Penn State CSE486, Penn State
Generalized Stereo
Lecture 18: Key idea: Any two images showing an overlapping
Generalized Stereo: view of the world can be treated as a stereo pair...
Epipolar Geometry
... we just have to figure out how the
two views are related.
Some of the most “beautiful” math in vision concerns
describing how multiple views are related, geometrically.
Robert Collins Robert Collins
CSE486, Penn State
Recall: Epipolar Constraint CSE486, Penn State
Review : Simple Stereo System
P
Important Stereo Vision Concept:
z Left camera
y
Given a point in the left image, we don’t have to
z
search the whole right image for a corresponding point.
y Right camera
x
The “epipolar constraint” reduces the search space
to a one-dimensional line. Tx
Epipolar line
x
Same Y Coord!
baseline
depth
Equation relating
depth and disparity
disparity
Robert Collins Robert Collins
CSE486, Penn State
Review: Epipolar Constraint CSE486, Penn State
General Stereo
C2 y2
x1 y1 C1
z1 z2
O2
O1
x2
In general, the cameras may be related by
an arbitrary transformation (R,T)
Epipolar Matrix
Corresponding features are constrained to
lie along conjugate epipolar lines (on the In general, intrinsic camera parameters
may be different, and even unknown
same row in the case of our simple setup).
Fundamental Matrix
1

2. Robert Collins Robert Collins
CSE486, Penn State Epipolar Geometry
CSE486, Penn State
P
pl pr
el er Or
Ol
EPIPOLAR Epipoles:
Epipoles:
GEOMETRY • el: left image of Or
• er: right image of Ol
Epipolar plane:
• Three points: Ol,Or, and P define an epipolar plane
Epipolar lines and epipolar constraint:
• Intersections of epipolar plane with the image planes
• Corresponding points are on “conjugate” epipolar lines
conjugate”
The following slides are from Dr.Camps, PSU
Robert Collins Robert Collins
CSE486, Penn State CSE486, Penn State
BORING!!! EPIPOLAR
Let’s try again...
GEOMETRY
Robert Collins
CSE486, Penn State Epipolar Geometry Robert Collins
CSE486, Penn State
Rays to Points in Scene
A Visualization
Tie threads on
to the pins
Would would Pinhead’s
and connect
eye look like close up?
focal point
to scene
points
Now what would this look like to a second observer?
2

3. Robert Collins Robert Collins
Rays Seen from Second Observer
CSE486, Penn State CSE486, Penn State
Rays Seen by the First Viewer
epipolar lines epipolar lines
Q u i
c k T i
m e ™ a n d a
T I F F ( L Z W ) d e c o m p r e s s o r
a r e n e e d e d t o s e e t h i
s p i
c t u r e .
epipole epipole
Image 2 Image 1
Robert Collins Robert Collins
CSE486, Penn State
Epipolar Geometry CSE486, Penn State
Epipolar Geometry
image1 image 2 image1 image 2
Epipole : location of cam2 Epipole : location of cam1 Corresponding points
as seen by cam1. as seen by cam2. lie on conjugate epipolar lines
Robert Collins Robert Collins
CSE486, Penn State
Epipolar Geometry CSE486, Penn State
Epipole not Necessarily in Image
image1 image 2
epipolar lines
Conjugate epipolar lines induce a
T
r
Q
I
e
F
u
F
n
i
c
e
k
(
e
T
L
d
i
m
Z
e
W
d
e ™
)
t o
d
a
e
s
n
c
e
d
o
e
m
a
t
p
h
r
i
s
e s
p
s
i
c
o
t
r
u r e .
a generalized 1D “scan-line” ordering epipole
on the images (analogous to traditional
scan line ordering of rows in an image)
Image 2
3

4. Robert Collins Robert Collins
Epipolar Geometry
CSE486, Penn State
Epipolar Constraint:
CSE486, Penn State
P P
pl pr pl pr
el er Or el er Or
Ol Ol
Epipoles:
Epipoles: Epipoles:
Given Epipoles:
• el: left image of Or • el: left image of Or
• er: right image of Ol • er: right image of Ol
Epipolar plane: Given pl:
• Three points: Ol,Or, and P define an epipolar plane •consider its epipolar line: pl el
Epipolar lines and epipolar constraint: •find epipolar plane: Ol,pl,el
• Intersections of epipolar plane with the image planes •intersect the epipolar plane with the right image plane
• Corresponding points are on “conjugate” epipolar lines
conjugate” •search for pr on the right epipolar line
The following slides are from Dr.Camps, PSU
Robert Collins Robert Collins
CSE486, Penn State
Essential Matrix CSE486, Penn State
Essential Matrix
P P
Pl Pr Pl Pr
pl pr pl pr
el er Or el er Or
Ol Ol
T T
Does this look familiar? Recall world to camera
transformation by (R,T). Here, we are transforming
from camera to camera.
Robert Collins Robert Collins
CSE486, Penn State
Essential Matrix CSE486, Penn State
Essential Matrix
Epipolar constraint: Pl, T and Pl - T are coplanar: Epipolar constraint: Pl, T and Pl - T are coplanar:
P P
Pl Pr Pl Pr
pl pr pl pr
el er Or el er Or
Ol Ol
T T
4

5. Robert Collins
Vector Product as a Matrix Robert Collins
CSE486, Penn State
Multiplication
CSE486, Penn State
Essential Matrix
Epipolar constraint: Pl, T and Pl - T are coplanar:
P
Pl Pr
pl pr
el er Or
Ol
T
S has rank 2 ; it depends only on T
Robert Collins Robert Collins
CSE486, Penn State
Essential Matrix CSE486, Penn State
Essential Matrix Properties
Epipolar constraint: Pl, T and Pl - T are coplanar:
P
Pl Pr
pl pr • has rank 2
el er Or • depends only on the EXTRINSIC Parameters (R & T)
Ol
T
Essential Matrix: We will discuss more of the wonderful
properties of this matrix next time...
5