2. What is Camera Calibration?
• Primarily, finding the quantities internal to
the camera that affect the imaging process
§ Position of image center in the image
• It is typically not at (width/2, height/2) of image
§ Focal length
§ Different scaling factors for row pixels
and column pixels
§ Skew factor
§ Lens distortion (pin-cushion effect)
3. Motivation
• Good calibration is important when we need
to
§ Reconstruct a world model: Virtual L.A. project
§ Interact with the world
• Robot, hand-eye coordination
Image plane Evaluation of position of a square for 2 focal
lengths (red and blue projection geometry)
We see a square of
known size
4. Scaling of Rows and Columns
in Image
• Camera pixels are not necessarily square
• Camera output may be analog (NTSC)
• Image may be obtained by digitizing card
§ A/D converter samples NTSC signal
Monitor
Camera NTSC signal Digitizing display
CCD/
CMOS
5. Compound Lens Imaging
• Inexpensive single lens systems distort image at
its periphery
• Compound lenses may be used to reduce
chromatic effects and pin-cushion effects
Camera
Image plane
Nodal Point
Principal planes
f
Image plane for equivalent
pinhole camera is f C
not camera image plane Center of Projection
6. Variety of Techniques
• VERY large literature on the subject
• Work of Roger Tsai influential
• Linear algebra method described here
§ Can be used as initialization for iterative non
linear methods.
• Some interesting methods use vanishing
points
8. Calibration Procedure
• Calibration target: 2 planes at right angle with
checkerboard patterns (Tsai grid)
§ We know positions of pattern corners only with
respect to a coordinate system of the target
§ We position camera in front of target and find
images of corners
§ We obtain equations that describe imaging and
contain internal parameters of camera
• As a side benefit, we find position and orientation of
camera with respect to target (camera pose)
9. Image Processing of
Image of Target
• Canny edge detection
• Straight line fitting to detected linked edges
• Intersecting the lines to obtain the image
corners
• Matching image corners and 3D target
checkerboard corners
§ By counting if whole target is visible in image
• We get pairs (image point)--(world point)
( xi , yi ) → ( X i , Yi , Z i )
10. Central Projection
If world and image points are represented by homogeneous
vectors, central projection is a linear transformation:
xs
xi = f u f 0
xs
0 0
zs v = 0 ys xi = u / w, yi = v / w
f 0 0
zs
ys w 0
0 1 0
1
yi = f Image plane Scene point
zs y Image point
(xs , ys , zs )
center of (xi , yi , f)
x
projection
z
C
f
11. Transformation From Lengths to Pixels
Transformation uses: y Image point
ypix
(xi , yi , f)
• image center (x0, y0)
y0 Image center x
• scaling factors kx and ky
x0 xpix
xs xs + zs x0
xi = f x pix = k x xi + x0 = f k x
zs zs
ys y s + z s y0
yi = f y pix = k y yi + y0 = f k y
zs zs
xs
u ' α x 0 x0 0 α x = f kx x pix = u ' / w'
v' = 0 α y0 ys
0
with then
y
zs α y = f ky y pix = v' / w'
w' 0
0 1 0
1
12. Internal Camera Parameters
xs
u ' α x s x0 0 α x = f kx x pix = u ' / w'
v' = 0 α y y0 0 ys
with
zs α y = − f ky y pix = v' / w'
w' 0
0 1 0
1
α x s x0 0 α x s x0 1 0 0 0
0 α y0 0 = 0 α y y0 0 1 0 0 = K [I 3 | 03 ]
y
0
0 1 0 0
0 1 0 0 1 0
• αx and αy “focal lengths” in pixels
• x0 and y0 coordinates of image center in pixels
•Added parameter s is skew parameter
• K is called calibration matrix. Five degrees of freedom.
•K is a 3x3 upper triangular matrix
13. From Camera Coordinates
y to World Coordinates
C x
Image plane
Image point Z
z
(xi , yi , f)
Y
(xs , ys , zs )
M O
(Xs , Ys , Z s ) X
14. From Camera Coordinates
to World Coordinates 2
x
X
i k z M
j (Xs , Ys , Z s ) in world cordinate system
C
I (xs , ys , zs ) in image cordinate system
CO = T translation vector J
O K Z
CM = CO + OM
xS i + y S j + zS k = Tx i + Ty j + Tz k + X S I + YS J + Z S K
xS = Tx + X S I.i + YS J.i + Z S K.i
xS Tx I.i J.iK.i X S
y = T + I.j J.j K.j YS
S y
z S Tz I.k
J.k K.k Z S
15. Homogeneous Coordinates
xS Tx I.i J.i K.i X S
y = T + I.j J.j K.j Y
S y S
z S Tz I.k J.k K.k Z S
xS I.i J.i Tx X S
K.i xS X S
y I.j y T YS
S = J.j Ty YS
K.j S = R
z S I.k J.k K.k Tz Z S z S 0 T
3
Z
1 S
1 0 0 0 1 1 1 1
16. From Camera Coordinates
to World Coordinates 3
x
X
i k z M
j (Xs , Ys , Z s )
C
I (xs , ys , zs )
J
O K Z
CM = OM - OC
xS i + yS j + z S k = ( X S - X C ) I + (YS - YC ) J + ( Z S - Z C ) K
xS = ( X S - X C ) I.i + (YS - YC ) J.i + ( Z S - Z C ) K.i
~ ~
x cam = R (X - C) (T = -R C)
~
C is vector OC expressed in world coordinate system
17. Homogeneous Coordinates 2
~
• Here we use - R C instead of T
xS XS
y ~
S = R - R C YS
z S 0 T 1 ZS
3
1 1
18. Linear Transformation from
World Coordinates to Pixels
• Combine camera projection and coordinate
transformation matrices into a single matrix P
xs
u' y
v ' = K [I | 0 3 ] s
3
zs XS
w'
u' ~
1 v' = K [I | 0 ] R
⇒
- R C YS
3 T
1 Z S
3
xS XS w' 0 3
y ~ 1
S = R - R C YS
XS
z S 0 T 1 ZS u' Y
⇒ v' = P S
3
x = PX
1 1 ZS
w'
1
19. Properties of Matrix P
• Further simplification of P:
R ~
- R C
x = K [I 3 | 0 3 ] T X
0 3 1
~
[I 3
R
| 0 3 ] T
- R C
[ ~
] [ ~
= R - R C = R I3 | -C ]
0 3 1
[ ~
x = K R I3 | -C X ]
[
~
P = K R I3 | - C ]
• P has 11 degrees of freedom: ~
• 5 from triangular calibration matrix K, 3 from R and 3 from C
• P is a fairly general 3 x 4 matrix
•left 3x3 submatrix KR is non-singular
20. Calibration
• 1. Estimate matrix P using scene points and
their images
• 2. Estimate the interior parameters and the
exterior parameters
[ ~
P = K R I3 | - C ]
§ Left 3x3 submatrix of P is product of upper-
triangular matrix and orthogonal matrix
21. Finding Camera Translation
• Find homogeneous coordinates of C in the
scene
• C is the null vector of matrix P
§ P C = 0: X c 0
1 0 0 X c
P = K R I3[ ~
| -C ] 0 1 0
Yc Yc = 0
Z c 0
0 0 1
Zc
1
0
• Find null vector C of P using SVD
§ C is the unit singular vector of P corresponding to the
smallest singular value (the last column of V, where P
= U D VT is the SVD of P)
22. Finding Camera Orientation and
Internal Parameters
• Left 3x3 submatrix M of P is of form M=K R
§ K is an upper triangular matrix
§ R is an orthogonal matrix
• Any non-singular square matrix M can be
decomposed into the product of an upper-
triangular matrix K and an orthogonal matrix R
using the RQ factorization
§ Similar to QR factorization but order of 2 matrices is
reversed
23. RQ Factorization of M
1 0 0 c' 0 s ' c' ' − s' ' 0
R x = 0 c − s , R y = 0 1 0 , R z = s ' ' c' ' 0
0 s c
− s ' 0 c'
0
0 1
m33 m32
• Compute c = − , s= 2
( m32 + m33 )
2 2 1/ 2
(m32 + m33 )1/ 2
2
• Multiply M by Rx. The resulting term at (3,2) is zero
because of the values selected for c and s
• Multiply the resulting matrix by Ry, after selecting c’ and
s’ so that the resulting term at position (3,1) is set to zero
• Multiply the resulting matrix by Rz, after selecting c’’ and
s’’ so that the resulting term at position (2,1) is set to zero
M RxR yR z = K ⇒ M = K R TRTRT = K R
z y x
24. Computing Matrix P
• Use corresponding image and scene points
§ 3D points Xi in world coordinate system
§ Images xi of Xi in image
• Write xi = P Xi for all i
• Similar problem to finding projectivity
matrix H (i.e. homography) in homework
25. Improved Computation of P
• xi = P Xi involves homogeneous
coordinates, thus xi and P Xi just have to be
proportional: x i × P Xi = 0
• Let p1T, p2T, p3T be the 3 row vectors of P
p 1 X i
T
v'i p T Xi − w'i p T Xi
3 2
T
P X i = p 2 X i xi × P X i = w'i p1 Xi − u 'i p 3 Xi
T T
p T X i u'i p T X i − v'i p1 X i
T
3 2
0T − w'i X T v 'i X T p 1 p1
4 i i
T
⇒ w 'i X Ti 0T4 − u'i X i p 2 = 0 p2 is a 12 × 1 vector
− v 'i X T u 'i X T 0T p 3 p3
i i 4
26. Improved Computation of P,
cont’d
• Third row can be obtained from sum of u’i
times first row - v’i times second row
0T 4 − w'i X T
i v 'i X T p 1
i
w'i X T i 0T4 − u 'i X T p 2 = 0
i
− v 'i X T u 'i X T 0T p 3
i i 4
• So we get 2 independent equations in 11
unknowns (ignoring scale)
• With 6 point correspondences, we get
enough equations to compute matrix P
Ap=0
27. Solving A p = 0
• Linear system A p = 0
• When possible, have at least 5 times as
many equations as unknowns (28 points)
• Minimize || A p || with the constraint
|| p || = 1
§ P is the unit singular vector of A corresponding
to the smallest singular value (the last column
of V, where A = U D V T is the SVD of A)
• Called Direct Linear Transformation (DLT)
28. Improving P Solution with
Nonlinear Minimization
• Find p using DLT
• Use as initialization for nonlinear
minimization of ∑ d ( x i , PX i ) 2
i
§ Use Levenberg-Marquardt iterative minimization
29. Radial Distortion
• We have assumed that lines are imaged as
lines
• Not quite true for real lenses
§ Significant error for cheap optics and for short
focal lengths
Scene Image Corrected image
30. Radial Distortion Modeling
• In pixel cordinates the correction is written
xc − x0 = L(r ) ( x − x0 )
(x, y) yc − y0 = L ( r ) ( y − y0 )
(xc, yc)
(x0, y0) with
Distorted line
r 2 = ( x − x0 ) 2 + ( y − y0 ) 2
And ideal line:
L(r ) = 1 + κ 1 r + κ 2 r 2 + ...
(x, y)
(x’, y’)
• Minimize f (κ 1 ,κ 2 ) = ∑ ( x'i − xci ) 2 + ( y 'i − yci ) 2
i
using lines known to be straight
(x’,y’) is radial projection of (x,y) on straight line
31. References
• Multiple View Geometry in Computer Vision, R.
Hartley and A. Zisserman, Cambridge University
Press, 2000, pp. 138-183
• Three-Dimensional Computer Vision: A
Geometric Approach, O. Faugeras, MIT Press,
1996, pp. 33-68
• “A Versatile Camera Calibration Technique for
3D Machine Vision”, R. Y. Tsai, IEEE J. Robotics
& Automation, RA-3, No. 4, August 1987, pp.
323-344
