1. Tracking faces using PCA
Stephen Amar & Xuyan Yan

2. Do we use markers ?
• NO !
• The whole point is to have something that do
not require any physical markers.

3. How is this working ?
• OpenCV implementation of Principal
Components Analysis
• 2 steps:
– Training
– Real-time tracking (webcam, movie …)

4. Training
• We have a training set:
– A bunch of pictures and a list of coordinates

5. Training
• We apply PCA on this train set.
– The coordinates are written in the last line of the
picture.
– Each picture is transformed into a giant vector
(128x128 pix => 16384 dimensions vector).
– The Average Image and the Covariance Matrix of
these pictures-vectors is computed .
– We find the eigenvectors and the eigenvalues of
the covariance matrix (OpenCV uses the Jacobi
eigenvalue algorithm but we could also use
SVD).

6. Real-time tracking
• Once we have some training data:
– Get a new image to track
– Decompose this new image using the
eigenvectors. From this operation, we get
[α1,…, αn] where n is the number of train pictures -
1.
– Now, we are going to compute:
Output  Avg   1 I 1  ...   1 I n
– Using the output, we just need to read the last
line to get our new coordinates.

7. Downsides
• A train set can work for only one face.
• Very sensitive to the lighting conditions.
• Very sensitive to the background.
• The “actor” must remain still.
• The OpenCV implementation is quite limited
(support only 8 bits grayscale pictures).

8. Possible Improvements
• Instead of using the OpenCV implementation,
do it ourselves.
– OpenCV offers all the basic functions
(Computation of the Covariance Matrix, SVD and
other Matrix multiplication algorithms)
– We could do this using 32 bits float pictures.