This document discusses facial recognition and 3D face modeling techniques used by computers. It begins by asking how computers recognize, see, detect, and identify human faces. It then provides examples of facial recognition APIs and discusses approaches like feature-based and holistic analysis. Common algorithms mentioned include EigenFace, FisherFace, neural networks and deep learning. The document also covers 3D face modeling methods like 3D scanners, constructing models from photos using morphable models, and potential future applications.

1. Facial Recognition
and
3D Face Modeling
by
Tanasai Sucontphunt
interactive media science @ nida
http://interactivemedia.nida.ac.th/

2. How a computer recognizes a human face?
How a computer sees a human face?
How a computer detects a human face?
How a computer identifies
a human identity?
http://chenlab.ece.cornell.edu/

3. How a computer sees a human face?
3D Surface?
http://gl.ict.usc.edu/Research/FaceScanning/images/EGSR2007_SGI_faces.png
Temperature?
http://matlab-recognition-code.com/infrared-face-recognition-system/
Image?
Input Choices
Most Favorite!

4. How a computer detects a human face?
API choices
Lambda Labs Face Recognition
Animetrics Face Recognition
Skybiometry Face Detection and Recognition
ImageVision Face Detection
Face++
FaceMark
EmoVu by Eyeris
Face and scene recognition by Rekognition.com
FaceRect
Infatics Face Detection
OpenCV Face Recognizer
Libface
Automatic naming of characters in video
CCV
OpenBR
Flandmark
Computer Expression Recognition Toolbox (CERT)
Nviso 3D facial imaging technology
FaceReader
Affdex
Faceware
Face detection in iOS
Face-Recognition-SDK
FaceSDK
FaceIT SDK
Stupeflix
Face.com
Microsoft Research Face SDK
etc.
http://cdn0.dailydot.com/cache/fc/3a/fc3aa93d4cf68ff5406f291efcd81b08.jpg
Detection + Landmark Extraction
http://www.wired.com/2012/11/auto-facial-recognition/
Face Tracking
Post Estimation
Facial Feature Tracking
Emotion Recognition

5. How a computer identifies a human identity?
Usage Choices
Feature-Based Holistic
for image = all the pixels
http://www.ait.gr/ait_web_site/faculty/apne/Images/FacePreproc.jpg
e.g. facial landmarks
https://raw.githubusercontent.com/ci2cv/face-analysis-sdk/stable/doc/avatar-annotation.png
Both?

6. How a computer identifies a human identity?
Algorithm or Classifier choices
EigenFace (PCA)
FisherFace (FisherLDA)
Support Vector Machine
Neural Network
Deep Neural Network
Convolution Neural Network
etc.
http://chenlab.ece.cornell.edu/projects/KinshipVerification/images/kinship.png

7. EigenFace (Classic)
Face recognition using eigenfaces, Matthew Turk and Alex Pentland in CVPR 1991
http://mikedusenberry.com/on-eigenfaces/
Face Dataset EigenVectors
Principle
Component
Analysis
(PCA)

8. EigenFace (Classic)
Face recognition using eigenfaces, Matthew Turk and Alex Pentland in CVPR 1991
http://mikedusenberry.com/on-eigenfaces/
EigenVectors
A Face
Projection
0.4 0.7 0.5
Score (PCA Coefficient)
0.1 0.6 0.3
0.40.5 0.7
Same Face = Same Score
Similar Face = Similar Score

9. EigenFace (Classic)
https://www.youtube.com/watch?v=N5GoaNPWXFw
Analogy

10. Famous Issues
• Lighting
• Pose
• Facial Expression
• Deep Learning requires a huge training data set to
reach an acceptable accuracy (~98%)
http://www.nap.edu/read/11577/chapter/3
http://www.christian-behrenberg.de/img/portfolio/bscthesis

11. 3D Face Modeling

12. How can we model a 3D Face?
• Crafting manually by artist
• 3D Scanner
• Construct from a photograph?
by Stanislav Klabík
ICT, USC

13. 3D Face Scanner
Light Stage X
Structured Light

14. From Photograph
A Morphable Model for the Synthesis of 3D Faces [Blanz andVetter 99] (Classic)

15. Input Image
w1 * w2 * w3 * w4 * w5 *+ + + +
=Synthesized Image (Rendered)
Update
Scores
Calculate differences
Score (PCA Coef.)
3D Morphable Model

16. 3D Morphable Applications
Total Moving Face Reconstruction [S, Suwajanakorn, I. Kemelmacher-Shlizerman, S. M. Seitz, 2014]

17. 3D Morphable Applications

18. More to come…
• Your identity in characters in Games
• Cartoon face from photograph
• Celebrity face similarity
• โหวงเฮ้ง Games
• 3D Face Scanner with a single camera
• etc.