Convolutional neural networks (CNN) have improved the state of the art in many applications, especially the face recognition area. In this work, we present a review on latest face verification techniques based on Convolutional Neural Networks. In addition, we give a comparison on these techniques regarding their architecture, depth level, number of parameters in the network, and the obtained accuracy in identification and/or verification. Furthermore, as the availability of large-scale training dataset has significantly affected the performance of CNN based recognition methods, we present a preface to the most common large-scale face datasets, and then we describe some of the successful automatic data collection procedures.

1. Face Recognition Methods
based on
Convolutional Neural Networks
Presenter:
Elaheh Rashedi
Advisor:
Prof. Xuewen Chen
Winter 2016
1

2. Contents
■ Background
– Neural network
– Convolutional neural network
– General CNN-based face recognition schema
■ Face recognition models based on CNN
– DeepFace model
– Web-scaled DeepFace model
– DeepID model series
– FaceNet model
– VGG model
– Lightened CNN Model
■ CNN training and testing dataset
– CASIA-WebFace, MegaFace, IJB-A, VGG, …
■ Summary
■ Future Work
2

3. Neural Network
Fig 1. (a) General structure of a Neural Network, (b) an example of a neuron
3
(a) (b)

4. Convolutional Neural Networks
■ CNN
– A kind of neural network where the input is image
– Contains less fully connectivity between neurons
■ CNN layers
– Input layer
– Convolutional layer
– Pooling layer
– Fully connected layer
– Loss layer
4

5. Convolutional Neural Networks (cont. …)
Fig 2. General CNN structure in face recognition problems
5

6. General CNN-based face recognition schema
■ Common steps:
– Face detection
 Viola-Jones, Cascade CNN, …
– Pre-processing
 Geometric & lighting normalization
– CNN training
 Supervised vs. unsupervised
– Face identification
 Classification problem
– Metric learning
 Joint-Bayesian, Cosine similarity, Triplet Similarity, Energy-based similarity, …
– Face Verification
■ There are various ways to perform each step!
6

7. 7
Fig3.Generalfacerecognitionpipeline

8. Face recognition models based on CNN
■ CNN based models are different
– Architecture of CNN
– Depth of neural network
– Number of parameters
– Scale of training dataset
– Similarity metric
– Alignment vs. non-alignment pre-processing
– …
8

9. DeepFace Model
■ First CNN-based face recognition method (2014)
– By Facebook research group
■ Includes 4 main steps
– Detection
– 3D Alignment
– Feature representation
– Classification
■ Similarity metric learning
– Siamese energy based neural network
9

10. DeepFace Model (cont. …)
10
Fig 4. DeepFace [1]
Fig 5. Siamese network [2]

11. Web-scaled DeepFace Model
■ Based on DeepFace (2015)
– Higher identification acc.
– Lower verification acc.
– Lower feature vector dimension
■ Applies bootstrapping on large training dataset
– Select harder recognition cases
– Ignore easy recognition cases
■ Claim
– High dimensional feature vectors do not necessarily result in better accuracy!
11

12. DeepID model series
■ Inspired by DeepFace
■ Model Series
– DeepID (2014)
– DeepID2 (2014)
– DeepID2+ (2015)
– DeepID3 (2015)
12

13. DeepID model
■ CNN structure
– 4 convolutional layers
– 3 max-pooling layers
– 1 fully connected layer
■ Alignment
– Center of two eyes, two corners of mouth, nose tip.
■ Multiple patches
– Extract different features for different part of face
■ Similarity metric learning
– Joint-Bayesian
13

14. DeepID2 Model
14
Fig 6. DeepID2 [3]
Fig 7. DeepID [4]

15. DeepID2+ Model
■ Based on DeepID2
– More deep network
– Uses supervisory signals
– Uses fully connected layers
■ Fully connected layers
– Early feature extraction
■ Claim
– Deep CNN based networks are more
robust to corruption of image!
15Fig 8. DeepID2+ [5]

16. DeepID3 Model
■ DeepID3
– More Deep than DeepID2+
– Less deep than FaceNet & VGG
■ Includes
– Very deep neural networks
– 15 feature extraction layers
– Early fully connected layers
■ Similarity metric learning
– Joint-Bayesian
16Fig 9. DeepID3 [6]

17. FaceNet Model
■ FaceNet model: by Google research group
– Same framework for identification and verification
– Very deep network
– No alignment
– Efficient representation of features
■ Similarity metric learning
– Triplet loss
17Fig 10. Triplet loss learning [7]

18. VGG Model
■ VGG model: by Visual Geometry Group
– Inspired by the very deep FaceNet network
– Very deep CNN
– 36 level of feature extraction
■ Similarity metric
– Triplet loss
■ Contributions
– Automatic collection of large face dataset
– Publically available pre-trained CNN model
18

19. Lightened CNN Model
■ Shallow network
– 4 convolutional layers
– 4M parameters
■ Less computational intensive
– 9 times less than VGG model
19

20. CNN Training and Testing Datasets
20

21. Summary
■ The size of training dataset
■ Alignment
■ Joint-Bayesian similarity
■ Multiple patches
■ Verification performance on video
■ Identification performance on larger number of identities
21

22. Future Work
■ Using face verification to outperform face detection
■ Designing a similarity metric learning
■ Designing a face tracking method based on face verification
22

23. References
1. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in The IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), June 2014.
2. S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in Computer Vision
and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, pp. 539–546, 2005.
3. Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identification verification,” in Advances in Neural
Information Processing Systems, pp. 1988–1996, 2014.
4. Y. Sun, X. Wang, and X. Tang, “Deep learning face representation from predicting 10,000 classes,” in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, pp. 1891–1898, 2014.
5. Y. Sun, D. Liang, X.Wang, and X. Tang, “Deepid3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, 2015.
6. Y. Sun, X.Wang, and X. Tang, “Deeply learned face representations are sparse, selective, and robust,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, pp. 2892–2900, 2015.
7. F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in The IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), June 2015.
23

24. Thank You!
24