An Enhanced Independent Component-Based Human Facial Expression Recognition from Video

1. An Enhanced Independent
Component-Based Human
Facial Expression Recognition
from Video

2. By:-
Umme Rumaan 57
Ansari Rumana 63

3. Introduction
• Facial Expression Recognition (FER) from video is an
essential research area in the field of Human Computer
Interfaces(HCI).
• Nowadays, consumer video cameras have become inexpensive
and are being extensively used in many consumer devices such
as laptops, mobile phones, etc. Lately, these cameras are used
for the face related applications such as
[1]face detection,
[2]face recognition and
[3]facial expression recognition (FER).

4. • FER has been regarded as one of the fundamental technologies
for HCI, which enables computers to interrelate with humans
in a way to human to human interactions.
• For feature extraction from the facial expression video images,
most of the early FER research works extracted useful features
using Principal Component Analysis (PCA).
• PCA is a second-order statistical method to derive the
orthogonal bases containing the maximum variability in an
unsupervised manner that provides global image features. It is
also commonly used for dimension reduction.

5. Method
• Our proposed FER system consists of preprocessing of
sequential facial expression images in video,
• feature extraction via EICA-FLDA,
• codebook generation via vector quantization algorithm and
• modeling and recognition via HMM.

7. A. Preprocessing
• In preprocessing of sequential images of facial
expressions,first image alignment is performed to realign the
common regions of the face.
• A face alignment approach is utilized by manually matching
the eyes and mouth of the faces in the designated coordinates.
• The typical realigned image consists of 60 by 80 pixels.
• Histogram equalization is then performed on the realigned
images for lighting correction.
• Afterwards, the first frame of each input sequence is
subtracted from the following frames to obtain the delta
images to produce the facial expression change differences in
the images over time.

9. B. Feature Extraction
• The key idea of the feature extraction from a set of time
sequential facial expression images is the combination of
EICA and FLDA.
• The purpose of this method is to find an optimal local
representation of facial expression images in a low
dimensional space and to lead the well separated time
sequential features for robust recognition.
• It has following steps:
(1)PCA is performed first for dimension reduction,
(2) ICA is applied on the reduced PCA subspace to find statistically
independent basis images for corresponding facial expression image
representation,
(3) FLDA is then employed to compress the same classes as close as
possible and to separate the different classes as far as possible.

14. C. Vector Quantization
• To decode the temporal variations of the facial expression
features, we have employed discrete HMMs.

15. D. Modeling and Recognition via HMM
• While training the HMMs, the facial expression image
sequences are projected on the feature space and symbolized
through vector quantization.
• Thus, for each training facial expression image sequence,
corresponding observation sequence O={O1,O2,O3...,OT}is
obtained where T indicates the sequence length.
• The obtained observation symbol sequences are then utilized
to train the corresponding expression HMM such as anger-
HMM, joy-HMM, sad-HMM, disgust-HMM,fear-HMM, and
surprise-HMM.

17. EXPERIMENTAL SETUPS
• A set of comparison experiments were performed under the
same procedure.
• To report the recognition performance, we prepared the
training and testing video clips of variable length utilizing the
well-known Cohn-Kanade facial expression database.
• Thus, we tried to recognize six universal facial expressions
such as anger, joy, sad, disgust, fear, and surprise.

18. A. Facial Expression Database
• The facial expression database used in our experiments is the Cohn-
Kanade AU-coded facial expression database consisting of facial
expression sequences with a neutral expression as an origin to a target
facial expression.
• The frontal view of the face and each subset is composed of several
sequential frames of the specific expression.
• There are six universal expressions to be classified and recognized via
the proposed approach.
• B. Experiments
• Further the steps given in the methodology are applied on these
images using the facial expression DB.
• To evaluate the performance of the proposed system we applied a total
of 15 and 40 image sequences per expression for training and testing
each expression respectively.

20. Application
• FER is used in markets where there is a sale put up.
• It is used in online games as a Real-time application.

21. CONCLUSION
We have presented a video-based robust FER system using
EICA-FLDA for facial expression feature extraction and HMM for
recognition. We have illustrated the performance of our proposed
method applied on sequential datasets for the six facial expression
recognition problems. The experimental results show that EICA-
FLDA, the linear discriminant approach on IC feature vectors from
optimal representation of PCs, improves the feature extraction task.
Furthermore, HMM, dealing with the EICA-FLDA processed
sequential facial expression images can provide superior recognition
rate over the conventional feature extraction approaches, reaching
up to the mean recognition rate of 93.23%. Our system could be
used in any consumer systems for better human computer
interaction.

22. References• [1] M. T. Rahman and N. Kehtarnavaz, “Real-Time
Face-Priority Auto
• Focus for Digital and Cell-Phone Cameras,” IEEE
Transactions on
• Consumer Electronics, vol. 54, no. 4, pp. 1506–1513,
2008.
• [2] D.-S. Kim, I.-J. Jeon, S.-Y. Lee, P.-K. Rhee, and D.-J.
Chung,
• “Embedded Face Recognition based on Fast Genetic
Algorithm for
• Intelligent Digital Photography,” IEEE Transactions
on Consumer
• Electronics, vol. 52, no. 3, pp. 726–734, 2006.
• *3+ C. Padgett and G. Cottrell, “Representation face
images for emotion
• classification,” Advances in Neural Information
Processing Systems,
• vol. 9, Cambridge, MA, MIT Press, 1997.
• [4] S. Mitra and T. Acharya, “Gesture Recognition: A
survey,” IEEE
• Transactions on Systems, Man, and Cybernetics-Part
C: Applications
• and Reviews, vol. 37, no. 3, pp. 311-324, 2007.
• [5] G. Donato, M. S. Bartlett, J. C. Hagar, P. Ekman,
and T. J. Sejnowski,
• “Classifying Facial Actions,” IEEE Transaction on
Pattern Analysis and
• Machine Intelligence, vol. 21, no. 10, pp. 974-989,
1999.
• [6] P. S. Aleksic and A. K. Katsaggelos, “Automatic facial expression
• recognition using facial animation parameters and multistream
HMMs,”
• IEEE Transaction on Information and Security, vol. 1, pp. 3-11, 2006.
• [7] L. R. Rabiner, “A Tutorial on Hidden Markov Modes and selected
• application in speech recognition,” in Proceedings of IEEE, vol. 77,
pp.
• 257-286, 1989.
• *8+ L. Zhang and G. W. Cottrell, “When Holistic Processing is Not
Enough:
• Local Features Save the Day,” in Proceedings of the Twenty-sixth
• Annual Cognitive Science Society Conference, 2004.
• [9] Y. Linde, A. Buzo, and R. Gray, “An Algorithm for Vector
Quantizer
• Design,” IEEE Transaction on Communications, vol. 28, no. 1, pp.
84–
• 94, 1980.
• [10] J. F. Cohn, A. Zlochower, J. Lien, and T. Kanade, “Automated
face
• analysis by feature point tracking has high concurrent validity with
• manual FACS coding,” Psychophysiology, vol. 36, pp. 35-43, 1999.
• [11] J. J. Lee, M. Z. Uddin, P. T. H. Truc, and T.-S. Kim,
“Spatiotemporal
• Depth Information-based Human Facial Expression Recognition
Using
• FICA and HMM,” in Proceedings of the International Conference on
• Ubiquitous Healthcare, pp. 105-106, 2008.
• [32] G. J. Iddan and G. Yahav, “3D imaging in the studio (and
• elsewhere…),” in Proceedings of SPIE, vol. 4298, pp 48-55, 2001.