1. Pattern Recognition and Applications group
Department of Electrical and Electronic Engineering (DIEE)
University of Cagliari, Italy
Robustness of multi-modal biometric
verification systems under realistic
spoofing attacks
Battista Biggio, Zahid Akthar, Giorgio Fumera,
Gian Luca Marcialis, and Fabio Roli
Int’l Joint Conf. On Biometrics, IJCB 2011

2. Biometrics
• Examples of body traits that can be used for biometric recognition
Face Fingerprint Iris Hand geometry
Palmprint Signature Voice Gait
• Enrollment and verification phases in biometric system
User User Identity
Feature XTemplate
Enrollment Sensor
Extractor
Database
User Claimed Identity
XQuery XTemplate
Sensor Feature Matcher Database
Verification Extractor
score
Decision Genuine/Impostor
12-11-2011 G.L. Marcialis, IJCB 2011 2

3. Biometric systems
• Multi-modal biometric verification systems
DB
true
genuine
s1
Sensor Face matcher
s
Score fusion rule s ≥ s∗
s2 f (s1 , s2 )
Sensor Fingerprint matcher
false
impostor
DB
– more accurate than unimodal
– more robust to spoof attacks?
12-11-2011 G.L. Marcialis, IJCB 2011 3

4. Direct (spoofing) attacks
• Spoofing attacks
– attacks at the user interface (sensor)
– fake biometric traits
• Countermeasures
– liveness detection
– multi-modal biometric systems
12-11-2011 G.L. Marcialis, IJCB 2011 4

5. Motivation and goal of this work
• Open problems
– Estimation of the FAR under spoof attacks for multi-modal
systems
– Construction of fake biometric traits (cumbersome task)
• State-of-the-art
– Fake scores are simulated under a worst-case scenario, re-
sampling genuine user scores
sifake : p(si | G) (when the i-th matcher is spoofed)
• Our goal
– To experimentally verify if this worst-case assumption holds
under realistic spoofing attacks
12-11-2011 G.L. Marcialis, IJCB 2011 5

6. Experiments
• Multi-modal system with face and fingerprint matchers
– Bozorth3 (fingerprint)
– Elastic Bunch Graph Matching, EBGM (face)
true
genuine
s1
Sensor Face matcher
s
Score fusion rule s ≥ s∗
s2 f (s1 , s2 )
Sensor Fingerprint matcher
false
impostor
12-11-2011 G.L. Marcialis, IJCB 2011 6

7. Experiments
score fusion rules
1. product s = s1 ⋅ s2
2. weighted sum (LDA) s = w0 + w1s1 + w2 s2
3. likelihood ratio (LLR) s = p(s1 , s2 | G) / p(s1 , s2 | I )
4. extended LLR* explicitly models the distribution of
[Rodrigues et al., JVLC 2009] spoof attacks (worst-case)
12-11-2011 G.L. Marcialis, IJCB 2011 7

8. Experiments
Fake biometric traits
• Fake fingerprints by “consensual method”
– mould: plasticine-like materials
– cast: latex, silicon, and two-compound mixture of liquid silicon
live fake (latex) fake (silicon)
• Fake faces by “photo attack” and “personal photo attack”
– Photo displayed on a laptop screen to camera
– Personal photos (like those appearing on social networks)
live fake (photo) fake (personal)
12-11-2011 G.L. Marcialis, IJCB 2011 8

9. Experiments
Data sets
12-11-2011 G.L. Marcialis, IJCB 2011 9

10. Experiments
Results
• Fakes: latex (fingerprints) and photo (faces)
• Worst case assumption (dashed lines) holds to some extent
12-11-2011 G.L. Marcialis, IJCB 2011 10

11. Experiments
Results
• Fakes: latex (fingerprints) and photo (faces)
• Worst case assumption (dashed lines) does not hold
12-11-2011 G.L. Marcialis, IJCB 2011 11

12. Experiments
Extended LLR can be less robust than LLR
Results
to realistic fingerprint spoof attacks!
• Fakes: latex (fingerprints) and photo (faces)
• Worst case assumption (dashed lines) does not hold
12-11-2011 G.L. Marcialis, IJCB 2011 12

13. Experiments
Results
• Fakes: silicone (fingerprint) and personal photos (face)
12-11-2011 G.L. Marcialis, IJCB 2011 13

14. Conclusions and future work
• Crucial issue
– performance evaluation of multimodal biometric systems under
spoofing attacks
• State-of-the-art: “worst-case” scenario
• Our results
– it may not provide an accurate model for fake score simulation
– Score fusion rules designed under this assumption may worsen the
system’s robustness!
• Future work
– experimental analysis involving other spoofing attacks
– more accurate modelling and simulation of fake score distributions
12-11-2011 G.L. Marcialis, IJCB 2011 14