The state of the art and how we applied it at Finnair. This presentation was held in Futurice Munich Beer & Tech event: Brave new world, AI applied. https://www.meetup.com/Munchen-Beer-Tech-Meetup/events/240575870/

1. Face
recognition_
The state of the art
and how we applied it
at Finnair

3. • Face Detection & Tracking
• Head pose and eye-gaze estimation
• Emotion recognition
• 2D & 3D Face recognition
• Face Morphing
All with real-time performance!
Background & State of the Art_

4. 3D Face Recognition_
Source: Ayonix.com
From face landmarks
into a 3D model and
feature vector
Terminology:
• Match Rate
• False Negative
• False Positive

5. • Security & Surveillance
• AI & Robotics
• Consumer behavior
• Smart Shopping
• Convenience
Business Cases_

6. Domain specific
problem: Human-made
Identification_
+ Interaction with another human is
natural
+ Reacting to extraordinary situations is
much easier.
- We are vulnerable to tiredness and
biases.

7. Fixed camera
Identification_
+ Precise, no biases or overload
+ Works 24/7
- Lacks human interaction, no room for
improvising

8. What has been figured
TECH
SOFTWARE
HARDWARE
Current State
Analysis_

9. Walk in experience
What did we do differently?

10. strength in pattern-based classification with
strength in making choices
O U R A P P R O A C H
0 1

11. An invisible
walk-paced identification
focusing on
human interaction & convenience
S E R V I C E V I S I O N

12. Registration
App_
Premium Passengers of Finnair will
register themselves at home using the
Android App deployed to their phones.
First digital touchpoint

13. Arrive to the airport_
Embedded into the normal UX of the
passenger.

14. Check in desk 206
interface_
The “Hands in the pocket” experience
for the platinum and gold passengers
of Finnair at the priority check in desk
Second digital touchpoint

15. Technical Architecture
OVERVIEW
• PoC system architecture is
separated into two clients and one
backend server
• Android application is used to
register new people to the system
• Airport Client Application is used for
detecting faces from camera stream
and sending those for recognition.
Recognize
Backend
Processes the recognitions and
stores the data
Android
Airport Client Application
• Detects faces in camera stream
• Sends facial images for recognition
• Visualizes results
• Gathers feedback from the agent
Register

16. How did we do?
R E S U L T S
0 2

17. Passenger Feedback
“It's amazing that new
technologies are being
sought and tested actively”
4.6/5
Overall experience rating
4.75/5
Ease of use of the App
4.1/5
Not worried about privacy

18. 01_Check in
attendants
"The app is easy to use,
fast and reliable. Would like
to use it in the future,
especially if it is integrated
to our systems.”
- Tiia, Check in agent

19. Publicity Overview
Our proof of concept has been featured in
various travel and aviation magazines
worldwide.
Big national and international media outlets
have featured the Proof of Concept such as
AP, HS and SVD.
Ten interview requests including big
technology and aviation magazines, including
PTW, ATW and ZD NET.

20. One more little thing..
03

21. Privacy_
• GRPD: Penalties up to 4% Revenue / 20MEur
• Scope: “Personal data is any information relating to an
individual, whether it relates to his or her private, professional
or public life. It can be anything from a name, a home address,
a photo, an email address, bank details, posts on social
networking websites, medical information, or a computer’s IP
address.“ Source: European Commission press release
• Privacy by design, Consent, Encryption,
Pseudonymisation, Right to be forgotten, …

22. What we learnt
04

23. Control the illumination
(as much as you can)
Study your
environment and users
Triggers can help to
achieve “real-time”

24. Control the number &
quality of the images
in the database
• Registration
• Self-adjusting

25. Test the algorithm
Test the code
Test the UX!!

26. Key Insights_
•We can do this in real time, in real world environment
•You can use an app for sourcing the images, but free form facial image sourcing does not work.
•Equal illumination on both side of the face is crucial
•Minimum 3 photos with the illumination variation in the database is a requirement
•Privacy is your major concern, not your customers
•We need to compromise from zero UI approach
•If you are wearing glasses, we have a problem 

27. Privacy_
Who cares?
• Data controller, Data processor
• System developers?
• End users, You and me?
How to “protect” yourself and your client?
• Privacy by Design
• Know the rules
• Know what you’re doing
• Don’t store ANY data you don’t need

28. Thank You!