Learning Objectives: - Learn about Amazon Rekognition - Learn about the new features released recently with Amazon Rekognition - Learn about use cases for Amazon Rekognition Amazon Rekognition is a service that makes it easy to add image analysis and facial recognition to your applications. You can use the service to implement a searchable image library, face-based user verification, visual sentiment analysis, and facial recognition. Recently, we launched new features that allow for the estimation of age range, as well as the ability to enable automatic image moderation workflows by providing confidence scores for suggestive and explicit content. This session will cover the core use cases for Rekognition to help you get started quickly with the service.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Pearson, AWS AI Services
May 2017
What’s New in Amazon Rekognition
Extract Rich Image Metadata from Visual Content

2. Amazon AI
Intelligent Services Powered By Deep Learning

3. Rich Metadata from Visual Content
objects, scenes, facial attributes, people
Amazon Rekognition
Deep Learning-Based Image Recognition Service

4. Deer 98.8%
Wildlife 95.1%
Conifer 95.1%
Spruce 95.1%
Wood 78.3%
Tree 63.5%
Forest 63.5%
Vegetation 61.9%
Pine 60.6%
Outdoors 54.0%
Flower 53.9%
Plant 52.9%
Nature 50.7%
Field 50.7%
Grass 50.7%

5. DetectLabels
"Labels": [
{
"Confidence": 98.9294204711914,
"Name": "Moss"
},
{
"Confidence": 98.9294204711914,
"Name": "Plant"
},
{
"Confidence": 97.35887908935547,
"Name": "Creek"
},
{
"Confidence": 97.35887908935547,
"Name": "Outdoors"
},
{
"Confidence": 97.35887908935547,
"Name": "Stream"
},
{
"Confidence": 97.35887908935547,
"Name": "Water"
},

6. Beard: False 84.3%
Emotion: Happy 86.5%
Eyeglasses: False 99.6%
Eyes Open: True 99.9%
Gender: Male 99.9%
Mouth Open: False 86.2%
Mustache: False 98.4%
Smile: True 95.9%
Sunglasses: False 99.8%
Bounding Box
Height: 0.36716..
Left: 0.40222..
Top: 0.23582..
Width: 0.27222..
Landmarks
EyeLeft
EyeRight
Nose
MouthLeft
MouthRight
LeftPupil
RightPupil
LeftEyeBrowLeft
LeftEyeBrowRight
LeftEyeBrowUp
:
Quality
Brightness 52.5%
Sharpness 99.9%

7. "BoundingBox": {
"Height": 0.3449999988079071,
"Left": 0.09666666388511658,
"Top": 0.27166667580604553,
"Width": 0.23000000417232513
},
"Confidence": 100,
"Emotions": [
{"Confidence": 99.1335220336914,
"Type": "HAPPY" },
{"Confidence": 3.3275485038757324,
"Type": "CALM"},
{"Confidence": 0.31517744064331055,
"Type": "SAD"}
],
"Eyeglasses": {"Confidence": 99.8050537109375,
"Value": false},
"EyesOpen": {Confidence": 99.99979400634766,
"Value": true},
"Gender": {"Confidence": 100,
"Value": "Female”}
DetectFaces
smart cropping
& ad overlays
sentiment
capture
demographic
analysis
face editing
& pixelation

8. Similarity 93%
Similarity 0%

9. "FaceMatches": [
{"Face": {"BoundingBox": {
"Height": 0.2683333456516266,
"Left": 0.5099999904632568,
"Top": 0.1783333271741867,
"Width": 0.17888888716697693},
"Confidence": 99.99845123291016},
"Similarity": 96
},
{"Face": {"BoundingBox": {
"Height": 0.2383333295583725,
"Left": 0.6233333349227905,
"Top": 0.3016666769981384,
"Width": 0.15888889133930206},
"Confidence": 99.71249389648438},
"Similarity": 0
}
],
"SourceImageFace": {"BoundingBox": {
"Height": 0.23983436822891235,
"Left": 0.28333333134651184,
"Top": 0.351423978805542,
"Width": 0.1599999964237213},
"Confidence": 99.99344635009766}
}
CompareFaces

11. Collection
IndexFaces
SearchFacesbyImage
Nearest neighbor
search
FaceID: 4c55926e-69b3-5c80-8c9b-78ea01d30690
Similarity: 97
FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d
Similarity: 92
FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d
Similarity: 85

12. New Features in Rekognition

13. Added to DetectFaces in
February 2017
Values returned as integers
for high and low estimates
Facilitates high scale,
demographic analysis
(paired with gender attribute)
Estimated Age Range
"AgeRange": {
"High": 43,
"Low": 26 }
"Gender": {
"Confidence": 99.91,
"Value": "Male” }
"AgeRange": {
"High": 43,
"Low": 26 }
"Gender": {
"Confidence": 100,
"Value": “Female” }

14. Demographic Analysis
• Touchless data gathering via in-store cameras
• Anonymous, high volume analysis of demographic (age range, gender)
• Extensible to sentiment analysis to measure service quality

15. Sentiment +
Demographic
Analysis
"AgeRange": {
"High": 68,
"Low": 48 }
"Gender": {
"Confidence": 99.926…,
"Value": "Male“ }
"Emotions": [
{ "Confidence": 99.449…,
"Type": "HAPPY” } …
"Smile": {
"Confidence": 73.576…,
"Value": true }
"AgeRange": {
"High": 55,
"Low": 35 }
"Gender": {
"Confidence": 100,
"Value": “Female“ }
"Emotions": [
{ "Confidence": 99.885…,
"Type": "HAPPY” } …
"Smile": {
"Confidence": 99.075…,
"Value": true }

16. Image Moderation
Detect images with explicit or suggestive adult content
Automate and optimize manual review processes
Hierarchical taxonomy provides greater control for geo-sensitive content
"ModerationLabels": [
{
"Confidence": 83.55088806152344,
"Name": "Suggestive",
"ParentName": ""
},
{
"Confidence": 83.55088806152344,
"Name": "Female Swimwear Or Underwear",
"ParentName": "Suggestive"
}
]
}
DetectModerationLabels

17. Image Moderation
Detect images with explicit or suggestive adult content
Automate and optimize manual review processes
Hierarchical taxonomy provides greater control for geo-sensitive content
Top-Level Category Second-Level Category
Explicit Nudity
Nudity
Graphic Male Nudity
Graphic Female Nudity
Sexual Activity
Partial Nudity
Suggestive
Female Swimwear Or Underwear
Male Swimwear Or Underwear
Revealing Clothes

18. Optimizing manual review processes
• Automating the detection of inappropriate content with Rekognition
• Reducing volume of images for human curation increases review quality

19. Amazon Rekognition Console
https://console.aws.amazon.com/rekognition/home

20. Amazon Rekognition Customers
• Law Enforcement and Public Safety
• Travel and Hospitality
• Digital Marketing and Advertising
• Media and Entertainment
• Internet of Things (IoT)

21. Amazon Rekognition
Customers
• Digital Asset Management
• Media and Entertainment
• Travel and Hospitality
• Influencer Marketing
• Systems Integration
• Digital Advertising
• Consumer Storage
• Law Enforcement
• Public Safety
• eCommerce
• Education

22. Law Enforcement and Public Safety
Washington County Sheriff (OR)
To follow leads from citizens & security cameras, a person
spends days manually searching thousands of images
The mobile and web app powered by Amazon Rekognition
compares new images with photos of previous offenders:
• Helps identify unknown theft suspects from security footage
• Provides leads by identifying possible witnesses & accomplices
• Identifies persons of interest who do not have identification

23. Travel and Hospitality
Anticipatory guest experiences for hotels using Amazon
Rekognition for facial recognition and sentiment capture
Kaliber is using Amazon Rekognition to help front desk agents
enhance relationships with guests:
• Recognize guests early for instant and personalized service
• Receive rich, contextualized guest information in real time
• Track guest sentiment throughout their stay
• Drive an 80% increase in guest satisfaction scores

24. Guest Workflow
Walk in Be recognized Be greeted
Capture sentiment to
trigger actionsEnjoy personalized serviceLeave with a fond farewell
“Kaliber allows us to bond with our guests from the
second they walk in my hotel.” – GM of a 5-star property

25. Influencer Marketing
Associate influencers with objects and scenes in social media
images in order to create high impact campaigns for clients
Using Amazon Rekognition for metadata extraction:
• Create rich media indexes of images from social media feeds, which
the application associates with influencers
• Enable analytics to profile environments where influence is strongest
• Connect client brands with the influencers most likely to have impact

26. Rekognition Demo with Video Frame Sampling

27. Media and Entertainment
Identify who is on camera for each of 8 networks so
that recorded video can be indexed and searched
Video frame-sampling facial recognition solution
using Amazon Rekognition:
• Indexed 97,000 people into a face collection in 1 day
• Sample frames every 6 secs and test for image variance
• Upload images to Amazon S3 and call Amazon Rekognition
to find best facial match
• Store time stamp and faceID metadata

28. C-SPAN Indexing Architecture
Video feeds encoded from 8
locations (3 networks and 5
federal courthouses)
Frames extracted into
JPGs and hosted in
Amazon S3
Amazon SQS provides
asynchronous decoupling
Search Amazon Rekognition
collection for high similarity
matches
Results cache drives
search and discovery
requests
R3 hashing detects if a scene
significantly changes

32. IoT Use Case
real-time facial recognition at the edge
AWS Advanced Consulting Partner
• Migrations
• DevOps
• Managed Services
• Software & Hardware Engineering
• User Experience & Visual Design
• Rapid Prototyping
AWS Competencies: DevOps, IoT, Healthcare

33. NERF CS-18 N-Strike Elite Rapidstrike
Adafruit 2.8”
PiTFT display
Raspberry Pi 3
Amazon Rekognition
https://sturdy.cloud/sting/
Training
Image

35. Amazon Rekognition Availability and Pricing
Free Tier: 5000 images processed per month for first 12 months
General Availability in 3 regions:
US East (N. Virginia), US West (Oregon); EU (Ireland)
Image Analysis Tiers Price per 1000
images processed
First 1 million images processed* per month $1.00
Next 9 million images processed* per month $0.80
Next 90 million images processed* per month $0.60
Over 100 million images processed* per month $0.40

36. Developer Resources and more…
https://aws.amazon.com/blogs/ai/
https://aws.amazon.com/rekognition

37. Thank You!
pearsond@amazon.com