3. search, verify, and organize millions of images
Amazon Rekognition
Deep Learning-Based Image Recognition Service
Object and Scene
Detection
Facial
Analysis
Face
Comparison
Facial
Recognition
6. Thousands of Objects and Scenes
• Search, filter, and
curate image
libraries
• Photo, real estate,
vacation rental,
travel, hospitality,
and more
High Rise
Coast
City
Pier
Water
Waterfront
Dawn
Outdoors
Harbor
Sky
Building
10. Facial Analysis
Locate faces within images and analyze face attributes to
detect emotion, pose, facial landmarks, and features
• Avoid faces when cropping
images and overlaying ads
• Capture user demographics
and sentiment
• Recommend the best photos
• Improve online dating match
recommendations
• Dynamic, personalized ads
14. Feature Location
add face overlays in social media apps
"FaceDetail": [
...
{
"Landmarks": [
{
"Type": "string",
"X": number,
"Y": number }
...
Response (excerpt)EyeLeft
EyeRight
Nose
MouthLeft
MouthRight
LeftPupil
RightPupil
LeftEyeBrowLeft
LeftEyeBrowRight
LeftEyeBrowUp
:
15. Demographic Analysis
measure the impact of targeted marketing campaigns
"FaceDetail": [
...
{
"AgeRange": {
"High": number,
"Low": number
},...
"Gender": {
"Confidence": number,
"Value": Boolean }
...
Response (excerpt)
Look Your Best All Day
Time for A New Look?
PersonAPersonB
Sees
Sees
17. Image Quality and Direction of Gaze
detect blurry or poor quality light; subject liveness
"FaceDetail": [
...
{
"Pose": {
"Pitch": number,
"Roll": number,
"Yaw": number
},
"Quality": {
"Brightness": number,
"Sharpness": number
}
Response (excerpt) "Pose": {
"Pitch": 8.250975608825684,
"Roll": -8.29802131652832,
"Yaw": 14.244261741638184
},
"Quality": {
"Brightness": 46.14684295654297,
"Sharpness": 99.9945297241211
},
18. Face Comparison
Measure the likelihood that faces in two images are of the
same person
• Add face verification to
applications and devices
• Extend physical security
controls
• Provide guest access to
VIP-only facilities
• Verify users for online
exams and polls
21. Facial Recognition
Identify people in images by finding the closest match for an
input face image against a collection of stored face vectors
• Add friend tagging to
social and messaging apps
• Assist public safety officers
find missing persons
• Identify employees as they
access sensitive locations
• Recognize celebrities in
historical image archives
22. Amazon Rekognition API
Facial Recognition
index and search faces in a collection
Index
Search
Collection
IndexFaces
SearchFacesByImage
27. AMAZON S3 AWS LAMBDA AMAZON
REKOGNITION
AMAZON
DYNAMODB
Triggered Media Indexing
When a new image is uploaded to S3…
1. Lambda function is triggered and calls Rekognition
2. Rekognition API retrieves the image from S3 and returns the detected labels,
attributes, and recognized persons
3. The output metadata is stored into DynamoDB for durability and speed of access
28. Fast Analysis of New Image Files in S3
Console Demo
Event-driven processing with Lambda
• Add new images to an S3 bucket
• Lambda log as it processes new images
• View metadata in DynamoDB table
30. Influencer Marketing Case Study
Associate influencers with objects and scenes in social media
images in order to create high impact campaigns for clients
Using Rekognition for metadata extraction:
• Create rich media indexes of images from social media feeds, which
the application associates with influencers
• Enable analytics to profile environments where influence is strongest
• Connect client brands with the influencers most likely to have impact
31. Law Enforcement Case Study
To service leads from citizens and security cameras, a
person spends days manually searching thousands of images
The new Rekognition-powered mobile and web app compares
field images with photos of previous offenders:
• Helps identify unknown theft suspects from security footage
• Provides leads by identifying possible witnesses and accomplices
• Identifies people of interest who do not have identification
32. Media Case Study
Identify who is on camera at what time for each of 8 networks
so that recorded video streams can be indexed and searched
Video frame-sampling facial recognition solution using
Amazon Rekognition:
• Indexed 97,000 people into a face collection in 1 day
• Sample frames every 6 secs and test for image variance
• Upload images to S3 and call Rekognition to find best facial match
• Store time stamp and faceID metadata
33. C-SPAN Architecture
Video feeds encoded from
8 locations (3 networks and
5 federal courthouses)
Frames extracted into
JPGs and hosted in S3
SQS provides
asynchronous decoupling
Search Rekognition collection
for high similarity matches
Results cache
drives search and
discovery requests
R3 hashing detects if a
scene significantly changes
34. Rekognition Customers
Digital Asset Management
Media and Entertainment
Influencer Marketing
Digital Advertising
Consumer Storage
Law Enforcement
Public Safety
eCommerce
Education
35. Amazon Rekognition Availability and Pricing
Free Tier: 5000 images processed per month for first 12 months
General Availability in 3 regions:
US East (N. Virginia), US West (Oregon); EU (Ireland)
Image Analysis Tiers Price per 1000
images processed
First 1 million images processed* per month $1.00
Next 9 million images processed* per month $0.80
Next 90 million images processed* per month $0.60
Over 100 million images processed* per month $0.40
36. Developer Resources and more…
https://aws.amazon.com/blogs/ai/
https://aws.amazon.com/rekognition
Introducing Amazon Rekognition - a fully managed deep learning based image recognition service. Rekognition was designed from the get-go to run at scale. It comprehends scenes, objects, concepts and faces. Given an image, it will return a list of labels. Given an image with one or more faces,it will return bounding boxes for each face, along with face attributes. Given two images with faces, it will compare the largest face from the source image and find similarity with faces found in the tagret image. Rekognition provides quality face recognition at scale, and supports creation of collection of millions of faces and search of similar faces in the collection.
Now lets dive into each of these features and look at the API that support these features.
In its simplest form – DetectLabels takes an image as input and returns a set of labels with confidence score.
Request
{ "Image": { "Bytes": blob, "S3Object": { "Bucket": "string", "Name": "string", "Version": "string" } }, "MaxLabels": number, "MinConfidence": number }
Response
{ "Labels": [ { "Confidence": number, "Name": "string" } ], "OrientationCorrection": "string" }
In the preceding example, the operation returns one label for each of the three objects. The operation can also return multiple labels for the same object in the image. For example, if the input image shows a flower (for example, a tulip), the operation might return the following three labels.
A mobile app uploads new images to S3
An S3 notification triggers a Lambda function which calls Rekognition’s DetectLabels API with an S3 url
DetectLabels analyzes the image and returns labels for objects and scenes detected in the image
This output is persisted as metadata into DynamoDB to ensure durability and into Elasticsearch to power a smart search index. The application serves selected images directly from S3.
Options on this configuration include: writing to Elasticsearch directly from the Lambda function, and using CloudFront to serve the image to the user.
DetectFaces taken an image with one or more faces and returns bounding box of the faces and some key landmarks and attributes for each face detected.
Input
{ "Attributes": [ "string" ], "Image": { "Bytes": blob, "S3Object": { "Bucket": "string", "Name": "string", "Version": "string" } } }
Talk about the attributes.
A mobile app uploads new images to S3
An S3 notification triggers a Lambda function which calls Rekognition’s DetectLabels API with an S3 url
DetectLabels analyzes the image and returns labels for objects and scenes detected in the image
This output is persisted as metadata into DynamoDB to ensure durability and into Elasticsearch to power a smart search index. The application serves selected images directly from S3.
Options on this configuration include: writing to Elasticsearch directly from the Lambda function, and using CloudFront to serve the image to the user.
A mobile app uploads new images to S3
An S3 notification triggers a Lambda function which calls Rekognition’s DetectLabels API with an S3 url
DetectLabels analyzes the image and returns labels for objects and scenes detected in the image
This output is persisted as metadata into DynamoDB to ensure durability and into Elasticsearch to power a smart search index. The application serves selected images directly from S3.
Options on this configuration include: writing to Elasticsearch directly from the Lambda function, and using CloudFront to serve the image to the user.
A mobile app uploads new images to S3
An S3 notification triggers a Lambda function which calls Rekognition’s DetectLabels API with an S3 url
DetectLabels analyzes the image and returns labels for objects and scenes detected in the image
This output is persisted as metadata into DynamoDB to ensure durability and into Elasticsearch to power a smart search index. The application serves selected images directly from S3.
Options on this configuration include: writing to Elasticsearch directly from the Lambda function, and using CloudFront to serve the image to the user.
The Amazon S3 inventory tool produces a .csv file, that lists the images that are stored in an images bucket, and saves the csv file to an S3 bucket called the inventory bucket. The inventory file is produced daily by the S3 service.
When a new version of the zipped S3 inventory .csv file is saved to the destination S3 inventory bucket, a Lambda function is called as the inventory bucket is configured to invoke the Lambda function on any object with a .csv.gz file extension.
The Lambda function reads the contents of the .csv file, and for each image it finds, it creates a new AWS Batch Job with the image bucket and name, and submits it to the AWS Batch queue. AWS Batch processes the batch jobs submitted to the job queue by launching EC2 instances and executing the batch jobs on those instances.
The Lambda function removes the S3 event trigger to avoid the AWS Batch backfill workflow to run more than once.
The AWS Batch jobs receive an image bucket and name as input parameters and check if the image has already been processed by querying the Amazon ES domain index. If the image has not been processed, the AWS Batch Job calls the Amazon Rekognition detect_label API.
The AWS Batch jobs save the labels that Rekognition returns for the image into the Amazon ES domain index.
Clients can request influencers in a key demographic. Rekognition partners use their metadata to identify high quality influencers for targeted campaigns, which may involve paying influencers for product use and social media posts featuring the product.
An influencer’s strength is measured by who is following them on social media
Future use for Reko: identify brands, measure impact, rate influencers
Quoted Interesting Facts
More than 300,000 photos of previous offenders indexed in 2 days
Each search requires seconds to return results
Entire process went from 2-3 days, to minutes
Within 1 week of going live, the app was used to identify and arrest a suspect who stole over $5,000 from local stores. There were no other leads prior to the app finding the match.
No other tool could scale to this volume.
Plans to expand to neighboring counties, and beyond.
Democratizing Image Analysis (key factors: affordable, accessible, scalable) that helps you get started in minutes
