Speaker: Herbert-John Kelly, AWS Customer Speaker: Data Prophet Level: 200 Join us to hear about our strategy for driving machine learning (ML) innovation for our customers and learn what's new from AWS in the machine learning space. We will discuss and demonstrate the latest new services for ML on AWS: Amazon SageMaker, AWS DeepLens, Amazon Rekogntion Video, Amazon Translate, Amazon Transcribe and Amazon Comprehend. Attend this session to understand how to make the most of machine learning in the cloud.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning State of the Union
Herbert-John Kelly
Peter Brookstein
Solutions Architect, AWS /in/herbertjkelly
Senior Engineer, Data Prophet /company/data-prophet

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon.com,1995
Amazon 1995

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
« Two Decades of Recommender Systems at Amazon.com » (2017)
https://www.computer.org/csdl/mags/ic/2017/03/mic2017030012.html
G.D. Linden, J.A. Jacobi, and E.A.
Benson, Collaborative
Recommendations Using Item-to-Item
Similarity Mappings, US Patent
6,266,649, to Amazon.com, Patent and
Trademark Office, 2001 (filed 1998).

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Put Machine Learning in
the hands of every
developer and data
scientist
Our mission ::

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bottom Layer: Frameworks & Interfaces
Tesla V100 GPUs
P3
AWS Deep Learning AMI
5,120 tensor cores
128 GB of memory
1 petaflop of compute
NVLink 2.0
~14X faster than P2
C5
3.0 GHz Intel Xeon
(Skylake) processors
72 vCPUs
144 GB of memory
Advanced Vector
Extension (AVX) 512
Nitro Hypervisor
25% improvement in
price/perf. than C4

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The Amazon ML Stack
Frameworks & Interfaces
Platform Services: Custom ML Models
Application Services: Vision, Language, Chatbots
Caffe2 CNTK
Apache
MXNet
PyTorch TensorFlow Torch Keras Gluon
AWS Deep Learning AMIs
Infrastructure
EC2 GPUs EC2 CPUs IoT Edge
Chainer

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
More AI/ML is built on AWS than anywhere else

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application Services

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
Deep Learning-based image analysis service

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Marinus Analytics
Marinus Analytics provides law enforcement with tools
founded in artificial intelligence.
Traffic Jam, is a suite of tools for use by law
enforcement agencies on sex trafficking
investigations.
Before using Amazon Rekognition, their only recourse
was manual processing; this was time-intensive or not
possible.
Now, investigators are able to take effective action by
searching through millions of records in seconds to find
victims.
http://www.marinusanalytics.com/articles/2
017/10/17/amazon-rekognition-helps-
marinus-analytics-fight-human-trafficking

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Video
Deep Learning-based video analysis service

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Video Analysis

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Royal Wedding

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Polly
Deep Learning-based text-to-speech service

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Welcome to the AWS Cape
Town Summit. The weather is a
Lekker 17 °C!
Amazon Polly: Text In, Life-like Speech Out

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Translate
Neural Machine Translation Service

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Do you want to go see
a movie tonight?”
Amazon Translate
Natural and fluent language translation
Amazon
Translate
‫الليلة؟‬ ‫فيلم‬ ‫لمشاهدة‬ ‫الذهاب‬ ‫تريد‬ ‫هل‬

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Real-time translation Powered by Deep
Learning
12 Language pairs
(more to come)
Language detection
Amazon Translate
Natural and fluent language translation

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hotels.com
Machine Translation
« At Hotels.com, we are committed to offering all of our customers the most relevant and up to
date information about their destination. To achieve that, we operate 90 localized websites in 41
languages. We have more than 25M customer reviews and more are coming in every day, making
a great candidate for machine translation. Having evaluated Amazon Translate and several other
solutions, we believe that Amazon Translate presents a quick, efficient and most importantly,
accurate solution »
Matt Fryer, VP and Chief Data Science Officer,
Hotels.com

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Transcribe
Automatic speech recognition service

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Hello, this is
Herbert speaking”
Automatic speech recognition service
Amazon
Transcribe

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Support for
telephony
audio
Timestamp
generation
Intelligent
punctuation
and formatting
Recognize multiple
speakers
Custom
vocabulary
Multiple
languages
Automatic speech recognition service

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Comprehend
Natural Language Processing

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fully managed natural language processing
Discover valuable insights from
text
Entities
Key Phrases
Language
SentimentAmazon
Comprehe
nd

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Topic modeling
STORM
WORLD SERIES
AUSTRALIASTOCK
MARKET
WASHINGTON
HEALTH
CRISIS
MACHINE
LEARNING
LIBRARY OF
NEWS ARTICLES *
Amazon
Comprehend

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Lex
Conversational Interfaces

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Intents
A particular goal that the
user wants to achieve
Utterances
Spoken or typed phrases that
invoke your intent
Slots
Data the user must provide to fulfill the
intent
Prompts
Questions that ask the user to input
data
Fulfillment
The business logic required to fulfill the
user’s intent
BookHotel
Lex Bots
Salesforce
Microsoft
Dynamics
Marketo
Zendesk
Web
Devices
Apps
Facebook Messenger,
Slack,
Amazon
Connect
Mobile
Mobile Hub
integration
Quickbooks

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Platform Services

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML on Amazon Elastic Map Reduce
Amazon EMR

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Collect and
prepare
training data
Choose and
optimize
your ML
algorithm
Set up and
manage
environment
s for training
Train and
tune model
(trial and
error)
Deploy
model
in
production
Scale and
manage the
production
environment
Easily build, train, and deploy machine learning models

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon SageMaker
Pre-built
notebooks
for
common
problems
K-Means Clustering
Principal Component Analysis
Neural Topic Modeling
Factorization Machines
Linear Learner - Regression
XGBoost
Latent Dirichlet Allocation
Image Classification
Seq2Seq
Linear Learner - Classification
ALGORITHMS
Apache MXNet
TensorFlow
Caffe2,
CNTK,
PyTorch,
Torch
FRAMEWORKS
Built-in,
high-
performance
algorithms Build
One-click
training
Hyperparameter
optimization
Train
Fully
managed
hosting with
auto-scaling
One-click
deployment
Deploy

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Digital Globe
http://blog.digitalglobe.com/industry/using-machine-learning-to-save-money-on-cloud-data-storage/
https://www.youtube.com/watch?v=mkKkSRIxU8M
In the last 18 years DigitalGlobe has been operating
Earth imaging satellites, they have collected over 100
PB of imagery.
There is a trade-off between how quickly data can be
accessed and how much it will cost to store.
Working with Amazon ML Solutions Lab, Digital Globe
built a predictive model that will reduce cloud storage
costs for their imagery archive by 50%.

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ML Solutions Lab
Lots of companies
doing Machine
Learning
Unable to unlock
business potential
Brainstorming Modeling Teaching
Lack ML
expertise
Leverage Amazon experts with decades
of ML experience with technologies
like Amazon Echo, Amazon Alexa,
Prime Air and Amazon Go
Amazon ML Solutions
Lab provides the
missing ML expertise

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS DeepLens
T h e w o r l d ’ s f i r s t d e e p
l e a r n i n g - e n a b l e d v i d e o
c a m e r a f o r d e v e l o p e r s
• A new way to learn ML though sample projects, with practical,
hands-on examples
• Run deep learning models locally on the camera to recognize
or classify without streaming to the cloud
• Easy to customize and fully programmable using AWS Lambda
• Integrated with Amazon SageMaker for custom model
deployment
• Runs on any deep learning framework, including Apache
MXNet, TensorFlow, and Caffe
Available now to pre-order from amazon.com for $249
• Intel Atom Processor
• Gen9 graphics
• Ubuntu OS- 16.04 LTS
• 100 GFLOPS performance
• Dual band Wi-Fi
• 8-GB RAM
• 16-GB storage (eMMC)
• 32-GB SD card
• 4 MP camera with MJPEG
• H.264 encoding at 1080p resolution
• 2 USB ports
• Micro HDMI
• Audio out
• AWS Greengrass preconfigured
• clDNN-optimized for MXNet
• Key Differentiators/Technologies
• Intel cLDNN Library optimized for MXNet
• Intel Deep Learning Deployment Toolkit
AWS DeepLens Specifications

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DEMO TIME!

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DoorBell Demo

39. Client Testimonial
Industry:
AI Optimization for Manufacturing Processes

40. DataProphet Background
What we have worked on
Manufacturing
Retail
Predictive & Prescriptive
Natural
Language
Processing
Computer
Vision
Defect Prediction
Automated process parameter
assignment
Next best product
Next basket prediction
Anomaly detection
Best time to contact
Emotion recognition
Customer
lapse
Other
Established
in 2014
with the goal of
providing
bleeding-edge
machine learning
solutions
Head Office
Cape Town
With a satellite
office
in Johannesburg
Team of
30+
consisting of PhD
engineers, computer
scientists and
statisticians
(42+ university
degrees in the team)
Suite of
Products
Numerous products
live and in production
with international and
local multinational
clients

41. What is OMNI
Vision
Examples of Clients
Optimal control parameters - with AI-enable
variable process control:
● Minimise capacity lost on rework
Active defect prediction - with AI-enable
learnt simulation of production
environment:
● Continuous, pre-emptive monitoring of
defects and probable reasons
● Learnt simulation to test defect causes
Machine Vision Quality Control – Detects
and locates surface defects.
● Works on previously unseen components
● Minimise waste due to defects

42. How DataProphet uses AWS
Leverage AWS for all stages of model development through deployment
Great tools and libraries for rapid data exploration and modeling.
Secure, compartmentalized environments & deployments
Rapidly scale compute and storage to match the challenge.
Deterministic infrastructure with Terraform and SaltStack.

43. Lifecycle of a project
Explore
Raw data in S3. High memory
(r4) + GPU (p3) EC2 instances
for data scientists to play with
and explore data.
In house libraries for reliable
persistence of models and
feature extraction code.
Train
GPU instances (p3) for CNN’s
and DNN’s.
High memory / CPU for RNNs
and LSTM models (R4 & c5)
Graph, weights & biases
exported and saved to S3.
Serve
Large CPU instances (c5 / on
prem)
Tensorflow Serving
+ Python (Flask) API Servers
Serve from both AWS, or for low
latency serving on premises in
internet starved environments.
Test
Full mock serving environment, with
TensorFlow Serving model server
and Flask JSON API server.
Validate model performance before
serving.
End to end API testing to ensure
the feature extraction modules are
working correctly.

44. Architecture of a project on AWS
Explore Train ServeTest
Data
Scientist
Sandbox
VPC
VPN
Gateway
p3 / r4
Instances
p3 / r4
Instances
Model &
Feature
Extraction
Model &
Feature
Extraction
TensorFlow
Serving
+ API
Test Runner
c4 Instance t2 Instance
Test Results Model &
Feature
Extraction
TensorFlow
Serving
Flask API
c5 Instance t2 Instance Elastic Load
Balancing
gRPC
Client
Serving
VPC
Internet
Gateway
S3 S3

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You!
Herbert-John Kelly - /in/herbertjkelly
Peter Brookstein - /company/data-prophet