Supercharge Your Machine Learning Solutions with Amazon SageMaker

Amazon SageMaker
Jhen-Wei Huang (黃振維)
Solutions Architect, AWS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solving Some Of The Hardest Problems In Computer Science
Learning Language Perception Problem
Solving
Reasoning

Put machine learning in the hands of every developer
and data scientist
ML @ AWS: Our mission

Customer Running ML on AWS Today

AWS ML Stack
FRAMEWORKS AND INTERFACES
AWS DEEP LEARNING API
Apache MXNet TensorFlowCaffe2 Torch KerasCNTK PyTorch GluonTheano
PLATFORM SERVICES
VISIO N
AWS DeepLensAmazon SageMaker
LANG UAG E
A P P L I C A T I O N S E R V I C E S
Amazon
Rekognition
Amazon Polly Amazon Lex
Amazon Rekognition
Video
Amazon Transcribe Amazon Translate Amazon Comprehend
Alexa for Business
VR/IR Amazon Sumerian
Amazon Kinesis
Video Streams

Amazon EC2 P3 Instances (October 2017)
• Up to eight NVIDIA Tesla V100 GPUs
• 1 PetaFLOPs of computational performance
– 14x better than P2
• 300 GB/s GPU-to-GPU communication
(NVLink) – 9X better than P2
• 16GB GPU memory with 900 GB/sec peak
GPU memory bandwidth
T h e f a s t e s t , m o s t p o w e r f u l G P U i n s t a n c e s i n t h e c l o u d

AWS Deep Learning AMI
• Get started quickly with easy-to-launch tutorials
• Hassle-free setup and configuration
• Pay only for what you use – no additional charge for
the AMI
• Accelerate your model training and deployment
• Support for popular deep learning frameworks

Amazon ML Lab
Lots of companies
doing Machine
Learning
Unable to unlock
business potential
Brainstorming Modeling Teaching
Lack ML
expertise
Leverage Amazon experts with decades of ML
experience with technologies like Amazon Echo,
Amazon Alexa, Prime Air and Amazon Go
Amazon ML Lab
provides the missing
ML expertise

Amazon ML Lab Customers

Let’s Review the ML Process

Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Discovery: The Analysts
Re-training
• Help formulate the right
questions
• Domain Knowledge

Analysis
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Integration: The Data Architecture
Retraining
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

Analysis
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
• Setup and manage
Notebook Environments
• Setup and manage
Training Clusters
• Write Data Connectors
• Scale ML algorithms to
large datasets
• Distribute ML training
algorithm to multiple
machines
• Secure Model artifacts
Why We built Amazon SageMaker: The Model Training Undifferentiated Heavy Lifting

Model Deployment
Monitoring &
Debugging
– Predictions
• Setup and manage Model
Inference Clusters
• Manage and Scale Model
Inference APIs
• Monitor and Debug Model
Predictions
• Models versioning and
performance tracking
• Automate New Model
version promotion to
production (A/B testing)
Why We built Amazon SageMaker: The Model Deployment Undifferentiated Heavy Lifting

A fully managed service that enables data scientists and developers to quickly and easily
build machine-learning based models into production smart applications.
Amazon SageMaker

Highly-optimized
machine learning
algorithms
Amazon SageMaker
BuildPre-built notebook
instances

Highly-optimized
machine learning
algorithms
One-click training
for ML, DL, and
custom algorithms
instances
Easier training with
hyperparameter
optimization
Train
Amazon SageMaker

One-click training
for ML, DL, and
custom algorithms
Easier training with
hyperparameter
optimization
Highly-optimized
machine learning
algorithms
Deployment
without
engineering effort
Fully-managed
hosting at scale
instances
Deploy
Train
Amazon SageMaker

End-to-End
Machine Learning
Platform
Zero setup Flexible Model
Training
Pay by the second
$
Amazon SageMaker
Build, train, and deploy machine learning models at scale

Amazon ECR
Model Training (on EC2)
Amazon SageMaker
Client application
Training code

Amazon ECR
Trainingdata
Training code Helper code
Client application
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Helper codeInference code
Client application
Inference code
Training code
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
Client application
Inference code
Training code
Inference requestInference
response
Inference Endpoint
Amazon SageMaker

Amazon ECR
Trainingdata
Modelartifacts
GroundTruth
Client application
Inference code
Training code
Inference requestInference
response
Inference Endpoint
Amazon SageMaker

Amazon SageMaker: Launch Customers

“With Amazon SageMaker, we can accelerate our Artificial
Intelligence initiatives at scale by building and deploying our
algorithms on the platform. We will create novel large-scale
machine learning and AI algorithms and deploy them on this
platform to solve complex problems that can power prosperity
for our customers.
"
- Ashok Srivastava, Chief Data Officer, Intuit

Key benefits of SageMaker at Intuit
Ad-hoc setup and management
of notebook environments
Limited choices for model
deployment
Competing for compute
resources across teams
Easy data exploration
in SageMaker notebooks
Building around virtualization
for flexibility
Auto-scalable model hosting
environment
From To

Model Hosting
(SageMaker)
N ear r eal - time fr aud detec tion in AW S us ing SageMak er
Calculate
Features
Reader
Cleanser
Processor
Data
Lookup
Training
Feature Store Model Training
(SageMaker)
Model
Client Service
Amazon
EMR
Amazon
SageMaker
Amazon
SageMaker

“As the world’s leading provider of high-resolution Earth
imagery, data and analysis, DigitalGlobe works with enormous
amounts of data every day. DigitalGlobe is making it easier for
people to find, access, and run compute against our entire
100PB image library, which is stored in AWS’s cloud, to apply
deep learning to satellite imagery. We plan to use Amazon
SageMaker to train models against petabytes of Earth
observation imagery datasets using hosted Jupyter
notebooks, so DigitalGlobe's Geospatial Big Data Platform
(GBDX) users can just push a button, create a model, and
deploy it all within one scalable distributed environment at
scale.
”
- Dr. Walter Scott, CTO of Maxar Technologies and founder of
DigitalGlobe

“We’re focused on making it faster and easier than ever to hire
and get hired, training our machine learning algorithms against
hundreds of millions of historical transactional activities in order
to deliver highly relevant job matches as quickly as possible.
Amazon SageMaker provided us with an answer to problems we
had with ML workflow management, allowing us to train,
evaluate and deploy models in a flexible way. In addition,
Amazon SageMaker's modularity provides the ability to build and
create models independently, which is a compelling feature for
ZipRecruiter.
”
- Avi Golan, VP of Engineering, ZipRecruiter

Amazon SageMaker
1 2 3 4
I I I I
Notebook Instances Algorithms ML Training Service ML Hosting Service

1
I
Notebook Instances
Zero Setup For Exploratory Data Analysis
Authoring &
Notebooks
ETL Access to AWS
Database services
Access to S3 Data
Lake
• Recommendations/Personalization
• Fraud Detection
• Forecasting
• Image Classification
• Churn Prediction
• Marketing Email/Campaign Targeting
• Log processing and anomaly detection
• Speech to Text
• More…
“Just add data”

2
I
Algorithms
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
IM Estimators in
Apache Spark Bring Your Own Algorithm (You build the Container)
Amazon SageMaker: 10x better algorithms

Built-in ML Algorithm
h t t p s : / / d o c s . a w s . a m a z o n . c o m / s a g e m a k e r / l a t e s t / d g / a l g o s . h t m l
Problem Algorithm Learning Typ
Discrete Classification,
Regression
Linear Learner Supervised
XGBoost Algorithm Supervised
Discrete Recommendations Factorization Machines Supervised
Image Classification Image Classification Algorithm Supervised, CNN
Neural Machine Translation Sequence to Sequence Supervised, seq2seq
Time-series Prediction DeepAR Supervised, RNN
Discrete Groupings K-Means Algorithm Unsupervised
Dimensionality Reduction PCA (Principal Component Analysis) Unsupervised
Topic Determination Latent Dirichlet Allocation (LDA) Unsupervised
Neural Topic Model (NTM) Unsupervised,
Neural Network Based

Streaming datasets,
for cheaper training
Train faster, in a
single pass
Greater reliability on
extremely large
datasets
Choice of several ML
algorithms
Amazon SageMaker: 10x better algorithms
2
I
Algorithms

Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Single
Machine
Distributed, with
Strong Machines

Streaming
GPU State

Streaming
Data Size
Memory
Data Size
Time/Cost

Distributed
GPU State
GPU State
GPU State

Shared State
GPU
GPU
GPU Local
State
Shared
State
Local
State
Local
State

Cost vs. Time
$$$$
$$$
$$
$
Minutes Hours Days Weeks Months
Best Alternative
Amazon SageMaker

Managed Distributed Training with Flexibility
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
Bring Your Own Algorithm (You build the Container)
3
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
Secured–
Amazon ECR
Save Inference Image
IM Estimators in
Apache Spark
CPU GPU HPO

4
I
ML Hosting Service
Amazon ECR
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
Versions of the same
inference code saved in
inference containers.
Prod is the primary one,
50% of the traffic must
be served there!

4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
be served there!
Create a Model
ModelName: prod
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Model versions
be served there!
Create versions of a Model
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
InitialInstanceCount: 3
ModelName: prod
VariantName: primary
InitialVariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
be served there!
Create weighted
ProductionVariants
Amazon SageMaker

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
be served there!
Create an
EndpointConfiguration from
one or many
ProductionVariant(s)EndpointConfiguration
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
be served there! Create an Endpoint from
one EndpointConfiguration
EndpointConfiguration
Inference Endpoint
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
Amazon ECR
30 50
10 10
ProductionVariant
Model Artifacts
Inference Image
Model versions
be served there!
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
ModelName: prod

4
I
ML Hosting Service
 Auto-Scaling Inference
APIs
 A/B Testing (more to
come)
 Low Latency & High
Throughput
 Bring Your Own Model
 Python SDK
Amazon SageMaker

demo

Amazon SageMaker – Call To Action
• Getting started with Amazon SageMaker: https://aws.amazon.com/sagemaker/
• Use the Amazon SageMaker SDK:
• For Python: https://github.com/aws/sagemaker-python-sdk
• For Spark: https://github.com/aws/sagemaker-spark
• SageMaker Examples: https://github.com/awslabs/amazon-sagemaker-examples
• Let us know what you build!

Amazon SageMaker
!GO BUILD!
End-to-End Managed ML Platform

Supercharge Your Machine Learning Solutions with Amazon SageMaker

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Supercharge Your Machine Learning Solutions with Amazon SageMaker

Similaire à Supercharge Your Machine Learning Solutions with Amazon SageMaker (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Supercharge Your Machine Learning Solutions with Amazon SageMaker