AWS SageMaker: Deploy ML Models from Notebook to Production

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Julien Simon
Principal Technical Evangelist, AI & Machine Learning,
AWS
@julsimon
From Notebook to Production
with Amazon SageMaker

© 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Put Machine Learning in the hands
of every developer and data scientist
Our mission

Application
Services
Platform
Services
Frameworks
& Infrastructure
API-driven services:Vision, Language & Speech Services, Chatbots
AWS ML Stack
Deploy machine learning models with high-performance machine learning algorithms,
broad framework support, and one-click training, tuning, and inference.
Develop sophisticated models with any framework, create managed, auto-scaling
clusters of GPUs for large scale training, or run prediction
on trained models.

Data Visualization &
Analysis
Business Problem
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training
Predictions

Amazon SageMaker
Pre-built
notebooks for
common
problems
K-MeansClustering
Principal Component Analysis
Neural TopicModelling
FactorizationMachines
Linear Learner
XGBoost
Latent Dirichlet Allocation
ImageClassification
Seq2Seq,
And more!
ALGORITHMS
Apache MXNet, Chainer
TensorFlow, PyTorch, scikit-learn
FRAMEWORKS Set up and manage
environments for training
Train and tune
model (trial and
error)
Deploy model
in production
Scale and manage the
production environment
Built-in, high-
performance
algorithms
Build
Git integration
Elastic inference

Git integration

Elastic Inference
https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/

Amazon SageMaker
Pre-built
notebooks for
common
problems
K-MeansClustering
Principal Component Analysis
Neural TopicModelling
FactorizationMachines
Linear Learner
XGBoost
Latent Dirichlet Allocation
ImageClassification
Seq2Seq,
And more!
ALGORITHMS
Apache MXNet, Chainer
TensorFlow, PyTorch, scikit-learn
FRAMEWORKS Set up and manage
environments for training
Train and tune
model (trial and
error)
Deploy model
in production
production environment
Built-in, high-
performance
algorithms
Build
New built-in algorithms
scikit-learn environment
Model marketplace
Search

Search training jobs

Machine Learning Marketplace

Amazon SageMaker
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Train
Deploy model
in production
production
environment
P3DN, C5N
TensorFlow on 256 GPUs
Resume HPO tuning job
Build

Amazon SageMaker
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Deploy
Model compilation
Elastic inference
Inference pipelines
TrainBuild

TheAmazon SageMakerAPI
• Python SDK orchestrating all Amazon SageMaker activity
• High-level objects for algorithm selection, training, deploying,
automatic model tuning, etc.
• Spark SDK (Python & Scala)
• AWS CLI: ‘aws sagemaker’
• AWS SDK: boto3, etc.

Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
GroundTruth
Client application
Inference code
Training code
Inference requestInference
response
Inference Endpoint

Training code
Factorization Machines
Linear Learner
PrincipalComponent Analysis
K-Means Clustering
XGBoost
And more
Built-inAlgorithms BringYour Own ContainerBringYour Own Script
Model options

Built-in algorithms
orange: supervised, yellow: unsupervised
Linear Learner: regression, classification Image Classification: Deep Learning (ResNet)
Factorization Machines: regression, classification,
recommendation
Object Detection (SSD): Deep Learning
(VGG or ResNet)
K-Nearest Neighbors: non-parametric regression
and classification
NeuralTopic Model: topic modeling
XGBoost: regression, classification, ranking
https://github.com/dmlc/xgboost
Latent DirichletAllocation: topic modeling (mostly)
K-Means: clustering BlazingText:GPU-basedWord2Vec,
and text classification
Principal ComponentAnalysis: dimensionality
reduction
Sequence to Sequence: machine translation, speech
speech to text and more
RandomCut Forest: anomaly detection DeepAR: time-series forecasting (RNN)
Object2Vec: general-purpose embedding IP Insights: usage patterns for IP addresses
Semantic Segmentation: Deep Learning

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Blazing Text
https://dl.acm.org/citation.cfm?id=3146354

Demo:
Text Classification with BlazingText
https://github.com/awslabs/amazon-sagemaker-
examples/tree/master/introduction_to_amazon_algorithms/blazingtext_text_classification_dbpedia

OptimizingTensorFlow
https://aws.amazon.com/blogs/machine-learning/faster-training-with-
optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3-instances/
(March 2018)
Training a ResNet-50 benchmark with
the synthetic ImageNet dataset using
our optimized build ofTensorFlow
1.11 on a c5.18xlarge instance type is
11x faster than training on the stock
binaries.
https://aws.amazon.com/about-aws/whats-
new/2018/10/chainer4-4_theano_1-0-
2_launch_deep_learning_ami/ (October 2018)

Demo:
Elastic Inference withTensorFlow
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/tensorflow_iris_dnn_classifier_using_estimators/

Automatic Model Tuning
Finding the optimal set of hyper parameters
1. Manual Search (”I know what I’m doing”)
2. Grid Search (“X marks the spot”)
• Typically training hundreds of models
• Slow and expensive
3. Random Search (“Spray and pray”)
• Works better and faster than Grid Search
• But… but… but… it’s random!
4. HPO: use Machine Learning
• Training fewer models
• Gaussian Process Regression and Bayesian Optimization
• You can now resume from a previous tuning job

Demo: HPO withTensorFlow
https://github.com/awslabs/amazon-sagemaker-
examples/tree/master/hyperparameter_tuning/tensorflow_mnist
CONV1
POOL1
CONV2
POOL2
DENSE1
DROPOUT1
OUTPUT
Class
probabilities
INPUT

Optimizing for the underlying hardware
https://aws.amazon.com/blogs/aws/amazon-sagemaker-neo-train-your-machine-learning-models-once-run-them-anywhere/
• Train once, run anywhere
• Frameworks and algorithms
• TensorFlow, Apache MXNet, PyTorch, ONNX, and XGBoost
• Hardware architectures
• ARM, Intel, and NVIDIA starting today
• Cadence, Qualcomm, and Xilinx hardware coming soon
• Amazon SageMaker Neo will be released as open source enabling hardware
vendors to customize it for their processors and devices.

Demo:
Compiling ResNet-50 for the Raspberry Pi
Configure the compilation job
{
"RoleArn":$ROLE_ARN,
"InputConfig": {
"S3Uri":"s3://jsimon-neo/model.tar.gz",
"DataInputConfig": "{"data": [1, 3, 224, 224]}",
"Framework": "MXNET"
},
"OutputConfig": {
"S3OutputLocation": "s3://jsimon-neo/",
"TargetDevice": "rasp3b"
},
"StoppingCondition": {
"MaxRuntimeInSeconds": 300
}
}
Compile the model
$ aws sagemaker create-compilation-job
--cli-input-json file://config.json
--compilation-job-name resnet50-mxnet-pi
$ aws s3 cp s3://jsimon-neo/model-
rasp3b.tar.gz .
$ gtar tfz model-rasp3b.tar.gz
compiled.params
compiled_model.json
compiled.so
Predict with the compiled model
from dlr import DLRModel
model = DLRModel('resnet50', input_shape,
output_shape, device)
out = model.run(input_data)

Inference Pipelines
• Linear sequence of 2-5 containers that process inference requests
• Feature engineering with scikit-learn or SparkML (on AWS Glue or Amazon EMR)
• Predict with built-in or custom containers
• The sequence is deployed as a a single model
• Useful to preprocess, predict, and post-process
• Available for real-time prediction and batch transform

Demo:
Inference with scikit-learn and linear learner
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/scikit_learn_inference_pipeline/

Amazon SageMaker
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Build Train Deploy

Selected Amazon SageMaker customers

Resources
http://aws.amazon.com/free
https://ml.aws
https://aws.amazon.com/sagemaker
https://github.com/aws/sagemaker-python-sdk
https://github.com/aws/sagemaker-spark
https://github.com/awslabs/amazon-sagemaker-examples
https://gitlab.com/juliensimon/ent321 SageMaker workshop at AWS re:Invent 2018
https://medium.com/@julsimon
https://gitlab.com/juliensimon/dlnotebooks

Thank you!
Julien Simon
Principal Technical Evangelist, AI and Machine Learning
@julsimon

AWS SageMaker: Deploy ML Models from Notebook to Production

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à AWS SageMaker: Deploy ML Models from Notebook to Production

Similaire à AWS SageMaker: Deploy ML Models from Notebook to Production (20)

Plus de Codiax

Plus de Codiax (20)

Dernier

Dernier (20)

AWS SageMaker: Deploy ML Models from Notebook to Production