Contenu connexe Similaire à Amazon SageMaker (December 2018) (20) Plus de Julien SIMON (20) Amazon SageMaker (December 2018)1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Julien Simon
Principal Technical Evangelist, AI & Machine Learning,
AWS
@julsimon
From Notebook to Production
with Amazon SageMaker
2. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Put Machine Learning in the hands
of every developer and data scientist
Our mission
3. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Application
Services
Platform
Services
Frameworks
& Infrastructure
API-driven services:Vision, Language & Speech Services, Chatbots
AWS ML Stack
Deploy machine learning models with high-performance machine learning algorithms,
broad framework support, and one-click training, tuning, and inference.
Develop sophisticated models with any framework, create managed, auto-scaling
clusters of GPUs for large scale training, or run prediction
on trained models.
4. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Application
Services
Platform
Services
Frameworks
& Infrastructure
API-driven services:Vision, Language & Speech Services, Chatbots
AWS ML Stack
Deploy machine learning models with high-performance machine learning algorithms,
broad framework support, and one-click training, tuning, and inference.
Develop sophisticated models with any framework, create managed, auto-scaling
clusters of GPUs for large scale training, or run prediction
on trained models.
5. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training
Predictions
6. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon SageMaker
Pre-built
notebooks for
common
problems
K-MeansClustering
Principal Component Analysis
Neural TopicModelling
FactorizationMachines
Linear Learner
XGBoost
Latent Dirichlet Allocation
ImageClassification
Seq2Seq,
And more!
ALGORITHMS
Apache MXNet, Chainer
TensorFlow, PyTorch, scikit-learn
FRAMEWORKS Set up and manage
environments for training
Train and tune
model (trial and
error)
Deploy model
in production
Scale and manage the
production environment
Built-in, high-
performance
algorithms
Build
Git integration
Elastic inference
7. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Git integration
8. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Elastic Inference
https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/
9. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon SageMaker
Pre-built
notebooks for
common
problems
K-MeansClustering
Principal Component Analysis
Neural TopicModelling
FactorizationMachines
Linear Learner
XGBoost
Latent Dirichlet Allocation
ImageClassification
Seq2Seq,
And more!
ALGORITHMS
Apache MXNet, Chainer
TensorFlow, PyTorch, scikit-learn
FRAMEWORKS Set up and manage
environments for training
Train and tune
model (trial and
error)
Deploy model
in production
Scale and manage the
production environment
Built-in, high-
performance
algorithms
Build
New built-in algorithms
scikit-learn environment
Model marketplace
Search
10. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Search training jobs
11. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Machine Learning Marketplace
12. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon SageMaker
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Train
Deploy model
in production
Scale and manage the
production
environment
P3DN, C5N
TensorFlow on 256 GPUs
Resume HPO tuning job
Build
13. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon SageMaker
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Deploy
Model compilation
Elastic inference
Inference pipelines
TrainBuild
15. TheAmazon SageMakerAPI
• Python SDK orchestrating all Amazon SageMaker activity
• High-level objects for algorithm selection, training, deploying,
automatic model tuning, etc.
• Spark SDK (Python & Scala)
• AWS CLI: ‘aws sagemaker’
• AWS SDK: boto3, etc.
16. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
GroundTruth
Client application
Inference code
Training code
Inference requestInference
response
Inference Endpoint
17. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Training code
Factorization Machines
Linear Learner
PrincipalComponent Analysis
K-Means Clustering
XGBoost
And more
Built-inAlgorithms BringYour Own ContainerBringYour Own Script
Model options
18. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Built-in algorithms
orange: supervised, yellow: unsupervised
Linear Learner: regression, classification Image Classification: Deep Learning (ResNet)
Factorization Machines: regression, classification,
recommendation
Object Detection (SSD): Deep Learning
(VGG or ResNet)
K-Nearest Neighbors: non-parametric regression
and classification
NeuralTopic Model: topic modeling
XGBoost: regression, classification, ranking
https://github.com/dmlc/xgboost
Latent DirichletAllocation: topic modeling (mostly)
K-Means: clustering BlazingText:GPU-basedWord2Vec,
and text classification
Principal ComponentAnalysis: dimensionality
reduction
Sequence to Sequence: machine translation, speech
speech to text and more
RandomCut Forest: anomaly detection DeepAR: time-series forecasting (RNN)
Object2Vec: general-purpose embedding IP Insights: usage patterns for IP addresses
Semantic Segmentation: Deep Learning
19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Blazing Text
https://dl.acm.org/citation.cfm?id=3146354
20. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Demo:
Text Classification with BlazingText
https://github.com/awslabs/amazon-sagemaker-
examples/tree/master/introduction_to_amazon_algorithms/blazingtext_text_classification_dbpedia
21. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
OptimizingTensorFlow
https://aws.amazon.com/blogs/machine-learning/faster-training-with-
optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3-instances/
(March 2018)
Training a ResNet-50 benchmark with
the synthetic ImageNet dataset using
our optimized build ofTensorFlow
1.11 on a c5.18xlarge instance type is
11x faster than training on the stock
binaries.
https://aws.amazon.com/about-aws/whats-
new/2018/10/chainer4-4_theano_1-0-
2_launch_deep_learning_ami/ (October 2018)
22. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Demo:
Elastic Inference withTensorFlow
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/tensorflow_iris_dnn_classifier_using_estimators/
24. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Automatic Model Tuning
Finding the optimal set of hyper parameters
1. Manual Search (”I know what I’m doing”)
2. Grid Search (“X marks the spot”)
• Typically training hundreds of models
• Slow and expensive
3. Random Search (“Spray and pray”)
• Works better and faster than Grid Search
• But… but… but… it’s random!
4. HPO: use Machine Learning
• Training fewer models
• Gaussian Process Regression and Bayesian Optimization
• You can now resume from a previous tuning job
25. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Demo: HPO withTensorFlow
https://github.com/awslabs/amazon-sagemaker-
examples/tree/master/hyperparameter_tuning/tensorflow_mnist
CONV1
POOL1
CONV2
POOL2
DENSE1
DROPOUT1
OUTPUT
Class
probabilities
INPUT
27. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Optimizing for the underlying hardware
https://aws.amazon.com/blogs/aws/amazon-sagemaker-neo-train-your-machine-learning-models-once-run-them-anywhere/
• Train once, run anywhere
• Frameworks and algorithms
• TensorFlow, Apache MXNet, PyTorch, ONNX, and XGBoost
• Hardware architectures
• ARM, Intel, and NVIDIA starting today
• Cadence, Qualcomm, and Xilinx hardware coming soon
• Amazon SageMaker Neo will be released as open source enabling hardware
vendors to customize it for their processors and devices.
28. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Demo:
Compiling ResNet-50 for the Raspberry Pi
Configure the compilation job
{
"RoleArn":$ROLE_ARN,
"InputConfig": {
"S3Uri":"s3://jsimon-neo/model.tar.gz",
"DataInputConfig": "{"data": [1, 3, 224, 224]}",
"Framework": "MXNET"
},
"OutputConfig": {
"S3OutputLocation": "s3://jsimon-neo/",
"TargetDevice": "rasp3b"
},
"StoppingCondition": {
"MaxRuntimeInSeconds": 300
}
}
Compile the model
$ aws sagemaker create-compilation-job
--cli-input-json file://config.json
--compilation-job-name resnet50-mxnet-pi
$ aws s3 cp s3://jsimon-neo/model-
rasp3b.tar.gz .
$ gtar tfz model-rasp3b.tar.gz
compiled.params
compiled_model.json
compiled.so
Predict with the compiled model
from dlr import DLRModel
model = DLRModel('resnet50', input_shape,
output_shape, device)
out = model.run(input_data)
30. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Inference Pipelines
• Linear sequence of 2-5 containers that process inference requests
• Feature engineering with scikit-learn or SparkML (on AWS Glue or Amazon EMR)
• Predict with built-in or custom containers
• The sequence is deployed as a a single model
• Useful to preprocess, predict, and post-process
• Available for real-time prediction and batch transform
31. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Demo:
Inference with scikit-learn and linear learner
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python-
sdk/scikit_learn_inference_pipeline/
32. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon SageMaker
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common
problems
Built-in, high-
performance
algorithms
One-click
training
Hyperparameter
optimization
Build Train Deploy
33. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Selected Amazon SageMaker customers
35. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Resources
http://aws.amazon.com/free
https://ml.aws
https://aws.amazon.com/sagemaker
https://github.com/aws/sagemaker-python-sdk
https://github.com/aws/sagemaker-spark
https://github.com/awslabs/amazon-sagemaker-examples
https://gitlab.com/juliensimon/ent321 SageMaker workshop at AWS re:Invent 2018
https://medium.com/@julsimon
https://gitlab.com/juliensimon/dlnotebooks
Notes de l'éditeur SageMaker makes it easy to build ML models and get them ready for training by providing everything you need to quickly connect to your training data, and to select and optimize the best algorithm and framework for your application. Amazon SageMaker includes hosted Jupyter notebooks that make it is easy to explore and visualize your training data stored in Amazon S3. You can connect directly to data in S3, or use AWS Glue to move data from Amazon RDS, Amazon DynamoDB, and Amazon Redshift into S3 for analysis in your notebook.
To help you select your algorithm, Amazon SageMaker includes the 10 most common machine learning algorithms which have been pre-installed and optimized to deliver up to 10 times the performance you’ll find running these algorithms anywhere else. Amazon SageMaker also comes pre-configured to run TensorFlow and Apache MXNet, two of the most popular open source frameworks, or you have the option of using your own framework.
SageMaker makes it easy to build ML models and get them ready for training by providing everything you need to quickly connect to your training data, and to select and optimize the best algorithm and framework for your application. Amazon SageMaker includes hosted Jupyter notebooks that make it is easy to explore and visualize your training data stored in Amazon S3. You can connect directly to data in S3, or use AWS Glue to move data from Amazon RDS, Amazon DynamoDB, and Amazon Redshift into S3 for analysis in your notebook.
To help you select your algorithm, Amazon SageMaker includes the 10 most common machine learning algorithms which have been pre-installed and optimized to deliver up to 10 times the performance you’ll find running these algorithms anywhere else. Amazon SageMaker also comes pre-configured to run TensorFlow and Apache MXNet, two of the most popular open source frameworks, or you have the option of using your own framework.
You can begin training your model with a single click in the Amazon SageMaker console. The service manages all of the underlying infrastructure for you and can easily scale to train models at petabyte scale. To make the training process even faster and easier, Amazon SageMaker can automatically tune your model to achieve the highest possible accuracy.
Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results.
For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.
Seq2Seq: used by Amazon Translate and AWS Sockeye
LDA: used by Amazon Comprehend Once your model is trained and tuned, SageMaker makes it easy to deploy in production so you can start generating predictions on new data (a process called inference). Amazon SageMaker deploys your model on an auto-scaling cluster of Amazon EC2 instances that are spread across multiple availability zones to deliver both high performance and high availability. It also includes built-in A/B testing capabilities to help you test your model and experiment with different versions to achieve the best results.
For maximum versatility, we designed Amazon SageMaker in three modules – Build, Train, and Deploy – that can be used together or independently as part of any existing ML workflow you might already have in place.