In this workshop, we step through the process of deploying and hosting machine learning (ML) models with AWS Lambda and get on-demand inferences. Given a demonstrative dataset, we build and train a simple ML classification model with Amazon SageMaker. Then, we host this model in an AWS Lambda function and expose an inference endpoint through Amazon API Gateway. Finally, we build a pipeline for automating model deployment to Lambda leveraging AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Your Own ML Application with
AWS Lambda & Amazon SageMaker
S R V 4 0 4
Diego Natali
Solutions Architect
Amazon Web Services EMEA
Giuseppe Angelo Porcelli
Sr. Solutions Architect
Amazon Web Services EMEA

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Repeats
Wednesday, 28
SRV404
Building Your Own ML Application with AWS Lambda & Amazon SageMaker
12:15 – PM | Bellagio, Level 1, Grand Ballroom 9

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE MACHINE LEARNING PROCESS
Business Problem -
ML problem
framing

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE MACHINE LEARNING PROCESS
Business Problem -
ML problem
framing
Data Collection
Data Integration
Data Preparation &
Cleaning

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE MACHINE LEARNING PROCESS
Data Visualization
& Analysis
Business Problem -
ML problem
framing
Data Collection
Data Integration
Data Preparation &
Cleaning
Feature
Engineering
Model Training &
Parameter Tuning
Model Evaluation
Experiment, Train, Tune and Evaluate
• Setup and manage Notebook
Environments
• Setup and manage Training Clusters
• Write Data Connectors
• Scale ML algorithms to large datasets
• Distribute ML training algorithm to
multiple machines
• Secure Model artifacts

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE MACHINE LEARNING PROCESS
Data Visualization
& Analysis
Business Problem -
ML problem
framing
Data Collection
Data Integration
Data Preparation &
Cleaning
Feature
Engineering
Are
Business
goals met?
Monitoring &
Debugging
- Predictions
Yes
Re-training
Model Training &
Parameter Tuning
Model Evaluation Model Deployment
Deploy, Monitor and Debug
• Setup and manage Model Inference
Clusters
• Manage and Auto-Scale Model
Inference APIs
• Monitor and Debug Model Predictions
• Models versioning and performance
tracking
• Automate New Model version
promotion to production (A/B testing)

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE MACHINE LEARNING PROCESS
Data Visualization
& Analysis
Business Problem -
ML problem
framing
Data Collection
Data Integration
Data Preparation &
Cleaning
Feature
Engineering
Are
Business
goals met?
Monitoring &
Debugging
- Predictions
YesNo
DataAugmentation
Feature
Augmentation
Re-training
Model Training &
Parameter Tuning
Model Evaluation Model Deployment

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WHERE SHOULD WE SPEND TIME?
BUI L D T R AI N D E P L O Y
T U NE

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER
End-to-end encryption with KMS
End-to-end VPC support
Compliance and audit capabilities
Metadata and experiment management capabilities
Pay as you go
Fully managed
hosting with auto-
scaling
One-click
deployment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training
BUI L D T R AI N & T UNE D E P L O Y
Hyperparameter
optimization

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EXECUTING INFERENCES
Amazon SageMaker Deployment AWS Lambda
Real-time Sync Use managed real-time inference cluster On-demand request-based model
Real-time Async
Use Amazon Kinesis and AWS Lambda to
invoke the real-time inference endpoint
Use Amazon Kinesis and AWS Lambda to
host the model and execute inferences in
Lambda
Batch Use batch transform Need to implement batch processing
Edge (GreenGrass)
Use built-in AWS Greengrass model
hosting
Deploy a Lambda function on AWS
Greengrass to execute inferences

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
USING AWS LAMBDA
PROS
Develop Modern Applications serverless
Charged per request and execution duration
Effortless, limitless scalability, high availability, fault tolerance,
etc.
Lower the cost of inferences
No infrastructure to maintain
1 million free executions
Event-based model
Example: 100M requests for about $54/month
CONS
GPU not yet available
512 MB /tmp limit
250 MB limit for uncompressed size of code/dependencies
Need to create your deployment package and pipeline to
automate the process
Cold start (~5s TF, ~2s MXNet)
Infrastructure to mantain if not using Amazon SageMaker for
training
Need to include and manage all dependencies
Need to create your own API for serving

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AMAZON SAGEMAKER vs AWS LAMBDA
Amazon SageMaker Deployment AWS Lambda
Workload
Well suited for constant and predictable
workloads, with regular and frequent traffic
Well suited for variable or unpredictable
workloads, with intermittent and spiky traffic
Scaling
Configure auto-scaling on the real-time
endpoint
Automatic scaling
Hardware GPU and CPU instances CPU only
Model architecture Any complexity [CNNs, RNNs, etc.] Simple models
A/B Testing Yes Yes
Deployment Fast, one-click deployment
Need to build package and create inference API
from scratch
Monitoring and Debugging Amazon CloudWatch / CloudWatch Logs Amazon CloudWatch / CloudWatch Logs

14. IT’S A MATTER OF CHOICE

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
THE DATASET
SMS Spam Collection Data Set
• A public set of 5574 SMS labeled messages that have been collected for mobile phone spam
research
• 747 SPAM, 4827 non SPAM
https://archive.ics.uci.edu/ml/datasets/sms+spam+collection
ham Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there
got amore wat...
ham Ok lar... Joking wif u oni...
spam Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive
entry question(std txt rate)T&C's apply 08452810075over18’s
ham U dun say so early hor... U c already then say...
ham Nah I don't think he goes to usf, he lives around here though
spam FreeMsg Hey there darling it's been 3 week's now and no word back! I'd like some fun you up for
it still? Tb ok! XxX std chgs to send, £1.50 to rcv
ham Even my brother is not like to speak with me. They treat me like aids patent.
ham As per your request 'Melle Melle (Oru Minnaminunginte Nurungu Vettam)' has been set as your
callertune for all Callers. Press *9 to copy your friends Callertune

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
WORKFLOW
BUI L D T R AI N & T UNE D E P L O Y

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TRAINING THE MODEL
SMS Spam
Training Data
SMS Spam
model artifacts
Amazon SageMaker
Training
Amazon ECR
Training Script
SageMaker MXNet
container

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DEPLOYING TO LAMBDA
Get model artifacts from S3
Run an Amazon Linux EC2
Instance or Cloud9
environment
Download pre-built MXNet
Lambda packages from
GitHub in a folder
Write Lambda handler,
package and save Lambda
deployment package to S3
Copy model artifacts in the
folder
https://docs.aws.amazon.com/sagemaker/latest/dg/API_ModelArtifacts.html
https://docs.aws.amazon.com/cloud9/latest/user-guide/welcome.html
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html
https://github.com/diegonat/smwshop/
https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-
how-to-create-deployment-package.html

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AUTOMATING DEPLOYMENT
AWS
CodePipeline
bucket
with
objects
event
(event-
based)
AWS
CloudTrail
Lambda
function
bucket
with
objects
event
(event-
based)
AWS
CloudTrail
Amazon
SageMaker
AWS
CodeBuild
template
stack
change set
bucket
with
objects
Amazon API
Gateway*
Lambda
function
AWS
CodeDeploy
Hook
Lambda
function
Checker

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.