In this session, we will take a deep-dive into the DevOps process that comes with Azure Machine Learning service, a cloud service that you can use to track as you build, train, deploy and manage models. We zoom into how the data science process can be made traceable and deploy the model with Azure DevOps to a Kubernetes cluster. At the end of this session, you will have a good grasp of the technological building blocks of Azure machine learning services and can bring a machine learning project safely into production.

1. DevOps and
Machine Learning
Henk Boelman
Cloud Advocate @ Microsoft
@hboelman
github.com/hnky
henkboelman.com

2. Machine Learning
Ability to learn without being
explicitly programmed.

3. Programming
Algorithm
Data
Answers

4. Machine Learning
Algorithm
Data
Answers

5. Machine Learning
Model
Data
Answers

6. Machine Learning
Model
Data
Answers

7. Machine Learning
Predictions
Data
Model
Data
Answers

8. Sophisticated pretrained models
To simplify solution development
Azure
Databricks
Machine Learning
VMs
Popular frameworks
To build advanced deep learning solutions TensorFlow Keras
Pytorch Onnx
Azure
Machine Learning
Language
Speech
…
Azure
Search
Vision
On-premises Cloud Edge
Productive services
To empower data science and development teams
Powerful infrastructure
To accelerate deep learning
Flexible deployment
To deploy and manage models on intelligent cloud and edge
Machine Learning on Azure
Cognitive Services

10. DevOps is the union of people,
process, and products to
enable continuous delivery of
value to your end users.
“
”

11. Developers
Data Scientists
Operations

12. Ask a sharp question Collect the data Prepare the data
Select the algorithm Train the model Use the answer
The data science process

13. Azure Machine Learning
A fully-managed cloud service that enables you to easily build, deploy, and share
predictive analytics solutions.

14. What is Azure Machine Learning?
Set of Azure
Cloud Services
Python
SDK
Prepare Data
Build Models
Train Models
Manage Models
Track
Experiments
Deploy Models
That enables
you to:

15. Prepare
your environment
Experiment
with your model & data
Deploy
Your model into production

16. Step 1: Prepare your environment

18. Datasets – registered, known data sets
Experiments – Training runs
Pipelines – Training workflows
Models – Registered, versioned models
Endpoints:
Real-time Endpoints – Deployed model endpoints
Pipeline Endpoints – Training workflow endpoints
Compute – Managed compute
Environments – defined training and inference environments
Datastores – Connections to data
Azure Machine Learning

19. Demo: Azure Machine Learning

20. Step 2: Create a ML pipeline to deliver a model

21. Pipelines
Azure ML Service Pipelines Azure Pipelines

22. Azure Machine Learning Pipelines
Workflows of steps that can
use Data Sources, Datasets
and Compute targets
Unattended runs
Reusability
Tracking and versioning

23. Azure Pipelines
Orchestration for Continuous Integration
and Continuous Delivery
Gates, tasks and processes for quality
Integration with other services
Trigger on code and non-code events

24. Create a pipeline step
Input Output
Runs a script
on a Compute Target
in a Docker container.
Parameters

25. Create a pipeline
Dataset of
Simpsons
Images
Prepare data
Train the Model
with PyTorch
Processed
dataset
model Register the
model
Blob Storage
Account
Model
Management

26. Submit the pipeline to the cluster

27. Demo: Create an Azure ML Pipeline

28. Jupyter Notebook
Compute Target
Docker Image
Data store
1. Snapshot folder and
send to experiment
2. create docker image
3. Deploy docker
and snapshot to
compute
4. Mount datastore
to compute
6. Stream
stdout,
logs,
metrics 5. Launch the script
7. Copy over
outputs
Experiment

29. Azure Machine Learning Service Pipelines
A repeatable process to deliver a ML model

31. Continuous Integration

32. Code and comments only (not Jupyter output)
Plus every part of the pipeline
And Infrastructure and dependencies
And maybe a subset of data
Source Control

33. Everything should be in source control!
Except your training data
which should be a known, shared data source

34. Triggered on code change
Refresh and execute AML Pipeline
Code quality, linting, and unit testing
Pull request process
Continuous Integration

35. Demo: Setup Azure Pipeline for AMLS
Pipeline

36. Step 3: Deploy your model

37. Trigger on model registration
Deploy to test and staging environments
Run integration and load tests
Control: rollout, feature flags, A/B testing
Continuous Delivery

38. Control model rollout!
The same way you do with other software

39. AMLS to deploy
The Model Score.py Environment file
Docker Image

40. Demo: Deploy with Azure DevOps

41. Complete Pipeline

42. @hboelman
github.com/hnky
henkboelman.com
Thank you!
Read more on: henkboelman.com