Introduction to Machine Learning Operations (MLOps) - Build pipelines with Tensor Flow Extended & Kubeflow

1. Prof. Dr. Jan Kirenz
Machine Learning Operations (MLOps)
Usage of Pipelines in the ML Lifecycle with
Tensor Flow Extended (TFX) and Kubeflow
Prof. Dr. Jan Kirenz
HdM Stuttgart

2. Prof. Dr. Jan Kirenz
80-85% PoC Factory
The Proof of Concept Factory
Most companies...
● … conduct AI experiments and pilots
but achieve a low scaling success
rate
● … have significant under investment,
yielding low returns
Source: Accenture (2019) https://www.accenture.com/us-en/insights/artificial-intelligence/ai-investments

3. Prof. Dr. Jan Kirenz
https://www.gartner.com/smarterwithgartner/gartner-top-10-data-and-analytics-trends-for-2021/
Scalable AI

4. Prof. Dr. Jan Kirenz
ML Project Code
The problem with scaling AI: ML code is only a
fraction of a production-ready ML project code
ML
Code 5-10%

5. Prof. Dr. Jan Kirenz
Monitoring
Hidden technical debt in machine learning systems
Sculley, D. et al. (2015). Hidden technical debt in machine learning systems. Advances in neural information processing systems, 28, pp. 2503-2511
Data Collection
Configuration
Feature Engineering
Data
Verification
Metadata Management
Model Analysis
Serving
Infra-
structure
Automation
Process Management
Machine Resource
Management
Testing and Debugging
ML
Code

6. Prof. Dr. Jan Kirenz
Machine learning operations (MLOps)
● ML Engineering culture and practice that
aims at unifying ML System development
(Dev) and ML system operations (Ops)
● Tools and principles to support workflow
standardization and automation through
the ML system lifecycle (e.g. with pipelines)

7. Prof. Dr. Jan Kirenz
Prof. Dr. Jan Kirenz
Machine learning
lifecycle

8. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Lifecycle
of an ML System
Plan | Data | Model | Deployment

9. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Lifecycle
of an ML System
Plan | Data | Model | Deployment

10. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Data splitting
Anomaly
detection
Data
preprocessing
Lifecycle
of an ML System
Plan | Data | Model | Deployment

11. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Evaluate
model
Model Training
& tuning
Select
algorithm
Data splitting
Anomaly
detection
Data
preprocessing
Lifecycle
of an ML System
Plan | Data | Model | Deployment

12. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Retrain
triggers
Evaluate
model
Model Training
& tuning
Monitor model
Select
algorithm
Data splitting
Anomaly
detection
Data
preprocessing
Lifecycle
of an ML System
Plan | Data | Model | Deployment

13. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Retrain
triggers
Evaluate
model
Model Training
& tuning
Monitor model
Select
algorithm
Data splitting
Anomaly
detection
Data
preprocessing
Lifecycle
of an ML System
Plan | Data | Model | Deployment
Common issues which lead to a PoC to production gap
● Lack of reuse and duplication
● Inconsistency (data, code, models)
● Manual and slow transition from PoC to production

14. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Retrain
triggers
Evaluate
model
Model Training
& tuning
Monitor model
Select
algorithm
Data splitting
Anomaly
detection
Data
preprocessing
Model
management
Model registry
Data and feature
management
Feature store
Pipeline
management
Pipeline orchestration
Metadata
management
Metadata store
Lifecycle
of an ML System
Plan | Data | Model | Deployment
MLOps components

15. Prof. Dr. Jan Kirenz
What is a pipeline?
● Description of an ML workflow
● A pipeline component is a self-contained
set of user code that performs one step in
the pipeline
● Includes the definition of the configuration
and inputs required to run the pipeline (e.g.
model hyperparameters)
… do this
… than that
Start
...
… the end
The workflow is
also called directed
acyclic graph (DAG)
This is a component
Complete workflow of the ML
system lifecycle

16. Prof. Dr. Jan Kirenz
Source: Baer & Ngahane (2019)
… do this
… than that
Start
… the end

17. Prof. Dr. Jan Kirenz
TensorFlow Extended (TFX)
● Google-production-scale machine learning
(ML) platform based on TensorFlow
● Portable to multiple environments (Azure,
AWS, Google Cloud, IBM, ...)
● Python based toolkit; can be used with
notebooks
● Helps you orchestrate your ML process:
Apache Airflow, Apache Beam or Kubeflow
pipelines
Source: TensorFlow (2021)

18. Prof. Dr. Jan Kirenz
TFX 1.0 (19.05.21)
● Enterprise-grade support
● Security patches and select bug fixes for
up to three years
● Guaranteed API & Artifact backward
compatibility
Source: Google (2021)

19. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
KerasTuner
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store: ML Metadata (MLMD)
TFX Options for Pipeline
Orchestration

20. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
KerasTuner
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration
Data
preprocessing

21. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
KerasTuner
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration

22. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration

23. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration

24. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
KerasTuner
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration

25. Prof. Dr. Jan Kirenz
Tuner
Evaluator
InfraValidator
ExampleGen
StatisticsGen
SchemaGen
Example
Validator
Transform
Trainer
Pusher
HUB / JS / LITE / SERVING
Model Server
KerasTuner
BulkInferrer
Metadata Store (ML Metadata)
TF
Data
Validation
(TFDV)
TFT
TF
TFX Options for Pipeline
Orchestration
TensorFlow
Model Analysis
(TFMA)
TensorFlow Lite is a set of tools that enables on-device
machine learning by helping developers run their models
on mobile, embedded, and IoT devices.

26. Prof. Dr. Jan Kirenz
Plan
Model
Deployment Data
Identify use
case
Frame
problem
Identify
variables
Define metrics
Business Analyst
Data
Engineer
Software
Developer
Data Scientist
Data ingestion
Analyze &
clean data
Define schema
Feature
engineering
Validate model
Deploy model
Serve model
Retrain model
Evaluate
model
Model Training
& tuning
Monitor model
ExampleGen
Select
algorithm
StatisticsGen
SchemaGen
Example
Validator
Transform
Data splitting
Trainer
Tuner
Evaluator
InfraValidator
Anomaly
detection
Pusher
HUB / JS / LITE / SERVING
Model Server
TF
Data
Validation
(TFDV)
TFT
TF
KerasTuner
TensorFlow
Model Analysis
(TFMA)
BulkInferrer
Data
preprocessing
Metadata Store (ML Metadata)
TFX Options for Pipeline
Orchestration
Production phase:
automate the execution
of the ML pipeline based
on a schedule or certain
triggering conditions.
Development phase: run the ML experiment, instead of
manually executing each step.
Data preparation
phase:
automatically
ingest, validate
and transform
data and provide
features to models

27. Prof. Dr. Jan Kirenz
Prof. Dr. Jan Kirenz
Pipeline orchestration

28. Prof. Dr. Jan Kirenz
TFX & Apache Airflow
● Programmatically author, schedule and
monitor workflows with Python code.
● User interface to visualize pipelines
running in production, monitor progress,
and troubleshoot issues.

29. Prof. Dr. Jan Kirenz
TFX & Apache Beam
● Provides a framework for running batch
and streaming data processing jobs that
run on a variety of runners (Spark, Flink, ...).
● Beam provides an abstraction layer which
enables TFX to run on any supported
runner without code modifications
● TFX only uses the Beam Python API

30. Prof. Dr. Jan Kirenz
TFX & Kubeflow pipelines
The Kubeflow Pipelines platform consists of:
● An engine for scheduling multi-step ML
workflows (using Kubernetes).
● User interface (UI) for managing and
tracking experiments, jobs, and runs.
● Python SDK for defining and manipulating
pipelines and components.
● Notebooks for interacting with the system
using the SDK
Kubeflow Pipelines is available as a core component of Kubeflow or as
a standalone installation.

31. Prof. Dr. Jan Kirenz

32. Prof. Dr. Jan Kirenz

33. Prof. Dr. Jan Kirenz

34. Prof. Dr. Jan Kirenz
KubeFlow

35. Prof. Dr. Jan Kirenz
ML toolkit for Kubernetes

36. Prof. Dr. Jan Kirenz
Google’s Vertex AI
Launched in May 2021

37. Prof. Dr. Jan Kirenz
ML Pipelines | wrap-up
Source: TensorFlow (2021)
By using a ML pipeline, you can:
● Automate your ML process, which lets you
regularly retrain, evaluate, and deploy your
model.
● Utilize distributed compute resources for
processing large datasets and workloads.
● Increase the velocity of experimentation by
running a pipeline with different sets of
hyperparameters.
To learn more visit the following tutorials @:
https://kirenz.github.io/
MLOps tutorials on how to:
● Install TF and TFX
● Build your first TFX pipeline
● Install Kubeflow
● Build your first Kubeflow pipeline
Jan Kirenz
www.kirenz.com