Set up A/B testing for MLOps with Kubernetes, Seldon, and Helm.

1. 1
A/B Testing
in MLOps
Sadik Bakiu, Data Max

2. 2
Agenda
What is A/B testing
1
What is Helm
3
What is seldon
4
Hands On
5
What is Kubernetes
2

3. 3
Sadik Bakiu
S A D I K B A K I U
Co-Founder & Co-CEO, Data Max
Graduated 2016, MSc, RWTH Aachen University
Working with data for 10+ years now
Find me in LinkedIn: linkedin.com/in/sbakiu
Email: sadik@data-max.io

4. What is A/B
Testing
1

5. 6
A/B Testing
A / B T E S T I N G
Determine which variant performs better
Used in advertisement a lot
Important: Objectively measure the results
Pick a metric
Compares two variants of one variable

6. What is
Kubernetes
2

7. 8
Kubernetes
K U B E R N E T E S
Controller
Pods
Containers
Services
Container Orchestrator

8. What is Helm
3

9. 10
Helm
H E L M
Like Pip
Has Charts
Charts have versions
Package manager for Kubernetes

10. What is
Seldon-core
4

11. 12
Seldon-core
S E L D O N C O R E
Deploy and monitor ML models on Kubernetes
Create containerized microservices
Out-of-the-box inference server
Metrics and monitoring
Python app server tailored to ML

12. Bonus:
Streamlit
5

13. 14
Streamlit
S T R E A M L I T
All in Python
Easy to use
Build data apps

14. What will we
be doing
6

15. 20
Aim

16. 21
Where are we now

17. 22
Where are we now

18. 23
Where are we now

19. 24
Where are we now

20. 25

21. 26
A/B testing tests versions of a variable
MLOps tooling is mature now
With microservice architecture it is easy to start
A/B testing has limits
Summary

22. THANK YOU!
www.data-max.io
sadik@data-max.io