3. Docker @ OVH
● 8 clusters
○ Internal production : 5
○ External beta/gamma : 2
○ Preproduction : 1
● 2500 running containers
● 800 Hosts
● 3000 cpu cores
● 12TB RAM
● 2 docker registry (internal & external)
● 100s of deployment per day
● 5 people
4. The beginning : 1 - 20 containers
Everything fit on 1 host
+ Easy to deploy docker run
+ Easy to build docker build
+ Easy to debug docker exec/logs/stats
- No high availability
- Not scalable
5. Scaling up : +20 containers, 1 team
You need an orchestrator :
● Docker swarm (Docker inc.)
● Mesos & Marathon (Apache Software Foundation)
● Kubernetes (Google then CNCF)
Don’t fit on 1 host ?
Need high availability ?
6. Mesos & marathon in 60 seconds
Linux kernel : provide abstraction of
hardware
Linux kernel
Your application
Linux scheduler
Scheduler : say which application run
when with which resources
7. Mesos & marathon in 60 seconds
Mesos : provide abstraction of
multiple hosts
Mesos
Your (Docker)
application
Framework
Framework (Marathon) : say which
application run when with which resources
8. Needs :
● Isolate team/client
● Be cost efficient, mutualize what we can
Scaling up : 20+ containers, 2+ team
More team !
9. Scaling up : 20+ containers, 2+ team
Marathon : no multitenancy out of the box
The solutions we try :
1) One marathon per team
2) One marathon with logical isolation (middleware)
3) One mesos & marathon per team
10. Multitenancy : One marathon per team
+ Good isolation
+ Mesos mutualization
+ Fair blast radius
- Does not scale well
Mesos
Marathon B
Marathon CMarathon A
11. Multitenancy : One marathon and logical isolation
+ Scale
+ Give us control on user input
- Less isolation
- Lots of maintenance
- Hard to keep up to date with
marathon API
- Big blast radius
Mesos
Marathon
Middleware
12. Multitenancy : One marathon & mesos cluster per team
+ Scale even better
+ Little blast radius
+ Best isolation
+ Modular
- Less/no mutualisation
- Costly
- Lot of parts to deploy and
maintain
Mesos B
Marathon B
Mesos A
Marathon A
13. Deploy mesos and marathon
using mesos and marathon
Lot of parts to deploy and
maintain ?
we already have a tool for that :
Mesos and Marathon !
Mesos B
Marathon B
Mesos A
Marathon A
Mesos
Admin
Marathon
Admin