This document discusses Arkena's migration from using Heroku to host their production video-on-demand platform to using Kubernetes. It outlines the key features they needed like high availability, scalability, log management, monitoring and network performance. It then describes how Kubernetes was able to provide these features through tools like replication controllers, autoscaling, daemonsets, and Influxdb/Heapster. The document details their migration steps and issues encountered. It concludes that the migration to Kubernetes on their own hardware was successful and reduced infrastructure costs.
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Arkena from heroku_to_k8s
1. From Heroku to Kubernetes :
A production use case
Antoine Legrand
Smaine Kahlouch
Arkena 2016
2. 2
AGENDA
• Presentation of Arkena
• Former Hosting plateform on Heroku, why to leave ?
• Is kubernetes able to provide similar features as
Heroku ?
• Migration steps
• Project lessons
• Deployment tools
3. WHO WE ARE
YOUR TRUSTED MEDIA PARTNER
3
A TDF Group
Business Unit
• 16 POPs CDN
• 1 Tbps connectivity
• 400 live radios & 360 live TVs
• 630 hours of On Demand video processed daily
• United Kingdom
• Norway
• USA
• Finland
• Denmark
• Poland
• France
• Spain
• Sweden
13 Offices in 9 Countries
A team of 400 employees
At a glance
4. ARKENA OTT / CDN
A UNIQUE EUROPEAN PRESENCE, ESPECIALLY FRANCE AND NORDICS
5
Content management
User accounts management
Content processing & protection
5. 6
HEROKU KEY FEATURES
• No need to worry about the underlying infrastructure
• Easy to scale apps
• Full application stacks are easy to install with Heroku addons
• Easy to iterate from dev to production
6. 7
CONTEXT
• Our VOD plateform gets increasingly successful with our
clients
• The Heroku Costs became significant
• Are we able to keep the same features as Heroku on Bare-
Metal ?
10. 10
HIGH AVAILABILITY & SCALABILITY
• Pods and Replication controllers
• Quickly increase the number of replicas
• Horizontal pod autoscaling
• Multi datacenters architecture
• Database replication
16. 16
ARKENA'S KUBERNETES CLUSTER
• 2 distinct kubernetes clusters
(1 per datacenter)
• 12 nodes (Total: 288 cpus
and 1152G of memory)
• Haproxy loadbalancers
• Multi-tenant database server
• Network plugin: Calico
17. 19
MIGRATION STEPS FROM HEROKU
1. Deploy application stack on kubernetes
2. Copy Heroku's database on Arkena's db servers
3. Configure Heroku app to use Arkena's database server
4. QA tests (rollback if any problem)
5. Update the DNS
6. Keep both platforms running during a validation period
18. 23
PROJECT PROGRESS ISSUES
• It's important to involve all teams.
• Not clear recomandations on High availability and
upgrade procedure
• Ecosystem tools are still young
19. 20
KUBESPRAY
• Open Source project to deploy a kubernetes cluster
(Baremetal, AWS and GCE)
• Current supported network plugins : Weave, Flannel &
Calico.
Coming soon : Romana and Opencontrail
• Support most popular Linux distributions (CoreOS,
Centos/RHEL, Debian/Ubuntu)
• Continuous Integration tests.
About 60 3-nodes clusters per day
20. 21
KPM APPS DEPLOYMENT
• Deploy full application stack
• Dependency management
• Reproducible and versionned deployment
• Ready to use application registry
21. 22
KPM KEY FEATURES
• One command line to install an app stack
• Super easy packaging
http://kpm.kubespray.io
kpm install heapster
22. 22
NEXT STEPS
• Ubernetes
• Continuous Deployment (Openshift, DEIS …)
• Network policies enforcement
• Custom metrics Autoscaling
23. 23
CONCLUSION
• Successful migration from Heroku to Kubernetes on
production
• Technical teams are very interested on Kubernetes
• Better resources management
• Greatly reduced infrastructure costs
26. 26
About us
System engineer and Devops
Work Experience
Smaine is a senior technical lead and devops
engineer at Arkena with 10 years experience
in Devops, Linux and open source
softwares.He built several streaming
plateforms (CDN, OTT) and he's currently in
charge of the engineering of Arkena's OTT
Plateform.Furthermore he's the Ansible Paris
meetup organizer and had talks on different
topics (Docker, Kubernetes, Ansible).
Smaine Kahlouch Antoine Legrand
Lead software engineer
Work Experience
Antoine Legrand is a senior software
engineer with 7 years experience.He
designs and leads the development of
Arkena's video-on-demand platform
used by large Broadcasters (Tf1, BeIn
Sports, Elisa....).He's an open source
softwares addict and co-author of
kubespray.io project.