Continuous delivery with Jenkins, Docker and Mesos/Marathon - jbcnconf

1. CONTINUOUS DELIVERY
WITH JENKINS, DOCKER
AND MARATHON/MESOS
Julia Mateo
@juliamateodc
@duchessswiss http://jduchess.ch/
Monday, June 29, 15

2. GOAL
Monday, June 29, 15

3. CONTINUOUS DEPLOYMENT
VS CONTINUOUS DELIVERY
• Continuous delivery : series of practices designed to ensure that
code can be rapidly and safely deployed to production
• Continuous deployment is the next step of continuous delivery:
Every change that passes the automated tests is deployed to
production automatically
Monday, June 29, 15

4. CONTINUOUS DELIVERY
Source : http://xebia.github.io/cd-with-docker/#/
Monday, June 29, 15

5. From https://www.docker.com/whatisdocker/
Using Docker for
Continuous Delivery
• Docker is an open platform for developers
and sysadmins to build, ship, and run
distributed applications.
Monday, June 29, 15

6. DOCKERVSVMS
Server
Host OS
Hypervisor
Guest
OS
Guest
OS
Guest
OS
Guest
OS
Libs/
Bins
Libs/
Bins
Libs/
Bins
App 1 App 2 App 3 App 4
Libs/
Bins
Server
Host OS
Docker Engine
Libs/Bins
Libs/
Bins
App 1 App 2 App 3 App 4
c1 c2 c3 c4
VM1 VM2 VM3 VM4
Monday, June 29, 15

7. CONTAINERS AND
IMAGES
Docker
Images
Docker
Images
libcontainer, Union Filesystem
Centos Ubuntu
Jetty
add App.war
Container 1
OracleDB
Container 2
Read
only
Writable
Writable
Read
only
Monday, June 29, 15

8. TO SUM UP :
• Simple : container as deployment unit
• Fast : Only need to build/upload the ∆ image
• Robust : less scope of error than custom deployment scripts
• Other advantages : docker facilitates fast rollback
Monday, June 29, 15

9. FIRST STEP : DEPLOYTO
ONE NODE
Monday, June 29, 15

10. THE WEB APPTO DEPLOY
Monday, June 29, 15

11. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
Monday, June 29, 15

12. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
Monday, June 29, 15

13. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
Monday, June 29, 15

14. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
Monday, June 29, 15

15. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
Monday, June 29, 15

16. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
Monday, June 29, 15

17. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

18. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

19. DOCKER PUBLISH PLUGIN
Monday, June 29, 15

20. USING DOCKER INTO
DOCKER
Monday, June 29, 15

21. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

22. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

23. DOCKER PRIVATE
REPOSITORY
• Add the registry’s location to the repository name :
my.registry.address:port/repositoryname
Monday, June 29, 15

24. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

25. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Monday, June 29, 15

26. JENKINS SSH PLUGIN
Monday, June 29, 15

27. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Node 1
Node 2
Monday, June 29, 15

28. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
ssh jenkins plugin
Node 1
Node 2
Container 1
Container 2
Container 3
Container 4
Container 5
Monday, June 29, 15

29. PROBLEMS
• Scalability
• Fault tolerance
Monday, June 29, 15

30. NEXT STEP :
DEPLOYTO A MESOS
CLUSTER
Monday, June 29, 15

31. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish pluginnode1
Container 1
Container 3
Container 4
Container 5
Container 2
Http request plugin
Master
Executor
Marathon
Executor
Monday, June 29, 15

32. • Abstraction of cluster resources
• Share resources across multiple frameworks (versions of the
same fwk)
• Resource fair sharing : alternative to static partitioning
• Data locality
What is Mesos ?
Monday, June 29, 15

33. MESOS
ZK
Master Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Monday, June 29, 15

34. HOW DOES IT WORK ?
http://mesos.apache.org/documentation/latest/mesos-architecture/
Monday, June 29, 15

35. USING MARATHON WITH
MESOS
• Marathon is a Mesos framework written in Scala
• Provides easy deployment of Docker containers
• Manages of long running apps
• Rest API for developers
Monday, June 29, 15

36. MARATHON
ZK
Master Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Monday, June 29, 15

37. MARATHON
ZK
Master Master
Executor Executor Executor
Executor ExecutorExecutor
Executor Executor Executor
Slaves
Marathon
Monday, June 29, 15

38. DEPLOY WITH DOCKER :
TEST ENVIRONMENT
Localhost
docker publish plugin
Http request plugin
Master
Executor
Marathon
Executor
Monday, June 29, 15

39. JENKINS : HTTP REQUEST
PLUGIN
Monday, June 29, 15

40. DEMO
Monday, June 29, 15

41. USING HAPROXY FOR LOAD
BALANCING
Monday, June 29, 15

42. DEPLOYINGTO
PRODUCTION
• Canary releasing
• Blue Green deployment
Monday, June 29, 15

43. FEEDBACK FROM DEVELOPER
PERSPECTIVE
• Many technologies in a short time : ansible, docker, mesos,
marathon..
• All of them are quite recent technologies
Monday, June 29, 15

44. FEEDBACK FROM DEVELOPER
PERSPECTIVE
• Docker, Mesos/Marathon : developer friendly
• Big and growing community
Monday, June 29, 15

45. NEXT IMPROVEMENTS
• Fine grained proxy configuration, service discovery...
Monday, June 29, 15

46. THANK YOU !
QUESTIONS
?
Monday, June 29, 15