Un jenkins amélioré avec docker mesos et marathon à Devoxx 2015
•
0 j'aime•2,358 vues
Jenkins est aujourd’hui un outil d’intégration continue incontournable. Cependant, ses mises à jour peuvent rapidement devenir fastidieuses, la gestion des plugins chaotique et la configuration de vos slaves pas vraiment optimal. À travers ce Tools in Action, vous découvrirez comment en containérisant Jenkins et en utilisant les resources d’un cluster Mesos vous pouvez considérablement améliorer la stabilité et les performances de votre usine logicielle. Speaker : Jean-Louis Rigau à Devoxx France 2015
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (17)
Similaire à Un jenkins amélioré avec docker mesos et marathon à Devoxx 2015
Similaire à Un jenkins amélioré avec docker mesos et marathon à Devoxx 2015 (20)
Plus de Publicis Sapient Engineering
Plus de Publicis Sapient Engineering (20)
Dernier
Dernier (20)
Un jenkins amélioré avec docker mesos et marathon à Devoxx 2015
- 1. @jlrigau#containerized_jenkins Un Jenkins amélioré avec Docker, Mesos et Marathon Expert Continuous Delivery Coach DevOps @ Xebia IT Architects jlrigau@jlrigau
- 2. Objectifs
- 3. @jlrigau#containerized_jenkins Objectifs • Simplifier la gestion de Jenkins en le conteneurisant avec Docker et en le déployant au sein d’un cluster Mesos avec le framework Marathon • Utiliser les ressources du cluster Mesos pour créer des slaves Jenkins à la demande en s’appuyant sur des conteneurs Docker
- 4. Outils
- 5. @jlrigau#containerized_jenkins Jenkins • Serveur d’intégration continue • Projet open-source maintenu par CloudBees • Écrit en Java “An extensible open source continuous integration server”
- 6. @jlrigau#containerized_jenkins Docker • Gestionnaire de conteneurs • Projet open-source maintenu par Docker Inc. • Écrit en Go “Build, Ship and Run Any App,Anywhere”
- 7. @jlrigau#containerized_jenkins Mesos • Gestionnaire de cluster • Projet open-source maintenu par la Fondation Apache • Écrit en C++ “Program against your datacenter like it’s a single pool of resources”
- 8. @jlrigau#containerized_jenkins Marathon • Framework Mesos • Projet open-source maintenu par Mesosphere • Écrit en Scala “Deploy and manage containers on top of Apache Mesos at scale”
- 9. Design
- 14. @jlrigau#containerized_jenkins FROM jenkins:weekly ADD init/*.groovy /usr/share/jenkins/init.groovy.d/ USER root RUN apt-get update && apt-get -y install libsvn1 ADD http://downloads.mesosphere.io/master/debian/7/mesos_0.21.1-1.0.debian77_amd64.deb /tmp/mesos.deb RUN dpkg -i /tmp/mesos.deb && rm /tmp/mesos.deb RUN mkdir -p /usr/share/jenkins/plugins && chown -R jenkins /usr/share/jenkins/plugins /usr/share/jenkins/init.groovy.d ADD plugins.awk plugins.csv /usr/share/jenkins/ RUN awk -f /usr/share/jenkins/plugins.awk /usr/share/jenkins/plugins.csv | sh ADD jenkins-init.sh /usr/local/bin/jenkins-init.sh VOLUME ["/var/jenkins_home"] ENTRYPOINT ["/usr/local/bin/jenkins-init.sh"] Jenkins Dockerfile
- 15. @jlrigau#containerized_jenkins FROM jenkins:weekly ADD init/*.groovy /usr/share/jenkins/init.groovy.d/ USER root RUN apt-get update && apt-get -y install libsvn1 ADD http://downloads.mesosphere.io/master/debian/7/mesos_0.21.1-1.0.debian77_amd64.deb /tmp/mesos.deb RUN dpkg -i /tmp/mesos.deb && rm /tmp/mesos.deb RUN mkdir -p /usr/share/jenkins/plugins && chown -R jenkins /usr/share/jenkins/plugins /usr/share/jenkins/init.groovy.d ADD plugins.awk plugins.csv /usr/share/jenkins/ RUN awk -f /usr/share/jenkins/plugins.awk /usr/share/jenkins/plugins.csv | sh ADD jenkins-init.sh /usr/local/bin/jenkins-init.sh VOLUME ["/var/jenkins_home"] ENTRYPOINT ["/usr/local/bin/jenkins-init.sh"] Jenkins Dockerfile
- 16. @jlrigau#containerized_jenkins import hudson.model.*; import jenkins.model.*; println "--> disabling master executors" Jenkins.instance.setNumExecutors(0) master-executors.groovy
- 17. @jlrigau#containerized_jenkins FROM jenkins:weekly ADD init/*.groovy /usr/share/jenkins/init.groovy.d/ USER root RUN apt-get update && apt-get -y install libsvn1 ADD http://downloads.mesosphere.io/master/debian/7/mesos_0.21.1-1.0.debian77_amd64.deb /tmp/mesos.deb RUN dpkg -i /tmp/mesos.deb && rm /tmp/mesos.deb RUN mkdir -p /usr/share/jenkins/plugins && chown -R jenkins /usr/share/jenkins/plugins /usr/share/jenkins/init.groovy.d ADD plugins.awk plugins.csv /usr/share/jenkins/ RUN awk -f /usr/share/jenkins/plugins.awk /usr/share/jenkins/plugins.csv | sh ADD jenkins-init.sh /usr/local/bin/jenkins-init.sh VOLUME ["/var/jenkins_home"] ENTRYPOINT ["/usr/local/bin/jenkins-init.sh"] Jenkins Dockerfile
- 18. @jlrigau#containerized_jenkins FROM jenkins:weekly ADD init/*.groovy /usr/share/jenkins/init.groovy.d/ USER root RUN apt-get update && apt-get -y install libsvn1 ADD http://downloads.mesosphere.io/master/debian/7/mesos_0.21.1-1.0.debian77_amd64.deb /tmp/mesos.deb RUN dpkg -i /tmp/mesos.deb && rm /tmp/mesos.deb RUN mkdir -p /usr/share/jenkins/plugins && chown -R jenkins /usr/share/jenkins/plugins /usr/share/jenkins/init.groovy.d ADD plugins.awk plugins.csv /usr/share/jenkins/ RUN awk -f /usr/share/jenkins/plugins.awk /usr/share/jenkins/plugins.csv | sh ADD jenkins-init.sh /usr/local/bin/jenkins-init.sh VOLUME ["/var/jenkins_home"] ENTRYPOINT ["/usr/local/bin/jenkins-init.sh"] Jenkins Dockerfile
- 20. @jlrigau#containerized_jenkins BEGIN { FS="," } { print "curl -sSL --create-dirs -o /usr/share/jenkins/plugins/" $1 ".hpi https://updates.jenkins-ci.org/" ($2 == "latest" ? "latest/" : "download/plugins/" $1 "/" $2 "/") $1 ".hpi" } plugins.awk
- 22. @jlrigau#containerized_jenkins FROM jenkins:weekly ADD init/*.groovy /usr/share/jenkins/init.groovy.d/ USER root RUN apt-get update && apt-get -y install libsvn1 ADD http://downloads.mesosphere.io/master/debian/7/mesos_0.21.1-1.0.debian77_amd64.deb /tmp/mesos.deb RUN dpkg -i /tmp/mesos.deb && rm /tmp/mesos.deb RUN mkdir -p /usr/share/jenkins/plugins && chown -R jenkins /usr/share/jenkins/plugins /usr/share/jenkins/init.groovy.d ADD plugins.awk plugins.csv /usr/share/jenkins/ RUN awk -f /usr/share/jenkins/plugins.awk /usr/share/jenkins/plugins.csv | sh ADD jenkins-init.sh /usr/local/bin/jenkins-init.sh VOLUME ["/var/jenkins_home"] ENTRYPOINT ["/usr/local/bin/jenkins-init.sh"] Jenkins Dockerfile
- 27. @jlrigau#containerized_jenkins { "id": "jenkins", "container": { "type": "DOCKER", "docker": { "image": "jlrigau/jenkins", "network": "HOST" }, "volumes": [ { "containerPath": "/var/jenkins_home", "hostPath": "/var/data/jenkins", "mode": "RW" } ] }, "cpus": 0.2, "mem": 128.0, "instances": 1 } jenkins.json
- 28. @jlrigau#containerized_jenkins { "id": "jenkins", "container": { "type": "DOCKER", "docker": { "image": "jlrigau/jenkins", "network": "HOST" }, "volumes": [ { "containerPath": "/var/jenkins_home", "hostPath": "/var/data/jenkins", "mode": "RW" } ] }, "cpus": 0.2, "mem": 128.0, "instances": 1 } jenkins.json
- 29. @jlrigau#containerized_jenkins { "id": "jenkins", "container": { "type": "DOCKER", "docker": { "image": "jlrigau/jenkins", "network": "HOST" }, "volumes": [ { "containerPath": "/var/jenkins_home", "hostPath": "/var/data/jenkins", "mode": "RW" } ] }, "cpus": 0.2, "mem": 128.0, "instances": 1 } jenkins.json
- 30. @jlrigau#containerized_jenkins { "id": "jenkins", "container": { "type": "DOCKER", "docker": { "image": "jlrigau/jenkins", "network": "HOST" }, "volumes": [ { "containerPath": "/var/jenkins_home", "hostPath": "/var/data/jenkins", "mode": "RW" } ] }, "cpus": 0.2, "mem": 128.0, "instances": 1 } jenkins.json
- 31. @jlrigau#containerized_jenkins $curl -X POST -H "Content-Type: application/json” http:// MARATHON_PUBLIC_HOST:8080/v2/apps -d@jenkins.json Déploiement de Jenkins
- 33. @jlrigau#containerized_jenkins … mais sur quel slave ?
- 35. @jlrigau#containerized_jenkins root@ip-172-31-14-112:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d85414d1ad4f jlrigau/jenkins:latest "/usr/local/bin/jenk 11 minutes ago Up 11 minutes mesos-f1c49c55-8075-4865-b644-7d35465c8a96 docker ps
- 36. @jlrigau#containerized_jenkins root@ip-172-31-14-112:~# docker logs $(docker ps -lq) Copying init scripts Copying plugins /usr/share/jenkins/ref/init.groovy.d/tcp-slave-angent-port.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-angent-port.groovy -> init.groovy.d/tcp-slave-angent-port.groovy copy init.groovy.d/tcp-slave-angent-port.groovy to JENKINS_HOME Running from: /usr/share/jenkins/jenkins.war webroot: EnvVars.masterEnvVars.get("JENKINS_HOME") Apr 07, 2015 11:33:28 PM winstone.Logger logInternal INFO: Beginning extraction from war file Apr 07, 2015 11:33:30 PM org.eclipse.jetty.util.log.JavaUtilLog info INFO: jetty-winstone-2.8 Apr 07, 2015 11:33:32 PM org.eclipse.jetty.util.log.JavaUtilLog info INFO: NO JSP Support for , did not find org.apache.jasper.servlet.JspServlet Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME") Apr 07, 2015 11:33:32 PM org.eclipse.jetty.util.log.JavaUtilLog info INFO: Started SelectChannelConnector@0.0.0.0:8080 Apr 07, 2015 11:33:32 PM winstone.Logger logInternal INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled Apr 07, 2015 11:33:33 PM jenkins.InitReactorRunner$1 onAttained INFO: Started initialization Apr 07, 2015 11:33:58 PM jenkins.InitReactorRunner$1 onAttained INFO: Listed all plugins Apr 07, 2015 11:33:58 PM jenkins.InitReactorRunner$1 onAttained INFO: Prepared all plugins Apr 07, 2015 11:33:58 PM jenkins.InitReactorRunner$1 onAttained INFO: Started all plugins Apr 07, 2015 11:33:59 PM jenkins.InitReactorRunner$1 onAttained INFO: Augmented all extensions Apr 07, 2015 11:34:02 PM jenkins.InitReactorRunner$1 onAttained INFO: Loaded all jobs Apr 07, 2015 11:34:02 PM jenkins.util.groovy.GroovyHookScript execute INFO: Executing /var/jenkins_home/init.groovy.d/master-executors.groovy Apr 07, 2015 11:34:02 PM org.jenkinsci.main.modules.sshd.SSHD start INFO: Started SSHD at port 60955 --> disabling master executors Apr 07, 2015 11:34:03 PM jenkins.util.groovy.GroovyHookScript execute INFO: Executing /var/jenkins_home/init.groovy.d/tcp-slave-angent-port.groovy Apr 07, 2015 11:34:03 PM jenkins.InitReactorRunner$1 onAttained INFO: Completed initialization Apr 07, 2015 11:34:03 PM hudson.WebAppMain$3 run INFO: Jenkins is fully up and running --> setting agent port for jnlp docker logs $(docker ps -lq)
- 50. @jlrigau#containerized_jenkins root@ip-172-31-14-111:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ee5455595cc5 java:7 "/bin/sh -c 'java -D 15 minutes ago Up 15 minutes mesos-d6692494-0d90-4782-a887-702f6a6cd3fa docker ps
- 60. @jlrigau#containerized_jenkins root@ip-172-31-14-112:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f5d271f37950 jlrigau/maven-git:latest "/bin/sh -c 'java -D 43 seconds ago Up 42 seconds mesos-ed390fd9-6abc-452a-a5f2-cf195151b9a6 78bd7391c980 java:7 "/bin/sh -c 'java -D 45 seconds ago Up 45 seconds mesos-85d716dd-c887-42ce-a4bd-8b1883f89a71 9ab7272b32da jlrigau/jenkins:latest "/usr/local/bin/jenk About an hour ago Up About an hour mesos-9082d0e2-3ff8-4133-9e1e-6959d9ebbfcb docker ps
- 61. @jlrigau#containerized_jenkins root@ip-172-31-14-112:~# docker exec -it f5d271f37950 /bin/bash root@ip-172-31-14-112:/# ls -l /jenkins/workspace/Projet Maven/ total 20 -rw-r--r-- 1 root root 2229 Apr 8 01:12 README.md drwxr-xr-x 3 root root 4096 Apr 8 01:12 jenkins -rw-r--r-- 1 root root 342 Apr 8 01:12 jenkins.json drwxr-xr-x 2 root root 4096 Apr 8 01:12 maven-git drwxr-xr-x 3 root root 4096 Apr 8 01:12 sample docker exec
- 63. @jlrigau#containerized_jenkins Liens • Sources du Tools in Action • https://github.com/jlrigau/ci-mesos • Sources de la configuration du cluster Mesos • https://github.com/WeScale/mesosphere-walking-tutorial • Sources d’inspiration • Delivering eBay’s CI Solution with Apache Mesos - Part I • Delivering eBay’s CI Solution with Apache Mesos - Part II