SlideShare une entreprise Scribd logo
1  sur  36
Télécharger pour lire hors ligne
Continuous Integration
with Hudson

Arun Gupta
Sun Microsystems, Inc.
Acknowledgements

        Originally presented by
   Kohsuke Kawaguchi & Jesse Glick
              at JavaOne

     Scheduled to be presented by
          Fabiane Nardon
             At FISL 10
Rise of Continuous Integration
>   Offload from people, push to computers
$
       computers



        us



                                             time
                                                    3
“Never use humans for the job of a machine”
Spend more CPU power to help you
>   … even if it only helps a little

>   First on your laptops and workstations
       IDEs are at the forefront
>   And then to the servers
       a.k.a. “Continuous Integration”
       More frequent build/test executions
       Static code analysis tools
       And more to come


                                              5
Hudson                               https://hudson.dev.java.
                                                         net/
>   Open-source CI server at java.net

>   Emphasis on ease of installation and use
       “java -jar hudson.war” execution
       Configure everything from browsers
>   Extensibility
       140+ community-developed public plugins
       By 150+ contributors

>   Estimated 13,000 installations
                                                                6
It basically does builds and tests
>   Check out the source code
       Subversion, Perforce, Git, Mercurial, CVS, …
>   Do builds and/or tests
       Ant, Maven, MSBuild, shell script, …
>   Record results
       Binary, test results, code coverage, static analysis
>   Notify people
       E-mail, IM, RSS, tray apps, IDEs



                                                               7
Hudson Plugin Ecosystem




New Plugins

2008: 55
2009: 44 so far




                          8
Localized to 8 languages




                           9
And hopefully more to come…




                              10
Adoption in all kinds of businesses




                                      11
Eclipse Community Survey




                           12
13
Why Distributed Builds?
>   You need to use multiple computers because…
       You need different environments
       You need isolation

>   There’s only so much you can do with 1 computer




                                                      14
Installing new slaves
>   For first 20 or so slaves, we did it manually
       Insert CD, click, type, click, type, click, …
       But that doesn’t scale

>   Then we automated
       Available as “Hudson PXE Plugin”




                                                        15
Automated System Installations
           >   Hudson + PXE plugin
                ISO images of OS




      >   Slaves
           Power on, hit F12

           PC boots from network (PXE)




                                          16
Automated System Installations
           >   Hudson + PXE plugin
                ISO images of OS

                                          Your corporate IT guy &
                                             his DHCP server




      >   Slaves
           Power on, hit F12

           PC boots from network (PXE)

           Choose OS from menu

           Installs non-interactively



                                                            17
Automated System Installations
>   Supports OpenSolaris, Ubuntu, CentOS, Fedora
       Trivial with most Linux
>   Cooperate with Windows, too

>   Quite useful outside Hudson, too
       No more broken CD drives
       No more CD-Rs




                                                   18
Distributed builds with Hudson
>   Master
       Serves HTTP requests
       Stores all important info
>   Slaves
       170KB single JAR
       Assumed to be unreliable
       Scale to at least 100
>   Link
       Single bi-di byte stream
       No other requirements


                                    19
Heterogeneous Cluster Challenge
>   Your builds/tests need to run in specific
    environment
>   Dependency on individual nodes hurts utilization
           jobs                       slaves
         Wombat                      Window
        Windows test
                                       s #1
         GlassFish                   Window
        Windows test
                                       s #2
          Hudson                     Solaris
        Windows test
                                        #1
        Hudson Solaris
             test


                                                       20
Labels to rescue
>   Label is a group of slaves
>   Tie jobs to labels

           jobs                     slaves
         Wombat                    Window
        Windows test
                                     s #1
                         Windows
         GlassFish                 Window
        Windows test
                                     s #2
          Hudson                   Solaris
        Windows test
                         Solaris      #1
        Hudson Solaris             Window
             test
                                     s #3
                                             21
Coming to
                                                                 your
                                                                Hudson

Making builds sticky                                             soon




>   We want jobs to be mostly on the same slave
       Faster check out
       Consistent results
       Minimizes disk consumption
>   But does it softly

>   Hudson uses consistent hash* to achieve this
>   More schedule controls become possible:
       Use faster machines more frequently
       Slowly ramp up newly installed slaves

           * http://en.wikipedia.org/wiki/Consistent_hashing               22
Forecasting failures
>   Hudson monitors key health metrics of slaves
       Low disk space, insufficient swap
       Clock out of synch
       Extensible
>   Slaves go offline automatically
>   Catch problems before they break builds




                                                   23
Load Statistics Monitoring




                             24
When it’s time to add more slaves




                                    25
Hudson made this extensible
>   Hudson detects excessive workload
>   Hudson notifies plugins
>   Plugins can provision more slaves
       … assuming that you have that infrastructure




                                                       26
27
Amazon EC2: The Good
>   Pay as you go (10¢/h or so)
       Loads on Hudson tend to be spiky
>   Programmable API
>   Instances launch at machine-speed
>   EC2 instances are forgetful




                                           28
Amazon EC2: The Bad
>   Your data is still inside your firewall
       Takes time to check out code
       … or to archive build artifacts
       Some data just can’t be moved
>   EC2 instances are forgetful
>   Can your tests run in parallel?




                                              29
Hudson EC2 plugin
>   Built on top of typica*
>   What does it do?
       Automatically provisions slaves on EC2 on demand
       Picks the right AMI depending on demand
       Starts slave agent
       Shuts down unused instances




           * http://code.google.com/p/typica/              30
Hudson “Appliance” on EC2
>   Run the master in the cloud too, if you like
       Hudson on stock OpenSolaris AMI
       Data stored persistently in Elastic Block Storage
           Dynamically expandable thanks to ZFS
           Online, too
>   Packaged as a wizard




                                                            31
Hudson EC2 plugin usage
>   Tell Hudson what AMIs you want to start




                                              32
33
Conclusion
>   CI is here to stay
       We’ll continue to push more workload to servers
>   Hudson makes this easy for you
>   Reap the benefit of a cluster in multiple ways




                                                          34
Resources
>   http://hudson.dev.java.net/

>   Support Subscription
       hudson@sun.com




                                  35
Arun Gupta
http://blogs.sun.com/arungupta
http://hudson.dev.java.net/




                                 36

Contenu connexe

Tendances

Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12dotCloud
 
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16 What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16 Docker, Inc.
 
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...Eric Smalling
 
Introduction to Docker and deployment and Azure
Introduction to Docker and deployment and AzureIntroduction to Docker and deployment and Azure
Introduction to Docker and deployment and AzureJérôme Petazzoni
 
Docker and Containers for Development and Deployment — SCALE12X
Docker and Containers for Development and Deployment — SCALE12XDocker and Containers for Development and Deployment — SCALE12X
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
 
Continuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsContinuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsB1 Systems GmbH
 
Docker presentation | Paris Docker Meetup
Docker presentation | Paris Docker MeetupDocker presentation | Paris Docker Meetup
Docker presentation | Paris Docker MeetupdotCloud
 
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...Docker, Inc.
 
DCSF19 CMD and Conquer: Containerizing the Monolith
DCSF19 CMD and Conquer: Containerizing the Monolith  DCSF19 CMD and Conquer: Containerizing the Monolith
DCSF19 CMD and Conquer: Containerizing the Monolith Docker, Inc.
 
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...Edureka!
 
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGHDeploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGHErica Windisch
 
Best Practices for Mission-Critical Jenkins
Best Practices for Mission-Critical JenkinsBest Practices for Mission-Critical Jenkins
Best Practices for Mission-Critical Jenkinsmrooney7828
 
How Jenkins Builds the Netflix Global Streaming Service
How Jenkins Builds the Netflix Global Streaming ServiceHow Jenkins Builds the Netflix Global Streaming Service
How Jenkins Builds the Netflix Global Streaming ServiceGareth Bowles
 
Intro to Docker
Intro to DockerIntro to Docker
Intro to DockerTim Haak
 
Modern Continuous Delivery with Docker and Liferay
Modern Continuous Delivery with Docker and LiferayModern Continuous Delivery with Docker and Liferay
Modern Continuous Delivery with Docker and LiferayManuel de la Peña Peña
 
Optimizing Docker Images
Optimizing Docker ImagesOptimizing Docker Images
Optimizing Docker ImagesBrian DeHamer
 
Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?John Rofrano
 
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena Tapia
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena TapiaFrom Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena Tapia
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena TapiaDocker, Inc.
 

Tendances (20)

Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
 
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16 What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16
What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16
 
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
 
Introduction to Docker and deployment and Azure
Introduction to Docker and deployment and AzureIntroduction to Docker and deployment and Azure
Introduction to Docker and deployment and Azure
 
Docker and Containers for Development and Deployment — SCALE12X
Docker and Containers for Development and Deployment — SCALE12XDocker and Containers for Development and Deployment — SCALE12X
Docker and Containers for Development and Deployment — SCALE12X
 
Continuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsContinuous Integration using Docker & Jenkins
Continuous Integration using Docker & Jenkins
 
Docker presentation | Paris Docker Meetup
Docker presentation | Paris Docker MeetupDocker presentation | Paris Docker Meetup
Docker presentation | Paris Docker Meetup
 
Docker zero
Docker zeroDocker zero
Docker zero
 
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
Build, Publish, Deploy and Test Docker images and containers with Jenkins Wor...
 
DCSF19 CMD and Conquer: Containerizing the Monolith
DCSF19 CMD and Conquer: Containerizing the Monolith  DCSF19 CMD and Conquer: Containerizing the Monolith
DCSF19 CMD and Conquer: Containerizing the Monolith
 
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
 
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGHDeploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
Deploying Docker (Provisioning /w Docker + Chef/Puppet) - DevopsDaysPGH
 
Best Practices for Mission-Critical Jenkins
Best Practices for Mission-Critical JenkinsBest Practices for Mission-Critical Jenkins
Best Practices for Mission-Critical Jenkins
 
How Jenkins Builds the Netflix Global Streaming Service
How Jenkins Builds the Netflix Global Streaming ServiceHow Jenkins Builds the Netflix Global Streaming Service
How Jenkins Builds the Netflix Global Streaming Service
 
Introduction To Docker
Introduction To DockerIntroduction To Docker
Introduction To Docker
 
Intro to Docker
Intro to DockerIntro to Docker
Intro to Docker
 
Modern Continuous Delivery with Docker and Liferay
Modern Continuous Delivery with Docker and LiferayModern Continuous Delivery with Docker and Liferay
Modern Continuous Delivery with Docker and Liferay
 
Optimizing Docker Images
Optimizing Docker ImagesOptimizing Docker Images
Optimizing Docker Images
 
Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?
 
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena Tapia
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena TapiaFrom Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena Tapia
From Zero Docker to Hackathon Winner - Marcos Lilljedahl and Jimena Tapia
 

Similaire à Hudson at FISL 2009

Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the SourceLearn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the SourceMarakana Inc.
 
Jenkins talk at Silicon valley DevOps meetup
Jenkins talk at Silicon valley DevOps meetupJenkins talk at Silicon valley DevOps meetup
Jenkins talk at Silicon valley DevOps meetupCloudBees
 
Hudson@java one2010
Hudson@java one2010Hudson@java one2010
Hudson@java one2010InfraDNA
 
Austin Web Architecture
Austin Web ArchitectureAustin Web Architecture
Austin Web Architecturejoaquincasares
 
Continuous integration and deployment with docker
Continuous integration and deployment with dockerContinuous integration and deployment with docker
Continuous integration and deployment with dockerpebble {code}
 
Docker: Testing to Production
Docker: Testing to ProductionDocker: Testing to Production
Docker: Testing to ProductionEdwin Fuquen
 
Deploying Windows Containers on Windows Server 2016
Deploying Windows Containers on Windows Server 2016Deploying Windows Containers on Windows Server 2016
Deploying Windows Containers on Windows Server 2016Ben Hall
 
Agile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Partner S.A.
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISORVanika Kapoor
 
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé MochtarTestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé MochtarXebia Nederland BV
 
Automating Dev Environment - Introduction to Docker and Chef
Automating Dev Environment - Introduction to Docker and ChefAutomating Dev Environment - Introduction to Docker and Chef
Automating Dev Environment - Introduction to Docker and Chefkamalikamj
 
RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...
 RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do... RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...
RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...Thoughtworks
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
 
Docker intro
Docker introDocker intro
Docker introOleg Z
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014Carlo Bonamico
 
Running Docker in Production - The Good, the Bad and The Ugly
Running Docker in Production - The Good, the Bad and The UglyRunning Docker in Production - The Good, the Bad and The Ugly
Running Docker in Production - The Good, the Bad and The UglyKontena, Inc.
 

Similaire à Hudson at FISL 2009 (20)

Learn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the SourceLearn About Continuous Integration With Hudson Directly From the Source
Learn About Continuous Integration With Hudson Directly From the Source
 
Jenkins talk at Silicon valley DevOps meetup
Jenkins talk at Silicon valley DevOps meetupJenkins talk at Silicon valley DevOps meetup
Jenkins talk at Silicon valley DevOps meetup
 
Hudson@java one2010
Hudson@java one2010Hudson@java one2010
Hudson@java one2010
 
Austin Web Architecture
Austin Web ArchitectureAustin Web Architecture
Austin Web Architecture
 
Continuous integration and deployment with docker
Continuous integration and deployment with dockerContinuous integration and deployment with docker
Continuous integration and deployment with docker
 
Docker: Testing to Production
Docker: Testing to ProductionDocker: Testing to Production
Docker: Testing to Production
 
Scalable QA With Docker
Scalable QA With DockerScalable QA With Docker
Scalable QA With Docker
 
Deploying Windows Containers on Windows Server 2016
Deploying Windows Containers on Windows Server 2016Deploying Windows Containers on Windows Server 2016
Deploying Windows Containers on Windows Server 2016
 
Agile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: Introduction
 
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISORLOAD BALANCING OF APPLICATIONS  USING XEN HYPERVISOR
LOAD BALANCING OF APPLICATIONS USING XEN HYPERVISOR
 
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé MochtarTestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
TestWorks Conf Scalable QA with docker - Maarten van den Ende and Adé Mochtar
 
Automating Dev Environment - Introduction to Docker and Chef
Automating Dev Environment - Introduction to Docker and ChefAutomating Dev Environment - Introduction to Docker and Chef
Automating Dev Environment - Introduction to Docker and Chef
 
RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...
 RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do... RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...
RootConf 2014 Bangalore: Automating the Dev Environment - Introduction to Do...
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
 
Hudson ci
Hudson ciHudson ci
Hudson ci
 
Java one 2015 - v1
Java one   2015 - v1Java one   2015 - v1
Java one 2015 - v1
 
Docker intro
Docker introDocker intro
Docker intro
 
J+s
J+sJ+s
J+s
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014
 
Running Docker in Production - The Good, the Bad and The Ugly
Running Docker in Production - The Good, the Bad and The UglyRunning Docker in Production - The Good, the Bad and The Ugly
Running Docker in Production - The Good, the Bad and The Ugly
 

Plus de Arun Gupta

5 Skills To Force Multiply Technical Talents.pdf
5 Skills To Force Multiply Technical Talents.pdf5 Skills To Force Multiply Technical Talents.pdf
5 Skills To Force Multiply Technical Talents.pdfArun Gupta
 
Machine Learning using Kubernetes - AI Conclave 2019
Machine Learning using Kubernetes - AI Conclave 2019Machine Learning using Kubernetes - AI Conclave 2019
Machine Learning using Kubernetes - AI Conclave 2019Arun Gupta
 
Machine Learning using Kubeflow and Kubernetes
Machine Learning using Kubeflow and KubernetesMachine Learning using Kubeflow and Kubernetes
Machine Learning using Kubeflow and KubernetesArun Gupta
 
Secure and Fast microVM for Serverless Computing using Firecracker
Secure and Fast microVM for Serverless Computing using FirecrackerSecure and Fast microVM for Serverless Computing using Firecracker
Secure and Fast microVM for Serverless Computing using FirecrackerArun Gupta
 
Building Java in the Open - j.Day at OSCON 2019
Building Java in the Open - j.Day at OSCON 2019Building Java in the Open - j.Day at OSCON 2019
Building Java in the Open - j.Day at OSCON 2019Arun Gupta
 
Why Amazon Cares about Open Source
Why Amazon Cares about Open SourceWhy Amazon Cares about Open Source
Why Amazon Cares about Open SourceArun Gupta
 
Machine learning using Kubernetes
Machine learning using KubernetesMachine learning using Kubernetes
Machine learning using KubernetesArun Gupta
 
Building Cloud Native Applications
Building Cloud Native ApplicationsBuilding Cloud Native Applications
Building Cloud Native ApplicationsArun Gupta
 
Chaos Engineering with Kubernetes
Chaos Engineering with KubernetesChaos Engineering with Kubernetes
Chaos Engineering with KubernetesArun Gupta
 
How to be a mentor to bring more girls to STEAM
How to be a mentor to bring more girls to STEAMHow to be a mentor to bring more girls to STEAM
How to be a mentor to bring more girls to STEAMArun Gupta
 
Java in a World of Containers - DockerCon 2018
Java in a World of Containers - DockerCon 2018Java in a World of Containers - DockerCon 2018
Java in a World of Containers - DockerCon 2018Arun Gupta
 
The Serverless Tidal Wave - SwampUP 2018 Keynote
The Serverless Tidal Wave - SwampUP 2018 KeynoteThe Serverless Tidal Wave - SwampUP 2018 Keynote
The Serverless Tidal Wave - SwampUP 2018 KeynoteArun Gupta
 
Introduction to Amazon EKS - KubeCon 2018
Introduction to Amazon EKS - KubeCon 2018Introduction to Amazon EKS - KubeCon 2018
Introduction to Amazon EKS - KubeCon 2018Arun Gupta
 
Mastering Kubernetes on AWS - Tel Aviv Summit
Mastering Kubernetes on AWS - Tel Aviv SummitMastering Kubernetes on AWS - Tel Aviv Summit
Mastering Kubernetes on AWS - Tel Aviv SummitArun Gupta
 
Top 10 Technology Trends Changing Developer's Landscape
Top 10 Technology Trends Changing Developer's LandscapeTop 10 Technology Trends Changing Developer's Landscape
Top 10 Technology Trends Changing Developer's LandscapeArun Gupta
 
Container Landscape in 2017
Container Landscape in 2017Container Landscape in 2017
Container Landscape in 2017Arun Gupta
 
Java EE and NoSQL using JBoss EAP 7 and OpenShift
Java EE and NoSQL using JBoss EAP 7 and OpenShiftJava EE and NoSQL using JBoss EAP 7 and OpenShift
Java EE and NoSQL using JBoss EAP 7 and OpenShiftArun Gupta
 
Docker, Kubernetes, and Mesos recipes for Java developers
Docker, Kubernetes, and Mesos recipes for Java developersDocker, Kubernetes, and Mesos recipes for Java developers
Docker, Kubernetes, and Mesos recipes for Java developersArun Gupta
 
Thanks Managers!
Thanks Managers!Thanks Managers!
Thanks Managers!Arun Gupta
 
Migrate your traditional VM-based Clusters to Containers
Migrate your traditional VM-based Clusters to ContainersMigrate your traditional VM-based Clusters to Containers
Migrate your traditional VM-based Clusters to ContainersArun Gupta
 

Plus de Arun Gupta (20)

5 Skills To Force Multiply Technical Talents.pdf
5 Skills To Force Multiply Technical Talents.pdf5 Skills To Force Multiply Technical Talents.pdf
5 Skills To Force Multiply Technical Talents.pdf
 
Machine Learning using Kubernetes - AI Conclave 2019
Machine Learning using Kubernetes - AI Conclave 2019Machine Learning using Kubernetes - AI Conclave 2019
Machine Learning using Kubernetes - AI Conclave 2019
 
Machine Learning using Kubeflow and Kubernetes
Machine Learning using Kubeflow and KubernetesMachine Learning using Kubeflow and Kubernetes
Machine Learning using Kubeflow and Kubernetes
 
Secure and Fast microVM for Serverless Computing using Firecracker
Secure and Fast microVM for Serverless Computing using FirecrackerSecure and Fast microVM for Serverless Computing using Firecracker
Secure and Fast microVM for Serverless Computing using Firecracker
 
Building Java in the Open - j.Day at OSCON 2019
Building Java in the Open - j.Day at OSCON 2019Building Java in the Open - j.Day at OSCON 2019
Building Java in the Open - j.Day at OSCON 2019
 
Why Amazon Cares about Open Source
Why Amazon Cares about Open SourceWhy Amazon Cares about Open Source
Why Amazon Cares about Open Source
 
Machine learning using Kubernetes
Machine learning using KubernetesMachine learning using Kubernetes
Machine learning using Kubernetes
 
Building Cloud Native Applications
Building Cloud Native ApplicationsBuilding Cloud Native Applications
Building Cloud Native Applications
 
Chaos Engineering with Kubernetes
Chaos Engineering with KubernetesChaos Engineering with Kubernetes
Chaos Engineering with Kubernetes
 
How to be a mentor to bring more girls to STEAM
How to be a mentor to bring more girls to STEAMHow to be a mentor to bring more girls to STEAM
How to be a mentor to bring more girls to STEAM
 
Java in a World of Containers - DockerCon 2018
Java in a World of Containers - DockerCon 2018Java in a World of Containers - DockerCon 2018
Java in a World of Containers - DockerCon 2018
 
The Serverless Tidal Wave - SwampUP 2018 Keynote
The Serverless Tidal Wave - SwampUP 2018 KeynoteThe Serverless Tidal Wave - SwampUP 2018 Keynote
The Serverless Tidal Wave - SwampUP 2018 Keynote
 
Introduction to Amazon EKS - KubeCon 2018
Introduction to Amazon EKS - KubeCon 2018Introduction to Amazon EKS - KubeCon 2018
Introduction to Amazon EKS - KubeCon 2018
 
Mastering Kubernetes on AWS - Tel Aviv Summit
Mastering Kubernetes on AWS - Tel Aviv SummitMastering Kubernetes on AWS - Tel Aviv Summit
Mastering Kubernetes on AWS - Tel Aviv Summit
 
Top 10 Technology Trends Changing Developer's Landscape
Top 10 Technology Trends Changing Developer's LandscapeTop 10 Technology Trends Changing Developer's Landscape
Top 10 Technology Trends Changing Developer's Landscape
 
Container Landscape in 2017
Container Landscape in 2017Container Landscape in 2017
Container Landscape in 2017
 
Java EE and NoSQL using JBoss EAP 7 and OpenShift
Java EE and NoSQL using JBoss EAP 7 and OpenShiftJava EE and NoSQL using JBoss EAP 7 and OpenShift
Java EE and NoSQL using JBoss EAP 7 and OpenShift
 
Docker, Kubernetes, and Mesos recipes for Java developers
Docker, Kubernetes, and Mesos recipes for Java developersDocker, Kubernetes, and Mesos recipes for Java developers
Docker, Kubernetes, and Mesos recipes for Java developers
 
Thanks Managers!
Thanks Managers!Thanks Managers!
Thanks Managers!
 
Migrate your traditional VM-based Clusters to Containers
Migrate your traditional VM-based Clusters to ContainersMigrate your traditional VM-based Clusters to Containers
Migrate your traditional VM-based Clusters to Containers
 

Dernier

IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 

Dernier (20)

IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 

Hudson at FISL 2009

  • 1. Continuous Integration with Hudson Arun Gupta Sun Microsystems, Inc.
  • 2. Acknowledgements Originally presented by Kohsuke Kawaguchi & Jesse Glick at JavaOne Scheduled to be presented by Fabiane Nardon At FISL 10
  • 3. Rise of Continuous Integration > Offload from people, push to computers $ computers us time 3
  • 4. “Never use humans for the job of a machine”
  • 5. Spend more CPU power to help you > … even if it only helps a little > First on your laptops and workstations  IDEs are at the forefront > And then to the servers  a.k.a. “Continuous Integration”  More frequent build/test executions  Static code analysis tools  And more to come 5
  • 6. Hudson https://hudson.dev.java. net/ > Open-source CI server at java.net > Emphasis on ease of installation and use  “java -jar hudson.war” execution  Configure everything from browsers > Extensibility  140+ community-developed public plugins  By 150+ contributors > Estimated 13,000 installations 6
  • 7. It basically does builds and tests > Check out the source code  Subversion, Perforce, Git, Mercurial, CVS, … > Do builds and/or tests  Ant, Maven, MSBuild, shell script, … > Record results  Binary, test results, code coverage, static analysis > Notify people  E-mail, IM, RSS, tray apps, IDEs 7
  • 8. Hudson Plugin Ecosystem New Plugins 2008: 55 2009: 44 so far 8
  • 9. Localized to 8 languages 9
  • 10. And hopefully more to come… 10
  • 11. Adoption in all kinds of businesses 11
  • 13. 13
  • 14. Why Distributed Builds? > You need to use multiple computers because…  You need different environments  You need isolation > There’s only so much you can do with 1 computer 14
  • 15. Installing new slaves > For first 20 or so slaves, we did it manually  Insert CD, click, type, click, type, click, …  But that doesn’t scale > Then we automated  Available as “Hudson PXE Plugin” 15
  • 16. Automated System Installations > Hudson + PXE plugin  ISO images of OS > Slaves  Power on, hit F12  PC boots from network (PXE) 16
  • 17. Automated System Installations > Hudson + PXE plugin  ISO images of OS Your corporate IT guy & his DHCP server > Slaves  Power on, hit F12  PC boots from network (PXE)  Choose OS from menu  Installs non-interactively 17
  • 18. Automated System Installations > Supports OpenSolaris, Ubuntu, CentOS, Fedora  Trivial with most Linux > Cooperate with Windows, too > Quite useful outside Hudson, too  No more broken CD drives  No more CD-Rs 18
  • 19. Distributed builds with Hudson > Master  Serves HTTP requests  Stores all important info > Slaves  170KB single JAR  Assumed to be unreliable  Scale to at least 100 > Link  Single bi-di byte stream  No other requirements 19
  • 20. Heterogeneous Cluster Challenge > Your builds/tests need to run in specific environment > Dependency on individual nodes hurts utilization jobs slaves Wombat Window Windows test s #1 GlassFish Window Windows test s #2 Hudson Solaris Windows test #1 Hudson Solaris test 20
  • 21. Labels to rescue > Label is a group of slaves > Tie jobs to labels jobs slaves Wombat Window Windows test s #1 Windows GlassFish Window Windows test s #2 Hudson Solaris Windows test Solaris #1 Hudson Solaris Window test s #3 21
  • 22. Coming to your Hudson Making builds sticky soon > We want jobs to be mostly on the same slave  Faster check out  Consistent results  Minimizes disk consumption > But does it softly > Hudson uses consistent hash* to achieve this > More schedule controls become possible:  Use faster machines more frequently  Slowly ramp up newly installed slaves * http://en.wikipedia.org/wiki/Consistent_hashing 22
  • 23. Forecasting failures > Hudson monitors key health metrics of slaves  Low disk space, insufficient swap  Clock out of synch  Extensible > Slaves go offline automatically > Catch problems before they break builds 23
  • 25. When it’s time to add more slaves 25
  • 26. Hudson made this extensible > Hudson detects excessive workload > Hudson notifies plugins > Plugins can provision more slaves  … assuming that you have that infrastructure 26
  • 27. 27
  • 28. Amazon EC2: The Good > Pay as you go (10¢/h or so)  Loads on Hudson tend to be spiky > Programmable API > Instances launch at machine-speed > EC2 instances are forgetful 28
  • 29. Amazon EC2: The Bad > Your data is still inside your firewall  Takes time to check out code  … or to archive build artifacts  Some data just can’t be moved > EC2 instances are forgetful > Can your tests run in parallel? 29
  • 30. Hudson EC2 plugin > Built on top of typica* > What does it do?  Automatically provisions slaves on EC2 on demand  Picks the right AMI depending on demand  Starts slave agent  Shuts down unused instances * http://code.google.com/p/typica/ 30
  • 31. Hudson “Appliance” on EC2 > Run the master in the cloud too, if you like  Hudson on stock OpenSolaris AMI  Data stored persistently in Elastic Block Storage  Dynamically expandable thanks to ZFS  Online, too > Packaged as a wizard 31
  • 32. Hudson EC2 plugin usage > Tell Hudson what AMIs you want to start 32
  • 33. 33
  • 34. Conclusion > CI is here to stay  We’ll continue to push more workload to servers > Hudson makes this easy for you > Reap the benefit of a cluster in multiple ways 34
  • 35. Resources > http://hudson.dev.java.net/ > Support Subscription  hudson@sun.com 35