SlideShare a Scribd company logo
1 of 45
Download to read offline
Jenkins User Conference San Francisco #jenkinsconf 
Seven Habits of 
Highly Effective Jenkins Users 
Andrew Bayer 
Build and Tools Architect, Cloudera 
Twitter: @abayer 
October 23, 2014 
#jenkinsconf
Jenkins User Conference San Francisco #jenkinsconf 
Who Am I? 
2 
Build and tools architect at 
Cloudera. 
Contributor to Jenkins core and 
author of plugins since spring 2009. 
Committer and PMC member of 
multiple Apache projects, inc. 
jclouds, Whirr, Bigtop… 
ASF Member and volunteer for 
maintaining builds.apache.org.
Jenkins User Conference San Francisco #jenkinsconf 
What’s this talk about? 
These are lessons learned from maintaining 
multiple large Jenkins instances over the 
years. 
3 
Cloudera’s three masters with 1000+ 
jobs each (4800+ total! Ouch!) with 
dozens running at a time. 
builds.apache.org’s 1200+ jobs from 
~120 different project teams. 
Oh, and my time on IRC, working on 
core and plugins, etc.
Jenkins User Conference San Francisco #jenkinsconf 
Your mileage may vary 
• These habits can be valuable on every 
Jenkins instance. 
• Some will be more relevant for larger 
instances, those with more complex jobs, 
and/or production-critical instances. 
• But these are *my* recommendations - you 
need to learn what’s best for your Jenkins 
setup. 
4
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 1: MAKE YOUR MASTER 
STABLE AND RESTORABLE 
5
Jenkins User Conference San Francisco #jenkinsconf 
Use LTS Releases 
• LTS release trains created every 12 weeks. 
• Active train updated three times before the 
next one starts. 
• Avoid bleeding edge instability. 
• LTS releases go through automated 
acceptance testing and a manual testing 
matrix before going out. 
6
Jenkins User Conference San Francisco #jenkinsconf 
Be conservative about 
upgrading plugins 
• Plugins can change a lot without it being 
obvious. 
• Backwards compatibility can sometimes 
break. 
• Example - Extended Email plugin 
recipient/trigger settings recently. 
• New features can be unstable/problematic 
in the wild. 
7
Jenkins User Conference San Francisco #jenkinsconf 
Have an upgrade testbed 
• Test out upgrades and new plugins in a 
testbed environment before going live in 
production. 
• Set up jobs to cover your plugin usage. 
• If possible, test your usage at scale. 
• Give significant changes a few days, at 
least, of builds running before going live. 
8
Jenkins User Conference San Francisco #jenkinsconf 
Back up your Jenkins configuration 
• Kind of obvious, isn’t it? =) 
• Lots of possible solutions 
• Within Jenkins, I recommend the 
thinBackup plugin. 
• Full copies of $JENKINS_HOME work great, 
but can be slow and use lots of disk. 
• Config files can be copied without 
copying all the builds, etc as well - see an 
example here 
9
Jenkins User Conference San Francisco #jenkinsconf 
Avoid using the Maven job type 
• Maven build steps are perfectly fine in 
freestyle jobs, but the Maven plugin’s Maven 
job type is…questionable. 
• Implementation leads to some potential 
problems in plugin support, lazy loading of 
projects, interception of Maven execution 
internals, etc… 
• I’ve seen a lot of strange edge case 
problems show up with it at scale. Be careful 
with it. 
10
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 2: BREAK UP THE BLOAT 
11
Jenkins User Conference San Francisco #jenkinsconf 
Multiple Masters 
• If you have a lot of projects and teams, 
multiple masters give you a lot more agility 
and control. 
• Split up masters by team, function, access, 
etc. 
• Makes it easier to restart for plugin 
installs/upgrades, etc without disrupting 
everyone. 
• More masters with less jobs each are more 
stable, less prone to edge case bugs. 12
Jenkins User Conference San Francisco #jenkinsconf 
Break up your jobs 
• Modularization and reuse are good in 
programming - and good in Jenkins too. 
• Multi-job builds allow reusability of generic 
jobs across multiple projects, releases, etc. 
• Few things more frustrating than a 10 hour 
build crashing 9.5 hours in. 
• Multi-job builds can be relaunched at any 
step in the process, if designed properly. 
13
Jenkins User Conference San Francisco #jenkinsconf 
Tools for breaking up your jobs 
• Just some examples - there are a ton of ways you 
are able to do this in Jenkins. 
• Parameterized Trigger + Conditional Build Step, 
Copy Artifact, Promoted Builds: 
• Very powerful, not as easy to configure. 
• Build Pipeline plugin: 
• Visualize your workflow, integrate manual steps. 
• Workflow plugin: 
• Define the relationship between your steps in a 
DSL. 
14
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 3: AUTOMATE JENKINS 
TASKS! 
15
Jenkins User Conference San Francisco #jenkinsconf 
The script console and Scriptler 
• Why do things by hand? 
• Get deep into Jenkins itself - control the 
internals and get full visibility of what’s 
happening. 
• Access the entire Jenkins model - make 
changes to jobs, find problem configurations 
and more. 
• Use Scriptler to store and share Groovy scripts 
for reuse. 
16
Jenkins User Conference San Francisco #jenkinsconf 
Some examples from the Scriptler 
catalogs 
• Disable/enable jobs matching a pattern 
• Clear the build queue 
• Set log rotation/discard old builds 
configuration across all jobs 
• Disable SCM polling at night across all jobs 
• Run the log rotator (actually discard old 
builds) for all jobs 
• etc… 
17
Jenkins User Conference San Francisco #jenkinsconf 
System Groovy build steps 
• Run system Groovy scripts as part of your 
actual build. 
• Note - gives full access to Jenkins to the 
build, so be careful about who can run it 
and what it does. 
• Good way to pilot plugin concepts or do 
things not big enough to be worth a plugin 
on their own. 
• Run Scriptler scripts as build steps - reuse 
system scripts in multiple builds easily. 18
Jenkins User Conference San Francisco #jenkinsconf 
Generate jobs programmatically 
• Jenkins REST API and CLI let you post new 
jobs and changes to jobs. 
• Or you can define your whole job and/or 
workflow of multiple jobs in a DSL. 
19
Jenkins User Conference San Francisco #jenkinsconf 
Some DSL-like plugins 
• Job DSL plugin 
• Full Groovy DSL for job definitions - check in 
your DSL and create your jobs as a build 
step. 
• DotCI plugin 
• Define your jobs in YAML and check them in 
- jobs created automatically. 
• The upcoming Literate plugin 
• Markdown-like syntax for defining your job in 
your project source. 
20
Jenkins User Conference San Francisco #jenkinsconf 
Workflow Plugin 
• Define multiple complex steps in just one 
relatively simple DSL. 
• Bleeding edge! This is *new*, so I haven’t 
really used it yet. 
• Requires a fairly new Jenkins version - 1.580+. 
No LTS supporting it yet. 
• New job type - you need to create your jobs 
over again. 
• Go see Kohsuke’s talk for a lot more 
information. 21
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 4: TEND YOUR PLUGIN 
GARDEN 
22
Jenkins User Conference San Francisco #jenkinsconf 
Dear Mr Jenkins, 
There are too many plugins these days. 
Please eliminate three hundred. 
P.S. - I am not a crackpot. 
23
Jenkins User Conference San Francisco #jenkinsconf 
Do you really need that plugin? 
• Don’t install plugins on the master if you 
aren’t going to actually use them. 
• Lots of duplication of functionality across 
plugins - pick the right one for the job. 
• Plugins can cause instability in areas you 
don’t expect, and can add to load and run 
time for jobs - why take a hit from plugins you 
don’t use? 
24
Jenkins User Conference San Francisco #jenkinsconf 
Clean up old plugins and their data 
• Uninstall unused/unneeded plugins. 
• In Manage Jenkins, watch for the note 
about old data - clear it out when you 
uninstall plugins, to slim down your job and 
build config files. 
• Speeds up loading of the master and 
individual jobs. 
25
Jenkins User Conference San Francisco #jenkinsconf 
My essential plugins 
• Job Config History 
• Disk Usage 
• Not any more - newer versions don’t scale well 
at all! 
• Static analysis plugins 
• xUnit 
• Translates lots of test output into junit format 
for Jenkins to consume. 
• Parameterized Trigger and Conditional Build Step 
• My Swiss Army Knife for build workflows! 
26
Jenkins User Conference San Francisco #jenkinsconf 
My essential plugins 
• Tool Environment 
• Use Jenkins’ tool auto installation from shell 
steps. 
• EnvInject 
• Seems to be the best option for setting env 
vars for your build in various ways. 
• Rebuild 
• Re-run parameterized builds easily. 
• Build Timeout 
• Builds hang. This plugin deals with hung builds. 27
Jenkins User Conference San Francisco #jenkinsconf 
Don’t take my word for it 
• These are *my* essential plugins, from my 
experience and for my use cases. 
• You may not need these plugins, you may 
need other plugins completely. 
• But these are plugins I think have a lot of 
versatility and value, and little risk. 
28
Jenkins User Conference San Francisco #jenkinsconf 
Remember the global configuration 
• Some plugins have global configuration 
settings you should remember to look at. 
• The defaults may not always work for you - 
and sometimes the defaults aren’t great 
choices. 
• Job Config History, for example 
• By default, saves “changes” for every 
Maven module separately! Ouch! 
29
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 5: INTEGRATE WITH OTHER 
TOOLS AND SERVICES 
30
Jenkins User Conference San Francisco #jenkinsconf 
Jenkins plays nicely with others 
• Thanks to Jenkins’ plugins and REST API, 
services and tools can easily interact with 
Jenkins and vice versa. 
• Trigger builds based on GitHub pull requests, 
update JIRA upon successful builds and 
much, much more. 
• I’ll only touch on a few such tools and 
services - you can find many more on the 
Jenkins wiki. 
31
Jenkins User Conference San Francisco #jenkinsconf 
Source Control! 
• …Well, yeah. 
• Moving on… 
32
Jenkins User Conference San Francisco #jenkinsconf 
Gerrit and GitHub pull requests 
• Gerrit Trigger (Hi, Robert!), GitHub Pull 
Request Builder, Jenkins Enterprise’s version 
of GitHub pull request builder - all very useful. 
• Build every proposed change, report back to 
the review tool. 
• With this, you can enable automatic 
merging of changes, promotion from branch 
to branch, and much more. 
33
Jenkins User Conference San Francisco #jenkinsconf 
JIRA 
• Update JIRA issues when commits with 
messages containing the issues are built. 
• Follow build fingerprints to update issues in 
related projects as well. 
• Generate JIRA release notes as part of the 
build process. 
34
Jenkins User Conference San Francisco #jenkinsconf 
Artifactory 
• Define credentials for deployment and 
artifact resolution globally across Jenkins 
jobs. 
• Override Maven distributionManagement on 
a per-job basis. 
• Restrict where Maven jobs and build steps 
will look to resolve artifacts. 
• Capture build info and relationship to 
artifacts in Artifactory. 
35
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 6: MAKE YOUR SLAVES 
FUNGIBLE 
36
Jenkins User Conference San Francisco #jenkinsconf 
Fungible? What does that mean? 
• “Fungibility is the property of a good or a 
commodity whose individual units are 
capable of mutual substitution.” 
• A fungible slave is a slave you can replace 
easily with another slave. 
• If one dies or is busy, no problem - just add 
another one. 
• The easier it is to add slaves, the easier your 
life is. 
37
Jenkins User Conference San Francisco #jenkinsconf 
How do you make your slaves 
fungible? 
• Make creating the environments easily 
repeatable. 
• Config management - Puppet, Chef, Ansible, 
etc. 
• Pre-baked images - cloud, PXE, etc, using 
something like Packer and config 
management to build them. 
• I have no opinion on config management tools 
- to be honest, it doesn’t really matter. Anything 
that can set up your environment consistently is 
good enough! 38
Jenkins User Conference San Francisco #jenkinsconf 
Reusability and flexibility 
• Try to make your slaves general purpose 
• Don’t make them customized solely for use 
by one job or set of jobs if you can avoid it. 
• Interchangeable slaves allow for more 
efficient usage. 
• If you need specific custom slaves, make 
them on-demand. 
• Don’t tie up static resources with slaves 
that won’t be used all the time. 
39
Jenkins User Conference San Francisco #jenkinsconf 
To the cloud! 
• More efficient usage of your resources. 
• Private cloud or public cloud - the goal is to 
avoid idle resources that can’t be used for 
anything else. 
• Mesos plugin with Docker is very intriguing - 
run your slaves as containers in a general 
purpose cluster! 
• Pre-bake your cloud slave images - faster 
startup time, more consistency. 
40
Jenkins User Conference San Francisco #jenkinsconf 
HABIT 7: JOIN THE COMMUNITY 
41
Jenkins User Conference San Francisco #jenkinsconf 
Get involved! 
• Write plugins. 
• Extend existing plugins! 
• Open JIRAs. 
• Fix bugs. 
• Get help on the mailing lists or IRC. 
• Help others too! 
42
Jenkins User Conference San Francisco #jenkinsconf 
QUESTIONS? 
43
Jenkins User Conference San Francisco #jenkinsconf 
Thank you for attending! 
44
Jenkins User Conference San Francisco #jenkinsconf 
Thank You To Our Sponsors 
Platinum Gold 
Silver Corporate

More Related Content

What's hot

Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Edureka!
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To JenkinsKnoldus Inc.
 
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...Edureka!
 
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...Edureka!
 
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaWhat is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaEdureka!
 
CI and CD with Jenkins
CI and CD with JenkinsCI and CD with Jenkins
CI and CD with JenkinsMartin Málek
 
Jenkins Introduction
Jenkins IntroductionJenkins Introduction
Jenkins IntroductionPavan Gupta
 
CI/CD Overview
CI/CD OverviewCI/CD Overview
CI/CD OverviewAn Nguyen
 
Introduce to Git and Jenkins
Introduce to Git and JenkinsIntroduce to Git and Jenkins
Introduce to Git and JenkinsAn Nguyen
 
Jenkins tutorial for beginners
Jenkins tutorial for beginnersJenkins tutorial for beginners
Jenkins tutorial for beginnersBugRaptors
 
Run Jenkins as Managed Product on ECS - AWS Meetup
Run Jenkins as Managed Product on ECS - AWS MeetupRun Jenkins as Managed Product on ECS - AWS Meetup
Run Jenkins as Managed Product on ECS - AWS MeetupPhilipp Garbe
 
Getting started with Jenkins
Getting started with JenkinsGetting started with Jenkins
Getting started with JenkinsEdureka!
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkinsAbe Diaz
 
Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CDHoang Le
 
Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration Amazon Web Services
 
Introduction to Docker Compose
Introduction to Docker ComposeIntroduction to Docker Compose
Introduction to Docker ComposeAjeet Singh Raina
 

What's hot (20)

Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
Jenkins Pipeline Tutorial | Continuous Delivery Pipeline Using Jenkins | DevO...
 
An Introduction To Jenkins
An Introduction To JenkinsAn Introduction To Jenkins
An Introduction To Jenkins
 
Jenkins tutorial
Jenkins tutorialJenkins tutorial
Jenkins tutorial
 
Jenkins
JenkinsJenkins
Jenkins
 
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...
CI CD Pipeline Using Jenkins | Continuous Integration and Deployment | DevOps...
 
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...
 
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | EdurekaWhat is Jenkins | Jenkins Tutorial for Beginners | Edureka
What is Jenkins | Jenkins Tutorial for Beginners | Edureka
 
CI and CD with Jenkins
CI and CD with JenkinsCI and CD with Jenkins
CI and CD with Jenkins
 
Jenkins Introduction
Jenkins IntroductionJenkins Introduction
Jenkins Introduction
 
Jenkins CI presentation
Jenkins CI presentationJenkins CI presentation
Jenkins CI presentation
 
CI/CD Overview
CI/CD OverviewCI/CD Overview
CI/CD Overview
 
Introduce to Git and Jenkins
Introduce to Git and JenkinsIntroduce to Git and Jenkins
Introduce to Git and Jenkins
 
Jenkins tutorial for beginners
Jenkins tutorial for beginnersJenkins tutorial for beginners
Jenkins tutorial for beginners
 
Run Jenkins as Managed Product on ECS - AWS Meetup
Run Jenkins as Managed Product on ECS - AWS MeetupRun Jenkins as Managed Product on ECS - AWS Meetup
Run Jenkins as Managed Product on ECS - AWS Meetup
 
Getting started with Jenkins
Getting started with JenkinsGetting started with Jenkins
Getting started with Jenkins
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
 
Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
 
Introduction to CI/CD
Introduction to CI/CDIntroduction to CI/CD
Introduction to CI/CD
 
Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration Continuous Delivery, Continuous Integration
Continuous Delivery, Continuous Integration
 
Introduction to Docker Compose
Introduction to Docker ComposeIntroduction to Docker Compose
Introduction to Docker Compose
 

Viewers also liked

DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and ComposeDockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and ComposeDocker, Inc.
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for TestingCarlos Sanchez
 
Rise of the Machines - Automate your Development
Rise of the Machines - Automate your DevelopmentRise of the Machines - Automate your Development
Rise of the Machines - Automate your DevelopmentSven Peters
 
Bamboo - an introduction
Bamboo - an introductionBamboo - an introduction
Bamboo - an introductionSven Peters
 
Ic maven jenkins_sonar
Ic maven jenkins_sonarIc maven jenkins_sonar
Ic maven jenkins_sonarRocío Muñoz
 
Dockercon2015 bamboo
Dockercon2015 bambooDockercon2015 bamboo
Dockercon2015 bambooSteve Smith
 
GitFlow, SourceTree and GitLab
GitFlow, SourceTree and GitLabGitFlow, SourceTree and GitLab
GitFlow, SourceTree and GitLabShinu Suresh
 
Git and GitHub for Documentation
Git and GitHub for DocumentationGit and GitHub for Documentation
Git and GitHub for DocumentationAnne Gentle
 
Master Continuous Delivery with CloudBees Jenkins Platform
Master Continuous Delivery with CloudBees Jenkins PlatformMaster Continuous Delivery with CloudBees Jenkins Platform
Master Continuous Delivery with CloudBees Jenkins Platformdcjuengst
 
Game of Codes: the Battle for CI
Game of Codes: the Battle for CIGame of Codes: the Battle for CI
Game of Codes: the Battle for CIAtlassian
 
Gitlab Training with GIT and SourceTree
Gitlab Training with GIT and SourceTreeGitlab Training with GIT and SourceTree
Gitlab Training with GIT and SourceTreeTeerapat Khunpech
 
Continuous Delivery with Jenkins and Wildfly (2014)
Continuous Delivery with Jenkins and Wildfly (2014)Continuous Delivery with Jenkins and Wildfly (2014)
Continuous Delivery with Jenkins and Wildfly (2014)Tracy Kennedy
 
Git 101 - Crash Course in Version Control using Git
Git 101 - Crash Course in Version Control using GitGit 101 - Crash Course in Version Control using Git
Git 101 - Crash Course in Version Control using GitGeoff Hoffman
 
Introduction to Git/Github - A beginner's guide
Introduction to Git/Github - A beginner's guideIntroduction to Git/Github - A beginner's guide
Introduction to Git/Github - A beginner's guideRohit Arora
 
Jenkins Docker
Jenkins DockerJenkins Docker
Jenkins DockerAlex Soto
 
Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners HubSpot
 
DevOps and Continuous Delivery reference architectures for Docker
DevOps and Continuous Delivery reference architectures for DockerDevOps and Continuous Delivery reference architectures for Docker
DevOps and Continuous Delivery reference architectures for DockerSonatype
 
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) ItalyClustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) ItalyGiovanni Toraldo
 

Viewers also liked (20)

DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and ComposeDockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
 
Using Docker for Testing
Using Docker for TestingUsing Docker for Testing
Using Docker for Testing
 
Rise of the Machines - Automate your Development
Rise of the Machines - Automate your DevelopmentRise of the Machines - Automate your Development
Rise of the Machines - Automate your Development
 
Bamboo - an introduction
Bamboo - an introductionBamboo - an introduction
Bamboo - an introduction
 
Ic maven jenkins_sonar
Ic maven jenkins_sonarIc maven jenkins_sonar
Ic maven jenkins_sonar
 
Dockercon2015 bamboo
Dockercon2015 bambooDockercon2015 bamboo
Dockercon2015 bamboo
 
GitFlow, SourceTree and GitLab
GitFlow, SourceTree and GitLabGitFlow, SourceTree and GitLab
GitFlow, SourceTree and GitLab
 
Git and GitHub for Documentation
Git and GitHub for DocumentationGit and GitHub for Documentation
Git and GitHub for Documentation
 
Master Continuous Delivery with CloudBees Jenkins Platform
Master Continuous Delivery with CloudBees Jenkins PlatformMaster Continuous Delivery with CloudBees Jenkins Platform
Master Continuous Delivery with CloudBees Jenkins Platform
 
Game of Codes: the Battle for CI
Game of Codes: the Battle for CIGame of Codes: the Battle for CI
Game of Codes: the Battle for CI
 
Gitlab Training with GIT and SourceTree
Gitlab Training with GIT and SourceTreeGitlab Training with GIT and SourceTree
Gitlab Training with GIT and SourceTree
 
Continuous Delivery with Jenkins and Wildfly (2014)
Continuous Delivery with Jenkins and Wildfly (2014)Continuous Delivery with Jenkins and Wildfly (2014)
Continuous Delivery with Jenkins and Wildfly (2014)
 
Git best practices workshop
Git best practices workshopGit best practices workshop
Git best practices workshop
 
Git 101 - Crash Course in Version Control using Git
Git 101 - Crash Course in Version Control using GitGit 101 - Crash Course in Version Control using Git
Git 101 - Crash Course in Version Control using Git
 
Introduction to Git/Github - A beginner's guide
Introduction to Git/Github - A beginner's guideIntroduction to Git/Github - A beginner's guide
Introduction to Git/Github - A beginner's guide
 
Jenkins Docker
Jenkins DockerJenkins Docker
Jenkins Docker
 
Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners Git 101: Git and GitHub for Beginners
Git 101: Git and GitHub for Beginners
 
DevOps and Continuous Delivery reference architectures for Docker
DevOps and Continuous Delivery reference architectures for DockerDevOps and Continuous Delivery reference architectures for Docker
DevOps and Continuous Delivery reference architectures for Docker
 
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) ItalyClustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
 
Tao zhang
Tao zhangTao zhang
Tao zhang
 

Similar to Seven Habits of Highly Effective Jenkins Users (2014 edition!)

JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...CloudBees
 
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...CloudBees
 
Jenkins Meetup Pune
Jenkins Meetup PuneJenkins Meetup Pune
Jenkins Meetup PuneUmesh Kumhar
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins UsersJules Pierre-Louis
 
Building your Continuous Delivery Toolkit @ JUC SF 2014
Building your Continuous Delivery Toolkit @ JUC SF 2014Building your Continuous Delivery Toolkit @ JUC SF 2014
Building your Continuous Delivery Toolkit @ JUC SF 2014XebiaLabs
 
Version your build process as you version your code
Version your build process as you version your codeVersion your build process as you version your code
Version your build process as you version your codeVincent Latombe
 
Jenkins User Conference: Building Your Continuous Delivery Toolkit
Jenkins User Conference: Building Your Continuous Delivery ToolkitJenkins User Conference: Building Your Continuous Delivery Toolkit
Jenkins User Conference: Building Your Continuous Delivery ToolkitXebiaLabs
 
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryCodifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryAlvin Huang
 
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...CloudBees
 
Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life DevOps.com
 
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsTYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsMichael Lihs
 
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/Hudson
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/HudsonEclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/Hudson
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/HudsonVladLica
 
JUC Europe 2015: Hey! What Did We Just Release?
JUC Europe 2015: Hey! What Did We Just Release?JUC Europe 2015: Hey! What Did We Just Release?
JUC Europe 2015: Hey! What Did We Just Release?CloudBees
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using JenkinsRogue Wave Software
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins UsersAndrew Bayer
 

Similar to Seven Habits of Highly Effective Jenkins Users (2014 edition!) (20)

JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
JUC Europe 2015: Jenkins-Based Continuous Integration for Heterogeneous Hardw...
 
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...
JUC Europe 2015: From Virtual Machines to Containers: Achieving Continuous In...
 
From Virtual Machines to Containers
From Virtual Machines to ContainersFrom Virtual Machines to Containers
From Virtual Machines to Containers
 
Jenkins Meetup Pune
Jenkins Meetup PuneJenkins Meetup Pune
Jenkins Meetup Pune
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users
 
Juc boston2014.pptx
Juc boston2014.pptxJuc boston2014.pptx
Juc boston2014.pptx
 
Basic Jenkins Guide.pptx
Basic Jenkins Guide.pptxBasic Jenkins Guide.pptx
Basic Jenkins Guide.pptx
 
Building your Continuous Delivery Toolkit @ JUC SF 2014
Building your Continuous Delivery Toolkit @ JUC SF 2014Building your Continuous Delivery Toolkit @ JUC SF 2014
Building your Continuous Delivery Toolkit @ JUC SF 2014
 
Version your build process as you version your code
Version your build process as you version your codeVersion your build process as you version your code
Version your build process as you version your code
 
Jenkins User Conference: Building Your Continuous Delivery Toolkit
Jenkins User Conference: Building Your Continuous Delivery ToolkitJenkins User Conference: Building Your Continuous Delivery Toolkit
Jenkins User Conference: Building Your Continuous Delivery Toolkit
 
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared LibraryCodifying the Build and Release Process with a Jenkins Pipeline Shared Library
Codifying the Build and Release Process with a Jenkins Pipeline Shared Library
 
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...
JUC Europe 2015: How to Optimize Automated Testing with Everyone's Favorite B...
 
Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life Achieving Full Stack DevOps at Colonial Life
Achieving Full Stack DevOps at Colonial Life
 
Build Time Hacking
Build Time HackingBuild Time Hacking
Build Time Hacking
 
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source ToolsTYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
TYPO3 Camp Stuttgart 2015 - Continuous Delivery with Open Source Tools
 
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/Hudson
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/HudsonEclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/Hudson
Eclipse DemoCamp Bucharest 2014 - Continuous Integration Jenkins/Hudson
 
JUC Europe 2015: Hey! What Did We Just Release?
JUC Europe 2015: Hey! What Did We Just Release?JUC Europe 2015: Hey! What Did We Just Release?
JUC Europe 2015: Hey! What Did We Just Release?
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using Jenkins
 
7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users7 Habits of Highly Effective Jenkins Users
7 Habits of Highly Effective Jenkins Users
 
Contineous integration
Contineous integrationContineous integration
Contineous integration
 

Recently uploaded

Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9Jürgen Gutsch
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Neo4j
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기Chiwon Song
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 

Recently uploaded (20)

Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Sustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire ThornewillSustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire Thornewill
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Program with GUTs
Program with GUTsProgram with GUTs
Program with GUTs
 
20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 

Seven Habits of Highly Effective Jenkins Users (2014 edition!)

  • 1. Jenkins User Conference San Francisco #jenkinsconf Seven Habits of Highly Effective Jenkins Users Andrew Bayer Build and Tools Architect, Cloudera Twitter: @abayer October 23, 2014 #jenkinsconf
  • 2. Jenkins User Conference San Francisco #jenkinsconf Who Am I? 2 Build and tools architect at Cloudera. Contributor to Jenkins core and author of plugins since spring 2009. Committer and PMC member of multiple Apache projects, inc. jclouds, Whirr, Bigtop… ASF Member and volunteer for maintaining builds.apache.org.
  • 3. Jenkins User Conference San Francisco #jenkinsconf What’s this talk about? These are lessons learned from maintaining multiple large Jenkins instances over the years. 3 Cloudera’s three masters with 1000+ jobs each (4800+ total! Ouch!) with dozens running at a time. builds.apache.org’s 1200+ jobs from ~120 different project teams. Oh, and my time on IRC, working on core and plugins, etc.
  • 4. Jenkins User Conference San Francisco #jenkinsconf Your mileage may vary • These habits can be valuable on every Jenkins instance. • Some will be more relevant for larger instances, those with more complex jobs, and/or production-critical instances. • But these are *my* recommendations - you need to learn what’s best for your Jenkins setup. 4
  • 5. Jenkins User Conference San Francisco #jenkinsconf HABIT 1: MAKE YOUR MASTER STABLE AND RESTORABLE 5
  • 6. Jenkins User Conference San Francisco #jenkinsconf Use LTS Releases • LTS release trains created every 12 weeks. • Active train updated three times before the next one starts. • Avoid bleeding edge instability. • LTS releases go through automated acceptance testing and a manual testing matrix before going out. 6
  • 7. Jenkins User Conference San Francisco #jenkinsconf Be conservative about upgrading plugins • Plugins can change a lot without it being obvious. • Backwards compatibility can sometimes break. • Example - Extended Email plugin recipient/trigger settings recently. • New features can be unstable/problematic in the wild. 7
  • 8. Jenkins User Conference San Francisco #jenkinsconf Have an upgrade testbed • Test out upgrades and new plugins in a testbed environment before going live in production. • Set up jobs to cover your plugin usage. • If possible, test your usage at scale. • Give significant changes a few days, at least, of builds running before going live. 8
  • 9. Jenkins User Conference San Francisco #jenkinsconf Back up your Jenkins configuration • Kind of obvious, isn’t it? =) • Lots of possible solutions • Within Jenkins, I recommend the thinBackup plugin. • Full copies of $JENKINS_HOME work great, but can be slow and use lots of disk. • Config files can be copied without copying all the builds, etc as well - see an example here 9
  • 10. Jenkins User Conference San Francisco #jenkinsconf Avoid using the Maven job type • Maven build steps are perfectly fine in freestyle jobs, but the Maven plugin’s Maven job type is…questionable. • Implementation leads to some potential problems in plugin support, lazy loading of projects, interception of Maven execution internals, etc… • I’ve seen a lot of strange edge case problems show up with it at scale. Be careful with it. 10
  • 11. Jenkins User Conference San Francisco #jenkinsconf HABIT 2: BREAK UP THE BLOAT 11
  • 12. Jenkins User Conference San Francisco #jenkinsconf Multiple Masters • If you have a lot of projects and teams, multiple masters give you a lot more agility and control. • Split up masters by team, function, access, etc. • Makes it easier to restart for plugin installs/upgrades, etc without disrupting everyone. • More masters with less jobs each are more stable, less prone to edge case bugs. 12
  • 13. Jenkins User Conference San Francisco #jenkinsconf Break up your jobs • Modularization and reuse are good in programming - and good in Jenkins too. • Multi-job builds allow reusability of generic jobs across multiple projects, releases, etc. • Few things more frustrating than a 10 hour build crashing 9.5 hours in. • Multi-job builds can be relaunched at any step in the process, if designed properly. 13
  • 14. Jenkins User Conference San Francisco #jenkinsconf Tools for breaking up your jobs • Just some examples - there are a ton of ways you are able to do this in Jenkins. • Parameterized Trigger + Conditional Build Step, Copy Artifact, Promoted Builds: • Very powerful, not as easy to configure. • Build Pipeline plugin: • Visualize your workflow, integrate manual steps. • Workflow plugin: • Define the relationship between your steps in a DSL. 14
  • 15. Jenkins User Conference San Francisco #jenkinsconf HABIT 3: AUTOMATE JENKINS TASKS! 15
  • 16. Jenkins User Conference San Francisco #jenkinsconf The script console and Scriptler • Why do things by hand? • Get deep into Jenkins itself - control the internals and get full visibility of what’s happening. • Access the entire Jenkins model - make changes to jobs, find problem configurations and more. • Use Scriptler to store and share Groovy scripts for reuse. 16
  • 17. Jenkins User Conference San Francisco #jenkinsconf Some examples from the Scriptler catalogs • Disable/enable jobs matching a pattern • Clear the build queue • Set log rotation/discard old builds configuration across all jobs • Disable SCM polling at night across all jobs • Run the log rotator (actually discard old builds) for all jobs • etc… 17
  • 18. Jenkins User Conference San Francisco #jenkinsconf System Groovy build steps • Run system Groovy scripts as part of your actual build. • Note - gives full access to Jenkins to the build, so be careful about who can run it and what it does. • Good way to pilot plugin concepts or do things not big enough to be worth a plugin on their own. • Run Scriptler scripts as build steps - reuse system scripts in multiple builds easily. 18
  • 19. Jenkins User Conference San Francisco #jenkinsconf Generate jobs programmatically • Jenkins REST API and CLI let you post new jobs and changes to jobs. • Or you can define your whole job and/or workflow of multiple jobs in a DSL. 19
  • 20. Jenkins User Conference San Francisco #jenkinsconf Some DSL-like plugins • Job DSL plugin • Full Groovy DSL for job definitions - check in your DSL and create your jobs as a build step. • DotCI plugin • Define your jobs in YAML and check them in - jobs created automatically. • The upcoming Literate plugin • Markdown-like syntax for defining your job in your project source. 20
  • 21. Jenkins User Conference San Francisco #jenkinsconf Workflow Plugin • Define multiple complex steps in just one relatively simple DSL. • Bleeding edge! This is *new*, so I haven’t really used it yet. • Requires a fairly new Jenkins version - 1.580+. No LTS supporting it yet. • New job type - you need to create your jobs over again. • Go see Kohsuke’s talk for a lot more information. 21
  • 22. Jenkins User Conference San Francisco #jenkinsconf HABIT 4: TEND YOUR PLUGIN GARDEN 22
  • 23. Jenkins User Conference San Francisco #jenkinsconf Dear Mr Jenkins, There are too many plugins these days. Please eliminate three hundred. P.S. - I am not a crackpot. 23
  • 24. Jenkins User Conference San Francisco #jenkinsconf Do you really need that plugin? • Don’t install plugins on the master if you aren’t going to actually use them. • Lots of duplication of functionality across plugins - pick the right one for the job. • Plugins can cause instability in areas you don’t expect, and can add to load and run time for jobs - why take a hit from plugins you don’t use? 24
  • 25. Jenkins User Conference San Francisco #jenkinsconf Clean up old plugins and their data • Uninstall unused/unneeded plugins. • In Manage Jenkins, watch for the note about old data - clear it out when you uninstall plugins, to slim down your job and build config files. • Speeds up loading of the master and individual jobs. 25
  • 26. Jenkins User Conference San Francisco #jenkinsconf My essential plugins • Job Config History • Disk Usage • Not any more - newer versions don’t scale well at all! • Static analysis plugins • xUnit • Translates lots of test output into junit format for Jenkins to consume. • Parameterized Trigger and Conditional Build Step • My Swiss Army Knife for build workflows! 26
  • 27. Jenkins User Conference San Francisco #jenkinsconf My essential plugins • Tool Environment • Use Jenkins’ tool auto installation from shell steps. • EnvInject • Seems to be the best option for setting env vars for your build in various ways. • Rebuild • Re-run parameterized builds easily. • Build Timeout • Builds hang. This plugin deals with hung builds. 27
  • 28. Jenkins User Conference San Francisco #jenkinsconf Don’t take my word for it • These are *my* essential plugins, from my experience and for my use cases. • You may not need these plugins, you may need other plugins completely. • But these are plugins I think have a lot of versatility and value, and little risk. 28
  • 29. Jenkins User Conference San Francisco #jenkinsconf Remember the global configuration • Some plugins have global configuration settings you should remember to look at. • The defaults may not always work for you - and sometimes the defaults aren’t great choices. • Job Config History, for example • By default, saves “changes” for every Maven module separately! Ouch! 29
  • 30. Jenkins User Conference San Francisco #jenkinsconf HABIT 5: INTEGRATE WITH OTHER TOOLS AND SERVICES 30
  • 31. Jenkins User Conference San Francisco #jenkinsconf Jenkins plays nicely with others • Thanks to Jenkins’ plugins and REST API, services and tools can easily interact with Jenkins and vice versa. • Trigger builds based on GitHub pull requests, update JIRA upon successful builds and much, much more. • I’ll only touch on a few such tools and services - you can find many more on the Jenkins wiki. 31
  • 32. Jenkins User Conference San Francisco #jenkinsconf Source Control! • …Well, yeah. • Moving on… 32
  • 33. Jenkins User Conference San Francisco #jenkinsconf Gerrit and GitHub pull requests • Gerrit Trigger (Hi, Robert!), GitHub Pull Request Builder, Jenkins Enterprise’s version of GitHub pull request builder - all very useful. • Build every proposed change, report back to the review tool. • With this, you can enable automatic merging of changes, promotion from branch to branch, and much more. 33
  • 34. Jenkins User Conference San Francisco #jenkinsconf JIRA • Update JIRA issues when commits with messages containing the issues are built. • Follow build fingerprints to update issues in related projects as well. • Generate JIRA release notes as part of the build process. 34
  • 35. Jenkins User Conference San Francisco #jenkinsconf Artifactory • Define credentials for deployment and artifact resolution globally across Jenkins jobs. • Override Maven distributionManagement on a per-job basis. • Restrict where Maven jobs and build steps will look to resolve artifacts. • Capture build info and relationship to artifacts in Artifactory. 35
  • 36. Jenkins User Conference San Francisco #jenkinsconf HABIT 6: MAKE YOUR SLAVES FUNGIBLE 36
  • 37. Jenkins User Conference San Francisco #jenkinsconf Fungible? What does that mean? • “Fungibility is the property of a good or a commodity whose individual units are capable of mutual substitution.” • A fungible slave is a slave you can replace easily with another slave. • If one dies or is busy, no problem - just add another one. • The easier it is to add slaves, the easier your life is. 37
  • 38. Jenkins User Conference San Francisco #jenkinsconf How do you make your slaves fungible? • Make creating the environments easily repeatable. • Config management - Puppet, Chef, Ansible, etc. • Pre-baked images - cloud, PXE, etc, using something like Packer and config management to build them. • I have no opinion on config management tools - to be honest, it doesn’t really matter. Anything that can set up your environment consistently is good enough! 38
  • 39. Jenkins User Conference San Francisco #jenkinsconf Reusability and flexibility • Try to make your slaves general purpose • Don’t make them customized solely for use by one job or set of jobs if you can avoid it. • Interchangeable slaves allow for more efficient usage. • If you need specific custom slaves, make them on-demand. • Don’t tie up static resources with slaves that won’t be used all the time. 39
  • 40. Jenkins User Conference San Francisco #jenkinsconf To the cloud! • More efficient usage of your resources. • Private cloud or public cloud - the goal is to avoid idle resources that can’t be used for anything else. • Mesos plugin with Docker is very intriguing - run your slaves as containers in a general purpose cluster! • Pre-bake your cloud slave images - faster startup time, more consistency. 40
  • 41. Jenkins User Conference San Francisco #jenkinsconf HABIT 7: JOIN THE COMMUNITY 41
  • 42. Jenkins User Conference San Francisco #jenkinsconf Get involved! • Write plugins. • Extend existing plugins! • Open JIRAs. • Fix bugs. • Get help on the mailing lists or IRC. • Help others too! 42
  • 43. Jenkins User Conference San Francisco #jenkinsconf QUESTIONS? 43
  • 44. Jenkins User Conference San Francisco #jenkinsconf Thank you for attending! 44
  • 45. Jenkins User Conference San Francisco #jenkinsconf Thank You To Our Sponsors Platinum Gold Silver Corporate

Editor's Notes

  1. Yes, I made up the title
  2. builds.a.o is particularly challenging due to having 600+ jobs using the Maven plugin’s project type - see later
  3. - Might be wise to wait for .2 LTS releases, but rarely an issue - Plugins generally are dependent on LTS releases, so if you’re working on a plugin, inherit from an LTS unless you absolutely need something newer
  4. Disk usage could be mentioned but that’d be rude. Incompat sometimes shown with semver, i.e., git 2.0, but not always
  5. Something like JOC can help with this, presumably.
  6. Gist shows selective rsyncing to a git repo. Git repo gets stupid big, mind you. Probably could use a better way to do plugin backups.
  7. My issues - less with Maven interaction, more with scaling and plugin interaction due to the fact that each individual module is represented by a class inheriting from AbstractProject itself.
  8. JOC can definitely help with the management issues here. In practice, in my experience, 3 masters with x jobs each are more performant sharing a decent physical host than 1 master with 3x jobs.
  9. - There are definitely more I’m not mentioning because I’m not familiar with them. Sorry!
  10. Laziness is a virtue.
  11. Often you can do any of this with an existing plugin, but sometimes…
  12. DSL plugin more powerful than DotCI plugin - stupid amount of plugins supported, Groovy XML builder syntax makes it easy to do arbitrary ones. I like it more, I admit it.
  13. - Last count of my update center - 1000+ plugins. Yowza. That’s a lot. - Check usage count - help see what the most popular plugin for your functionality is
  14. Some plugins are particularly bad about cleaning up their own old data. Watch out for that.
  15. Disk usage plugin - may get better, but right now, it’s ugly with lazy loading of builds (i.e., it ends up causing way too much loading), it holds onto the executor to do the workspace size check at the end of each build, etc…
  16. Note - toolenv and envinject don’t work together - can’t use toolenv vars in envinject vars And envinject is kinda wonky sometimes.
  17. Personal preference for GitHub PR builder - the Jenkins Enterprise one. Less opinionated, ties into the validated merge plugin if you want to automate the merge process further.
  18. I do love me the word fungible. Yessir.