CloudBees' webinar slides: 7 Ways to Optimize Hudson in Production. Webinar delivered by Kohsuke Kawaguchi - the founder of Hudson. Video of the Webinar available on http://www.youtube.com/cloudbeestv

1. 7 Ways To Optimize Hudson/Jenkins
for Production Installation
Kohsuke Kawaguchi
Creator of the Hudson project
©2011 Cloud Bees, Inc. All Rights Reserved

2. About CloudBees
Our
Mission
Become
the
leading
Java™
Pla3orm
as
a
Service
(PaaS)
Why
We’re
CloudBees
services
the
complete
lifecycle
of
Cloud
Different
applica>on
development
and
deployment.
No
Servers.
No
Virtual
Machines.
No
IT.
Strategy
§ DEV@cloud
–
Cloud
Services
for
Developers
§ RUN@cloud
–
Fric@onless
run@me
PaaS
for
Java
apps
©2011 Cloud Bees, Inc. All Rights 2
Reserved

3. Continuous Integration - Hudson
Nectar
–
On-­‐Premise
Enterprise
Hudson
• Support
from
the
Hudson
experts.
Hudson Adoption, by
• VMware
scale
your
Hudson
environment.
Builds
25,000
• Enterprise
Features
extend
Hudson
for
large
environments.
20,000
• Integrate
with
the
Cloud
integra@on
with
DEV@Cloud
and
RUN@Cloud
coming
15,000
Benefits
of
DEV@cloud
Hudson
Service:
10,000
• Scale
your
Hudson
environment
with
the
power
of
the
Cloud
5,000
• Ease
your
Hudson
management
overhead
• Speed
your
Hudson
builds
0
• Save
money
with
on-­‐demand
Hudson
Service.
Source: Hudson Labs
©2011 Cloud Bees, Inc. All Rights 3
Reserved

4. Idea Behind This Webinar
• 7 relatively easy techniques you can
apply today
• It helps to spend some time setting it up
“right”
– Even if Hudson is easy enough to deploy
– Especially for serious production use
• It’s not too late even if you are already
running Hudson
©2011
CloudBees,
Inc.
AInc. All Reserved
©2011 CloudBees, ll
Rights
Rights 4
Reserved

5. #1: Backup
• Better late than never
• Useful for
– Disaster recovery
– Accidental configuration changes
©2011
CloudBees,
Inc.
All
Rights
Reserved
5

6. #1: Backup Planning
• Everything is under $HUDSON_HOME
– Check system config to see where it is
– No need to backup slaves
• Live vs offline
– Live backup is OK
– Configuration writes are atomic
©2011
CloudBees,
Inc.
All
Rights
Reserved
6

7. #1: Subsetting backup
• /war (exploded war)
• /cache (downloaded tools)
• /tools (extracted tools)
• /jobs/*
– builds (build records)
– builds/*/archive (archived artifacts)
– workspace
©2011
CloudBees,
Inc.
All
Rights
Reserved
7

8. #1: More on backup
• File system-level snapshot for maximum
consistency
– LVM
– Other storage-level snapshots
• Don’t forget fsfreeze
– ZFS, if you are on Solaris
• Incremental, too
• Nectar can help you
©2011
CloudBees,
Inc.
All
Rights
Reserved
8

9. #1: Testing restore
• Just copy it somewhere and “java -jar
hudson.war --httpPort=9999”
– HUDSON_HOME is relocateable
©2011
CloudBees,
Inc.
All
Rights
Reserved
9

10. #2: Disk
• Prepare for disk usage growth
– Especially when you start to host jobs from
people who aren’t close to you
– Make sure you can throw more disks at the
problem later
• No need to waste money on 15000rpm
SCSI disks
– But bigger disk is nice
©2011
CloudBees,
Inc.
All
Rights
Reserved
10

11. #2: Expandable volume
• “Spanned volume” on Windows
• LVM on Linux
– This needs the most upfront planning
• ZFS
– Shame there’s no viable free Solaris flavor
– $HUDSON_HOME should be its own FS
• If it’s too late, symlink is your friend
©2011
CloudBees,
Inc.
All
Rights
Reserved
11

12. #3: Native packages
• Comes in .deb/.rpm
– Much easier to install/upgrade
– Reproducible
– init script & daemon user
• Configuration in
– /etc/default/hudson
– /etc/sysconfig/hudson
• Windows --- work in progress
• Existing $HUDSON_HOME can be
migrated
©2011
CloudBees,
Inc.
All
Rights
Reserved
12

13. #4: Distributed builds
• You will grow beyond a single system
– Load is not the only factor
• Better isolation between builds
• Diversity in the test/build platform
– Cheaper to scale horizontally
• Let Hudson use multiple computers
©2011
CloudBees,
Inc.
All
Rights
Reserved
13

14. Demo

15. #4: Slave reconnectivity
• Let master launch slaves (if you can)
– Via SSH or DCOM
– Easier to keep the cluster up & running
– It’s amazing how easily a computer gets sick
• Set up SSH public key authentication
– Login to systems without typing password
– Must-have! Inexcusable!
– Only takes 5 minutes or so to set up
©2011
CloudBees,
Inc.
All
Rights
Reserved
15

16. #5: Labels
• Treat build machines like livestock, not
like pets
– Don’t tie builds to specific build machine
– Build machines should be interchangeable
• Labels, instead of names
– Tag slaves by their capabilities/environments
– Jobs can be tied to boolean expression of
labels
• Better resource utilization, manageability
©2011
CloudBees,
Inc.
All
Rights
Reserved
16

17. Demo

18. #6: Invest in good URL
• If your users can’t see Hudson, much of
the benefit is lost
– Make the URL easier to remember
hVp://sca14-­‐3530.sca.cloudbees.com:8080/hudson/
hVp://hudson.cloudbees.com/
©2011
CloudBees,
Inc.
All
Rights
Reserved
18

19. #6: Use service name
• Get the host alias, instead of using
primary machine name
– If your IT isn’t helping, use dynamic DNS
– This also makes your service relocatable
©2011
CloudBees,
Inc.
All
Rights
Reserved
19

20. #6: Share port 80 with other apps
• Apache reverse proxy
– Let you run Hudson in non-root, too
browser
Apache
Hudson
• For Windows
– IIS7+URL Rewrite+Application Request
Routing
©2011
CloudBees,
Inc.
All
Rights
Reserved
20

21. #6: No “/hudson”
• Use virtual host to distinguish multiple
apps, not context path
Hudson
browser
Apache
/
IIS
hVp://hudson.cloudbees.com/
Redmine
hVp://redmine.cloudbees.com/
©2011
CloudBees,
Inc.
All
Rights
Reserved
21

22. #7: Keeping build records under control
• Discard old build records if you can
– Helps start-up time, memory usage
– Often feasible, for example CI builds
• Setting is per-project basis
• The point is to avoid unbounded
consumption. Don’t go overboard
©2011
CloudBees,
Inc.
All
Rights
Reserved
22

23. Demo

24. Conclusion
• 7 tips
– Backup
– Disk planning
– Native packages
– Distributed builds
– Labels
– Reverse proxy
– Discard build records
©2011
CloudBees,
Inc.
All
Rights
Reserved
24

25. CloudBees Resources
• CloudBees
– http://www.cloudbees.com/
• Nectar
– http://nectar.cloudbees.com/
• Try Dev@Cloud
– https://grandcentral.cloudbees.com/account/signup
• Register for news from CloudBees
– http://www.cloudbees.com/company.cb
• CloudBees on YouTube: new videos soon
– http://www.youtube.com/user/CloudBeesTV
©2011
CloudBees,
Inc.
All
Rights
Reserved
25

26. Coming Soon: 1 Day Hudson Training
• Instructor-led
• More comprehensive, all-round
• London, New York, and Tokyo
• To receive updates:
– http://www.cloudbees.com/company.cb
©2011
CloudBees,
Inc.
All
Rights
Reserved
26

27. Q&A
©2011 CloudBees, Inc. All Rights
Reserved