5. Spend more CPU power to help you
> … even if it only helps a little
> First on your laptops and workstations
IDEs are at the forefront
> And then to the servers
a.k.a. “Continuous Integration”
More frequent build/test executions
Static code analysis tools
And more to come
5
6. Hudson https://hudson.dev.java.
net/
> Open-source CI server at java.net
> Emphasis on ease of installation and use
“java -jar hudson.war” execution
Configure everything from browsers
> Extensibility
140+ community-developed public plugins
By 150+ contributors
> Estimated 13,000 installations
6
7. It basically does builds and tests
> Check out the source code
Subversion, Perforce, Git, Mercurial, CVS, …
> Do builds and/or tests
Ant, Maven, MSBuild, shell script, …
> Record results
Binary, test results, code coverage, static analysis
> Notify people
E-mail, IM, RSS, tray apps, IDEs
7
14. Why Distributed Builds?
> You need to use multiple computers because…
You need different environments
You need isolation
> There’s only so much you can do with 1 computer
14
15. Installing new slaves
> For first 20 or so slaves, we did it manually
Insert CD, click, type, click, type, click, …
But that doesn’t scale
> Then we automated
Available as “Hudson PXE Plugin”
15
16. Automated System Installations
> Hudson + PXE plugin
ISO images of OS
> Slaves
Power on, hit F12
PC boots from network (PXE)
16
17. Automated System Installations
> Hudson + PXE plugin
ISO images of OS
Your corporate IT guy &
his DHCP server
> Slaves
Power on, hit F12
PC boots from network (PXE)
Choose OS from menu
Installs non-interactively
17
18. Automated System Installations
> Supports OpenSolaris, Ubuntu, CentOS, Fedora
Trivial with most Linux
> Cooperate with Windows, too
> Quite useful outside Hudson, too
No more broken CD drives
No more CD-Rs
18
19. Distributed builds with Hudson
> Master
Serves HTTP requests
Stores all important info
> Slaves
170KB single JAR
Assumed to be unreliable
Scale to at least 100
> Link
Single bi-di byte stream
No other requirements
19
20. Heterogeneous Cluster Challenge
> Your builds/tests need to run in specific
environment
> Dependency on individual nodes hurts utilization
jobs slaves
Wombat Window
Windows test
s #1
GlassFish Window
Windows test
s #2
Hudson Solaris
Windows test
#1
Hudson Solaris
test
20
21. Labels to rescue
> Label is a group of slaves
> Tie jobs to labels
jobs slaves
Wombat Window
Windows test
s #1
Windows
GlassFish Window
Windows test
s #2
Hudson Solaris
Windows test
Solaris #1
Hudson Solaris Window
test
s #3
21
22. Coming to
your
Hudson
Making builds sticky soon
> We want jobs to be mostly on the same slave
Faster check out
Consistent results
Minimizes disk consumption
> But does it softly
> Hudson uses consistent hash* to achieve this
> More schedule controls become possible:
Use faster machines more frequently
Slowly ramp up newly installed slaves
* http://en.wikipedia.org/wiki/Consistent_hashing 22
23. Forecasting failures
> Hudson monitors key health metrics of slaves
Low disk space, insufficient swap
Clock out of synch
Extensible
> Slaves go offline automatically
> Catch problems before they break builds
23
26. Hudson made this extensible
> Hudson detects excessive workload
> Hudson notifies plugins
> Plugins can provision more slaves
… assuming that you have that infrastructure
26
28. Amazon EC2: The Good
> Pay as you go (10¢/h or so)
Loads on Hudson tend to be spiky
> Programmable API
> Instances launch at machine-speed
> EC2 instances are forgetful
28
29. Amazon EC2: The Bad
> Your data is still inside your firewall
Takes time to check out code
… or to archive build artifacts
Some data just can’t be moved
> EC2 instances are forgetful
> Can your tests run in parallel?
29
30. Hudson EC2 plugin
> Built on top of typica*
> What does it do?
Automatically provisions slaves on EC2 on demand
Picks the right AMI depending on demand
Starts slave agent
Shuts down unused instances
* http://code.google.com/p/typica/ 30
31. Hudson “Appliance” on EC2
> Run the master in the cloud too, if you like
Hudson on stock OpenSolaris AMI
Data stored persistently in Elastic Block Storage
Dynamically expandable thanks to ZFS
Online, too
> Packaged as a wizard
31
32. Hudson EC2 plugin usage
> Tell Hudson what AMIs you want to start
32
34. Conclusion
> CI is here to stay
We’ll continue to push more workload to servers
> Hudson makes this easy for you
> Reap the benefit of a cluster in multiple ways
34
35. Resources
> http://hudson.dev.java.net/
> Support Subscription
hudson@sun.com
35