4. Where were we for CI? Compile 11,000 Junit Tests 3000 Functional Tests Deploy JIRA Artifacts Minimum 8 Hours in Serial
5. Where were we for CI? Compile 11,000 Junit Tests 3000 Functional Tests Deploy JIRA Artifacts We had to break things up
6. Where were we for CI? Test 1 SCM Trigger Test 6 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15
7. We still had to…. Compile 11,000 Junit Tests 3000 Functional Tests Deploy JIRA Artifacts Minimum 8 Hours in Serial
8. Unable to Aggregate Results 15 builds with 15 results Test 1 SCM Trigger Test 6 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15 ? ? ? ? ?
9.
10.
11.
12.
13.
14. JIRA CI Design Goals Functional Tests Selenium Web Browser Tests Performance Tests Database Tests Supported Plugins Tests Inner Loop CI Outer Loop CI Distribution Tests
15.
16.
17. Seems like a mountain of tests to climb and is out of reach
20. Stages transforms our old CI 15 builds with 15 results Test 1 SCM Trigger Test 6 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15
21. We still had to…. Compile 11,000 Junit Tests 3000 Functional Tests Deploy JIRA Artifacts Minimum 8 Hours in Serial
22. Stages transforms our old CI 15 builds with 15 results Test 1 SCM Trigger Test 3 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15 1 build 15 jobs 1 result
23. New CI Plan: Add a Stage Test 1 Test 3 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15 Deploy Artifacts to Maven Nexus Repository Continue to Next Stage if Previous Passes One Build Plan
30. Remove Redundancies Test 1 Test 3 Test 11 Test 2 Test 7 Test 12 Test 3 Test 8 Test 13 Test 4 Test 9 Test 5 Test 10 Test 14 Test 15 Produce Application Artifacts
45. Our CI Future Infrastructure Improvements DVCS Bamboo upgrade from 3.0 to 3.1.1: Tasks Personal Builds Bamboo 3.2 Amazon enhancements Bamboo 3.2 restart failed tasks
46. The Story… JIRA CI Past Problems Slow, unreliable Created New CI Goals Faster promotional CI Bamboo Staging Aggregated Results, Deployment Bamboo Artifact Passing Speed, Re-use, Deployment Outsourcing Infrastructure Amazon, Admin, Attractive Pricing Deliver Awesome
Editor's Notes
Finally if we got a successful CI build, we would have our artifacts deployed to Maven, but it took us 8 hours to run this build in addition to any previous failed builds.
We can’t run CI in 8 hours
So we had to break things up. But the sad story is we have no way of aggregating the results of all 15 jobs. Though to get test feedback we did, of course, speed up this part of the CI loop.
Though we broke up our build, we still had to run our serialized testing build to test and deploy our maven jira artifacts. We wanted to have artifacts fully tested because so many of our other applications depend on these artifacts to build like the numerous great plugins that are available.
So we had to break things up. But the sad story is we have no way of aggregating the results of all 15 jobs. Though to get test feedback we did, of course, speed up this part of the CI loop.
A developer validates change locally, updates their checkout workspace, runs tests, then commits. Bamboo detects the changes in SCM and launches our functional tests and selenium web browser tests. We have these both in two plans and then we deploy our artifacts to maven if our functional test plan passes. On a nightly basis we then can consume the latest passing JIRA artifacts into our four sets of testing. We do performance regression tests for obvious reasons. By testing the SAME functional tests now against different databases and different distributions, we can ENSURE that any tests failures here are because of the database or distribution itself failing. We especially get this with tests against Oracle. Finally, we also want to test JIRA plugins that need to consume the JIRA artifacts as a dependency to ensure that these also do not break with the changes to JIRA.
If our Outer Loops passes we can then be confident about dogfooding JIRA inside Atlassian on our production environments to get soak time. This will then lead to more confidence to release JIRA to customers.
Wow that’s a lot of tests in your Outer Loop CI!!!
We have Bamboo 3.1!
Adding stages and jobs are fast, especially when we can clone jobs. Each stage must pass in order to go onto the next.
Recall our old build… and now we give it an extreme makeover
Though we broke up our build, we still had to run our serialized testing build to test and deploy our maven jira artifacts. We wanted to have artifacts fully tested because so many of our other applications depend on these artifacts to build like the numerous great plugins that are available.
Recall our old build… and now we give it an extreme makeover
We can have our broken up functional tests and now be able to act upon those results because they are aggregated through stages. One build now runs faster and we can do more with those results.
Our old selenium CI plan with stages has a timing of 61 minutes +/- 5 minutes depending on my elastic agent startup times and IO on those agents.
Each of our test jobs checkout JIRA source from SVN and then compiles all of the JIRA artifacts. We think that’s a waste of time that each job does this and would like to use artifact passing for re-use and to reduce times.
Adding a producer job to build a set of shared artifacts is straight-forward in Bamboo
Our new selenium CI plan is reduced to 38 minutes +/- 5 minutes depending on my elastic agent startup times and IO on those agents.
Adding stages and jobs are fast, especially when we can clone jobs. Each stage must pass in order to go onto the next.
I can get some cheap infrastructure with amazon and spot pricing now in Bamboo 3.1!
JIRA’s build server is able to get a spot instance for each linux and windows at greatly reduced prices. We had one here, that was not able to be a spot instance and was still able to boot with regular pricing to give us the infrastructure we need.
I save $2000 per month by using spot instances for my JIRA bamboo server!!!!
Which one of these factors do you want to start with to get your builds going in the right direction or reliable, faster, and cheaper?