Presentation about the new gen CI delivered at POSS 2015

1. CI is dead, long live CI!
Frédéric Lepied
Senior Engineering Manager
Member of Technical Staff
Nov 2015

2. POSS 20152
AGENDA
● Staging CI
● Gating CI
● DIY

3. Staging CI

4. POSS 2015
Definition
4
Staging CI
Pipeline of jobs triggered on a new commit in a VCS
Review Change Commit Change Build Artifacts Run Tests

5. POSS 2015
What do we do when a job fails?
5
Staging CI
According to Jez Humble and David Farley in the reference book
Continuous Delivery:
● “Don’t check in on a broken Build”
● To fix the issue:
○ “Never Go Home on a Broken Build”
○ “Always Be Prepared to Revert to the Previous Revision”

6. POSS 2015
Mitigation Strategies
6
Staging CI
● Use an integration branch before merging to master
● Run all the tests on the developer workstation before submitting

7. POSS 2015
Next?
7
Staging CI
So you need discipline to avoid a broken CI.
● Difficult at scale
● Difficult on Free Software projects as the level of implication is not
equal between participants

8. Gating CI

9. POSS 2015
Introduction to next gen CI
9
Gating CI
CI is now centered on the code review system:
Review Change Merge ChangeBuild Artifacts Run Tests
● No more broken builds
● Ease the reviewing process by filtering with the tests
● Promote parallel work

10. POSS 2015
Example of OpenStack
10
Gating CI
This CI process has been implemented by the OpenStack project
successfully. Every day:
● 1,000 proposed patchsets
● 7,500 posted comment and votes on the code review system
● 16,000 test environments spawn
● 250 changes merged

11. POSS 2015
Components used by the OpenStack CI
11
Gating CI
Local changes
submitted with
git-review
Gerrit code
review system
git.openstack.org
mirror
github
mirror
Zuul Gearman
Jenkins
servers
Nodepool
Jenkins
slaves

12. POSS 2015
Benefits of the OpenStack CI
12
Gating CI
● No broken master
● Parallel testing using speculative merging
● Cross project dependencies at the changeset level
● Optimized costs by provisioning the slaves on-demand

13. DIY

14. POSS 2015
Components
14
DIY
All the components are Free Software projects
● Download from source or pre-built packages
● Puppet modules for easy configuration

15. POSS 2015
Software Factory
15
DIY
Alternatively, you can use the pre-built integration from softwarefactory-project.io
● ready to use VM with the OpenStack CI components: Gerrit, Zuul, Jenkins and
Nodepool
● Extras:
○ prebuilt integration and delivery pipelines in Zuul
○ SSO
○ web menu and dashboard
○ configuration API and CLI
○ bug tracking system (Redmine)
○ collaboration tools: paste and etherpad

16. POSS 2015

17. POSS 2015

18. POSS 2015
Conclusion
18
DIY
No more broken master!
● re-use OpenStack CI components for your own needs
● help us improve Software Factory

19. Questions?

20. Contact: frederic.lepied@redhat.com
@flepied

21. THANK YOU
plus.google.com/+RedHat
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNews