These are the slides of my talk at Foss.IN 2012 on Continuous Integration for Open Source Distribution ecosystems. I have covered how existing Continuous Integration efforts can be improved to ensure greate collaboration between open source projects, developers and distributions to make earlier r

1. Continuous Integration
For
Open Source Distributions
Sriram Narayanan
sriram@belenix.org
@sriramnrn

2. About me
Belenix Tester + Developer
Thoughtworker

3. Why we work on distros
● Innovate
● more versatile installation
● faster boot up times
● experimental drivers
● Learn package management (Rolling updates!)
● Learn how a distro works (LFS,BLFS + more)

4. Pain points for a Distro Maintainer
● Repeat Tasks
● Dependency Tracking and Management
● Manual Testing

5. Pain points for a Distro Maintainer
● Repeat Tasks
● Track upstream changes and fixes
● Regular Distro builds
● Changelog and Release Notes
● Documentation Updates
● Dependency Tracking and Management
● Manual Testing

6. Pain points for a Distro Maintainer
● Repeat Tasks
● Dependency Tracking and Management
● Dependencies Expressed by not Tested
● Insufficient Integration Tests
● Heavily dependent on manual testing
● Manual Testing

7. Pain points for a Distro Maintainer
● Repeat Tasks
● Dependency Tracking and Management
● Manual Tests
● Package Integration
● Installation
● Desktop Application Functionality
● Service Functionality
● Testing at too many levels
● Not just exploratory (More exploratory is good)

8. Continuous Integration - Basics
● “Integrate changes continuously”
● Checkout source, make, make test, package
● Usual Implementation
● Automated Agents that perform tasks
● Farm of Agents depending upon scale and budget
– Great for packaging multiple apps in parallel
– Parallel Tests

9. Continuous Integration - Benefits
● Detect issues earlier
● Package software earlier
● Automate various tasks
● Packaging
● Testing
● Integration
● Generate ChangeLog + Release Notes skeleton
● Pending Documentation Hints to Documentation
Team
● Build the ISO!

10. Continuous Integration Today
● Isolated build environments
● Based on chroot jails
● Used by Fedora, Debian, Suse, et al
● Continuous Integration servers
● Jenkins/Hudson
● Koji
● Others (distro specific)

11. Continuous Integration
– What's missing
● Dependency awareness
● A “Green” upstream should trigger downstream
builds
● Complex Dependency graph builds and tests
● Prevents Automated Consolidation Releases
● A Variety of Testing
● “make test” is not enough!

12. Continuous Integration
– What's missing
● Dependency Awareness
httpd
apr
subversion
Image Credits: http://cd-with-thoughtworks-go.blogspot.in/

13. Why “make test” is not enough
./configure make make test package
● Tests the app only on the compiled system
● No tests post installation
● Packaging occurs after make test
● Packaging errors are not detected
– Incorrect placement, ownership and permissions
– Conflicting requirements
– Missing files
● Integration Tests needed

14. Testing
The Test Pyramid – Mike Cohn
User
Interface
Service Layer
(Integration)
Unit
http://martinfowler.com/bliki/TestPyramid.html

15. About Unit Tests
● Unit Tests
● Requirements in executable form
● Detect what exact functionality has broken

16. About Integration Tests
● Service Level Tests
● Is ComponentA correctly calling ComponentB?
● Is ComponentA fulfilling its public API contract?

17. About UI Level Tests
● Test End to End Functionality
● e.g. Sahi based tests

18. What Tests tell you
● UI Tests - “Something in this feature broke”
● Integration Tests - “Component B returned an
unexpected response to Component A”
● Unit Tests - “When we introduce functionality
requirement #13 into Component B, then our
code causes requirements #7 and #9 to break”.

19. Post – Installation Integration Tests
● CentOS package level tests
● A good start!
● Some useful tests for distro maintainers
– Is httpd able to serve files ? (selinux issues!)
– Is SSH accepting connections ?
– Did the network subsystem initialize correctly?
– Did the installer set the MBR + active partition correctly?
– Is the DNS server accepting queries ?
● Is the application behaving as required?

20. CI in Belenix
nginx
httpd
postgres WebStack
Distro + package level
python smoke tests
shell
httpd
vi Userland
Integration Automated Post Install
awk Tests Install Tests
ssh
● Distro tests +
Consolidation level
kernel
smoke tests

21. Roadmap - CI and Belenix
● Package early
● Consolidations – kernel, userland, subsystems
(storage, firewall, load balancer)
● Custom distros – CoreOS, storage, firewall,
load balancer
● Each with its own functional test suite!
● CI for open source projects that we bundle
● Potential inter-distro inter-project collaboration

22. Thanks!
www.belenix.org
@belenix
@sriramNRN