Continuous Delivery presents a compelling vision of builds that are automatically deployed and tested until ready for production. Most teams aren't there yet. Some never want to go that far. Others want to push the envelope further. This deck presents a model for scoring yourself on the continuum and examples of how companies can decide what parts of CD to adopt first, later and not at all.

1. Continuous Delivery
Maturity Model
Eric Minick, Lead Consultant
1

2. Eric Minick
Lead Consultant
• 8 Years at Urbancode
• Commits on three
generations of UC products
• Experience as a Dev, Tester
and build guy
• Currently focused on best
practices and industry
trends
2

3. The plan for today
• Short Intro to Continuous Delivery
• The Maturity Model
– Building, Deploying, Testing, and Reporting
• Applying the Model: Personas
– El Emeno Investments: Balancing agility and control
– All-Green United Systems: Large scale Scrum
– Diversity Handhelds: Dozens of platforms
• Additional Resources and Q&A
3

4. The plan for today
• Short Intro to Continuous Delivery
• The Maturity Model
• Appling the Model: Personas
• Additional Resources and Q&A
4

5. CI: it’s about Quality Determination
• Integrating frequently is less painful than
infrequently.
– But, If we commit frequently blindly, we have a
broken code base
• Implied condition: each integration will not
decrease the code quality.
– This is the key to CI
– The rest is implementation details.
5 http://www.urbancode.com/blogs/anthillpro-
blog/2011/09/07/continuous_integration_why_you_dont_really_get_it.html

6. Continuous Delivery: Beyond the Team Build
• Continuation of the Quality Determination
• Functional tests require deployment.
• Progression through environments on a march
to production, with “the build” subjected to
more exhaustive tests in each environment.
• Beyond the functional team, but also across
projects
Build & Deploy to Functional Deploy to Integration Deploy to Deploy to
Unit Test DEV Tests SIT Tests UAT PROD
6

7. Why a CD maturity model?
• How mature are our Build, Deploy, Test
practices?
• Where can we get the most improvement
based on our specific problems and needs?
• How are other organizations solving these
same problems?
“The future is here. It’s just not evenly distributed yet.”
– William Gibson
7

8. Maturity Model: Parts and personas
Building, Deploying, Testing, and Reporting
– Base, Beginner, Intermediate, Advanced, Extreme
• Industry Norm vs. Target
– What do we see most people doing today?
– A general target to shoot for (and how to pick a good target
for your own team)
8

9. Poll: Our CI or CD system handles:
• Building our code (at least daily)
• … and deploying after build
• … and testing after deployment
• … and reporting across building, deploying and
testing
• … and more!
• We have no CI system
9

10. Building
10

11. Deploying
11

12. Testing
12

13. Reporting
13

14. Standard Target
Based on this model, the standard target is:
• Continuous builds on a distributed grid using
dependency management
• Builds automatically deployed to first test environment
• Build runs extensive unit tests; extensive functional
tests automatically executed in test environments
• Test results and trends are widely visible and available
• Information from the build system informs manual
risk-based testing
• Self-service deployments to other environments
– Those deployments coordinated between tiers / services
14

15. The plan for today
• Short Intro to Continuous Delivery
• The Maturity Model
• Appling the Model: Personas
• Additional Resources and Q&Aof platforms
15

16. Apply the maturity model: Personas
El Emeno Investments: Balancing agility & control
All-Green United Systems: Large-scale Scrum
Diversity Handhelds: Dozens of platforms
16

17. Poll: Our main problem we hope CD can help
with is:
• Improving our delivery speed
• Improving quality and compliance
• Supporting multiple platforms or devices
• Multiple of these
• We have other problems
• We have no problems
17

18. El Emeno Investments
Trading system for securities
– Time to market is a key competitive advantage
– Strict legal requirements & audits
Problem: slow heavyweight process
– Numerous sign-offs, handheld
deployments, lengthy manual test cycle
– Pressure to work overtime to compensate
– Late mistakes in deployment
– High turnover
“How do we balance agility and control?”
18

19. El Emeno Investments: Deploying
Needed automated, traceable deployments
– Secure push-button deployments
– Deploy to production
Benefits
– Self-service deployments to test take pressure off
release engineers
– Using uniform automated deployments reduces
errors & delays
– Security and traceability make auditors happy
Advanced level deployments
19

20. El Emeno Investments: Building
Needed Audit Proof Builds
– Automatic, Controlled Build Process
– Builds stored in an artifact repository
(ITIL: Definitive Software Library)
Benefits
– Clear understanding of what is in each build lowers risk
– Supports accurate deployments by guaranteeing no
changes to tested artifacts
– Faster approvals and audits with lower risk of failure
Beginner level builds
with some Intermediate practices
20

21. El Emeno Investments: Testing
Needed Rapid Regression Testing and Security
– Automated unit tests of core functionality on all
builds
– Automated security scans of select builds
Benefits
– Less time on manual regression testing allows quicker
detection of problems in new functionality
– Security scans lower business risk, increasing
compliance
Beginner level testing
with advanced security scans
21

22. El Emeno Investments: Reporting
Needed historic release information, consolidated
reports
– Audit log of who deployed what build where
– Produce test, security scans, approval logs for releases
Benefits
– Consolidating reports yields faster, easier audit compliance
– Easy review of a build’s history reduces deployment errors
– Risk is more easily assessed
Intermediate level reporting
22

23. El Emeno Investments: Results
“How do we balance agility and control?”
– Time to market is a key competitive advantage
– Problem: slow, heavyweight process
“Our continuous delivery system provides us an
end-to-end automation across our build lifecycle.
We now have a faster cycle time, fewer manual
errors and an easier time in audits.”
23

24. All-Green United Systems
Large global IT with business critical applications
– Large scale scrum with cross-functional teams
– Traditional integration QA & release engineering
– Recent move to SOA
Problem: releases are a bottleneck
– Legacy “big bang” release process
– Lack of synchronization across applications creates backlog
– QA finding cross-team problems in integration testing
– Apps fail in product, after appearing to work in QA
“How do we realize the benefits of Scrum in a large
organization?”
24

25. All-Green United Systems: Building
Needed unification of build systems across scrum teams
– Consolidate from numerous build systems to one build grid
– Dependency management between components
– Scrum of Scrums yields Build of Builds
Benefits
– Build grid more efficiently uses hardware, lowering costs
and improving response time
– Build of Builds more quickly detects cross-team integration
problems and improves component traceability
– Single build system lowers learning curve when switching
scrum teams
Intermediate level building
25

26. All-Green United Systems: Deployments
Needed to speed the release process to keep pace
– Automated release across teams and environments
– Coordination of SOA releases – Deploy what was tested
– New applications with extensive testing using continuous
deployment to production
Benefits
– Unified deployment scripts avoid errors in production
– Self-Service deployments remove burden from release
engineers
– Continuous deployment give some systems maximum
throughput
Advanced and Extreme level deployments
26

27. All-Green United Systems: Testing
Needs vary by scrum team and application
– Investment in expanded automatic testing focused on
mission critical, frequently changing, and new applications
– Component interfaces gain automated regression tests
Benefits
– Fewer breakages in shared components improves developer
productivity
– Mission critical applications gain stability, lowering risk of
costly defects
Varying levels of testing
27

28. All-Green United Systems: Reporting
Needed better visibility between cooperating small
teams
– Consolidated build systems put all build information in one place
– Include test and deployment reports in CD system
– Trend report results
Benefits
– Reduced coordination overhead between scrum teams
– Single report repository improves coordination between
development and testing
– Trending improves the scrum team’s understanding of how done
they are
Advanced level reporting
28

29. All-Green United Systems: Results
“How do we realize the benefits of Scrum in our
organization?”
– Large scale scrum with cross-functional teams
– Problem: releases are a bottleneck
“Our continuous delivery system allowed our
small, globally distributed scrum teams to
cooperate in large system builds automatically.
By keeping the code integrated we eliminated
late errors, improved coordination across
teams, and eliminated our release bottleneck.”
29

30. Diversity Handhelds
Software platform for mobile devices
– Partners work with device manufacturers
– Each product ported across a wide range of platforms
Problems: breaking existing ports & tracking build
content
– Porting to a new platform might break an inactive port;
hard to track down cause later
– Tracking deliverable contents manually is time consuming;
hard for support to find information when needed
“How can we reduce the cost of adding a new platform?”
30

31. Diversity Handhelds: Building
Needed regular builds of many build configurations
– Diverse, multi-platform, multi-tool chain build farm
– Regular, automatic builds for all configurations
– Upload of build product into build artifact repository
Benefits
– Building all configurations at least daily saves developer
time with quick detection of platform specific compile
problems
– Support has easy access to builds delivered to customers
– Automated Bill of Materials tracking reduces manual effort
and errors.
Intermediate level building
31

32. Diversity Handhelds: Testing
Needs automated cross-platform testing
– Extensive unit testing run regularly
– Automated functional tests run in available, representative
device simulators
– Static analysis and security checks regularly run
Benefits
– Faster defect detection with shorter QA cycles leading to
improved time to market
– Lower risk of potentially expensive platform specific defects
– Manual testing can find deep bugs rather than focus on
smoke testing dozens of variants
Advanced level testing
32

33. Diversity Handhelds: Deployments
Needed deployments to simulators to support
testing
– Scheduled deployments integrated to build
management and testing
Benefits
– Advanced testing that provides rapid feedback
from functional testing is dependant on basic
deployments
Intermediate level deployments
33

34. Diversity Handhelds: Reporting
Needed to understand stability and historical information
– Gather build, various testing and release logs centrally
– Align reports to individual builds
– Maintain reports related to released builds through that
version’s end of life
Benefits
– Provides data to support and engineering regarding what
was tested on old releases customers have problems with
– Lowers effort in understanding test results for wide array of
platforms / configurations
Advanced level reporting
34

35. Diversity Handhelds: Result
“How can we reduce the cost of adding a new
platform?”
– Partners work with device manufacturers
– Problems: breaking existing ports & tracking build
contents
“Ourcontinuous delivery system handles our
wide-range of platforms and prevents our ports
from going stale. Our bill-of-materials is now
generated automatically and our support people
can access historical information any time.”
35

36. Key Takeaways
• Technology built for “extreme” uses can be
used by the rest of us
• Select a final goal and immediate first steps
• Interact across functional groups to project
teams to determine larger needs
36

37. The plan for today
• Short Intro to Continuous Delivery
• The Maturity Model
• Appling the Model: Personas
• Additional Resources and Q & A
37

38. Four partMaturity Model
38

39. More on CI, CD, and DevOps
Webinars and more: http://urbancode.com/resources
• Deployment Automation Basics
• Lean Techniques for Build & Deploy Automation
• Enterprise DevOps: Scaling Build, Deploy, Test, Release
You should follow us:
Urbancode.com/blogs/
Twitter.com/UrbanCodeSoft
Facebook.com/UrbanCodeSoft
Slideshare.net/UrbanCode
39

40. Yes, UrbanCode is a CD vendor
• uBuild(AnthillPro)
– Continuous integration and simple CD
• uDeploy
– Deployment and release management
• uCloud(soon)
– Environment provisioning from your private cloud
40

41. Questions?
Request a copy of the slides or ask more questions
eric@urbancode.com
mbz@urbancode.com
41