Presentation given to Sydney Tech Leaders Meetup 11-June-2015.
Focuses on behaviours and culture change that is required within the delivery team and throughout the enterprise to assist and complement the technical capability of the organisation.
3. Deliver more features value, more quickly
for less cost
Improves
•Focus
•Efficiency
•Quality
Reduces
•Cycle time
•Bureaucracy
•Release overheads
•Bug/feature management
•Cost
5. Can we do this faster?
TEST, TEST, TEST
100% Unit Test coverage
~75% Cucumber Tests (BDD)
Substantial integration tests in pre-prod
Deployment Automation
6. Global Release Vicious Circle
Confidence to release is
low despite automated
tests
Patching changes into RC
causes further delays
Delays cause more
features to be added to
RC
QA fall behind on
functional testing
Engineers become
careless
Anxiety from Release
Manager delays
releases
Difficult to identify
cause of regressions
9. How?
Clear definition of ownership
Shared responsibility for quality (not just QA)
Culture of personal responsibility
Collaboration rather than phase-gates and sign-off
Production releases focus the mind
13. Responsibility
Code is well-tested and production ready
Monitoring vital statistics (in all environments)
When to back-out and when to fix-forward
Nuanced understanding of risk and QA
Definition of Done == In production.
14. Change in Team
Less focus on points and commitment
More focus on output and why?
Shared ownership of product and quality
Teams judged on effectiveness rather than velocity
15. Finally..
It’s not “same same but faster”
You cannot solely automate and test your way to CD
You need to change your culture
You need trust and autonomy
Requires personal responsibility and accountability at every level
You cannot leave it to engineering
Today I’m focusing on culture change rather than technical implementation details
Quick reminder of benefits of CD
Traditionally Ops own responsibility for production systems and can only feel secure if everything has had a full regression test in a prod-like environment.
This made sense for code delivered on a CD-Rom but is not so relevant for server-hosted software.
At Westfield we attempted to speed up this cycle by investing in Test and build automation. You do need a lot of this but you need more than merely speeding up the old process.
Someone has to sign this release off!
Every change is perceived as a regression risk.
For Ops and QA to have certainty, everything gets tested together, each change has a huge overhead, that slows the process down further.
Releases always take days rather than hours or minutes which builds a new backlog/bottleneck of changes which set the next release up for delays.
This makes more sense if you are deploying infrequently.
Teams, and specifically engineers are trusted to make good judgements
In return, they accept and understand accountability
Transparency is delivered through frequent releases and performance data
Hard to do any of this without a de-coupled architecture
Assign ownership at the team and individual level
Engineers take responsibility for delivering code that has been properly tested and ask for help if necessary. The rest of the team commit to helping with testing and monitoring.
Personal responsibility is the idea that human beings choose, instigate, or otherwise cause their own actions
Formal sign-off becomes meaningless when you are releasing so many changes to so many moving parts
Changes that were observed in the team and individuals after changing our philosophy.
Backwards compatibility is critical.
At Westfield we had a very clear strategy that you couldn’t release changes to dependent components at the same time. A backward compatible component must go right through to production first.
This creates a robust system and process. Nothing is in the pipeline that could cause a regression if release order goes wrong.