1. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 1JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
Tim Noordhoek [CTO hoyhoy.nl]
Thomas Kruitbosch [Continuous delivery Consultant Xebia]

2. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 2JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

3. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 3JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

4. JULY 2015 | CONFIDENTIAL4
What is hoyhoy.nl?

5. JULY 2015 | CONFIDENTIAL5
Situation
• Hoyhoy.nl is a comparison website for insurances, CPA based business model
• Hoyhoy.nl was formerly a 15 year old existing company named verzekeringssite.nl
• Verzekeringssite.nl was acquired by BGL in 2012
• BGL is the founder of comparethemarket.com, a high traffic financial comparision website
• There was no (internal) IT team or stable architecture/infrastructure when I started
• Heavy investment to achieve explosive growth
Goal for IT
• Create a new IT team to support the business growth (interview, hire)
• Create a new architecture & hosting infrastructure (Strategy, direction)
Hoyhoy.nl – Situation & Goal

6. JULY 2015 | CONFIDENTIAL6
• After 3 years I built an Agile working delivery team consisting of 30 people (PO, BA, Dev, QA, DevOps)
• Develop and deliver a new backoffice & frontoffice, built in LAMP, MVC
• Make sure the new platform supports multi-device, os and browser
• Embed Agile working into the organisation together with Xebia
• Create a great working place where craftsmanship is natural (knowledge workshops, fun projects)
• Embed Continuous delivery into the daily working routine with Xebia (today’s topic)
Hoyhoy.nl – My approach

7. JULY 2015 | CONFIDENTIAL7
• Getting the right people (200+ interviews, still low attrition rate)
• Setting up the team structure once we got bigger and bigger
• Delivery slowed down, less features delivered…..
• Quality drop, more incidents / problems, chaotic delivery
• Resource fights (cross projects) .. everything is priority
• Unmeasurable velocity, time 2 market/delivery dates
• Being able to cope with heavy traffic during health season
Hoyhoy.nl – Challenges

8. JULY 2015 | CONFIDENTIAL8
The core hoyhoy.nl business application

9. JULY 2015 | CONFIDENTIAL9
Deliver more and deliver faster
> Releases with a turnaround time
of weeks
> Release at any desired
moment
Planned weekly releases Release any time

10. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 10JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

11. JULY 2015 | CONFIDENTIAL11
Approach: assessment of current delivery capabilities
Delivery value stream
analysis
Continuous Delivery &
Agile Maturity Scan
Architecture
Assessment
Benchmarked Continuous Delivery
maturity

12. JULY 2015 | CONFIDENTIAL12
Where did it really hurt?
• Chaotic release process
• Silo based, “scrum-ish” based delivery
• Lots of manual delivery support tasks
• Inconsistent test environments
• General lack of quality!

13. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 13JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

14. JULY 2015 | CONFIDENTIAL14
Implementation topics
• Dynamic Test Platform for feature driven delivery (lean and fast)
• Fully automated delivery pipeline
• Agile transition and Quality driven delivery for the full organisation
• Application Architecture (out of scope, was already in delivery)

15. JULY 2015 | CONFIDENTIAL15
Vision: release strategy and growth model
May 2015 Aug 2015 Dec 2015 Optional future
Value driven (as needed)

16. JULY 2015 | CONFIDENTIAL16
Roadmap: global implementation strategy
Delivery Automation MVP Platform
“Enables teams to deliver fast with high quality”
June July August September
1.6 Continuous Delivery maturity
Adoption & Improve delivery
“Teams improve and speed up their delivery”
Incremental enhancements
October November
~ 5
December
Continuous delivery
improvements

17. JULY 2015 | CONFIDENTIAL17
Results
Time to market
• 65% faster (from story
refinement to
production go live)
Reduce costs
• 10% higher Defect
Removal Efficiency
• 95% of the defects are
detected within 30
minutes (fail fast)
• 50% faster (from
idea/concept to cash)
Higher quality
• 25% productivity
improvement
Earlier Return on Investment

18. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 18JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

19. JULY 2015 | CONFIDENTIAL19
Why Docker?
• Portable containers (local and test environments)
• Automated consistency (using container images)
• Speed: (create new environments within 3 minutes)
• Capacity: 60+ feature environments on 3 VM’s

20. JULY 2015 | CONFIDENTIAL20
Branched pipelines
Create Feature
2 branch
Feature 1
Pipeline
Create Feature 1
branch
Feature 2
Pipeline

21. JULY 2015 | CONFIDENTIAL21
Feature pipeline process
Production like image

22. JULY 2015 | CONFIDENTIAL22
Feature pipeline overview

23. JULY 2015 | CONFIDENTIAL23
On average, we have 60 active feature test environments
Feature-hh-1164.dyn.local.hoyhoy.nl

24. JULY 2015 | CONFIDENTIAL24
From manual unstructured testing to automated structured testing

25. JULY 2015 | CONFIDENTIAL25
Automated structured user interface (system) testing
Feature container
Deployed
feature
containers- Test scripts
- Test Data
- Stubs
Behavior & Test Driven Development

26. JULY 2015 | CONFIDENTIAL26
Platform as a Service: with Core OS + “Platform service” containers
CoreOS Cluster
Node 1 Node 2 Node 3
NGiNX NGiNX NGiNX
Consul Consul templateConsul Consul templateConsul Consul template
Feature 2 Feature 1 Feature 3
Registrator Registrator RegistratorRegistry

27. JULY 2015 | CONFIDENTIAL27
Implementation: Components of the hoyhoy.nl delivery pipeline
Acceptance & Production platformhoyhoy.nl dynamic test platform
PHPCS
ESLint
Continuous
integration
Automated testing
Static tests Unit/component tests System / UI tests
Automated
Deployment
Cucumberjs
webdriverio
Automated
Provisioning
Istanbul
Hoyhoy.nl application
Hoyhoy.nl application
j2cli
Fleet
Registrator
MITMWEB
(mitmproxy)
Registry
Agile
BDD

28. Klik om de stijl te bewerken
Klik om de ondertitelstijl van het
model te bewerken
29-12-2015 28JULY 2015 | CONFIDENTIALDECEMBER 2015
Building a Docker® powered Continuous Delivery Pipeline
The hoyhoy.nl case
Delivery analysis
The vision, approach, roadmap and results
The feature driven delivery pipeline implementation
Lessons learned

29. JULY 2015 | CONFIDENTIAL29
Lessons learned - process
• Get your manual testers, but also developers ready to build test automation
• A pipeline only delivers value when it is used. Involve your customer (Delivery teams)
• Create a delivery vision and roadmap which is aligned with your company strategy
• Define a MVP and improve your product incrementally / iteratively
• Apply agile principles to deliver changes fast and incrementally, not scrumish
• Don’t over apply text-book Agile methodologies, it’s a learning process

30. JULY 2015 | CONFIDENTIAL30
Lessons learned - technical
• The test platform is an essential part of a delivery pipeline
• Automate everything
• Implement a single source of truth, everything under source control
• Don’t get stuck in “best practices”, Start simple and focus on real value
• Test automation forces application design quality

31. JULY 2015 | CONFIDENTIAL31
Thank you!
Questions?