1. 27 January 2018
Kari Kakkonen
Knowit Oy (Finland)
kari.kakkonen@knowit.fi
Twitter @kkakkonen
Linkedin karikakkonen
DevOps maturity models by Knowit
and DASA views
Knowit 1

2. Knowit 2

3. ROLES
• Knowit Oy, Director/Quality and Competences, Lead Consultant,
Trainer and Coach
• Treasurer of ISTQB Executive Committee
• Treasurer, ex-chairman of Finnish Software Testing Board (FiSTB)
• Auditor of Robot Framework Foundation.
ACHIEVEMENTS
• Influencing testing since 1996
• Ranked in 100 most influential IT persons in Finland (Tietoviikko magazine)
• Great number of presentations in Finnish and international conferences
• TestausOSY/FAST founding member.
• Co-author of Agile Testing Foundations book
• Regular blogger in Tivi-magazine.
EDUCATION
• ISTQB Expert Level Test Management Full, Advanced Full,
Agile Tester & Test Automation Engineer certified
• DASA DevOps Fundamentals, Scrum Master and SAFe certified
• SPICE provisionary assessor certified
• M.Sc.(Eng), Helsinki University of Technology (present Aalto University), Otaniemi, Espoo
• Marketing studies, University of Wisconsin-Madison, the USA.
BUSINESS DOMAINS
Wide spread of business domain knowledge
• Embedded, industry, public
• Training, telecom, commerce
• Insurance, banking, pension.
SERVICES
• ISTQB Advanced, Foundation and
Agile Testing, Knowit Quality Professional
• DASA DevOps Fundamentals
• Quality & Test process and organization development,
metrics
• Agile testing, Scrum, Kanban, Lean
• Leadership
• Test automation, mobile, cloud, DevOps
• Quality, cost, benefits.
3
Twitter: @kkakkonen
LinkedIn:
fi.linkedin.com/in/karikakkonen/
CV Kari Kakkonen - CONFIDENTIAL

5. Knowit is the Nordic company
that through a combination of
creativity, strong strategic
knowledge and a passion for
technology creates digital
opportunities.
Digital opportunities
Knowit 5

6. Knowit clients
Knowit 6

7. Testing and test management
• Fundamentals of testing
• Test planning and management
• Improving testing process
• Agile and Exploratory Testing
• Test case design
• Usability testing
• Security testing
Testing certifications
• ISTQB Foundation in Software Testing
• ISTQB Agile Tester
• ISTQB Advanced Level (Test Manager, Test
Analyst, Technical Test Analysit)
Robot Process Automation (RPA) & AI
• RPA for business representatives
• RPA for technical team
• RPA tools
• Robot Framework
• UI Path
• Artificial Intelligence briefings
Test automation
• Test tool selection and implementation
• Automation tools, i.e.
− Robot Framework Basics & Advanced
− HP UFT
− Jmeter & performance testing
− Management tools, i.e
− ALM, SilkCentral, Meliora
− Jira, VersionOne
Web and analytics
• Google Analytics basics & business
• Google Tag Manager
• SEO for communication professionals
Agile & DevOps
• DevOps briefing (CI ,Continuous Delivery,
DevOps Culture)
• DASA DevOps Fundamentals
• Scrum Master / Product Owner / Team
• Lean mindset
• User Story creation
• Scaled Agile Framework SAFe
727.1.2018
Examples of Knowit training offering
Software and web development
• Embedded C-programming
• Python Basics
• Security in Programming
• Cloud, Docker & Ansible

8. Knowit
• DevOps introduction
• Business benefits
• Terminology, e.g. Continuous Delivery
• Faster feedback
• Automate everything you can
• DevOps Culture
• DASA DevOps principles
• Organization
• Lean & Agile
• Autonomous teams
• Microservices and Cloud
• DevOps Maturity
• Step by Step to DevOps
• Maturity Model
8

9. Devops introduction
Knowit 9

10. Taking new technology into use gets faster exponentially
Knowit
How quickly a product/an application reached 50M users?
Television
Facebook
Internet
Instagram
Angry Birds
Pokemon Go
13 Years
4 Years
3,5 Years
35 days
9 days
6 months
10

11. Opportunities provided by DevOps (2016)
Knowit
https://puppet.com/resources/white-paper/2016-state-of-devops-report
200x
more frequent
deployments
2550x
shorter lead
times
50%
less time
spent remediating
security issues
3x
lower change
failure rate
22%
less time spent on
unplanned work and
rework
24x
faster recovery
from failures
11

12. Opportunities provided by DevOps (2017)
Knowit
https://puppet.com/resources/whitepaper/state-of-devops-report
46x
more frequent
deployments
440x
shorter lead
times
5x
lower change
failure rate
96x
faster recovery
from failures
12

13. How devops copes with software development challenges
Challenges in SW development
Long testing periods (weeks)
Long release cycle (months)
Quality problems
Unstable environments
Unused features
Lot of complicated surprises
High maintenance cost
Slow reaction to customer needs
Knowit
DevOps possibilites
Fast feedback via automation (in hours)
Faster time to market (continuous delivery)
Continuous Quality
More efficient use of environments
More profitability
Better visibility and predictability
Less maintenance effort
Higher customer satisfaction / more customers
13

14. DevOps and other terminology
Knowit
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps
Prepare Implement Build QA Deliver Deploy Operate
Continuous feedback
Agile development
14

15. Devops is about culture of high-performance IT:
Business is not separate from software & product
Agile & Lean
Way of
Working
Modern
Software
Architecture
Continuous
Delivery
Intelligent
Infrastructure
Talent
Management
Knowit 15

16. months
DevOps: from slow lead time to fast lead time?
Speed demands quality
Prepare Implement Build QA Deliver Deploy Operate
Knowit 16

17. months
DevOps: analyze and improve incrementally
Prepare Implement Build QA Deliver Deploy Operate
Measure
AnalyseOptimize
Prepare Implement Build QA Deliver Deploy Operate
Knowit 17

18. months
weeks
hours
DevOps:
faster cycles, visibility, better feedback and more profits
Speed demands quality
Prepare Implement Build QA Deliver Deploy Operate
Prepare Implement Build QA Deliver Deploy Operate
Knowit 18

19. Automate Everything you Can: Version Control
Knowit
Version-
control
code
commit
Development services
19

20. Automate Everything you Can: Continuous integration
Knowit
Build and installation
CI Install CI Install
UAT Performance
Continuous Integration (CI)
Version-
control
code
commit
Development services
CI Build
Integration
20

21. Automate Everything you Can:
Continuous integration and Test Automation
Knowit
Automated tests, build and installation
CI Install CI Install
UAT Performance
Continuous Integration (CI)
Version-
control
code
commit
Development services
CI Build
Integration
21

22. Automate Everything you Can:
Continuous integration, Test automation,Test data and Interfaces
Knowit
Automated tests, build and installation
CI Install CI Install
UAT Performance
Test data management
Virtualized Backend services
Continuous Integration (CI)
Version-
control
code
commit
Development services
CI Build
Integration
22

23. Automate Everything you Can:
Continuous integration and Continuous Delivery
Knowit
Automated tests, build and installation
CI Install CI Install Deliver
UAT Performance
Continuous Delivery (CD)
Test data management
Virtualized Backend services
Continuous Integration (CI)
Version-
control
code
commit
Development services
CI Build
Integration
23

24. Automate Everything you Can:
DevOps and Continuous Deployment
Knowit
Virtualized Backend services
Test data management
Version-
control
code
commit
CI Build CI Install Deliver
Integration
UAT
Performance
End user
monitoring
Deploy
Production
Quality metrics
Automated tests, build and installation
Robot framework tests
Development services
Virtualisation
24

25. But wait, can you automate everything?
• Some things are feasible still to execute manually
• Architecture
• Actual coding, including refactoring & optimization
• Test planning
• Risk evalution
• Test optimisation (until Artificial Intelligence replaces this...)
• Usability tests
• Exploratory tests (sometimes using scripts and automation as helper tools)
Knowit 25

26. Devops culture and DASA
Knowit 26

27. About the DevOps Agile Skills Association (DASA)
DASA is an independent and open
association supporting the development of
DevOps training and certification to the
global market.
Advocating the development of high-
performance IT professionals and
organizations through agile DevOps
initiatives.
DASA offers thought leadership as well as
practical guidance for competence
development for professionals and
organizations.
https://www.devopsagileskills.org/

28. The DASA DevOps Qualification Program
28
1
Novice
DevOps: Fundamentals
DevOps Specialization:
Create and Deliver
DevOps Specialization:
Specify and Verify
DevOps Specialization:
Enable and Scale
DevOps: Practitioner
5
Master
2
Competent
3
Proficient
4
Expert
3 days
2 days
2 days 3 days2 days

29. DASA Six Principles
29

30. 1. Customer-Centric Action
• Short feedback loops
• With real customers
• Act as lean startup
Knowit 30

31. 2. Create with the End in Mind
• See the complete picture
• Build for real customers
• Act as product companies
• Engineering mindset to find solutions
Knowit 31

32. 3. End-To-End Responsibility
• Accountable from concept to grave
• Organized vertically
• Responsibility leads to quality
• Culture
Knowit 32

33. 4. Cross-Functional Autonomous Teams
• Independent teams
• T-shaped all-around skill profiles
• Functional teams or platform teams
Knowit 33

34. 5. Continuous Improvement
• Adapt continuously
• Minimize waste
• Continuously minimize product and process
• Learning from failures
Knowit 34

35. 6. Automate Everything You Can
• Automate testing
• Automate software delivery process
• Automate infrastructure
• Container-based cloud platforms
Knowit 35

36. Devops organization
Knowit 36

37. Lean: Eliminate waste
• Defects
• Overproduction
• Waiting
• Not utilizing talent
• Transportation
• Inventory excess
• Motion waste
• Excess processing
Knowit 37

38. Lean: Value stream mapping
• Process from user need to fulfilling user need
• Aiming for flow efficiency: fast throughput
Knowit
Identify
need
Design Coding Testing Accept Deploy
20 days 2 days 5 days 10 days 1 days
1 days 5 days 10 days 10 days 3 days 2 days
Wait:
38 days
Add Value:
31 days
Flow efficiency = 31 / 69 = 45%
Total time:
69 days
Can we do something in parallel?
Can we remove waiting time (waste)?
38

39. Kanban - Visualize and Optimize the Flow Work within a value-
added chain
Knowit
• Work-in-Progress Limit: The amount of parallel
active tasks is strictly limited
• Whenever a station has free capacity, the
worker pulls a ticket from the previous station.
• Lead Time: Optimize the continuous flow of
tasks by minimizing the (average) lead time for
the complete value stream.
• Kanban Board: Each column
shows a station, which is a set
of related activities. The items to
be produced or tasks to be
processed are symbolized by
tickets moving from left to right
through the stations.
39

40. Continuous improvement
• Deming cycle PDCA (Plan-Do-Check-Act)
• DMAIC (Define-Measure-Analyze-Improve-Control)
• Kaizen
• Etc.
Knowit
Signals
Problems
HypothesisExperiment
Analyze
40

41. Autonomous Teams: in charge of everything
• How they develop code
• How they test
• How they connect to other systems or teams
• When they connect
• When they deploy to production
Knowit 41

42. Two types of teams
• If any team needs another team’s product or service, they can always get the latest ready to deploy
version automatically
Knowit
Functional team Functional team Functional team
Platform team Platform team
Self-service
interface
42

43. Microservices and cloud
• Move your architecture towards Microservices
• Enables more frequent delivery
• Consider cloud principles to your platform teams
• Helps you utilize environments as a service
• Less dependencies
• More frequent delivery
Knowit 43

44. Devops maturity
Knowit 44

45. Steps to DevOps
Knowit
Responsiveness
Agile methods in use
in development
scrum, backlog etc
Automation
deployed in larger
scale. Daily build
with automated
regression tests
Ability to deliver daily
SW quality metrics
support daily
activities
Changes deployed
automatically to
production
environment as soon
as available from
production pipe
Continuous
Integration
Continuous
delivery
Continuous
deployment
DevOps
Operational metrics
transparent, feature
throughput in days,
Processes fast and
lean, continuous
learning and
improvement
Automation in
product level,
automated
regression tests
Test Automation
Quality
Agile development
45

46. DevOps development example blocks
Knowit
Requirement Mgmt
Planning
Customer Data Mgmt
Release &
Configuration Mgmt
Architecture
Implementation Testing Delivery Deployment Production
Usage metrics
Fault MgmtAgile methods, Scrum,
backlog
Unit tests, static
analysis
Access Rights +
Version Control
Test Case Mgmt Customer Data
Mgmt
46

47. DevOps development example blocks
Knowit
Requirement Mgmt
Planning
Customer Data Mgmt
Release &
Configuration Mgmt
Architecture
Implementation
Agile methods, Scrum,
backlog
Technical debt
visualisation e.G.
SonarQube
Continuous Integration
/ Jenkins etc
Unit tests, static
analysis
Access Rights +
Version Control
Testing
Test Case Mgmt
Regression Test
Automation
Delivery Deployment
Customer Data
Mgmt
Production
Usage metrics
Fault Mgmt
47

48. DevOps development example blocks
Knowit
Backend service
virtualization
virtualisointi
Planning Implementation
Technical Debt
visualisation e.g.
SonarQube
Continuous Integration
/ Jenkins etc.
Virtualization/ cloud
environments
Testing
Test Case Mgmt
Regression Test
Automation
Test Data Mgmt
Installation Automation
Delivery
Automated transfer
KäyttöDeployment
Automated
update/upgrade
Production
Agile methods, Scrum,
backlog
Unit tests, static
analysis
Access Rights +
Version Control
Requirement Mgmt
Customer Data Mgmt
Release &
Configuration Mgmt
Architecture
Customer Data
Mgmt
Fault Mgmt
Usage metrics
48

49. DevOps development example blocks
Knowit
Käyttö
Käyttömittaristo
Planning Implementation
Virtualization/ cloud
environments
Testing
Test Data Mgmt
Installation Automation
Virtualization/ cloud
environments incl.
backend services
Full test automation
Optimized transfer
Delivery
Automated transfer
Deployment
Cloud environment
(Docker, Ansible
etc.)
Automated
update/upgrade
Production
Usage metricsRequirement Mgmt
Customer Data Mgmt
Release &
Configuration Mgmt
Architecture
Technical Debt
visualisation e.g.
SonarQube
Continuous Integration
/ Jenkins etc.
Agile methods, Scrum,
backlog
Unit tests, static
analysis
Access Rights +
Version Control
Test Case Mgmt
Regression Test
Automation
Customer Data
Mgmt
Fault Mgmt
49

50. Optimized transfer
DevOps development example blocks
Knowit
DeploymentImplementationPlanning Testing Delivery
Automated transfer
Microservices & Cloud
Optimized update/
Microservices
Production
Fault Mgmt
Usage metrics
Cloud environment
(Docker, Ansible
etc.)
Virtualization/ cloud
environments
Virtualization/ cloud
environments incl
backend services
Optimized
cloud
environments
Automated
update/upgrade
Transparency /optimization / KPI’s
Organization / feature teams
Full test automation
Requirement Mgmt
Customer Data Mgmt
Release &
Configuration Mgmt
Architecture
Technical Debt
visualisation e.g.
SonarQube
Continuous Integration
/ Jenkins etc.
Agile methods, Scrum,
backlog
Unit tests, static
analysis
Access Rights +
Version Control
Test Data Mgmt
Installation Automation
Test Case Mgmt
Regression Test
Automation
Customer Data
Mgmt
50

51. Devops/automation level visualisation
SW Development
- SW development
-Unit test
-Module test
-Static analysis & security analysis
-etc
SW Building
-Adding SW
modules/versions to a
build
-Static analysis
-etc
Install /Commissioning
-Reservation
-SW download
-SWBT /sanity test
-etc
QA / testing
-Regression set
-Entity testing
-Functional
-performance
-security
-Network Verification
Time Time Time Time
Deliver
- internal / external
- Delivery logic (pull vs push)
- optimized methods
Time
Deploy
-Installation / phased
-Update/upgrade
-Fallback
Time
Project SW Development SW Building Install /
Commissioning
QA / Testing Deliver Deploy
Org1 Team1
Org1 Team2
Org2 Team1
Org2 Team3
Full automation Partly automated Under planning Not reasonable to
automate
Manual
Knowit 51

52. 1. Test automation
Step by step forward, example
Sprint 1
Snapshot
Sprint 2
Roadmap
Sprint 3
Implementation
PO and team roles defined, agile
process desription
Test automation PoC, test automation
plan
CI pipeline modules implementation
Agile development understanding
Test automation implementation, phase 1 Test automation implementation, phase 2
Area to be developed
CI Process descriptions and gap
analysis
Trainings, workshops
2. Continuous Integration
3. DevOps/ agile
development steps
4. Competence
development
Competence development plans, chosen
courses
Knowit 52

53. Knowit DevOps Maturity Model
Knowit
Devops maturity 1 2 3 4
Organization and culture Planning, Development
and quality assurance in
own silos
Agile methods in use in
planning&development, QA in
separate organization
Feature team is responsible all the way to
production
Autonomous teams in both
feature and platform level
Continuous Delivery readiness (Dev &
Build & Test & Deliver)
No CI or only some
phases covered
CI in use, typical feedback loop over
a week.
CI pipeline fully automatic, includes all the
tests, cycle time max couple of days,
green build available for delivery
CI pipeline cycle less than 12
hours, test coverage and other
quality criteria meets the delivery
quality criteria
Fast deployment to production Deployment cycle
typically months
Requirement analysis and impact to
SW components takes weeks.
Development does the build ready
for acceptance tests.
Requirement analysis and impact to whole
product can be analysed in few hours.
Acceptance tests are automated,
Releasing/deployment to production done
manually.
Deployment automatic, when
critical failure in update,
automatic fallback happens and
feedback to development
delivered.
Visibility and reporting No visibility, missing or
manual reporting
Reporting of each phase exists, but
visibility only to that certain phase.
Test coverage and static code analysis
results visible for whole development
Product operative metrics visible
for developers, in case of
problems automatic alert to
development
Architecture Architecture unclear, lot of
surprises when something
changed
Product is one entity, installed as a
whole.
Product consists of subentities, and
interfaces are known and defined.
Microservices in use and
independently updateable
Product releasing and sales (pull vs
push)
No controlled feature or
configuration
management
Release content defined in advance
80%. Customer pays when release
acceptance is given and content is
inproduction in customer system.
Release content half fixed, half adapting
to order base. Product/update can be
ordered via internet.
Features done on demand, First
order and payment, then
development and delivery.
BDD/ATDD from customer
needs. Ordering and payment via
internet.

54. Kari Kakkonen
kari.kakkonen@knowit.fi
@kkakkonen
Thank you!
Knowit 54