13. Signals the org is getting better
Lead Times (and more predictable)
14. Signals the org is getting better
Lead Times (and more predictable)
MTTD (Mean Time To Detect)
15. Signals the org is getting better
Lead Times (and more predictable)
MTTD (Mean Time To Detect)
MTTR (Mean Time to Repair)
16. Signals the org is getting better
Lead Times (and more predictable)
MTTD (Mean Time To Detect)
MTTR (Mean Time to Repair)
Quality at the Source (Less scrap, caught faster)
18. Why DevOps?
Why Now?
(If we ignore DevOps, won’t it just blow over?)
19. We need the capability to learn
faster than our competitors
20. The Rise of a New IT Operations
Support Model
By 2015, DevOps will evolve from a niche strategy employed
by large cloud providers into a mainstream strategy employed
by 20% of Global 2000 organizations
Why DevOps will emerge:
!DevOps is not usually driven from
Why DevOps will not emerge:
!Cultural changes are the hardest to
organizations.
the top down and, thus, may be
more easily accepted by IT
operations teams.
implement, and DevOps requires a
significant rethinking of IT
operations conventional wisdom.
!ITIL and other best practices
frameworks are acknowledged to
have not delivered on their goals,
enabling IT organizations to look for
!There is a large body of work with
respect to ITIL and other best
practices frameworks that is already
accepted within the industry new models.
!The growing interest in tools such
as Chef, Puppet, etc., will help
industry.
!Open source (OSS) management
tools, which are more aligned with
this approach, have not seen pp p
stimulate demand for OSS-based
management
pp
significant enterprise market share
traction.
21.
22. 2014 State of
DevOps Survey
9,200+ Respondents
from 110 countries,
across all industries
23. 2014 State of
DevOps Survey
9,200+ Respondents
from 110 countries,
across all industries
24. DevOps gives us the capability
to learn faster than our
competitors
28. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
29. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
!
Handoff
!
Handoff
!
Handoff
30. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
Handoff
!
Application Knowledge
!
Handoff
!
Handoff
31. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
!
Application Knowledge
!
Operational Knowledge
!
Handoff
Handoff
Handoff
32. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
Handoff
!
Application Knowledge
!
Operational Knowledge
Business Intent
!
Handoff
Handoff
33. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
Handoff
!
Application Knowledge
!
Operational Knowledge
Business Intent
!
Handoff
Handoff
Ownership
but limited
Accountability
34. Slios are the #1 enemy of throughput and quality
Dev
Team
Release
Team
Ops
Team
Business
Team
Handoff
!
Application Knowledge
!
Operational Knowledge
Business Intent
!
Handoff
Handoff
Ownership
but limited
Accountability
Accountability
but no
Ownership
36. Redraw the organization to eliminate silos
Cross Functional Delivery Team
Dev
Team
Release
Team
Ops
Team
Business
Team
Cross Functional Delivery Team
Cross Functional Delivery Team
Aligned by value streams or
customer identifiable services
37. Redraw the organization to eliminate silos
Cross Functional Delivery Team
Dev
Team
Release
Team
Ops
Team
Business
Team
Cross Functional Delivery Team
Cross Functional Delivery Team
Aligned by value streams or
customer identifiable services
Freedom &
Responsibility
Culture is key
to enabling
38. Redraw the organization to eliminate silos
Cross Functional Delivery Team
Dev
Team
Release
Team
Ops
Team
Business
Team
Cross Functional Delivery Team
Cross Functional Delivery Team
Aligned by value streams or
customer identifiable services
Freedom &
Responsibility
Culture is key
to enabling Google:
“Cloud Operations at Netflix”
“Actionable Metrics Netflix”
Roy Rapoport
Different
Talk
40. Turn information flow into artifact flow
Customer
Shared Drive
Test
Shared
Drive Prod
Commits
Rollout
Schedule
README
MOP
Release
Schedule
PRD
PRD
Release
Memos
Tasks
QA Forum
Ticket
Remedy
Ticket
Estimates
Technical Support
Patch
Calendar
QA forum
MOP
EP(2)
README
ERR
ERR
MOP, SOP
PRD
Design
Specs
crit bugs
email
Lockdown
control
checklist
M
New Targets
Remedy
Ticket
Single
Image
Server
XML
BRD
ERR
BTS
QA
Environment
Documentum
Production
Packages
Customer
communication
L/T = 28d
P/T = 7d
H/C = 1
S/R =
Stephen / Xi
Product Program
Planning
L/T = 105d
P/T = 46d
H/C = 15
S/R = 100%
John Robert
Release Program
Management
L/T =
P/T =
H/C =
S/R =
Erica Smith
Engineering
Planning
Process
L/T = 45d
P/T = 18d
H/C = 23
S/R =
Preliminary Bob Smith
Development
L/T = 45d
P/T = 21d
H/C = 140
S/R =
Bob Smith
Full
Development
L/T = 75d
P/T = 43d
H/C = 130
S/R =
Bob Smith
Build
L/T = 1d
P/T = 0.3d
H/C = 2
S/R = 33%
John Doe
D
Selective
Promotion
L/T = 90d
P/T = 15d
H/C = 5
S/R =
Steve Young
QA Test
L/T = 105d
P/T = 11d
H/C = 42
S/R =
Sam Young
Engineering
Release
L/T = 60d
P/T = 1d
H/C = 1
S/R = >5%
Victoria Doe
Release
Promotion
L/T = 60d
P/T = 0.2d
H/C = 1
S/R = >5%
Victoria Doe
Cloud Services
Release
L/T = 60d
P/T = 16d
H/C = 3
S/R = 3%
Reggie / Carlos
Change Control
L/T = 42d
P/T =
H/C =
S/R =
Peter Lee
Deploy Release
L/T = 90d
P/T = 8d
H/C = 8
S/R = 2%
Lewis S./Peter Y.
Server
Provisioning
L/T = 24d
P/T = 4d
H/C = 3
S/R = 50%
Jen Garza
BRD
Server
Acceptance
L/T = 14d
P/T = 1d
H/C = 4.5
S/R = 15%
Lynn A. etc
derived
reqs.
PRD
QA Forum
Ticket
Service pack
review
L/T = 56d
P/T = 7d
H/C = 6
S/R = 100%
Suresh Wu
M
PD(3)
PD(3)
M EP
PD
M(3)
M W(2)
TS
M(3)
M(2)
W(2)
M(2)
EP
EP
EP(3)
W
W
EP
W
PD
TS(2)
M
M
M(2)
M W(2) EP
D
M(3) EP
W
EP
PD D(3)
Current state value stream map
L/T Lead time
P/T Process time
H/C Head count
S/R Scrap rate
D Defects
EP Extra processes
M Motion
PD Partially done
TS Task switching
W Waiting
Product Management
Engineering
Cloud Services
41. Turn information flow into artifact flow
Customer
Shared Drive
Test
Shared
Drive Prod
Commits
Rollout
Schedule
README
MOP
Release
Schedule
PRD
PRD
Release
Memos
Tasks
QA Forum
Ticket
Remedy
Ticket
Estimates
Technical Support
Patch
Calendar
QA forum
MOP
EP(2)
README
ERR
ERR
MOP, SOP
PRD
Design
Specs
crit bugs
email
Lockdown
control
checklist
M
New Targets
Remedy
Ticket
Single
Image
Server
XML
BRD
ERR
BTS
QA
Environment
Documentum
Production
Packages
Customer
communication
L/T = 28d
P/T = 7d
H/C = 1
S/R =
Stephen / Xi
Product Program
Planning
L/T = 105d
P/T = 46d
H/C = 15
S/R = 100%
John Robert
Release Program
Management
L/T =
P/T =
H/C =
S/R =
Erica Smith
Engineering
Planning
Process
L/T = 45d
P/T = 18d
H/C = 23
S/R =
Preliminary Bob Smith
Development
L/T = 45d
P/T = 21d
H/C = 140
S/R =
Bob Smith
Full
Development
L/T = 75d
P/T = 43d
H/C = 130
S/R =
Bob Smith
Build
L/T = 1d
P/T = 0.3d
H/C = 2
S/R = 33%
John Doe
D
Selective
Promotion
L/T = 90d
P/T = 15d
H/C = 5
S/R =
Steve Young
QA Test
L/T = 105d
P/T = 11d
H/C = 42
S/R =
Sam Young
Engineering
Release
L/T = 60d
P/T = 1d
H/C = 1
S/R = >5%
Victoria Doe
Release
Promotion
L/T = 60d
P/T = 0.2d
H/C = 1
S/R = >5%
Victoria Doe
Cloud Services
Release
L/T = 60d
P/T = 16d
H/C = 3
S/R = 3%
Reggie / Carlos
Change Control
L/T = 42d
P/T =
H/C =
S/R =
Peter Lee
Deploy Release
L/T = 90d
P/T = 8d
H/C = 8
S/R = 2%
Lewis S./Peter Y.
Server
Provisioning
L/T = 24d
P/T = 4d
H/C = 3
S/R = 50%
Jen Garza
BRD
Server
Acceptance
L/T = 14d
P/T = 1d
H/C = 4.5
S/R = 15%
Lynn A. etc
derived
reqs.
PRD
QA Forum
Ticket
Service pack
review
L/T = 56d
P/T = 7d
H/C = 6
S/R = 100%
Suresh Wu
M
PD(3)
PD(3)
M EP
PD
M(3)
M W(2)
TS
M(3)
M(2)
W(2)
M(2)
EP
EP
EP(3)
W
W
EP
W
PD
TS(2)
M
M
M(2)
M W(2) EP
D
M(3) EP
W
EP
PD D(3)
Current state value stream map
L/T Lead time
P/T Process time
H/C Head count
S/R Scrap rate
D Defects
EP Extra processes
M Motion
PD Partially done
TS Task switching
W Waiting
Product Management
Engineering
Cloud Services
42. Drive all changes through a SDLC
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
43. Drive all changes through a SDLC
Code
Dev Ops *
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Collaboration
Tests
44. Drive all changes through a SDLC
Versioned Release
Tests
Code
Dev Ops *
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Collaboration
45. Versioned Release
Tests
Code
Dev Ops *
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Collaboration
Dev Ops *
Execute
Operations
Procedures
Drive all changes through a SDLC
46. Versioned Release
Tests
Code
Dev Ops *
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Collaboration
Dev Ops *
Execute
Operations
Procedures
Drive all changes through a SDLC
Same People!!
47. Versioned Release
Tests
Code
Dev Ops *
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
SERVICE
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Collaboration
Dev Ops *
Execute
Operations
Procedures
Drive all changes through a SDLC
49. What about cross-cutting concerns?
Cross Functional Delivery Team
(PO • Dev • Test • SRE)
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Packages Environments
SOURCE
QA Security Environments Monitoring
---
Metrics
50. What about cross-cutting concerns?
Cross Functional Delivery Team
(PO • Dev • Test • SRE)
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Packages Environments
SOURCE
Metrics as a
Service
Monitoring
---
Metrics
QA as a
Service
Security as a
Service
Env. as a
Service
QA Security Environments
51. What about cross-cutting concerns?
Cross Functional Delivery Team
(PO • Dev • Test • SRE)
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Packages Environments
SOURCE
pull pull pull pull
Metrics as a
Service
Monitoring
---
Metrics
QA as a
Service
Security as a
Service
Env. as a
Service
QA Security Environments
52. Be a service provider
pull
Cross-Cutting
Concern X
✓ Standardized offerings
✓ Pulled by users (not pushed)
✓ On-demand and self-service
✓ Implementation knowledge not
necessary for normal use
✓ Provider spends their time building
service and coaching users
X as a Service
53. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
54. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
55. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
56. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
57. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
58. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
59. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
3 Setup self-service interfaces
60. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
3 Setup self-service interfaces
61. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
3 Setup self-service interfaces
4 Setup secure access
62. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
1 Define your offerings
2 Tame the tool sprawl
3 Setup self-service interfaces
4 Setup secure access
63. How to start being a service provider
pull
X as a Service
Cross-Cutting
Concern X
Plug: Give Rundeck a try --> rundeck.org
1 Define your offerings
2 Tame the tool sprawl
3 Setup self-service interfaces
4 Setup secure access
65. Good rule of thumb:
Tickets are for exceptions, not the daily work
X
X
Ticket
System
??
X
66. Good rule of thumb:
Tickets are for exceptions, not the daily work
Manual request queues lead to...
• Bottlenecks
• Increased lead times
• Reinforces organizational silos
• Misinterpretation or omissions
X
X
Ticket
System
??
X
67. How do we mitigate the negative
impact of manual request queues?
DevOps
68. Use a work management system like Kanban
Up Next
Service B
Service C
Service D
Service E
Doing
Plan it Do it Review it Post Mortem
Backlog
prioritized by
stakeholders
Ta s k
Task
Service A
Task
Task
Task
Task Task
Emergency - Type 1
Emergency - Type 2
Task
Task
Task
Task
Task
Task Task
Task Task Task
Task
Task
Task
Task Task
Task Task
Task
Task
69. Use a work management system like Kanban
Up Next
Service B
Service C
Service D
Service E
Doing
Plan it Do it Review it Post Mortem
Backlog
prioritized by
stakeholders
Ta s k
Task
Service A
Task
Task
Task
Task Task
Emergency - Type 1
Emergency - Type 2
Task
Task
Task
Task
Task
Task Task
Task Task Task
Task
Task
Task
Task Task
Task Task
Task
Task
Only works if you set
and enforce:
• Service catalog and
backlog rules
• WIP and SLA per
service type
• WIP per person
70. Use a work management system like Kanban
Your standardized
offerings
Up Next
Service B
Service C
Service D
Service E
Doing
Plan it Do it Review it Post Mortem
Backlog
prioritized by
stakeholders
Ta s k
Task
Service A
Task
Task
Task
Task Task
Emergency - Type 1
Emergency - Type 2
Task
Task
Task
Task
Task
Task Task
Task Task Task
Task
Task
Task
Task Task
Task Task
Task
Task
Only works if you set
and enforce:
• Service catalog and
backlog rules
• WIP and SLA per
service type
• WIP per person
71. Use a work management system like Kanban
Your standardized
offerings
Up Next
Service B
Service C
Service D
Service E
Doing
Plan it Do it Review it Post Mortem
Backlog
prioritized by
stakeholders
Ta s k
Task
Service A
Task
Task
Task
Task Task
Emergency - Type 1
Emergency - Type 2
Task
Task
Task
Task
Task
Task Task
Task Task Task
Task
Task
Task
Task Task
Task Task
Task
Task
Only works if you set
and enforce:
• Service catalog and
backlog rules
• WIP and SLA per
service type
• WIP per person
SLA per service type
72. Use a work management system like Kanban
Your standardized
offerings
Up Next
Service B
Service C
Service D
Service E
Doing
Plan it Do it Review it Post Mortem
Backlog
prioritized by
stakeholders
Ta s k
Task
Service A
Task
Task
Task
Task Task
Emergency - Type 1
Emergency - Type 2
Task
Task
Task
Task
Task
Task Task
Task Task Task
Task
Task
Task
Task Task
Task Task
Task
Task
Only works if you set
and enforce:
• Service catalog and
backlog rules
• WIP and SLA per
service type
• WIP per person
SLA per service type
Enforce WIP to protect capacity and hit commitments!
74. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
75. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Design and Code
Reviews
76. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
Design and Code
Reviews
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Code and Binary
Scanning
77. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
Design and Code
Reviews
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Code and Binary
Scanning
“Bake” security
tests into your
“immune system”
78. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
Design and Code
Reviews
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Code and Binary
Scanning
“Bake” security
tests into your
“immune system”
Component
vulnerability and
governance
79. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
Design and Code
Reviews
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
Code and Binary
Scanning
“Bake” security
tests into your
“immune system”
Component
vulnerability and
governance
Access policy and
operational security
checks
80. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
81. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
82. Security and Compliance Opportunity
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
How did you validate
the change?
83. Security and Compliance Opportunity
How did you validate
the change?
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
Where did the
change go?
84. Security and Compliance Opportunity
How did you validate
the change?
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
Where did the
change go?
Who has access to
what environment?
Who did what when
and where?
85. Security and Compliance Opportunity
How did you validate
the change?
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
What was
executed on the
box to make the
change?
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
Where did the
change go?
Who has access to
what environment?
Who did what when
and where?
86. Security and Compliance Opportunity
How did you validate
the change?
Tests Code
Source
Repo
Config Env
Spec
Run-book
Auto-mation
CI
Server
Package
Repo
Operations
Console
What was
executed on the
box to make the
change?
Shell
Powershell
Pre-Production
Environments
Shell
Powershell
Production
Environment
Packages
Operations
Development
SOURCE
What’s the
change?
Where did the
change go?
Who has access to
what environment?
Who did what when
and where?
Change things
here
Run / control
things here
88. Are you helping your company to...
Reduce cycle time AND improve quality?
89. Are you helping your company to...
Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of
those handoffs that can't be eliminated?
90. Are you helping your company to...
Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of
those handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and
eliminate manual information flow?
91. Are you helping your company to...
Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of
those handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and
eliminate manual information flow?
Eliminate manually-fulfilled request queues
and other sources of waiting?
92. Are you helping your company to...
Reduce cycle time AND improve quality?
Eliminate handoffs or reduce the friction of
those handoffs that can't be eliminated?
Improve tool-to-tool artifact flow and
eliminate manual information flow?
Eliminate manually-fulfilled request queues
and other sources of waiting?
Improve awareness and understanding of the
current state and desired state of the end-to-end
system?