Infosec at Ludicrous Speeds - Rugged DevOps

Security is Dead.
Long Live Rugged DevOps:

IT at Ludicrous Speed…

Gene Kim
IT Revolution Press
Session ID:

@RealGeneKim, genek@realgenekim.me

Act I: IT Ops Fixing Fragile Artifacts


Act 2: The Product Managers


Act 3: The Developers


Act 4: IT Ops And Dev At War

8

Act 5: Nothing Left For Infosec


The Downward
Spiral…

11

The IT Core Chronic Conflict
 Every IT organization is pressured to
simultaneously:
 Respond more quickly to urgent business needs
 Provide stable, secure and predictable IT service

Source: The authors acknowledge Dr. Eliyahu Goldratt, creator of the Theory of Constraints and
author of The Goal, has written extensively on the theory and practice of identifying and resolving
core, chronic conflicts.
13

Every Company Is An IT Company…
 95% of all capital projects have an IT
component…
 50% of all capital spending is technology-related

Where we need to
be…
IT is always in the way
(again…)

We are here…


There Must Be A Better
Way…

15

Source: John Allspaw

Source: Theo Schlossnagle

Source: James Wickett


Source: John Jenkins, Amazon.com @RealGeneKim, genek@realgenekim.me

The Three Ways
And Six Prescriptive Steps
Infosec Can Take

27

If I Could Wave A Magic Wand, Everyone Will…

 Become conversant with DevOps and recognize
the practices when you see them
 Be energized about how information
practitioners can contribute in this organizational
journey
 Leave with some concrete steps to get some
great outcomes
 Become a part of a team that starts putting
DevOps practices into place

28

The First Way:
Systems Thinking


The First Way:
Systems Thinking

(Business) (Customer)


The First Way:
Systems Thinking (Left To Right)

 Understand the flow of work
 Always seek to increase flow
 Never unconsciously pass defects downstream
 Never allow local optimization to cause global
degradation
 Achieve profound understanding of the system


“Annual business planning sessions can be
madding. They think IT Operations is an „all you
can eat buffet.‟”

-Ben Rockwood,
Director Systems Engineering,
Joyent


Practice #1: Define The Work and Make It
Visible

 Business projects (e.g., new order entry system)
 Internal IT projects (e.g., create new
environments, infosec remediation)
 Changes (e.g., deploys, improve database
performance)
 Unplanned work (e.g., site down, site impaired,
security incident)

33

Day 2: PMO Meeting


Practice #2: Create One Step Environment
Creation Process

 Make environments available early in the
Development process
 Make sure Dev builds the code and environment
at the same time
 Create a common Dev, QA and Production
environment creation process


Change the Agile sprint policy:

“At the end of each sprint, we must have working
code and the environment it runs in!”


Infosec Insurgency
 Find the automated infrastructure project team
(e.g., puppet, chef)
 Release managers can provide hardening guidance
 Integrate and extend their production configuration
monitoring
 Put ASSERTs to find misconfigurations, enforce https,
etc.
 Define what changes/deploys cannot be made
without triggering full retest

37

The First Way:
Outcomes
 Creating single repository for code and environments
 Determinism in the release process
 Consistent Dev, QA, Int, and Staging environments, all
properly built before deployment begins
 Decreased cycle time
 Reduce deployment times from 6 hours to 45 minutes
 Refactor deployment process that had 1300+ steps
spanning 4 weeks
 Faster release cadence


The Second Way:
Amplify Feedback Loops


The Second Way:
Amplify Feedback Loops (Right to Left)

 Understand and respond to the needs of all
customers, internal and external
 Shorten and amplify all feedback loops: stop the
line when necessary
 Create quality at the source
 Create and embed knowledge where we need it


The Toyota Andon Cord

41

“We found that when we woke up developers at
2am, defects got fixed faster than ever.”

Patrick Lightbody
CEO, BrowserMob


Pattern #3: Embed Dev Into IT Ops
 Embed Dev into IT Ops incident escalation
process
 Invite Dev to post-mortems/root cause analysis
meeting
 Have Dev and Infosec cross-train IT Operations
 Ensure application monitoring/metrics to aid in
Ops and Infosec work (e.g., incident/problem
management)


The Second Way:
Outcomes

 Defects and security issues getting fixed faster
than ever
 Reusable Ops and Infosec user stories now part
of the Agile process
 All groups communicating and coordinating
better
 Everybody is getting more work done


The Third Way:
Culture Of Continual Experimentation And
Learning


The Third Way:
Culture Of Continual Experimentation And
Learning

 Foster a culture that rewards:
 Experimentation (taking risks) and learning from
failure
 Repetition is the prerequisite to mastery
 Why?
 You need a culture that keeps pushing into the danger
zone
 And have the habits that enable you to survive in the
danger zone


Break Things Early And Often
 “Do painful things more frequently, so you can
make it less painful… We don‟t get pushback
from Dev, because they know it makes rollouts
smoother.”

-- Adrian Cockcroft, Architect, Netflix


48

Pattern #5: Inject Failures Often


You Don’t Choose Chaos Monkey…
Chaos Monkey Chooses You


Pattern #6: Break Things Before Production

 Enforce consistency in code, environments and
configurations across the environments
 Add your ASSERTs to find misconfigurations,
enforce https, etc.
 Add static code analysis to automated
continuous integration and testing process


Pattern #6: Allocate 20% Of Cycles To
Technical Debt Reduction


Recognize Compounding Technical Debt…


That Gets Worse…


And Fixing It…

Source: Pingdom

An Innovation Culture

“By installing a rampant innovation culture, they
now do 165 experiments in the three months of tax
season.

Our business result? Conversion rate of the
website is up 50 percent. Employee result?
Everyone loves it, because now their ideas can
make it to market.”

--Scott Cook, Intuit Founder

57

Why Do I Think This Is
Important?

58

The Downward
Spiral…

59

The Three Ways: Some Patterns

First Way Second Way Third Way
Define The Wake Up Break Things Early
Work And Make Developers And Often
It Visible

Make Embed Dev Into IT Reserve 20% Of
Environments Operations Cycles For
Available Early Technical Debt
Reduction

62

63

Help The Business Win…


With Support From Your Peers…


And Do More With Less Effort…


67

When IT Fails: A Business Novel and
The DevOps Cookbook
 Coming January 15, 2013 and Q1 2013

“The greatest IT management book of our generation.”
Branden Williams, CTO Marketing, RSA

“The lessons in When IT Fails might just save your business if IT fails
for you. Every IT executive should share this book with their business
peers.”
James Turnbull, VP Operations, Puppet Labs and author of “Pro
Puppet”

“This book will have a profound effect on IT, just as The Goal did for
manufacturing.‟
Jez Humble, co-author of the Jolt award-winning book Continuous
Delivery, and Principal at ThoughtWorks Studios.


Our Mission: Positively Impact The Lives Of
One Million IT Workers By 2017

 For these slides, the “Top 10 Things You
Need To Know About DevOps,” Rugged
DevOps resources, and updates on the
book:

Sign up at http://itrevolution.com
Email genek@realgenekim.me

 Or text “[email] 74730” to
+1 (858) 598-3980
 Visit:
http://www.instantcustomer.com/go/7473
0

Infosec at Ludicrous Speeds - Rugged DevOps

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Infosec at Ludicrous Speeds - Rugged DevOps

Similaire à Infosec at Ludicrous Speeds - Rugged DevOps (20)

Plus de Gene Kim

Plus de Gene Kim (7)

Dernier

Dernier (20)

Infosec at Ludicrous Speeds - Rugged DevOps

Notes de l'éditeur