Damon Edwards presentation at DevOps Connect: Rugged DevOps, RSA Conference 2016

1. Ops Happens:
Improve Security Without Getting in the Way
February 29, 2016 ● San Francisco
Damon Edwards
@damonedwards

2. Damon
Edwards
Operational Improvement
DevOps Consulting
Tools

3. Damon
Edwards
Operational Improvement
DevOps Consulting
Tools
Community

4. The Shared Plight of Ops and Security
OPS
&
SEC
“Go faster!”
“Open it up!”
“Be more secure!”
“Be more reliable!”

5. Deployment dominates the conversation
2013
Deployment. Deployment.
Continuous Delivery.
Deployment. Deployment.
Continuous Deployment.
Deployment. CI/CD.
Deployment. Deployment.
Deployment. PaaS.
Deployment. IaaS.
Deployment. Deployment.
Infrastructure as Code.
Deployment. Deployment.
Deployment. Deployment.
Containers. Containers.
Deployment. Deployment.
Deployment. Docker
Deployment. Docker. CaaS.
Deployment. Docker.
Docker. Docker. Docker.
Mesos. Deployment.
Kubernetes. Deployment.
Microservices. Deployment.
Deployment. Docker.
2016

6. What this sounds like to enterprise Ops & Sec
“What we always give you, but more of it… and a lot more frequently”

7. “What we always give you, but more of it… and a lot more frequently”
What this sounds like to enterprise Ops & Sec

8. “Shift Left” to avoid disaster (a.k.a “DevOps 101”)

9. Writing / Running Automated Tests
Writing / Exercising Deploy Automation
Running Security Scanning Tools
“Shift Left” to avoid disaster (a.k.a “DevOps 101”)

10. Writing / Running Automated Tests
Writing / Exercising Deploy Automation
Running Security Scanning Tools
Deploy.
Deploy.
Deploy.
“Shift Left” to avoid disaster (a.k.a “DevOps 101”)

11. But guess what...
Sh*t happens

12. But guess what...
Sh*t happens
Operations

13. How do you “shift left” incident response?

14. How do you “shift left” incident response?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

15. How do you “shift left” incident response?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

16. How do you “shift left” incident response?
But...
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

17. How do you “shift left” incident response?
But...
How do you safely and securely give out access?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

18. How do you “shift left” incident response?
But...
How do you safely and securely give out access?
How do you enable the experts to contribute remediations?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

19. How do you “shift left” incident response?
But...
How do you safely and securely give out access?
How do you enable the experts to contribute remediations?
How do you give visibility into operations?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

20. How do you “shift left” incident response?
But...
How do you safely and securely give out access?
How do you enable the experts to contribute remediations?
How do you give visibility into operations?
How do you do postmortems days/weeks/months later?
Those who build something define the procedures to fix it
Those who build something fix it when it breaks
1
2

21. Design pattern we’ve seen developing in the community...

22. Shift Left Step 1: Establish a Secure Ops Portal

23. Shift Left Step 2: Establish a SDLC for Ops Procedures

24. Shift Left Step 3: Connect with Enterprise Management Systems

25. Shift Left Step 4: Make Compliance Really Happy
Who created the procedure?
Who reviewed it? Who? When? Where? Approval trail?

26. Pay for it with ROI outside of Security
Mark
Maun
Jody
Mulkey
Ticketmaster’s “Support at the Edge” model
• Empowered support teams with self-service ops tasks
• Automated Ops procedures written/vetted by the delivery teams
• Expanded who could take action, but ops remained in full control of
the policy

27. Pay for it with ROI outside of Security
Mark
Maun
Jody
Mulkey
Ticketmaster’s “Support at the Edge” model
• Empowered support teams with self-service ops tasks
• Automated Ops procedures written/vetted by the delivery teams
• Expanded who could take action, but ops remained in full control of
the policy
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ
http://rundeck.org/stories/mark_maun.html
• Removed multiple days of effort from throughout the lifecycle
• Reduced escalations by 30% - 40% and overall support incident
costs by 55%
• Reduced mean time to repair (MTTR) by 50% - 150%

28. Want to talk more about “shift left” and operations?
@alexhonor
alex@simplifyops.com
My colleague who
thinks a lot about
these solutions

29. A word from today’s organizers…

30. A word from today’s organizers…

31. A word from today’s organizers…