Matt Tesauro discusses moving application security (AppSec) beyond traditional security testing towards event-based security using continuous integration/continuous delivery (CI/CD) pipelines and automation. Key points include: - Implementing AppSec pipelines that automate security tasks using tools like Docker to increase efficiency and consistency while reducing friction between AppSec and development teams. - Treating individual security findings as tests that are run continuously via tools like Jenkins to quickly determine when issues are fixed. - With increased automation and efficiency, one company increased the number of application assessments from 44 in 2014 to over 400 in 2016 while reducing AppSec staffing levels.

1. AppSec Pipelines and
Event based Security
Moving beyond a traditional
security test
Matt Tesauro
matt.tesauro@owasp.org

2. Hello!
I am Matt Tesauro
I think AppSec needs to change
And I’m going to tell you how is see it changing
matt.tesauro@owasp.org / @matt_tesauro

3. Custom Coachwork and
Bespoke AppSec

4. Who is
This
Guy?

5. Proposed
Traditional AppSec Programs
cannot scale to fit today's needs
and AppSec needs to change

6. The Phoenix Project
3 Ways of DevOps

7. AppSec
Our purpose is to make
the security posture
of apps visible to the
business

8. How AppSec sees itself

9. How Devs see AppSec

10. if you can’t find love,
change your appearance.[1]
As any dating-website veteran
will tell you,
[1] Economist, Jan 21, 2017 - http://sl.owasp.org/economist-quote

11. AppSec Pipelines
Using CI/CD as inspiration,
figure out your AppSec workflow

12. “Spending time
optimizing anything
other than the critical resource is
an illusion.
W. Edwards Deming

13. Key Goals of AppSec Pipelines
◈ Optimize the critical resource -
AppSec personnel
○ Automate the things that don’t require a human brain
○ Drive up consistency
○ Increase tracking of work status
○ Increase flow through the system
○ Increase visibility and metrics
○ Reduce any dev team friction with application security

14. Gen 1 Pipelines
Look at your team's purpose and
those processes which aid it

15. “To put the world in order,
we must first put the nation in order;
to put the nation in order,
we must first put the family in order;
to put the family in order;
we must first cultivate our personal life;
we must first set our hearts right.
Confucius

16. Custom
Made
With finite
Options

17. First, get your house in order...

18. Key Features of AppSec Pipelines
◈ Designed for iterative improvement
◈ Provides a reusable path for AppSec
activities to follow
◈ Provides a consistent process for both the
team and our constituency
◈ One way flow with well-defined states
◈ Relies heavily on automation
◈ Grow in functionality organically over time
◈ Gracefully interconnects with the
development process

19. Gen 2 Pipelines
Look outside your team's and
those processes which aid others

20. DevOps Pipeline AppSec Pipeline
Gen 2
AppSec
Pipeline

21. A call to action...

22. AppSec
Chat Ops
Making chat the way
you do security

23. Advice for Devs - 24x7

24. FYI: You’re being attacked

25. FYI: You’re being blocked

26. Weaponizing Jenkins
◈ Zero false positives
○ Anaphylactic shock
◈ Health Checks vs Scanning
○ Run these all the time
◈ Home of specific issue tests
○ Find a vuln, write a test
◈ Cadence for longer running tests
○ These NEVER break the build
○ Every X builds or every Y days

27. Scaling with
Docker Containers

28. docker run -it --name kali-pipeline kali-pipeline
/bin/bash /usr/local/bin/run.sh
'nikto localhost -h localhost -T 58' results.txt

29. Docker Security
Tool Launch
(python, Go)
ZAP
Nikto
Return ZAP IP
Run Scan, Push
Results to S3

30. Benefits
◈ Effectively Scales
◈ Build security tools once,
run anywhere
◈ Ease of deployment

31. Pull in or scale out, your choice
Pull in Docker containers
to your build server
ZAP
Nikto
Scale out to Docker Swarm
ZAP
Nikto

32. Jenkins Pipeline

33. Pipeline as Code

34. AppSec Pipeline
Math
CI/CD + Docker = Event based Security

35. AppSec Pipelines & Event based Security
◈ Security Findings
○ Turn each into a self-contained test
◈ Add those tests to Jenkins
○ Run hourly or at least daily
○ Turn green when they are fixed
◈ Tied alerts / Chat ops to those tests
○ Let them tell you when they are fixed
◈ Let the developer know that release X
fixed finding Y
○ Bonus points for connecting Jenkins test
passing to closing Jira bug
◈ 2 FTEs assessed 35 Apps in year 1

36. AppSec Pipeline
for OWASP

37. OWASP’s AppSec Pipeline for Projects
◈ Create an AppSec Pipeline of
OWASP Projects to assess
OWASP Projects
Use OWASP Zap
to scan OWASP Security Shepherd,
store the results in OWASP Defect Dojo
and push findings to Jira

38. OWASP Defect Dojo
◈ One-stop source of truth for findings
◈ AppSec Programs, QA, Pen Testers
○ Custom report generation
○ Metrics and Dashboards
○ App & Infrastructure findings supported
◈ New-ish OWASP Project
○ Code base is 3+ years - started at Rackspace
◈ Community and contributor friendly
○ Bugs triaged and verified in 4 hours - 8 to fix
○ 11 contributors from multiple companies
◈ Github: 178 stars, 62 forks, 196 watchers

39. OWASP & AppSec Pipelines

40. What can an
AppSec Pipeline
do for you?

41. 2014
◈ 44 assessments
~5x increase
2015
◈ ~200 assessments
Changes from 2014 to 2015:
- Created the AppSec Pipeline - initial launch in March 2015
- AppSec team numbers dropped - lost a couple of key people approx
3.5 FTEs
- Two of the AppSec team members went meta for most of 2015

42. 2015
◈ ~200 assessments
~2x increase
2016
◈ 414 assessments
Changes from 2015 to 2015:
- Lost 2 key FTE engineers
- AppSec team numbers dropped - not every vacant FTE position
was filled

43. 2014
◈ 44 assessments
9.4x increase
2016
◈ 414 assessments
Things to remember
- Year 1 may go slow - you need to build a solid foundation
- Get your house in order, THEN reach out to other teams
- Divide tests into
- Quick, low false-positive - these go into CI/CD
- Longer, less accurate tests

44. Company A
◈ Adopted DefectDojo
for their pipeline
◈ 4,000 employees
◈ 2,000+ issues tracked
◈ Manual Pen Tests
◈ Reporting
◈ Dashboard
Anonymous Co’s
Company B
◈ Migrated off COTS to
DefectDojo
◈ Imported 20k issues
◈ Currently at 50k+ issues
◈ Reporting
◈ Metrics/Dashboard
◈ API for automation
◈ Read-only for mgmt

45. How can you help?
Help fill the AppSec Toolbox
http://sl.owasp.org/pipeline

50. How can you help?
Help fill the AppSec Toolbox
http://sl.owasp.org/pipeline

51. Thanks!
Any questions?
Aaron Weaver
@weavera
aaron.weaver@owasp.org
/in/aweaver
github.com/aaronweaver
Matt Tesauro
@matt_tesauro
matt.tesauro@owasp.org
/in/matttesauro
github.com/mtesauro

52. CAMS / CALMS
◈ Culture, Automation,
Measurement, Sharing
○ CALMS = CAMS + Lean
◈ Measurement = Metrics => Visibility
◈ Automate the drudgery
○ Allows meaningful personal interactions
◈ What would you want if you were the dev
you’re talking to?

53. Credits
Special thanks to all the people who made and
released these awesome resources for free:
◈ Presentation template by SlidesCarnival
◈ Photographs by Unsplash
◈ Backgrounds by SubtlePatterns

54. Presentation design
This presentations uses the following typographies and colors:
◈ Titles: Playfair Display
◈ Body copy: Droid Sans
You can download the fonts on this page:
https://www.google.com/fonts#UsePlace:use/Collection:Droid+Sans:400,700|Playfair+Display:400,70
0,400italic,700italic
Click on the “arrow button” that appears on the top right
◈ Yellow #ffd900
◈ Light gray #f3f3f3
◈ Black #000000
You don’t need to keep this slide in your presentation. It’s only here to serve you as a design guide if you need to create
new slides or download the fonts to edit the presentation in PowerPoint®

55. SlidesCarnival icons are editable shapes.
This means that you can:
● Resize them without losing quality.
● Change line color, width and style.
Isn’t that nice? :)
Examples:

56. Now you can use any emoji as an icon!
And of course it resizes without losing quality and you can change the color.
How? Follow Google instructions
https://twitter.com/googledocs/status/730087240156643328
✋ ❤
and many more...