Penetration testing and code reviews are useful for identifying software security vulnerabilities, but for these vulnerabilities to actually be fixed they typically must be communicated to developers for remediation. This lunch and learn discusses real-world strategies for bundling security vulnerabilities into software defects and communicating them to development teams for maximum clarity and impact.

1. Treating Security Vulnerabilities As Software Defects
SANS What Works In AppSec 2010
Friday February 5th, 2010

2. Agenda
• Something Most Security Vendors Won’t Tell You
• The Wrong Way to Do It
• A More Excellent Way
• Strategies
• Demo
• Questions?
1

3. Something Most Security Vendors Won’t Tell You
2

4. Something Most Security Vendors Won’t Tell You
Finding Vulnerabilities is Easy
Fixing Vulnerabilities is Valuable
3

5. The Wrong Way to Do It
4

6. The Wrong Way to Do It
Dan: What is your application security strategy
A: We bought Scanner XYZ
Dan: Cool! Have you started using it?
A: Yes. The analyst who wanted us to buy it ran a bunch of scans when we got
the license key.
Dan: All right! Did you find anything?
A: Oh yeah! We found all sorts of scary stuff.
Dan: Well what did you do about it?
A: We sent the PDF report to the development team and told them to fix the
problems.
Dan: Were they successful?
A: I don’t know. I guess I should check in on that…
5

7. Why Is This Bad?
• PDFs are blobs
• Email is infinitely ignorable
• Lumps all vulnerabilities together
• No guidance for developers
• Just plain rude
6

8. A More Excellent Way
7

9. A More Excellent Way
• Treat (Application) Security Vulnerabilities as Software Defects
• Why?
– Developers have to fix the issues eventually
– Developers understand defects
– Even most “loosely-structured” development teams have defect tracking systems
8

10. What Makes a Good Security Defect?
9

11. What Makes a Good Security Defect?
• Why do I care?
• Scope
• Where is it?
• How do I fix it?
10

12. Why Do I Care?
11

13. Why Do I Care?
• Do not rely solely on the defect to communicate this
– Simply pumping defects into the defect tracking system is unlikely to be effective
• Provide context
• Provide steps to reproduce
– Automated if possible
• Transparency!
12

14. Scope
13

15. Scope
• Defects that take 5 minutes to fix take far longer to administer
– Especially with mature (elaborate) QA processes
• Maximum time: 16 hours
– http://www.joelonsoftware.com/items/2007/10/26.html
• Target: 1-16 hours
– Long enough to be an actual task, short enough to be predictable
– Defects for technical vulnerabilities should be shorter
– Defects for logical vulnerabilities can be longer
14

16. Where Is It?
15

17. Where Is It?
• Providing location information removes a “barrier” to fixing
• Better location information leads to quicker fix times
• Dynamic analysis: attack surface location
– Vulnerability type, URL, possibly parameter
– (For web applications)
• Static analysis: code location
– Filename
– Line (and hopefully column)
– Include actual code if possible in case underlying codebase has changed
16

18. How Do I Fix It?
17

19. How Do I Fix It?
• Prescriptive guidance is required here
– Removes a reason not to fix
– Leads to consistency
• Does your organization have an ESAPI? Does it address this issue?
18

20. Why Is This Approach Better?
• Defects are structured data
• Defects are durable
• Vulnerabilities have been portioned out into tractable chunks of “work”
• We have provided prescriptive guidance
• Communicates with developers via systems they already use
19

21. Strategies
20

22. Strategies
• Group by location
• Group by type
• Group by severity
21

23. Grouping By Location
• By file/URL or by directory
• Pros:
– Helpful if there is one “owner” for that area of the code
– Can help to minimize requirements for QA regression testing
• Cons:
– Different vulnerability types require different fixes
– Can be hard to keep things straight
22

24. Grouping By Type
• By vulnerability type (XSS, SQL injection, authorization issue, etc)
• Pros:
– Similar vulnerabilities often have very similar fixes
– Economies of assembly lines – get Henry Ford on vulnerabilities
– Approach with a “punchlist” mentality
• Cons:
– There can be LOTS of vulnerabilities of a given type if bad coding idioms are in use
23

25. Grouping By Severity
• High, medium, low
• Pros:
– Can help you game certain metric programs
• Cons:
– Least tied to how developers work
– Different types of vulnerabilities
– Cutting across functional areas
24

26. Strategies (continued)
• Combine more than one
– Group by type or severity and then by location
25

27. What About BIG Issues?
• Serious issues can map to multiple defects
• REALLY serious issues can map to enterprise change management
initiatives
26

28. What About Non-Software Vulnerabilities?
• Transition to change management systems rather than defect tracking
systems
27

29. Demo
28

30. Contact
Dan Cornell
dan@denimgroup.com
(210) 572-4400
@danielcornell
Web: www.denimgroup.com
Blog: blog.denimgroup.com
Vuln Mgr: vulnerabilitymanager.denimgroup.com
29