Why and How to build an Application Security Audit Program from the ISACA Chicago 2012 Boat Cruise Event

1. Copyright ©2009 Savid Technologies, Inc. All Rights Reserved
Application Security
Building The Audit Program
Michael A. Davis
Chief Executive Officer
Savid Technologies, Inc.
http://www.savidtech.com

2. Agenda
» Where Is The Application Security Problem?
» Secure Software Development Life Cycle
(SDLC)
» How to deploy a SDLC
» Tips on what not to do
» Steps to Build the Audit Program

3. Who am I?
» Michael A. Davis
– CEO of Savid Technologies
• IT Security, Risk Assessment, Penetration Testing
– Speaker
• Blackhat, Defcon, CanSecWest, Toorcon, Hack In The Box
– Open Source Software Developer
• Snort
• Nmap
• Dsniff
» Savid Technologies
– Risk Assessments, IT Security Consulting, Audit and Compliance

4. Author

5. InformationWeek Contributor

6. Where we got our data
» November 2011 Survey
» Over 450 Security and Audit Professionals
» Follow-up Interviews
» Wide Variety Of Industries
– Financial
– Healthcare
– Business Services

7. We All Know This But..
• 75% of attacks are at the Application Level -
Gartner
• 95% of all vulnerabilities are in software -
NIST
• 7 out of 10 web sites have serious
vulnerabilities - White Hat Security

8. If Cars Were Built Like Applications….
1. 70% of all cars would be built without following the
original designs and blueprints. The other 30%
would not have designs.
2. Car design would assume that safety is a function
of road design and that all drivers were
considerate, sober and expert drivers.
3. Cars would have no airbags, mirrors, seat belts,
doors, roll-bars, side-impact bars, or locks,
because no-one had asked for them. But they
would all have at least six cup holders.
4. Not all the components would be bolted together
securely and many of them would not be built to
tolerate even the slightest abuse.
5. Many safety features originally included might be
removed before the car was completed, because
they might adversely impact performance.
- Denis Verdon

9. Enterprises Inherit Majority of
Application Security Risk
© 2012 Veracode

10. Where is the problem?
Root Cause:
– Developers are not trained to write or test for secure code
– Network security (firewall, IDS, etc) does not protect the Web
Application Layer
– Business Goals do not match Security Goals
•Current State:
– Organizations test tactically when a vuln is found
– A communication gap exists between security and development as such
vulnerabilities are not fixed
– Testing coverage is incomplete and assume training will fix the problem
– We don’t measure or manage application security
– Only looking at Source Code

11. Lets get Specific

12. Language Really Doesn’t Matter

13. Application Security Challenges
• Are vague or too broad (OWASP, BITS)
• Are too detailed & myopic (CWE)
• Lack pragmatic guidance on metrics
• Ignore current threat landscape
• App Sec Program Metrics
– Confuse Risk with LOC
– Disenfranchise developers
– Fail to clearly communicate:
• Impact and Loss to Business
• Savings (remediation, lost opportunity cost)
• Positive progress over time (ROI)

14. #0: Inappropriate Scope
• Many different areas:
– SDLC
– Risk Assessment/Threat Modeling Processes
– Source Code Review
– Dynamic/Static Analysis Technologies
– Education
• What will you include?
– Risk Assessment
– Dynamic/Static Analysis Technologies

15. #1 - Start with Goals
• What are we trying to accomplish?
• Measurement is critical to success
– Outline this BEFORE you pick a technology or
change processes
• What lifecycle stage are most flaws originating in?
• What security mechanisms are we having trouble
implementing?
• What security vulnerabilities are we having trouble
avoiding?

16. #2 - Integrate with SDLC
Abuse
Cases
Security
Requirements
Risk
Analysis
Risk-based
Test Plans
Static
Analysis
Security Ops &
Vulnerability Mgt
Risk
Analysis
Design
Review
Requirements and
Use Cases
Plan Risk
Assessment
Design
Security
Design
Reviews
Application
Security
Testing
S/W Support
Scanning &
Remediation
Build Deploy
Architecture and
Detailed Design Code and Testing
Field Deployment and
Feedback
Organizations that provide security risk-based analysis throughout the
lifecycle will have more resilient software products and systems
Organizational Process Assets cover: governance, policies, standards, training, tailoring guidelines
Modifying the SDLC to incorporate security processes
and tools should be done in phases
Allow for time to change culture and processes
Avoid drastic changes to existing development environment
Balance benefits and determine best integration points
Penetration
Testing
* Adopted in part from “What to Test from a Security Perspective: An Introduction to Security Testing for the QA Professional” (Cigital) and
“Neutralizing the Threat: A Case Study in Enterprise-wide Application Security Deployments” (Fortify Software & Accenture Security Technology
Consulting)
Code
Review
“Build Security In” throughout the lifecycle

17. #3 – Properly Automate
» Quality is not just “Does it work”, Security is a measure of
quality also.
• QA IS THE SPOT
– Use existing ticketing/processes
– QA training will yield higher results (already focused on
negative testing)
– Comprehensive and used to backtest
• Don’t force the developers to do it
– Unless small team
• Leverage technology
– Code Coverage
– Decrease false negatives

18. #4 – Don’t forgot defense in depth
• Reduce impact by locking down the environment – YOU
CAN control these
Guards, locks, tracking devicesPhysical security
Application SecurityApplication
OS hardening, authentication,
update management, antivirus
updates, auditing
Host
Network segments, IPSec, NIDSInternal network
Firewalls, Web Application
Firewalls, Data Leak Prevention
Perimeter
Strong passwords,ACLs,
encryption, EFS, backup
and restore strategy
Data

19. Lack of Failure Analysis
• Failure analysis is the process of collecting and
analyzing data to identify the failed condition of a
complex system
• “Cause and Effect relationships govern everything that
happens and as such are the path to effective problem
Solving” – Dean Gano
• Every Problem in our lives have the three basic elements
connected through causality
• Each Effect, has at least two causes: an Action and
Condition

20. Conditions and Actions exist along a
continuum of Time and space
• Conditions can exist at various times, but the effect is the result
when the action occurs and the conditions exist at the same time
and space.

21. Building the Audit Program
• Start small and focus on implementation of risk
analysis and testing into SDLC
• Don’t attempt to force use of a framework right away
• Look at test plans and ensure security is
represented
• Critical Apps must be pen tested/scanned.
• If culture allows, look for ongoing training.
Preferably using a specific secure programming
course
• Focus on educating development to look at their
metrics and their problems
– Helps prove why SDLC is needed.

22. Review of App Sec Controls/Metrics
Process Metrics
Is a SDL Process used? Are
security gates enforced?
Secure application
development standards and
testing criteria?
Security status of a new
application at delivery (e.g., %
compliance with organizational
security standards and
application system
requirements).
Existence of developer support
website (FAQ's, Code Fixes,
lessons learned, etc.)?
% of developers trained, using
organizational security best
practice technology,
architecture and processes
Management Metrics
% of applications rated
“business-critical” that have
been tested.
% of applications which
business partners, clients,
regulators require be
“certified”.
Average time to correct
vulnerabilities (trending).
% of flaws by lifecycle phase.
% of applications using
centralized security services.
Business impact of critical
security incidents.

23. Conclusion
Contact Information
Michael A. Davis
mdavis@savidtech.com
708-532-2843
Twitter: @mdavisceo