Two models for running automated security tests in a CI/CD pipeline: either blocking or parallel security tests
Integration depends on the level of cultural integration of security into DevOps.
3 Models of test ownership:
1. Owned by Security team - least desirable
2. Owned by DevOps, overseen by security - better
3. Owned by SecDevOps, look Ma, no silos.
Overview of BDD-Security
Configuring Jenkins with BDD-Security as inline tests
Automating security tests for Continuous Integration
1. Automating Security Tests for
Continuous Integration
Stephen de Vries
@stephendv
www.continuumsecurity.net
2. About Continuum Security
• Founded 2012
• Services: Security Testing, BDD-Security jump start
• Products: Securing the SDLC
– Open Source
• BDD-Security Testing Framework
• OWASP ZAP integration with JUnit
• Nessus Java client API
– Commercial
• IriusRisk Risk Management for Application Security: www.iriusrisk.com
3. Security Testing
• Performed after build
• Uses external testers
• Process is opaque to
dev/opts
Unit/Integration/Functiona
l Testing
• Performed during build
• Owned by dev/test
• Tests visible to the team
7. • Everyone is responsible for
• Move testing closer to the code
• Continuous automated testing
• Tests are visible to the team
quality
quality
security
security
^
9. Design Build Integration Tests
Unit
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
Continuous Deployment with Semi-SecDevOps: Parallel tests
10. Who owns the security tests?
A) Security team
• Benefits of automation
• Fast feedback
• Poor collaboration
• Lack of ownership by DevOps
11. Who owns the security tests?
B) DevOps team with oversight by Security
• Better collaboration
• More sense of ownership of security
• Good stepping stone to…
12. Who owns the security tests?
C) Sec + Dev + Ops in a cross-functional
team
• Security testing is our problem
• We have the tools and skills to manage it
13. Automated Security Tests should:
• return either a pass or fail result
• execute quickly (similar to acceptance tests)
• test infrastructure and application tiers
• test functional security features, e.g. Login, Password Reset
• capture manual testing processes and automate them,
i.e. security regression tests
• be checked into version control along with the code
• be understandable by the whole team
25. Summary
• Security testing is just another form of software testing
• Automate as much as possible for faster feedback
• Security Tests can be treated as security requirements
• Self Verifying Requirements!
• Tests written in a BDD language foster collaboration between
sec, dev and ops
• Automated Security tests should include more than just
scanning
Story:
There is a gap between sec testing and every other type of software testing
Sec is still stuck in waterfall model
History of testing and sec testing
Benefits of agile, fast feedback, low cost tests = run them more often
TDD and BDD: Tests as communication mechanism
DevOps is Agile for the rest of the organisation
Don’t need to look far to find answers to integrating security into dev: quality
Sec testing is subset
Security testing > automated scanning
Security testing > Penetration Testing
BDD-Sec intro: objectives. Default set of security tests + allows building more specific tests
There is a considerable gap between how we use security testing and how we use almost every other type of software test.
The first great leap forward for software testing was Agile
But even though we had this benefit of fast feedback, it was constrained to the dev environment.
…there’s a manual step before deploying into production.
All security testing, automated scans and manual penetration testing.
And looking at how testing has evolved over time two things stand out:
we’re doing more automation,
and we’re doing that automated testing closer to the code. Our unit and integration tests are a key part to making continuous delivery possible.
And I claim that that we can and should take the same approach with security testing: automated as much as possible, so that we can run them continuously and get feedback as quick as possible.
Not to replace manual security testing, but to complement it.
The primary control is the automated security tests, the secondary control is the manual tests.
Automated tests can run in parallel, but at same pace as integration tests.
The first activity that springs to mind when talking about automated security testing is probably scanning.