7. BOSTON 10-11 SEPT 2018
Why do developers make mistakes?
• We need it ASAP
• Agile environment requires multiple release per day/week
• Security requirements are not followed
• Threat model not in place
• Rely on SAST tools for vulnerability detection
• Security testing is not embedded in the CI/CD
• Security is too late in the SDLC
8. BOSTON 10-11 SEPT 2018
Security teams challenges
• Provide security at the DevOps speed
• Have developers understand security
• Detect vulnerabilities in early stage
• Have Pentesters focus on “serious” stuff
9. BOSTON 10-11 SEPT 2018
Mobile security challenges
• Different Operating Systems
• Client side testing
• Different apps (native, hybrid, web)
• Different languages (poor SAST tool support)
• Different security controls
10. BOSTON 10-11 SEPT 2018
Improve the testing
• Security must be an accelerator and not a step back
• Extend the SDLC with security integration tests
14. BOSTON 10-11 SEPT 2018
Why BDD in security
• BDD offers more precise guidance on organizing the conversation between developers,
testers and security experts
• Notations originating in the BDD approach, in particular the given-when-then canvas,
are closer to everyday language and have a shallower learning curve
• Tools targeting a BDD approach generally afford the automatic generation of technical
and end user documentation from BDD “specifications"
19. BOSTON 10-11 SEPT 2018
OWASP MSTG to BDD
Test scripts
.features
steps
20. BOSTON 10-11 SEPT 2018
Process
Test scripts APK
.features
steps.rb
Emulator / Device
21. BOSTON 10-11 SEPT 2018
Full process in CI/CD
Security Requirements
Threat modelling
(abuse case generation)
Implement BDD standard
security tests
Implement BDD application
specific security tests
Test against acceptance
environment
MSTG Test casesMASVS Checklist
Manual PT
Identify what can be
automated
Requirements Design Code Build Test Release Deploy Operate
22. BOSTON 10-11 SEPT 2018
Attack surface
22
Application layer OS and architecture layer Network layer
AUTHENTICATION
ACCESS CONTROLS
SESSION MANAGEMENT
ENCRYPTION
OBFUSCATION
INPUT & ERROR VALIDATION
DATA PROTECTION
MORE
PERMISSION MODEL
SERVICES
LIBRARIES
RESIDUAL DATA
MORE
CERTIFICATE PINNING
ENCRYPTION
MITM
URL WHITELISTING
WEB SERVER ASSESSMENT
NETWORK SCAN
MORE
INTERACTION WITH OS
23. BOSTON 10-11 SEPT 2018
What are we going to do
• Automate MASVS using Calabash, Gherkin and Ruby
• Identify what we can automate from the MSTG
• Extend UI/UX testing framework to create security integration tests
• Write BDD tests
24. BOSTON 10-11 SEPT 2018
Why
• MASVS is becoming the standard de facto for security testing
• MSTG is the technical sister (thanks Sven Sneiler and Bernard Muller)
• All the checks are currently performed manually from pentesters, security
engineers, developers,
• ..or integrating SAST tools in the pipeline. But SAST is not too smart!
• With BDD security is pushed left in the SDLC
25. BOSTON 10-11 SEPT 2018
Benefits
• Increase security maturity of the teams
• Perform security integration tests on every build improves the code
• Simplify pentesterts life
• Decrease TTR (Time To Release) and enhance security
• Translate threats in tests
• Have a ready-to-use documentation
27. BOSTON 10-11 SEPT 2018
Setup
• Dockerfile
• Calabash
• Android SDK
• Android tools
• JDK
• Genymotion for Personal use (FREE)
• Emulate any Android device
• IDE / Text Editor of your choice
• Recommended: Sublime with Gherkin syntax plugin
28. BOSTON 10-11 SEPT 2018
Outcome
• .features
• A Feature File is an entry point to the Cucumber tests. This is a file
where you will describe your tests in Descriptive language (Like
English).
29. BOSTON 10-11 SEPT 2018
Steps
• security_steps.rb
• Implementation of the Gherkin syntax
• Ruby function with parameters in input
• We are going to use the android tools and to perform analysis on the
device
30. BOSTON 10-11 SEPT 2018
.features
Feature: Logs must not contain sensitive information
@first_scenario
Scenario: As a user I insert my sensitive information and I
check that they are not reflected in the logfiles
Given I clean "all" the application log
31. BOSTON 10-11 SEPT 2018
security_steps.rb
Given /^I clean "(.*)" the application log$/ do |log|
%x(adb logcat -b #{log} -c)
end
32. BOSTON 10-11 SEPT 2018
Clone the GitHub repository
https://github.com/ing-bank/bdd-mobile-security-automation-framework
Today we are going to touch three topics: Security, Mobile and Automation
Because this workshop is heavily focus on how to implement BDD test, the questions come along:
how many of you use BDD tests in their company?
How many of you have used BDD at least once?
how many of you know what BDD is?
Great so let’s introduce BDD
We are in the era of Agile right, this is the time where teams release once ,twice, three , four five six times per second right? But they also need to test at least once twice three times etc.
How do DevOps test?
As you know we have 3 main different type of testing
Avoid or limits the gap between development and security