StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects
•
0 j'aime•727 vues
Do you know why your software testing strategy isn't finding many of the "really big" bugs hidden in the web-based software your company churns out? Find out now...
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (17)
Similaire à StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects
Similaire à StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects (20)
Plus de Rafal Los
Plus de Rafal Los (20)
Dernier
Dernier (20)
StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects
- 1. 1 30 January 2015 Detective Work for Testers finding workflow-based defects Rafal M. Los HP ASC Sr. Security Solutions Expert
- 2. Security Quality is a Tough Business 2
- 3. 3 Workflow-based security defects in Web applications are especially difficult to identify because they evade traditional, point-and-scan vulnerability detection techniques. Understanding these potential defects and why black-box scanners typically miss them, are key to creating a testing strategy for successful detection and mitigation. Rafal Los describes the critical role that testers play in assessing application work flows and how business process-based testing techniques can uncover these flaws. Rafal demystifies the two main types of workflow-based application vulnerabilities—business process logic vulnerabilities and parameter-based vulnerabilities—and provides you with a sound basis to improve your testing strategies. Become a security testing sleuth and learn to find the workflow-based security defects before your system is compromised. Abstract
- 4. Background What we know for sure… − Security vulnerabilities exist in virtually all web sites − “Security in a vacuum” is your largest risk − A cooperative SDL mingling QA and IT Security is key QA’s advantage − Understanding site flow − Understanding business logic The advantage of knowing the application! 4
- 5. Challenges Traditional issues − “Security is IT Security’s problem, right?” − Domain separation (developers vs. QA vs. security) − Lack of tools… processes… education Web 2.0 challenges − Extremely complex technologies − “The Hydra” problem − “The Maze” problem 5
- 6. Workflow-Based Security Defects Defined: Any security defects which can only be identified by following a specific flow, or logic within the site or application. To be clear… − Not a new type of attack vector − Identifying old attacks in their hiding places − Workflows often hide untested defects (like land-mines) 6
- 7. Workflow-Based Security Defects Common examples Registration Form Legitimate Transactions (rewards points) 7 Form 1: Enter User Details Form 2: Enter Account Info (userID, pwd) Form 3: Enter email, finalize reg. Form 4: Confirm All Details Page 1: Select a retailer Page 2: Enter purchase details Page 3: Complete sale Page 4: Receive rewards points
- 8. An(almost)Real-Life Example! Just another online banking app Multiple steps = multiple hiding spots 8
- 9. An(almost)Real-Life Example! Just another online banking app Transferring money… Form inputs 2 drop-downs, 2 free-form 9
- 10. An(almost)Real-Life Example! Just another online banking app Can we spot logic flaws? confirmation page get free money? 10
- 11. An(almost)Real-Life Example! What do we learn? Defects “?” forks workflows use-cases − Most use-cases are non-linear* − *Non-linear = users have options/choices − Automation cannot intelligently trace non-linear use- cases on its own… human intellect is required 11
- 12. Workflow Complexity Workflow complexity is a critical factor Low-Order Workflow − One to several steps − No forks − Pre-defined selection fields (no free-form input) High-Order Workflows − Potentially dozens of steps − Multiple forks − Free-form input fields (with back-end logic) 12
- 13. Why Automation Fails • Automation relies on spidering technology − essentially link processing • Spiders submit data & click buttons − Most spiders submit garbage data fail − Good spiders allow user to pre-define a data set possibly fail − Great spiders submit a number of combinations per form 13
- 14. Spiders, Clicks and Data-Sets What about- highly complex forms? anti-automation technologies? 14 30 January 2015 Link: … Link: … Link: … Form__ Param1= Param2= Param3= [button] Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: …
- 15. Complexity vs. Automation The problem with highly complex structures… 15
- 16. High Complexity Case Study Online trading application − 100 forms+ − ~10,000 combinations each − >1,000,000 possible execution paths across the application… Can be solved… − Built an EFD… What’s an EFD you ask? Hold that thought! 16
- 17. Break the problem down • Over 1,000,000 possible permutations? • Where do you begin!? … it’ll make your head spin! Here’s the secret… 1. Build an EFD [Execution-Flow Diagram] 2. Identify pivotal inputs 3. Identify 2 additional non-pivotals 4. Test unique execution paths +2 Tackling Complexity 17
- 18. The EFD Tracking execution flow Think of an EFD as the ultimate cheat through a maze… • 2 possible exits to the maze • 1,000,000 possible ways through, only 2 get you out! • EFD is a cheat-sheet (or guide) for following the 2 paths out EFD… Business logic + Form Variations = Complete test 18
- 19. Building an EFD Step 1: Business logic Sample bill-pay flow 19 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
- 20. Building an EFD Step 2: Possible form inputs Pick a step and expand… 20 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
- 21. Building an EFD Step 2: Possible form inputs Pick a step and expand… Enter payment details 4 Form Fields Payee 1 Payee 2 New One-Time Recurring Notes Field Checking Savings Line-of- Credit 2 Pivotals Here
- 22. Focus on the Pivotals & Forks Maximize coverage Minimize effort • IT Security uses a hammer– don’t see flows − QA succeeds here, this is your core function • QA builds fork/flow –driven tests − Minimizes test cycle time (no time wasted repeating) − Maximizes test-case, workflow completeness • Much more complete approach to testing 22
- 23. EFD to Test Plan Execution Execute the test plan 1. Fire up your security testing “tool of choice” 2. Map the application EFD 3. Choose depth or breadth –first technique 4. Execute the test cases • Use your use-case “insider” knowledge • Emphasize forks & pivotals 5. Verify & validate results 23
- 24. Start to Finish Follow the yellow brick road… 24 Map out business logic Identify pivotals/forks Execute test cases Validate & Verify results
- 25. Bring It Home Identifying risks in applications involves • Security “breakers” or hackers • Quality “testers” … use-case knowledge • Developers “builders” … engineers … one cannot mitigate risks without the others. 25 30 January 2015
- 26. Rafal Los Security Specialist HP Application Security Center 26 Blog Following the White Rabbit http://www.communities.hp.com/securitysoftware/blogs/rafal/default.aspx Email Rafal.Los@hp.com Direct (404) 606-6056 Twitter http://twitter.com/RafalLos
Notes de l'éditeur
- … last point… “if only there was a name for that”… Requirements Management!
- Traditional Issues Security is slowly starting to permeate into the other aspects of the SDL… but still largely misunderstood as “security’s problem” IT Security cannot possibly handle the challenge of securing complex web technologies alone Code complexity, servers, systems, integrations are too much for IT Security to try and comprehend Domain separation continues to exist – development, qa/testing, security… none of these cooperatively work to reduce risks Domain separation causes the ‘downward spiral of infighting’… and the blame game Distinct lack of tools… that’s right – LACK OF TOOLS! So many vendors but… where is security? How can security folks bring security defect-finding tools to QA folks and make it comprehensible? Usable? Web 2.0 Challenges Extremely complex technologies are making testing miserable… creating test cases is an ever-increasing exercise in madness Flash and “rich media” applications AJAX , mash-ups and multi-point applications Complexity is the ancient enemy of good security… like a battlefield scenario The “Hydra Problem” There are too many “heads” in the development organization Consistency of thought (wrt security) is nearly impossible in some enterprises Many poor practices, ways to get the solution wrong… No matter how many heads you cut off, it seems like there is always another one popping up! The “Maze Problem” Complex web sites/applications are like a “maze” Many, many data ingress/egress points, interactive systems, partner data feeds, exports, etc… Complex execution paths make it almost a certainty that IT Security alone will miss critical issues Only QA has the map to navigate the maze (or at least the best chance to do so)
- Workflow-based defects are not a new type of attack, only a new way to find these types of attacks within an application (such as an XSS buried within a 3-part form, on page 3)… etc
- Multiple steps means that a “scanner” or automation must know how to proceed from one step to the next Scanners aren’t intelligent but make best-guesses… there is no guarantee they will succeed in stepping-through Automation cannot think therefore has the chance to fail
- Form-based fields can almost always be manipulated in applications where security isn’t integrated in These forms typically either echo-back (XSS) or access a database (SQLi)
- A logic flaw may allow us (in an extreme case) to make a “free transfer”… Transfer money, without losing it from the origin account! Many opportunities for logic flaws within these types of application flows
- Automation relies on “spidering” a site… Spiders fail at work-flows because spiders do not enter data (typically) Even GOOD spiders (which have the ability to enter data) fail because data must be specific One-time actions complicate workflows and fail even the best spiders
- Workflows can be tricky because they often require specific combinations of inputs (or worse… free-form fields) to continue Some workflows are low order (only a few steps, can be navigated using a spider) Some workflows are high order (spidering cannot navigate the flow)
- Automation alone will almost always fail… Simply following links and blindly submitting forms (like most spiders do) isn’t going to walk through a flow How does a spider know it’s actually moving forward and hasn’t forked or gone off the flow? Submitting garbage (pre-defined static data) FAIL Submitting user-defined static data still some FAIL Submitting one-time forms, fields the KEY! How do you account for a CAPTCHA? What about a one-time password? What about other form complexities? What does the audience have to work with??
- Looking at this page it’s easy to see why many black-box testing tools fail… Too many options… consume too many resources CPU + Memory Tables in memory have to handle each path, results, and diff… that’s not very likely in “high complexity” situations
- When you’re faced with a form or set of forms that allows for 100,000+ permutations, how do you solve the problem? Refer to the Data-Flow Diagram (DFD) Building an execution-flow-diagram (EFD) is absolutely critical! Identify “pivotal inputs” as – input which causes your execution flow to fork Execute 2 additional non-pivotal data paths to ensure understanding of execution path Make sure you’re only testing unique + 2 2 extras for certainty
- You can explain an EFD best by using the “maze” example… it’s like having a turn-by-turn map! You know which paths lead to nowhere (or the same place) Business logic diagrams (Data-flow diagrams or DFDs) are combined with form variations to complete the picture!
- Break out each form by the field and options for that field Highlight each of the options which are “pivotal” (which ones create a new fork?)
- IT Security will miss defects (false-negatives) by not fully exercising the application and all workflows QA does a better job natively by focusing on the execution, application understanding and coverage QA’s gap is the security aspect of testing – but automation can compensate for some, manual security analysis for the rest! QA’s role is crucial because… MINIMIZE TIME MAXIMIZE COMPLETENESS Save time, find more bugs… why isn’t everyone doing this already?