Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Exploratory Testing

13 109 vues

Publié le

Presentation on Exploratory Testing delivered by Anutthara Baradwaj at the June 09 Knowledge sharing session hosted at Oracle India

Publié dans : Technologie
  • Login to see the comments

Exploratory Testing

  1. 1. GSJGD Exploratory Testing Anutthara Bharadwaj Microsoft http://blogs.msdn.com/anutthara
  2. 2. if there was an Olympics for FAILURE
  3. 3. Or a World Cup for that which fails the most
  4. 4. surely SOFTWARE would reign supreme
  5. 5. GSJGD How do we find the *%&^ bugs?!!
  6. 6. Manual v. automated • Automation is cool, it gets respect … but … – Automation is code • like any code it will take time, have bugs, cause false positives and require maintenance • it will have an imperfect oracle • Manual testing is tedious … but … – It finds over 90% of the bugs in the system
  7. 7. Manual testing • Scripted manual testing – From rigid (click this button, enter this value) to flexible (achieve this result) – Requires preparatory work – Scripts act as their own documentation • Exploratory testing – Minimizes preparatory work – Documentation of testing is extra work
  8. 8. Manual testing • We need to get better at it … A LOT better at it • What is the right prep time v. test time? • What are the right parts of the app to focus on? – Can we pinpoint problematic functionality? – Can we focus on changed code? – Should we focus on buggy parts? • Where is the method in the madness?
  9. 9. Types of exploratory testing • Freestyle • Scenario • Strategy • Feedback
  10. 10. Exploratory testing • Freestyle exploratory testing – Ad hoc exploration of an application’s features – No rules, not necessary to account for coverage, past tests, etc. – Good for: • familiarizing yourself with an application to prepare for more sophisticated methods • quick smoke tests, sometimes finds bugs
  11. 11. Exploratory testing • Scenario-based exploratory testing – An extension of traditional scenario testing – Start with: • user stories • e2e scenarios • test scenarios – Then widen the scope and inject variation as you execute the scenarios
  12. 12. Exploratory testing • Strategy-based exploratory testing – Freestyle testing combined with known bug finding techniques • documented strategies like boundary values and combinatorial testing • intuitive instinct from experience or familiarity with the application – Focus is on learning tips and tricks and recognizing when to apply them
  13. 13. Exploratory testing • Feedback-based exploratory testing – Starts out freestyle until a test history is built up – That feedback then guides future exploration • Test history • Coverage • Churn • … – Tools are often an integral part of this type of testing as they ‘remember’ history for us
  14. 14. The tourist metaphor • You’re visiting Hyderabad for the first time – You can go it alone – You can buy a guidebook – You can hire a guide – You can take the bus tour • On return visits you can – Repeat the best tours – Choose to visit new places
  15. 15. The tourist metaphor • You’re testing an application – What do you use as a guide? – Don’t just wander the app • Always have a goal in mind • When presented with a choice, allow the goal to guide you • Track which goals work for you – Coverage of code, features, UI – Find important bugs – Force the software to best exhibit its functionality
  16. 16. The guidebook tour • Guidebooks help tourists navigate choices – Boil down the possibilities to a ‘best of’ list – Many tourist follow this advice exclusively • Use – User manual, follow its advice – Online help (the F1 tour) – Third party advice (the blogger’s tour) – Detractors (the pundit’s tour) – Competing products (the competitor’s tour)
  17. 17. The antisocial tour • Tours have rules and there are places off limits • Antisocial tourists ignore those rules – Enter illegal input (the crime spree tour) – Enter the least likely input (the opposite tour) – Choose an input, change your mind, use undo, use cancel, then do it all over again
  18. 18. The back alley tour • Explore the back alleys of your app • Do you track feature usage? – Tour the least popular features – Mix and match popular and unpopular features (mixed destination tour)
  19. 19. The couch potato tour • Being a nonparticipant on a tour is a waste of money • Sometimes for a tester, it finds bugs – Do as little as possible on every screen – Leave fields blank, enter as little info as possible, always take the shortest path • Why? – Default values have to be set, used – Decisions have to be made by the software – Just because you aren’t working hard doesn’t mean the software isn’t either
  20. 20. The obsessive compulsive tour • OCD tours would likely be quite unpopular for real tourists – Avoiding sidewalk cracks – Walking the same street over and over obsessively • But testers can use repetition to their advantage – Developers often code based on an expected path – Repeating inputs individually and in sequence can often lead to bugs
  21. 21. The all nighter tour • The ‘clubbing tour’ is about staying out all night, a real test of the constitution • Software should stay out all night too – Keep it working, never shut it down – Keep files open without saving or closing them • This tests time-out functionality, finds memory problems and leaks that are masked by shutdown-restart
  22. 22. The TOGOF tour • Test-One-Get-One-Free • Run multiple copies of the app • Put them through their paces • Try and open the same files, access the same resources … can they coexist? • Why TOGOF? Because if you find a bug in one copy, you’ve broken them all 
  23. 23. The saboteur • The sabotage tour • Ask the software to do something – Access a resource – Open a file – Perform a tasks • Understand the resource it uses – Memory – Files – Network • Then prevent it from doing so – Take away memory – Delete/rename files – Turn off the network
  24. 24. The collector’s tour • Some people need to do it all – Pictures of landmarks, samples at a wine tasting, signatures from the Disney characters – They don’t want to miss anything • Testers can learn from this – Force the software to produce every output • Buy items from every department • Put every special character in an email • Fill a document with every possible graphic • …
  25. 25. The supermodel tour • This tour goes only skin deep – Ignore the function and meaning and look only at the interface – It’s about first impressions • Watch the interface elements – Do they look good and render properly? – Are interface elements consistent across the app? – As you make changes does the UI refresh properly?
  26. 26. Types of exploratory testing • Scenario • Then inject variation – Start with a scenario – Freestyle guidance • Strategy – Tour metaphors – Start with accumulated testing knowledge • Feedback – Test for a while – Build up a history – Use the history to guide additional testing
  27. 27. Obstacles to exploratory testing • Where’s the accountability? • How do we track completeness? • What metrics are important? WE NEED TOOLS!
  28. 28. GSJGD Q/A