Summary from the STP Con Testing Conference: Documenting Tester instructions – a Survey of Successful approaches Track: Test Process Justin hunter, CEO, Hexawise This session provides an overview of several different approaches that are successfully used by testers to address the questions of: What type of testing documentation should you create; and how much detail is appropriate to include in order to guide testers and/or satisfy compliance requirements? This session will discuss pros and cons of several approaches, including: • Checklists • Mindmaps • Test ideas • Specification by example • Detailed test scripts In some software testing presentations, the speaker will explain why they think one particular approach is particularly attractive. This won’t be one of those talks. Justin is a test design specialist who has enjoyed teaching testers how to improve the efficiency and effectiveness of their test case selection and documentation approaches.

1. Documenting Software Testing
Instructions - A Survey of
Successful Approaches
STP Con October 25th, 2011
Justin Hunter

2. Topics
Considerations
Context Objectives Tradeoffs
Documents
“Straw Man” Mind Maps “Kan Ban Cards”
Idea Triggers
Personas Combinations Checklists

3. Topics
3

4. Context is Critical
Tester’s
experience
level /
Exploratory
Testing
skills
Importance of keeping detailed records (e.g.,
for regulatory compliance or lawsuit defense)
4

5. Context is Critical
Relatively little Enough detail to
detail satisfy lawyers
Tester’s
experience
level /
Exploratory
Testing
skills
Enough detail to
avoid tester Very
confusion detailed
Importance of keeping detailed records (e.g.,
for regulatory compliance or lawsuit defense)
5

6. Appropriate Amt. of Detail
Sources: Key Terms (“Cloud Level” / “Clam Level”, etc.)
6 http://alistair.cockburn.us/Use+cases
Image: accelerateddeliveryplatform.com

7. Objectives
7

8. Objectives & Assumptions
? W
o he
h n?
W
Prescriptive vs.
Exploratory?
8

9. Tradeoffs
9

10. Practical Tradeoffs
Reqs Docs and
the SUT
Test ideas
from
looking
at...
Reqs Docs
Only
Few Many
Details
in tests
10

11. Practical Tradeoffs
Reqs Docs Exploratory /
and the SUT Designed During
Test ideas
from
looking
at...
Prescriptive /
Reqs Docs
Only Designed Before
Few Details Many
in tests
11

12. Documentation Strategy
12

13. “Straw Man”
Every test script
MUST have:
• Test Name
• Test Number
• Tester’s Name
• Tester’s Dept.
• Type of Test
• Requirement Number
• Pre-conditions
• Configuration Details
• Test Data
• Specific Steps
• Designer
• Approver
• Execution Time (est.)
• Expected Results
13

14. Time Management
14

15. Overly-Detailed Documentation
Actual
Testing
15

16. Mind Maps
16

17. Mind Maps - Test Plan Template
http://www.softwaretestingclub.com/forum/topics/the-two-
17 page-test-plan?commentId=751045%3AComment%3A105353

18. Clear Communication Example
http://www.softwaretestingclub.com/forum/topics/the-two-page-
test-plan?commentId=751045%3AComment%3A105353
18

19. Evolution of a Mind Map
“1. Identify Properties you're interested in”
http://testerkiwi.blogspot.com/2011/04/building-
exploratory-test-plan-redux.html
19

20. Evolution of a Mind Map
“2. Start Collecting Intel”
http://testerkiwi.blogspot.com/2011/04/building-
exploratory-test-plan-redux.html
20

21. Evolution of a Mind Map
3. After Talking to Developer...
http://testerkiwi.blogspot.com/2011/04/building-
exploratory-test-plan-redux.html
21

22. Mind Maps
Benefits of mind maps so far...
Creation
Maintenance
Quick and Easy
Collaboration
Reporting
But would mind maps be detailed enough...
... For junior testers?
... For testers who will join team in six months?
22

23. “MindMapping 101 for Testers”
http://www.bettertesting.co.uk/content/?p=956
23

24. Test Details into Test Mgmt Tool
24

25. Collaboration is Easy
25

26. Planning & Prioritizing Risks
http://www.bettertesting.co.uk/content/wp-
content/uploads/2011/01/Test-Phase-3.4.jpg
26

27. Determining Scope - What’s In
http://www.bettertesting.co.uk/content/wp-
content/uploads/2011/01/Test-Phase-3.4.jpg
27

28. Determining Scope - What’s Out
http://www.bettertesting.co.uk/content/wp-
content/uploads/2011/01/Test-Phase-3.4.jpg
28

29. Documentation
29

30. Kan Ban Boards
30

31. http://www.flickr.com/photos/roxocubes/5786672702/
31

32. Paul Holland - (Testing-Focused)
Paul Holland
Alcatel-Lucent
(personal photo of his
testing board, used
with permission)
32

33. Paul Holland - (Testing-Focused)
Each note represents a half-day of testing
activity (via Session-Based Testing)
Highly recommended reading: For a detailed description of Paul’s
experiences with this approach and his lessons learned, check out Michael
Bolton’s “A Sticky Situation” in an upcoming version of STP Magazine.
Also, Google “Session-based Test Management”
33

34. Kan Ban-ish Tool: Pivotal Tracker
34

35. More Details on Hover-Overs
35

36. More Detail Still by Clicking
36

37. Including Recent Test Results
37

38. Test Descriptions / Size Estimates
38

39. 39

40. Soap Opera Testing
Tests should be fun and aggressive.
Write scenarios that are:
• Reality-based
• Exaggerated, and
• Condensed
http://www.logigear.com/logi_media_dir/Documents/
Soap_Opera_Testing.pdf
40

41. Soap Opera Testing
http://www.logigear.com/logi_media_dir/Documents/
Soap_Opera_Testing.pdf
41

42. Soap Opera Testing
http://www.logigear.com/logi_media_dir/Documents/
Soap_Opera_Testing.pdf
42

43. Soap Opera Example
http://www.logigear.com/logi_media_dir/
Documents/Soap_Opera_Testing.pdf
43

44. Personas and “Substruction”
http://www.software-testing.com.au/blog/2006/07/30/
personas-substruction-and-other-trades-tricks/
44

45. Behavior-Driven Development
45

46. Behavior-Driven Development
46

47. 47

48. How to Select?
48

49. ... How to Handle More Details?
49

50. ... and still MORE Details?
50

51. How to Document?
Test inputs
from all 3
mind maps
are entered.
51

52. jklhlkjhkjhkjh
Pairwise coverage in 19
highly-varied tests
52

53. Auto-populate Mad Libs-like tests
53

54. Detailed into Detailed Test Scripts
Turned test scripts: quickly
generated by Hexawise.
54

55. Checklists
55

56. Checklists
• Cem Kaner - The Value of Checklists and the Danger of
Scripts: What Legal Training Suggests for Testers http://
www.kaner.com/pdfs/ValueOfChecklists.pdf
• Justin Hunter - http://hexawise.wordpress.com/2009/11/04/
checklists-good-test-scripts-bad/
• Thoughts from the Test Eye - Software Quality
Characteristics
• Michael Hunter, “You Are Not Done Yet”
• Elizabeth Hendrickson - Testing Cheat Sheet
56

57. Software Quality Characteristics
http://thetesteye.com/blog/2010/11/
software-quality-characteristics-1-0/
57

58. Rapid Reporter (Free Tool)
http://testing.gershon.info/reporter/
58

59. Checklists
www.kaner.com/pdfs/ValueOfChecklists.pdf
59

60. “Easy-Access Executive Summary”
An organized summary of all the links
referred to in this presentation is
available in the link below:
http://bitly.com/bundles/hexawise/2
60

61. Invitation to SlideShare Viewers
If you have read this far, odds are very good that you’re the kind of tester
who is interested in trying out promising new testing tools.
Hexawise is a powerful test design tool that allows testers to zero in on a
small set of unusually-powerful tests. We’d like you to try it and let us
know what you think about it.
We have an unusual - some have even said bizarre - pricing policy. While
enterprise licenses of Hexawise start at $50,000 / year, we make fully-
featured versions of Hexawise available at no cost for teams of 1 to 5
testers. It’s like having a rich uncle with a $50,0000 BMW who gave you
the keys to his car indefinitely. And he wouldn’t get upset even if you kept
it or ruined it. Wouldn’t you at least want to take it out for a test drive?
Sign up now https://app.hexawise.com/signup-1-5
61