1. TE
AM Tutorial
10/1/2013 8:30:00 AM
"Discovering New Test Ideas:
Getting that Burst of Creativity"
Presented by:
Karen Johnson
Software Test Management, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Karen N. Johnson
Software Test Management, Inc.
Karen N. Johnson is a software test consultant, working on-site and remotely to help her clients
build testing teams and strengthen their testing practice. Karen is considered an expert in
software testing and has been part of the context-driven software testing community for a
number of years. She is a frequent speaker at testing conferences; a contributing author to the
book, Beautiful Testing; and co-founder of WREST, the Workshop on Regulated Software
Testing. Karen has published numerous articles and blogs about her experiences with software
testing.
3. 8/20/2013
Discovering
New
Test
Ideas
Karen N. Johnson
Copyright 2013
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
1
about: Karen N. Johnson
• Software Test Consultant
• Published Author
• Teach Software Testing
• Co-founder of WREST, the Workshop
on Regulated Software Testing
• Site: www.karennjohnson.com
• Twitter: karennjohnson
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
2
1
4. 8/20/2013
Is there space
in our work for
creativity?
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
3
What stops us?
Mental locks
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
The right answer
That’s not logical
Follow the rules
Be practical
Play is frivolous
That’s not my area
Avoid ambiguity
Don’t be foolish
To err is wrong
I’m not creative
– Roger von Oech, “A What on the Side of the Head”
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
4
2
5. 8/20/2013
agenda
•
•
Introduction
Exercise: the cup
•
•
Brainstorming explained
Exercise: brainstorming
•
•
Heuristics explained
Exercise: heuristics
•
•
The Phoenix Checklist explained
Exercise: The Phoenix Checklist
•
•
Tours Explained
Exercise: tours
•
•
$100 Test explained
Exercise $100 test
•
•
Exercise: Brainstorming Part 2
Team Debriefs
5
Exercise: the cup
The premise
We all get overloaded with
demands and distraction.
A solution
Empty what is on your mind
onto the paper provided. You
do not need to share or
discuss the paper, it is yours.
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
6
3
6. 8/20/2013
about: Brainstorming
• Originated by Alex F. Osborn
• Start with a specific question
• Guidelines:
1. Defer judgment
2. Quantity is wanted
3. “Free-wheeling” is
welcome (the wilder an idea,
the better)
4. Combination and
improvement to ideas
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
7
Brainstorming Blockers
• Free riding - not contributing
because you feel the group has it
“covered "without your input
• Criticism - fear of your idea being
evaluated harshly
• Social matching effect - shutting
down your contributions to match
the productivity of the group
• The illusion of group productivity thinking the group has found enough
ideas
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
8
4
7. 8/20/2013
Exercise: Brainstorming
1. Break into teams
2. Work with the problem
assigned to your team
3. Refine the problem. Make
the problem specific.
4. Collect a quantity of ideas
5. In a second session you
will have time to go back
and evaluate ideas
Discovering New Test Ideas v6
heu·ris·tic
© Karen N. Johnson, 2013
9
a Greek word meaning "find" or "discover"
Definition of HEURISTIC
: involving or serving as an aid to
learning, discovery, or problemsolving
by experimental and
especially trial-and-error methods
<heuristic techniques> <a heuristic
assumption>; also : of or relating to
exploratory problem-solving
techniques that utilize self educating
techniques (as the
evaluation of feedback) to improve
performance
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
10
5
8. 8/20/2013
Exercise:
Heuristic card deck
1.
2.
3.
4.
In groups, use the heuristic card deck.
In groups, review the heuristics cheat sheet.
Make a note of heuristics that helped.
Make a note of confusions and apprehensions using either
"tool."
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
11
The Phoenix Checklist
The Phoenix Checklist is list of
questions developed by the CIA
to encourage agents to consider a
problem from different angles by
using a series of questions.
Process:
1. Determine the problem you
want to resolve.
2. Write the problem down.
3. Use the questions to resolve
the problem.
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
12
6
10. 8/20/2013
The "tourist" metaphor
Here's a list of tours from Michael
Kelly. FCC CUTS VIDS is the
mnemonic he devised for:
•
•
•
•
•
•
•
•
•
•
•
Feature
Complexity
Claims
Configuration
User
Testability
Scenario
Variability
Interoperability
Data
Structure
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
15
Tours from James Whittaker
•
•
•
•
•
•
•
•
•
•
business district: where the business gets done
historical district: legacy code
tourist district: novice users
entertainment district: supportive features
seedy district: unsavory and illegal operations
landmark tour: create a landmark coverage map
FedEx tour: where data moves through
morning commute: startup tasks and procedures
after hours: maintenance tasks, backup files
intellectual tour: ask the software hard questions
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
16
8
11. 8/20/2013
Exercise:
tours testing
1. In groups, review the tours testing concept
2. Make a note of tours you feel would be helpful
3. Make a note of confusions and apprehensions using the
tours concept.
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
17
There are constraints
on nearly all projects.
Typical constraints:
• time
• money
• resources
• deadlines
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
18
9
12. 8/20/2013
Exercise: $100 constraint matrix
1. Complete the $100 matrix as a team.
2. Your team has $100 to spend and more than $100 of items
to “fund.”
3. Your team will need to choose what to fund and what not to
fund.
4. When your team is done, have a representative from your
team explain what your team chose and how your team
made those decisions
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
19
20
10
13. 8/20/2013
Exercise: Brainstorming Part 2
1.
2.
3.
4.
5.
Return to your teams
Review all the materials
provided today as well as the
idea list your team created at
the start of the class.
What testing techniques do
you feel your team can
implement?
Now is the time to judge the
ideas~
Designate a speaker(s) to
provide a team debrief.
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
21
Resources
Books
• Thinkertoys by Michael Michalko
• Gamestorming by Dave Gray, Sunni
Brown and James Macanufo
• Applied Imagination by Alex F.
Osborn
• Exploratory Software Testing by
James Whittaker
Discovering New Test Ideas v6
Phoenix Checklist
• Cem Kaner & Andy Tinkham:
http://www.kaner.com/pdfs/Explorin
gExploratoryTesting.pdf
• Michael Bolton
• http://www.developsense.com/blog/
2010/11/context-free-questions-fortesting/
© Karen N. Johnson, 2013
22
11
14. 8/20/2013
Brainstorming variations
•
Group passing - write an idea, pass
your paper onto the next person until
the paper is back to the originator
•
Team idea mapping - brainstorm alone
then share with the group, build a map
together of the ideas (may form an
affinity map).
•
6-3-5 - 6 participants, 3 ideas every 5
minutes.
•
3-12-3 - 3 minutes for generating a
pool of observations, 12 for combining
observations to ideas and 3 for
presenting concepts to the group.
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
23
Thank you
For being here!
Feedback is
appreciated.
Karen N. Johnson
Copyright 2013
Discovering New Test Ideas v6
© Karen N. Johnson, 2013
24
12
