The document discusses exploratory testing and provides an overview of the speaker's approach. Some key points include: exploratory testing combines experience, structure, and techniques; it involves chartered sessions with logging and debriefing; the tandem approach uses two testers; and it simultaneously designs, executes, and learns from tests. The goal is to gain confidence through a flexible process that fits agile and DevOps environments.

1. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 1
Exploring software;
Combine experience,
structure& techniques

2. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 2
|
Todays presentation
 Our flavour of Exploratory Testing
 A little theory
 Some exercises and dynamic testing
Exploratory Testing © Sogeti 2016 2
This little fellow is R0B3.
He will assist us in this
session.

3. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 3
Our flavour of
exploratory
testing.

4. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 4
|
E.T. sources:
Exploratory Testing © Sogeti 2016 4
Cem Kaner first used “Exploratory Testing” in 1984
Good books:
“Explore it!” by Elisabeth Hendrickson
“Exploratory Software Testing” by James Whittaker
Website:
www.TMap.net
And many more sources…

5. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 5
|
E.T. has many flavours, this is our flavour:
Exploratory Testing © Sogeti 2016 5
• Focus on confidence (risk-based)
• Structured (charter, log, debriefing)
• Session-based & Timeboxed (not too short, not too long)
• Tandem-approach (two people)
• Combine experience-based and coverage-based testing
• Simultaneous test design, test execution and learning
• Flexible (fit for Agile and DevOps)
• Prepared (test ideas, testing tours)
• Tools (heuristics, checklists, test design tooling)
• Serious Fun !!

6. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 6
Focus on
confidence
Confidence

7. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 7
|
Testing = Minimizing difference between
expected and delivered
Exploratory Testing © Sogeti 2016 7
Expected Delivered
Business value achieved
Business value missed
Product risks
materialized and
money wasted

8. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 8
|
Definition of testing
Exploratory Testing © Sogeti 2016 8
TMap® (Test Management approach):
Testing is a process
that provides insight into,
and advice on,
quality and the related risks

9. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 9
|
Testing contributes to gaining CONFIDENCE
© Sogeti 2016 9
Quality
Risks
Confidence
Quality attributes
Analysis and mitigation

10. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 10
|
Definition of exploratory testing
Exploratory Testing © Sogeti 2016 10
Elisabeth Hendrickson:
Exploratory Testing is
simultaneously designing and executing tests
to learn about the system,
using your insights from the last experiment
to inform the next.

11. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 11
|
What is the main deliverable of testing?
Exploratory Testing © Sogeti 2016 11
Go
No Go
ClientTester

12. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 12
|
How do you get the test report?
Investigate the situation
Prepare a test (input & expected result)
Execute a test (compare the actual result)
Use the outcome to report
Testing à la façon TMap © Sogeti 2016 12
Use a testing method

13. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 13
|
Purpose of exploratory testing
Exploratory Testing © Sogeti 2016 13
Fast Feedback Learning
Revealing the
unexpected
Gaining confidence

14. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 14
Tandem-
approach.

15. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 15
|15
How testing is often perceived
Two IT people are cycling up a hill on a tandem-bike.
When they finally reach the summit the one in front, the developer, says:
‘It was hard but we made it!’
Then the one at the back, the tester, comments:
‘Yes and I hope you appreciate that I was using the brakes all the time
because otherwise we would have rolled down the hill backwards’
Make sure you are a tester that helps cycling up the hill !!

16. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 16
|
Tandem approach: 2 people
Exploratory Testing © Sogeti 2016 16
Why work in pairs?
• Two people know more than one
• One subject-matter expert and one testing-expert
• One does the testing, the other the logging
Alternative: one expert and one trainee
Beware:
not suited for very dominant or very introvert people

17. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 17
|
Why 2 people see more than 1
Exploratory Testing © Sogeti 2016 17

18. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 18
|
Why 2 people see more than 1
Exploratory Testing © Sogeti 2016 18

19. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 19
Structured
(charter, log,
debriefing)

20. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 20
|
Information /
Conclusion /
Advice
Session log
Organize with charters: 3 major activities
Exploratory Testing © Sogeti 2016 20
Create
charters
Execute
charter
Debriefing
Backlog of
charters
Session log
Session log
Information /
Conclusion /
Advice
Information /
Conclusion /
Advice

21. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 21
|
Our pragmatic simple charter template
Exploratory Testing © Sogeti 2016 21
Charter
Log
(notes)
De-
briefing
Test ideas: e.g.
testing tours,
test data, etc.

22. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 22
|
Determine test units, assign test charters
Exploratory Testing © Sogeti 2016 22
Someone organizes exploratory testing upfront by creating charters
(e.g. a test manager, scrum master, team member, etc.)
Divide the test object in logical test units
Create one or more charters per test unit
This may involve any kind of risk analysis (e.g. risk poker)
Create a few extra “empty” charters that can be assigned based on
conclusion and advice in the debriefing.

23. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 23
Prepared
(testideas,
testing tours)

24. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 24
|
Test ideas
Inspired by: Elisabeth Hendrickson Exploratory Testing © Sogeti 2016 24
A test idea is
any useful thought, piece of data, technique,
heuristic or whatever
that you write down on your charter
so that during your test session you have an
abundance of possibilities to vary your testing.
Make sure you also think of non-functional quality characteristics!

25. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 25
|
Testing tours
Exploratory Testing © Sogeti 2016 25
Landmarks of Amsterdam
Rijksmuseum
Dam square
Red lights
Van Gogh museum

26. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 26
|
Testing tours
Exploratory Testing © Sogeti 2016 26
James Whittaker distinguishes many different possible testing tours.
Examples:
The landmark tour: think of all the “landmarks” that you must have
seen during your exploratory session, while travelling between these
landmarks you explore the rest of your test object.
The supermodel tour: Only the outside matters, focus testing on
the look-and-feel.
The intellectual tour: Prepare “hard questions” for the test object,
e.g. if you must open a file, find the hardest file to open.
(for more tours read: “Exploratory Software Testing” by James Whittaker)

27. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 27
|
Exercise: test ideas for line-following robot
Exploratory Testing © Sogeti 2016 27
R0B3 is a robot with a light sensor.
It can follow a line
It will stop if it detects a red color
What test ideas come to your mind?

28. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 28
Session-based
& Timeboxed.

29. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 29
|
Running the exploratory testing session
Exploratory Testing © Sogeti 2016 29
No distractions
No interruptions
Timebox:
½ hour – 3 hours
(as specified in charter)Charter

30. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 30
|
Structure: session based & chartered
Exploratory Testing © Sogeti 2016 30
Keep the scope of your charter in mind, but keep some room to follow your
insights to explore somewhat beyond the scope to learn more.
Room for wandering !!

31. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 31
|
Test log
Exploratory Testing © Sogeti 2016 31
Inputs Actions Expected output Actual output
Observations,
Defects, etc.

32. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 32
|
Possible debriefing outcomes
Exploratory Testing © Sogeti 2016 32
Debriefing
Not able to
execute test
New risks
and/or test
ideas
Return
charter to
backlog
New
charter on
backlog
Information /
Conclusion /
Advice
Information /
Conclusion /
Advice

33. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 33
Combine
Experience &
Coverage.

34. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 34
|
Always combine both approaches
combine
Exploratory Testing © Sogeti 2016

35. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 35
|
Four groups of Coverage based testing
Better know & use
one technique
of each group
than
all techniques
of only one group
Exploratory Testing © Sogeti 2016

36. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 36
|
Experience based approaches with TMap HD
Largely intuitive & ad hoc
Structured, chartered



Experience brought together
Exploratory Testing © Sogeti 2016

37. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 37
|
Test the triangle program
What would you want to test?
Why?
What is your expected result?
What is the actual result?
What is your conclusion?
© Sogeti 2016 37

38. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 38
|
Expected outcome is ESSENTIAL
Exploratory Testing © Sogeti 2016 38
Be sure FIRST to think of your expected outcome
before you execute a test.
1 + 1 = ???
Expected: 2
Actual:

39. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 39
Simultanuous
test design,
test execution,
and learning.

40. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 40
|
Why combine structure and flexibility?
Exploratory Testing © Sogeti 2016 40
Exercise
The trainer has an object in mind
(can be anything in the universe)
The participants ask a yes/no question
The trainer answers (yes or no only)
If a participant knows the object  shout it out!
Lee Copeland

41. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 41
|
Exciting…
© Sogeti 2016 41

42. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 42
|© Sogeti 2016 42

43. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 43
Flexible
(fit for Agile
and DevOps)

44. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 44
|
Exploration is a feedback loop
Exploratory Testing © Sogeti 2016 44
TEST
LEARN

45. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 45
|
Early testing
Exploratory Testing © Sogeti 2016 45
Also known as
“Reviewing”
Also known as
“Testing”
Start testing as early as
possible.
Combine static and
dynamic testing
whenever possible.

46. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 46
Tools
(heuristics, checklists,
test design tooling, test
execution tooling, etc.)

47. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 47
|
Use heuristics to get test ideas
Exploratory Testing © Sogeti 2016 47
A heuristic is a rule of thumb, simplification, or educated guess
Numbers:
- Zero
- One
- Many
- Too many
- Too few
Selecting:
- None
- One
- Some
- Many
- All
Timing:
- Always
- Sometimes
- Never
CRUD:
- Create
- Read
- Update
- Delete

48. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 48
|
Test design techniques are tools !
Exploratory Testing © Sogeti 2016 48
Use coverage based techniques as “test ideas”
(thus your start-set of test cases can be made long before …)

49. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 49
|
Tool: nightmare headline game
Exploratory Testing © Sogeti 2016 49

50. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 50
Serious fun !!

51. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 51
|
Bug hunt
Exploratory Testing © Sogeti 2016 51
Get a
charter
Analyse
Pair
testing
one hour
Judge
evalute
defects
Score
sheet
update
The “Friday afternoon-sport”:
competition in short iterations
to determine
which team finds
the most (real!) bugs
(of course there’s a prize!!)

52. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 52
|
Conclusion
source: TMap HD, Niki van Dreumel, ASR Exploratory Testing © Sogeti 2016 52
In the hands of experts,
Exploratory Testing is a structured way
to measure the quality of the information system
in a relatively short time
and to reach an objective view on the risks involved.

53. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 53
Happy
exploration !

54. Exploratory Testing, Techtalk with Capgemini Wroclaw (Poland), 7 December 2016
© Sogeti 2016 54
Questions ?
Answers !
Discussion 