Not all testers are lucky enough to get a good foundation for their testing—detailed requirements and system specifications. That, combined with the harsh reality of not having enough time, presents a challenge for the tester—How do I test to such an extent that I will be able to identify defects as early as possible and properly inform stakeholders about the quality of the product? Gitte Ottosen shares the approach she uses in agile projects—using a mind map to identify the structure of the system and its major workflows, and applying classic test design techniques and her experience to work in exploratory testing sessions. The result is documented as test notes, classification trees, process flow diagrams, defect reports, etc. Learn how a structured approach—using test design techniques in an exploratory approach—can help integrate continuous test into the agile lifecycle and ensure that the resulting system is fit for use.

1. T2
Concurrent Session
4/8/2014
10:30 AM
“Test Design Techniques in Exploratory Testing”
Presented by:
Gitte Ottosen
Sogeti Denmark
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Gitte Ottosen
Sogeti Denmark
Gitte Ottosen is a managing consultant at Sogeti Denmark, a company delivering test
consultancy to the local market. A career tester, Gitte has eighteen years of
experience in test engineering and test management, in both traditional and agile
contexts. She has worked with test process improvement, first as a test
architect/manager in Systematic and more recently as a test coach for a large
shipping-related IT company in Denmark. A self-confessed test evangelist who
preaches the need for a structured and committed approach to testing, Gitte is a
strong advocate for a context-driven approach, a role requiring profound professional
insight, passion, and persistence—qualities that Gitte holds in abundance.

3. 2014‐02‐04
1
Test Design Techniques in
Exploratory testing
Getting From Nothing to Something
Gitte Ottosen
Gitte.ottosen@sogeti.dk
@Godtesen
1
Sogeti Profile
• Sogeti is a leading provider of structured testing solutions
• Part of the Sogeti Group, which brings together more than 20,000
professionals in 15 countries and is present in over 100 locations in
Europe, USA and India
• Creators of the globally recognized methodologies TMap NEXT ®
and TPI NEXT®
• Well established cooperation with HP, IBM and Microsoft
© Sogeti

4. 2014‐02‐04
2
A bit about me ☺
• I AM CERTIFIED!!!
– ISEB Practitioner
– TMap test engineer
– Certified Scrum masterCertified Scrum master
– Certified Agile Test (CAT) Trainer
– Certificate TPI NEXT Foundation
– …And I am probably not done yet
• But I have also
– Been in the Royal Danish Air force for 9 years
– Tested complex mission critical systems the last 18 years
Read a lot of books about a lot of “stuff” ☺
© Sogeti
– Read a lot of books… about a lot of stuff ☺
– Been at conferences – to learn and share
– Been part of networks – to learn and share
– Been a scout for 10 years – learning by doing
– Been doing a lot of crafts and creative stuff that stimulates my brain
– Been looking at the fish in my aquariums …
3
Why This Presentation Anyway?
• Reactions from Agile testing days 2012
– It sounds simple but how do we do in practice?
– Like your presentation, great with a practical description ofy p , g p p
exploratory
– Have read and heard a lot about exploratory but how do I get
started?
• Less theory more practice
– I want to show you a practical example of how I have used
exploratory in a certain context in order to inspire
d
© Sogeti
you to get started.
– I haven’t got the ”best practice” – but I have a good practice
one from the context I was in at that point in time.
4

5. 2014‐02‐04
3
Let’s get started
5
Definition of Exploratory Test
”An interactive process of simultaneous learning,
test design, and test execution.”
Exploratory testing is not against the idea of scripting. In some contexts, you
will achieve your testing mission better through a more scripted approach; in
other contexts, your mission will benefit more from the ability to create and
improve tests as you execute them. I find that most situations benefit from a
mix of scripted and exploratory approaches.
James Bach
© Sogeti
James Bach
Exploratory Testing Explained

6. 2014‐02‐04
4
© Sogeti
7
The Case
• The Project
– Small agile project – (6 including me)
– Distributed team – developers in Kuala Lumpurp p
– Extension to system in production
• The System
– Web based application
– Complex capabilities for creating customized dialogues and
overviews
© Sogeti
o e e s
– Complex definitions of calculations from the GUI
– Connection to Data Warehouse
8

7. 2014‐02‐04
5
The Challenge
• Organization
– Requirements from TMO
– Product owner vs Project managerj g
• Test basis
– No system specifications for existing system
– Only limited user documentation (two slides)
– A lot of user stories but no overview and limited
business perspective
– No testware for the part of the system already in production
• The ”human challenge”
© Sogeti
g
– Product owner only part time available
– Could not see the value in a tester
– Don’t waste time on documenting – that’s overhead
– Only limited access to people in the project
9
Challenge Accepted
Then what?
10

8. 2014‐02‐04
6
Identifying the Stakeholders
Project
manager
Product
owner
Lead
developer
Users
Test
management
office (TMO)
© Sogeti 11
DevelopersSuper users
Requirement from TMO
• Define and document test strategy
• Continuous test during developmentg p
• Creation of regression test
• Document test in Quality Center
© Sogeti 12

9. 2014‐02‐04
7
Getting the Initial Overview
Product owner presenting the system
• Existing functionality – an overall presentation
• The new area – a more detailed presentation.The new area a more detailed presentation.
• Only verbal – no documentation
© Sogeti 13
From the Conversation - Initial Mind Map
© Sogeti 14

10. 2014‐02‐04
8
Getting an Overview
Aggregating stories to features
© Sogeti 15
But Why Bother?
System Feature
© Sogeti 16

11. 2014‐02‐04
9
Test Strategy in a page
• Objective
• Scope
– Product riskoduct s
– Project risk
• Test ”levels”
– Unit test
– Unit integration test
– Test on story level
– Test in the sprint
– Regressions test
© Sogeti
– Regressions test
• Tools and techniques
– Test design techniques
– Test documentation – QC
17
Getting the Overview
• Initial Exploration
– Started with the operational (stable) part of the application
– Getting acquainted with the conceptsg q p
– Identifying:
• General concepts
• General problems
© Sogeti 18
Taking lots of notesTaking lots of notes

12. 2014‐02‐04
10
Result after First Exploration
© Sogeti 19
Getting Structured
• A lot more exploration
• Identifying and using relevant test design techniques
• Identifying and using relevant heuristicsIdentifying and using relevant heuristics
EP
© Sogeti 20
Pair-wise

13. 2014‐02‐04
11
Where to Apply?
© Sogeti 21
EP/BVA and Classification Trees
Field1
Field 2
Field 3
Field 4
Field5
© Sogeti 22

14. 2014‐02‐04
12
Pair-wise
© Sogeti 23
Process Cycle Test
• Meeting the user
• Drawing and discussing
• Exploration in the systemExploration in the system
• Amending the drawing
• Presenting to PO
Yes Yes
© Sogeti 24
No
No

15. 2014‐02‐04
13
But Also Being Creative…
• I wonder whether they looked at field sizes in the database
when they specified field lengths?
• Have they remembered the support for Danish letters ÆØÅ?
• You can have so many types of measurements… what if you
combine the wrong ones in the calculations (one in KG the
other in Liter)
• When you can specify reporting frequency day, week, month…
what if you combine different frequencies in the summary?
• What if the user doesn’t add columns when creating the
dialogue?
© Sogeti
• What if the user gets interrupted and leaves for 15 minutes
without saving….
• What type of users? Do they understand English (error
messages were in English)… do they understand technical
English?
• ….. Etc etc
25
© Sogeti 26

16. 2014‐02‐04
14
Presenting the Findings
• Showing the classification tree
• Presenting the ”bugs”
• Discussed severity and priorityDiscussed severity and priority
• Documented in defect management tool (backlog)
© Sogeti 27
Supporting the Team
• Input to test design
• Continuous testing of stories
• Clarification of storiesClarification of stories
• Support creation of acceptance criteria
• Reproducing defects (sharing desktop)
• Verifying defect fixes
© Sogeti 28

17. 2014‐02‐04
15
How do we Make a Good Unit Integration Test?
© Sogeti 29
Automating what was originally manual test
Create a new meter with the following values…
Remember negative scenarios…
Regression Test:
From Mind map to Quality Center
• Attaching the test strategy
• From mind map to test tree
• Documenting the charter/missionDocumenting the charter/mission
• Attaching the output of test design techniques
• NO STEPS (or only one – use the attached….)
© Sogeti 30

18. 2014‐02‐04
16
The Result
• Maximum time doing testing minimum doing
documentation
• Thorough test of new functionalityg y
• Regression test of existing functionality
• Creation of “regression test suite”
• Support for unit integration testing
• Clarification of user stories
• Feedback to/from product owner
• A lot of defects in both new and existing software
© Sogeti
• And a product owner that now saw the value
in someone doing structured testing
• Can you come back next year?
31
What I Took With Me
• Re-discovered the joy in doing exploratory testing
• Focus on how to combine what I know – using my tool belt
• Once again recognizing; the world is not either black orOnce again recognizing; the world is not either black or
white – there are many colors… it is all about context.
© Sogeti 32

19. 2014‐02‐04
17
Questions?
© Sogeti 33