Slides from my talk at CEWT #3, http://qahiccupps.blogspot.co.uk/2016/11/testing-all-way-down-and-other.html The idea that testing is or can be a recursive activity - or even fractal - has some currency. In that view, a test or experiment generates some data, which suggests new experiments, which generate some data, which suggest new experiments and so on. The kinds of activities being done at each stage will be self-similar and testing is used as a kind of microscope to focus in on some aspect of the system under test. Testing all the way down. In this talk, I'll instead view testing as a number of different instruments that can be used in an arbitrary number of dimensions. Further, I'll suggest that testing can be applied not only to a system, but to descriptions of that system, to models of that system, to abstractions of that system, to a system which is testing that system, and to a system which is testing the system which is testing that system. And so on. It's testing all the way round. I'll finish by proposing a definition of testing that I think might capture this wide applicability.

1. TESTING ALL THE WAY
DOWN, AND OTHER
DIRECTIONS
James Thomas
@qahiccupps
CEWT #3. 6th November 2016

2. Tested = Checked + Explored
you need a test strategy that
answers two core questions:
1. Does the software behave
as intended …?
2. Are there any other risks?
Neither checking nor exploring
is sufficient on its own.

3. Some Things I was Thinking
Tested = Checked + Explored.
Values? Units? What operation is “+”?
Do we really always need both
checking and exploration? Always?
Past tense brings together both
testing and finishing testing.
"Oh!" he exclaimed. "I get it. I
hadn’t thought of it that way
before. I am testing the
requirements ..."
Testing: “[to] interact with the software
or system, observe its actual behavior,
and compare that to your
expectations"
Can this definition apply outside
software?

4. But was I Testing?
I was reading, re-reading, skim-reading,
searching, cataloguing, note-taking,
cross-referencing, correlating, sense-
making, model-building, consistency-
seeking, comparing, reflecting, filtering,
factoring, experimenting, evaluating,
being sceptical, thinking critically,
thinking laterally, …

5. But was I Testing?
I criticised the definition,
I challenged my model of the definition,
I analysed Elisabeth’s answers,
I reflected on the way I asked questions,
I wondered at why I cared about this,
I sought justification for all the above, …

6. But was I Testing?
Well, I was
… exploring
… a definition of testing
… using techniques from that definition
… which wouldn’t class my actions as testing
And it felt like testing, to me…

7. What is Testing?
Testing is the process of executing a
program with the intent of finding
errors.
Source: Arborosa
Testing is the process of evaluating a
product by learning about it through
exploration and experimentation …

8. Testing is Recursive
Testing is simple: you
understand what is important and
then you test it.
as each flaw ... is discovered ...
[a] mini exploration will result in a
more targeted testing exploration
around this feature area
Adam Knight, Fractal Exploratory Testing
Rikard Edgren, EuroSTAR 2015

9. Testing a sub-sub-feature Testing a sub-sub-feature
Testing a sub-feature
Testing a feature

10. TESTING ALL THE WAY
DOWN

11. Testing a sub-sub-feature
Testing a sub-feature
Testing a feature
Testing the model
Testing the PO’s view
Testing the links between
features
Testing another feature
Testing PO’s expression
of their view
Testing the way I’m
talking to the PO
Testing whether the PO is
the best person to talk to
Testing the end user need
Testing the feature testing
Testing the reason for
testing

12. TESTING ALL THE
OTHER DIRECTIONS

13. So What is Testing for Me?
Testing is the pursuit of actual or potential incongruity.

15. Selected Credits
• Explore It!: https://pragprog.com/book/ehxta/explore-it
• Rikard Edgren: https://www.linkedin.com/in/rikard-edgren-
6bb4612,http://qahiccupps.blogspot.co.uk/2015/11/testing
-is-simple-and-complicated.html
• Adam Knight: http://www.a-sisyphean-
task.com/2013/01/fractal-exploratory-testing.html
• Arborosa: https://arborosa.org/2015/06/14/what-is-testing/
• The Art of Software Testing:
https://www.amazon.co.uk/Art-Software-Testing-Glenford-
Myers/dp/1118031962
• Bolton, Bach: http://www.satisfice.com/blog/archives/856