A History class for software testers, discussing some important books, the development of test techniques and some concepts in software testing.

1. History Class
For software testers
Capgemini, CoP Testing
Joris Meerts, 27 May 2013
v1.2

2. About myself
• Joris Meerts
• Software tester since 2007
• Capgemini, Testing public, since March 2012
• Context-driven school of testing
• www.testingreferences.com
• Member of the Dutch Exploratory Workshop
on Testing (DEWT)

3. Test all things, hold fast that which is good.
Thessalonians 5:21 (AD 52)

6. Topics
• Some Famous Books
• My Favourite Techniques
• Tracing Some Root Concepts
• So What’s New?

7. The growth of a testing discipline
The growth of a testing discipline has been painfully slow,
despite the acknowledged need for better quality software.
Even the scope of what is or is not a testing activity is not
well defined.
Terminology in the field is unclear and literature that
would establish some foundations has not been written.
Testing as an activity needs and requires more attention.

8. Some Famous Books
Program Test Methods (1973)
• ‘Program’ testing, not
‘software’ testing
• Acceptance testing
• Test levels
• Design for testability
• Test specification
languages
• Test automation

9. Some Famous Books
The Mythical Man-Month (1975)
• Adding people to a late
project only makes it
later (unpartitionable
task).
• Plan to throw one away;
you will, anyhow.
• Let testers scrutinize
specifications long
before coding starts.

10. Some Famous Books
The Art of Software Testing (1979)
• Black box software
testing
• Higher order testing
(reliability,
configuration,
installability, usability
testing)

11. “Testing is an
extremely
creative and
intellectually
challenging
task.”
Some Famous Books
The Art of Software Testing (1979)
“Good software
testing is a
challenging
intellectual
process.”
Cem Kaner, James Bach, Bret
Pettichord – Lessons Learned in
Software Testing (2001)

12. Some Famous Books
Software Engineering Economics (1981)
• Famous cost-of-change
curve.
• The curve is an
important reason for
waterfall development.

14. My Favourite Techniques
Decision Table (1961)
• Burton Grad - Tabular Form in Decision Logic (1961)
• Decision rules for complex applications
• Eliminate inconsistencies and redundancies

15. My Favourite Techniques
Cause-Effect Graph (1973)
• William Elmendorf -
Cause-Effect Graphs in
Functional Testing

16. My Favourite Techniques
Equivalence partitioning (1975)
• John B. Goodenough,
Susan L. Gerhart –
Toward a Theory of Test
Data Selection
• We prove a
fundamental theorem
showing that properly
structured tests are
capable of
demonstrating the
absence of errors in a
program.

17. My Favourite Techniques
State Transition Testing (1978)
• Tsun S. Chow - Testing
Software Design
Modeled by Finite-State
Machines

18. My Favourite Techniques
CRUD Analysis (1981)
• James Martin, Clive
Finkelstein -
Information Engineering

19. My Favourite Techniques
Exploratory Testing (1988)
• Cem Kaner – Testing Computer Software
• Exploratory testing is not a testing technique. It's a way of
thinking about testing.

20. My Favourite Techniques
Classification Trees (1993)
• Matthias Grochtmann,
Klaus Grimm -
Classification Trees for
Partition Testing
• CTE XL

21. My Favourite Techniques
Pairwise Testing (1996)
• David M. Cohen, Siddhartha R.
Dalal, Jesse Parelius, Gardner C.
Patton - The Combinatorial
Design Approach to Automatic
Test Generation
• An empirical study of user
interface software at Telcordia
found that most field faults were
caused by either incorrect single
values or by an interaction of
pairs of values.

22. My Favourite Techniques
Soap Opera Testing (2004)
• Hans Buwalda – Soap
Opera Testing
• Tests should be fun and
aggressive

23. Tracing Some Root Concepts
Taylorism (1912)
• Frederick Winslow
Taylor – Principles
of Scientific
Management
• Measurement,
standards,
mechanization

24. Tracing Some Root Concepts
Waterfall (1970)
Winston Royce - Managing the Development of Large Software
Systems

25. “I believe in this
concept, but the
implementation
described above is
risky and invites
failure.”
Tracing Some Root Concepts
Waterfall (1970)

26. Tracing Some Root Concepts
V-model (1986)
Paul E. Rook - Controlling Software Projects

27. Tracing Some Root Concepts
Standards (1)
• IEEE 829 Standard for Software and System Test
Documentation (1983)
• IEEE 1012 Standard for Software Verification and
Validation Plans (1986)
• Capability Maturity Model (1988)
• ISO 9126 Software engineering - Product quality
(1991)
• TMap (1995)
• Testing Maturity Model (1996)

28. Tracing Some Root Concepts
Standards (2)
• BCS Standard for Software Component Testing (1998)
• International Software Testing Qualifications Board
(2002)
• ISO 29119 Software Testing (2012)

29. Tracing Some Root Concepts
Definition of testing
The process consisting of all lifecycle activities,
both static and dynamic, concerned with
planning, preparation and evaluation of
software products and related work products
to determine that they satisfy specified
requirements, to demonstrate that they are fit
for purpose and to detect defects.
ISTQB - Standard glossary of terms used in
Software Testing (2012)

30. So What’s New?
Test is dead
Julian Harty – Traditional Testing R.I.P. (2010)
Goranka Bjedov – The Future of Quality (2011)
James Whittaker – All That Testing is Getting in the
Way of Quality (2011)
Alberto Savoia – Test is Dead (2011)
• Testing slows projects down
• Most testing is futile (incorrect
diagnosis, irrelevant bugs)
• Value of quality < Price of quality
• It’s cheaper to fix the bug than to find it
• Users are better testers!
• Only the code matters
• The value of testing is not in the artifacts

31. So What’s New?
Shifting paradigms
Testing is not a phase
Jack Reeves – What Is Software Design?
(1992)
Les Hatton – Testing is not a phase (1999)
Elisabeth Hendrickson – Agile Testing,Nine
Principles and Six Concrete Practices for
Testing on Agile Teams (2008)

32. So What’s New?
Shifting paradigms
Sapient testing
Cem Kaner, James Bach, Bret Pettichord –
Lessons Learned in Software Testing (2001)
James Bach – Sapient Processes (2007)

33. So What’s New?
Shifting paradigms
Testing skills / competences:
• Communicative
• Precise
• Convincing
• Objective
• Creative
• Sensitive
Tim Koomen, Leo van der Aalst, Bart
Broekman, Michel Vroon - Tmap Next
(2006), page 464 - 465

34. So What’s New?
Shifting paradigms
Testing is the art of skillful
investigation by experimentation

35. So What’s New?
Shifting paradigms
Testing is the infinite process
of comparing the invisible
to the ambiguous
so as to avoid the unthinkable
happening to the anonymous.
James Bach - Becoming a Software Testing
Expert (2006)

36. So What’s New?
Shifting paradigms
“A tester is someone who knows that things
could be different.”
“It's an awkward field. If you wanted solid
ground to stand on, you chose the wrong
vocation."
James Bach – The Gift of Time (2008)

37. So What’s New?
Roles in testing
• Auditor
• Acceptance manager
• Test facilitator
• Cultural host
• Business consultant
• Quality assurance
manager