Reference Graham et.al (2006)

2. What Is Testing ?
Syllabus learning objectives for 1.2 What is testing?
1. Recall the common objectives of testing. (Kl)
2. Describe the purpose of testing in software development, maintenance and
operations as a means to find defects, provide confidence and infor mation,
and prevent defects. (K2)
In this section, we will review the common objectives of testing. We'll explain
how testing helps us to find defects, provide confidence and information, and
prevent defects. We will also introduce additional fundamental principles of
testing. As you read this section, you'll encounter the terms code, debugging,
development of software, requirement, review, test basis, test case, testing
and test objective.

3. Let's use an analogy to help us: driving tests. In a driving test, the examiner
critically assesses the candidate's driving, noting every mistake, large or small,
made by the driver under test. The examiner takes the driver through a route
which tests many possible driving activities, such as road junctions of different
types, control and maneuvering of the car, ability to stop safely in an
emergency, and awareness of the road, other road users and hazards. Some of
the activities must be tested. For example, in the UK, an emergency stop test is
always carried out, with the examiner simulating the moment of emergency by
hitting the dashboard at which point the driver must stop the car quickly,
safely and without loss of control.
The Driving Test - an Analogy
For Software Testing

4. At the end of the test, the examiner makes a judgment about the driver's
performance. Has the driver passed the test or failed? The examiner bases the
judgment on the number and severity of the failures identified, and also
whether the driver has been able to meet the driving requirements. A single
severe fault is enough to fail the whole test, but a small number of minor
faults might still mean the test is passed. Many minor faults would reduce the
confidence of the examiner in the quality —of the driving to the point where
the driver cannot pass.
The Driving Test - an Analogy
For Software Testing

5. Defining Software Testing
Let's break the definition down into parts; the definition has some key phrases
to remember. The definition starts with a description of testing as a process and
then lists some objectives of the test process. This is a suitable definition of testing
for any level of testing, from compo-nent testing through to acceptance testing,
provided that we remember to take the varying objectives of these different levels
of testing into account.

6. We can see that the software test is very like a driving test in many ways,
although of course it is not a perfect analogy! The driving examiner becomes the
software tester. The driver being examined becomes the system or software under
test, and you'll see as we go through this book that the same approach broadly
holds.
So, test activities exist before and after test execution. As a tester or test
manager, you will be involved in planning and control of the testing, choosing test
conditions, designing test cases based on those test conditions, executing them
and checking results, evaluating whether enough testing has been done by
Examining completion (or exit) criteria, reporting on the testing process and
system under test, and presenting test completion (or summary) reports.
Software Test and Driving Test Compared

7. We can use both dynamic testing and static testing as a means for
achieving similar test objectives. Both provide information to improve both
the system to be tested, and the development and testing processes. We
mentioned above that testing can have different goals and objectives, which
often include:
 Finding defects;
 Gaining confidence in and providing information about the level of quality;
 preventing defects.
When Can We Meet Our Test Objectives ?

8. Reviewing defects and failures in order to improve processes allows us to
improve our testing and our requirements, design and development
processes. One phenomenon that many testers have observed is that defects
tend to cluster. This can happen because an area of the code is particularly
complex and tricky, or because changing software and other products tends to
cause knock-on defects. Testers will often use this information when making
their risk assessment for planning the tests, and will focus on known 'hot
spots'.
Focusing On Defects Can Help
Us Plan Our Tests

9. A main focus of reviews and other static tests is to carry out testing as early
as possible, finding and fixing defects more cheaply and preventing defects
from appearing at later stages of this project. These activities help us find out
about defects earlier and identify potential clusters. Additionally, an important
outcome of all testing is information that assists in risk assess-ment; these
reviews will contribute to the planning for the tests executed later in the
software development life cycle. We might also carry out root cause analysis to
prevent defects and failures happening again and perhaps to identify the
cause of clusters and potential future clusters.
Focusing On Defects Can Help
Us Plan Our Tests

10. Over time, as we improve our whole software development life cycle and
the early static testing, we may well find that dynamic test levels find fewer
defects. A typical test improvement initiative will initially find more defects as
the testing improves and then, as the defect prevention kicks in, we see the
defect numbers dropping. The first part of the improvement enables us to
reduce failures in operation; later the improve-ments are making us more
efficient and effective in producing the software with fewer defects in it.
The Defect Clusters Change Over Time

11. As the 'hot spots' for bugs get cleaned up we need to move our focus else-
where, to the next set of risks. Over time, our focus may change from finding
coding bugs, to looking at the requirements and design documents for
defects, and to looking for process improvements so that we prevent defects
in the product.
The Defect Clusters Change Over Time

12. When a test finds a defect that must be fixed, a programmer must do some
work to locate the defect in the code and make the fix. In this process, called
debug-ging, a programmer will examine the code for the immediate cause of the
problem, repair the code and check that the code now executes as expected. The
fix is often then tested separately (e.g. by an independent tester) to confirm the
fix. Notice that testing and debugging are different activities. Developers may test
their own fixes, in which case the very tight cycle of identifying faults, debugging,
and retesting is often loosely referred to as debugging. However, often following
the debugging cycle the fixed code is tested independently both to retest the fix
itself and to apply regression testing to the surrounding unchanged software.
Debugging Removes Defects

13. This principle arises from the theory of the process of scientific
experimenta-tion and has been adopted by testers; you'll see the idea in many
testing books. While you are not expected to read the scientific theory
[Popper] the analogy used in science is useful; however many white swans we
see, we cannot say 'All swans are white'. However, as soon as we see one black
swan we can say 'Not all swans are white'. In the same way, however many
tests we execute without finding a bug, we have not shown 'There are no
bugs'. As soon as we find a bug, we have shown 'This code is not bug-free'.
Is The Software Defect Free ?

14. There is another important principle we must consider; the customers for
soft-ware - the people and organizations who buy and use it to aid in their
day-to-day tasks - are not interested in defects or numbers of defects, except
when they are directly affected by the instability of the software. The people
using soft-ware are more interested in the software supporting them in
completing tasks efficiently and effectively. The software must meet their
needs. It is for this reason that the requirements and design defects we
discussed earlier are so important, and why reviews and inspections are such a
funda-mental part of the entire test activity.
If We Don’t Find Defects Does That Mean
The Users Will Accept The Software

15. “THANK YOU”