There are several techniques for black box software testing that can help reduce the number of tests needed while still providing good coverage. Equivalence partitioning involves dividing inputs and outputs into partitions containing sets of values expected to behave the same way. Boundary analysis refines equivalence partitioning by testing partition boundaries. State transition testing uses a model of component states, transitions, events and actions to systematically test all state transitions. Decision tables present conditions and corresponding actions to define test cases. Together, these techniques help test complex systems efficiently without knowledge of internal design or code.
2. PREVIEW
the problem …
very large or infinite number of test scenarios
+
finite amount of time
=
impossible to test everything
the solution …
Software test techniques exist to reduce the number of tests to be
run whilst still providing sufficient coverage of the system under test
3. BLACK-BOX VS. WHITE-BOX
Test cases derived from specifications
The focus is not the design, nor the implementation
The focus is on the logic of implementation
4. EQUIVALENCE PARTITION
create partitions of the input and output values of the component
each partition shall contain a set or range of values, chosen such
that all values can reasonably expected to be treated by the
component in the same way
both valid and invalid values are partitioned in this way
For each test case specify:
Input to the component
Partition exercised
The expected outcome of the test case
Test completeness criteria: test at least one input/output pair
for each equivalence partition
5. EQUIVALENCE PARTITION
Values between 0 and 15 : not achieved
Values between 16 and 50 : partially achieved
Values between 51 and 85 : largely achieved
Values between 86 and 100 : fully achieved
Variable Equivalence partition Value
percentage value EP1: 0 ≤ x ≤ 15 10
(valid)
EP2: 16 ≤ x ≤ 50 33
EP3: 51 ≤ x ≤ 85 80
EP4: 86 ≤ x ≤ 100 100
percentage value EP5: x < 0 -5
(invalid)
EP6: x > 100 150
EP7: x no integer 1,5
EP8: x non numeric spice
6. BOUNDARY ANALYSIS
refinement of equivalence partitioning for which each edge
of an equivalence class is a representative element of the
class
invalid-input elements are found just beyond the ends
For each test case specify:
the input(s) to the component
the partition boundaries exercised
The expected outcome of the test case
Test completeness criteria: test at least one input/output
pair for each equivalence partition and the “borders”
between the equivalence partitions
7. BOUNDARY ANALYIS
Values between 0 and 15 : not achieved
Values between 16 and 50 : partially achieved
Values between 51 and 85 : largely achieved
Values between 86 and 100 : fully achieved
Variable Equivalence partition Value
percentage value EP1: 0 ≤ x ≤ 15 0, 15
(valid)
EP2: 16 ≤ x ≤ 50 16, 50
EP3: 51 ≤ x ≤ 85 51, 85
EP4: 86 ≤ x ≤ 100 86, 100
percentage value EP5: x < 0 -1
(invalid)
EP6: x > 100 101
EP7: x no integer 1,5
EP8: x non numeric spice
8. EXERCISE 1
Use equivalence partition and boundary analysis to
identify the values that have to be tested for the
following specification:
Up until 500 € value of goods, there is no rebate. From
500 up to 1000 €, there is a 2,5% rebate. Over 1000 up
to 5000 € there is a 5,0% rebate, above this an 8,5%
rebate applies
Up to a weight of 11 kg, sending it by normal mail will
cost 6 €, sending it by express mail costs 12 €. Up until
a weight of 30 kg shipping cost is 11 € (normal) and 18
€ (express). Above 30 kg shipping cost are 17 and 25 €,
respectively.
9. STATE TRANSITION TESTING
use a model of the states the component may occupy,
transitions between those states, the events which cause
those transitions, and the actions which may result from
those transitions
the model shall comprise states, transitions, events, actions
and their relationships
For each test case specify:
the starting state of the component
the input(s) to the component
the expected outputs from the component
the expected final state
Test completeness criteria: 100% of the state transition
diagram
10. STATE TRANSITION TESTING
Start push
pop Start
init
push push init
Empty Filled Full
pop pop Empty
delete
delete push
Filled End
End
push pop
pop push
Filled Filled Full Empty
pop
Filled
State transition tree
11. EXERCISE 2
Create the test transition
tree for the ATM diagram
Identify the unique paths
to achieve 100%
completeness criteria
12. DECISION TABLE
Decision Table
A decision table is a tabular form that presents a set of conditions and their
corresponding actions.
Condition Stubs
Condition stubs describe the conditions or factors that will affect the decision or
policy. They are listed in the upper section of the decision table.
Action Stubs
Action stubs describe, in the form of statements, the possible policy actions or
decisions. They are listed in the lower section of the decision table.
Rules
Rules describe which actions are to be taken under a specific combination of
conditions. They are specified by first inserting different combinations of
condition attribute values and then putting X's in the appropriate columns of the
action section of the table.
13. DECISION TABLE METHODOLOGY
1. Identify Conditions & Find the data attribute each condition tests and all of the attribute's
Values values.
2. Compute Max Multiply the number of values for each condition data attribute by
Number of Rules each other.
3. Identify Possible Determine each independent action to be taken for the decision or
Actions policy.
4. Enter All Possible Fill in the values of the condition data attributes in each numbered
Rules rule column.
5. Define Actions for For each rule, mark the appropriate actions with an X in the
each Rule decision table.
6. Verify the Policy Review completed decision table with end-users.
7. Simplify the Table Eliminate and/or consolidate rules to reduce the number of
columns.
14. EXAMPLE
A marketing company wishes to construct a decision
table to decide how to treat clients according to three
characteristics: Gender, City Dweller, and age group: A
(under 30), B (between 30 and 60), C (over 60). The
company has four products (W, X, Y and Z) to test
market.
Product W will appeal to female city dwellers.
Product X will appeal to young females.
Product Y will appeal to Male middle aged shoppers who do
not live in cities.
Product Z will appeal to all but older females.
15. EXAMPLE
3. Identify Possible Actions:
1. Identify Conditions & Values
2. Compute Maximum W
• gender: M, F
Number of Rules: X
• city dweller: Y, N
2 x 2 x 3 = 12 Y
• age group: A, B, C
Z
1 2 3 4 5 6 7 8 9 10 11 12
Gender F M F M F M F M F M F M
4. Enter All Possible City Y Y N N Y Y N N Y Y N N
Rules Age A A A A B B B B C C C C
1 2 3 4 5 6 7 8 9 10 11 12
W X X X
X X X
5. Define Actions Y X
for each Rule Z X X X X X X X X X X
16. EXAMPLE
Simplify the Table
rules 2, 4, 6, 8, 10, 12 have the same action pattern
rules 2, 6 and 10 have two of the three condition values (gender and city
dweller) identical and all three of the values of the non- identical value (age)
are covered, so they can be condensed into a single column 2
The rules 4 and 12 have identical action pattern, but they cannot be
combined because the indifferent attribute "Age" does not have all its values
covered in these two columns. Age group B is missing
Conditions
Gender F M F M F F M F F M
City Y Y N N Y N N Y N N
Age A - A A B B B C C C
Actions
W X X X
X X X
Y X
Z X X X X X X X X
1 2 3 4 5 6 7 8 9 10
17. BLACK BOX - SUMMARY
Equivalence partition
create partitions of the input and output values of the component
each partition shall contain a set or range of values, chosen such that all
values can reasonably expected to be treated by the component in the same
way
both valid and invalid values are partitioned in this way
Boundary Analysis
refinement of equivalence partitioning for which each edge of an equivalence
class is a representative element of the class
invalid-input elements are found just beyond the ends
State Transition Testing
use a model of the states the component may occupy, transitions between
those states, the events which cause those transitions, and the actions which
may result from those transitions
the model shall comprise states, transitions, events, actions and their
relationships
18. CONCLUSIONS
Advantages of Black Box Testing
more effective on larger units of code than glass box
testing
tester needs no knowledge of implementation, including
specific programming languages
tester and programmer are independent of each other
tests are done from a user's point of view
will help to expose any ambiguities or inconsistencies in
the specifications
test cases can be designed as soon as the
specifications are complete
19. CONCLUSIONS
Disadvantages of Black Box Testing
only a small number of possible inputs can actually be
tested, to test every possible input stream would take
nearly forever
without clear and concise specifications, test cases are
hard to design
there may be unnecessary repetition of test inputs if the
tester is not informed of test cases the programmer has
already tried
may leave many program paths untested
cannot be directed toward specific segments of code which
may be very complex (and therefore more error prone)
most testing related research has been directed toward
glass box testing