A Test Generation Tool for Specifications in the Form of State Machine

1. A Test Generation Tool
for Specifications in the
Form of State Machine
Zheng-Wen Shen
2004/11/23

2. References
 A Test Generation Tool for Specifications in the
Form of State Machines
 Q. M. Tan, A. Petrenko and G. v. Bochmann
 To appear in IEEE International Conference on
Communications -- 96, Dallas, USA, 1996.

3. Outline
1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

4. 1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

5. 1. Introduction
 TAG tool - Test Automatic Generation
 Automatically generates test case for an FSM
specification.
 Transition identification approach for test derivation
 Output test cases in the form of an SDL skeleton.

6.  FSM has been extensively used in the testing
phases of system developments.
 E.g. conformance testing of communication
protocols.
 Communication protocols are the rules that
govern the communication between the
different components within a distributes system.
 7-layer OSI model

7. Protocol conformance testing
 A protocol specification generally can lead to
several implementations in software and/or
hardware.
 Assure the compatibility with other
implementations of the same protocol.

8. The existing test derivation methods
 Check each transition in an FSM specification at
least once (branch coverage criteria.)
 Verify the tail state of the transition to obtain
high fault coverage and to guarantee
conformance in the context of a more general
fault model.
 Using state identification techniques
 Defect the fault that the FSM enters a different state than
specified.

9.  Most of the existing protocols are not
completely specified.
 Not all the sequences of interactions are foreseen.
 The existing test derivation methods for
protocols are limited to completely specified
specifications.
 TAG working directly for deterministic, partially
specified FSM specifications.

10. 1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

11. 2. Using the tool
 A complete test suite that guarantees full fault
coverage may be derived
 A set of test cases that cover a given test
purpose may be derived.
 Output format
 Mnemonic format
 SDL skeleton

12. 2.1 An example
 To achieve a particular test purpose which is a certain
transition to be tested:
1. Bring the FSM from its initial state to the starting state of
the transition under test using the shortest input sequence
possible. – a preamble of the test case
2. Execute the transition and check the observed output
3. Check a tail state of the transition by observing its reaction
to a pre-selected set of state identification sequences.
4. Apply an input sequence to return to the initial state of the
FSM. – a postamble of the test case

13.  State cover
 The set of all preambles.
 Transition cover
 The set of sequences used to execute all specified
transitions.
 State identification sequences
 Distinguish states by their output reactions.
 TAG implements the HSI method

14.  HSI method
 Similar to the widely used W-method
 A characterization set is used for state identification.
 A tuple of subsets of a W set.
 Can be applied to partially specified FSMs.

15. The INRES responder

16.  Input alphabet
 1- CR; 2- IDISreq; 3- ICONrsp; 4- DT0; 5- DT1
 Output alphabet
 0- NULL; 1- ICONind; 2- DR; 3- CC; 4- ACK0;
5- ACK0,IDATind; 6- ACK1; 7- ACK1,IDATind
 States
 S0- Closed (the initial state); S1- Opening; S2-
Waiting_DT0; S3- Waiting_DT1

17. Test suite and intermediate results
 State Identifier = {{41}, {41}, {4}, {4}}
 Preamble = {ε, 1, 13, 135}
 Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15,
131, 132, 134, 135, 1351, 1352, 1354, 1355}
 Postamble = {ε, 2, 2, 2}
 Test Suite = {41, 441, 541, 1241, 13241, 135241,
141, 1141, 1441, 1541, 134, 1314, 13514, 13514,
13554}

18. 2.2 Functions provided by the tool
 A text file containing the FSM specification with
suffix “.fsm”
 Symbol table (.tbl)
 FSM structure (.cpl)
 Complete test derivation
 A complete test suite is generated.
 Selective test derivation
 Give the test purpose, which specifies one transition
in the FSM.

19.  The test output is written in a text file.
 Mnemonic format (.mnc)
 SDL skeleton (.sdl)

20. 2.3 the FSM specification
1. the state definitions
2. The input definitions
3. The output definitions
4. The transition definitions
5. The variable declaration (optional)
6. The homing sequence definition (optional)
7. “end;”

21.  The state definitions are a list of state names.
 The input definition are a list of input names.
 The output definitions are list of output names.
 The transition definitions define the FSM state table
itself by a list of transition specifications.
 Current state name, input name, output name and next state
name.
 May be followed by a set of the comments.

22.  The variable definitions define the variables in
parameters.
 Integer, Charstring, Octetstring, Boolean
 Using Keyword “homing” to give a sequence of
input names as a homing sequence.

23. 1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

24. 3.1 Preambles
 A tree with the initial state as its root is
constructed such that the tail states of the
outgoing transitions from the state
corresponding to a current node.
 All nodes in this tree must become a current
node once and only once in the order that they
enter this tree.
 The path from the root to a given node is
preamble for the corresponding state.

25. S0
1
S1
3
5
S2 S3
Preamble = {ε, 1, 13, 135 }

26. 3.2 Postambles
 A tree with given state as its root is constructed.
 Once the initial state has been added to the tree,
the procedure stops.
 The path from the root to the last added node is
a postamble from the given state.
 Maybe no postamble from the given state.

27. S0
S1
2
S0
S2 S3
2 2
S0 S0 Preamble = {ε, 2, 2, 2 }

28. 3.3 Transition Cover
 The transition cover can be obtained by appending
each outgoing transition from this state to its preamble.
Preamble = {ε, 1, 13, 135 }
 S0:ε::1, ε::4, ε::5
 S1: 1::1, 1::2, 1::3, 1::4, 1::5
 S2: 13::1, 13::2, 13::4, 13::5
 S3: 135::1, 135::2, 135::4, 135::5
 Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15, 131, 132,
134, 135, 1351, 1352, 1354, 1355}

29. 3.4 State Identification Sequences.
 The characterization set W is a set of input
sequence.
 Harmonized state identification sets (HSI sets),
subsets of W.
 {D0, D1, …, Dn-1}
 Di is a set of prefixes of sequences in W
 n is the number of states of S.

30.  For any two distinguishable states Si and Sj of S, there
exists a sequence σ that is a prefix of both σi ∈ Di and
σj ∈ Dj such that σ can be accepted by these two states
and produces different outputs.

31.  How is Optimal ?
1. The number of sequences in W is minimal.
2. The sum of their lengths is minimal.
 To obtain a minimal characterization set for a
give FSM maybe an NP-Hard problem.

32. A heuristic solution
A1: A set P0 of all the state pairs that are
distinguishable for the given FSM is produced.
A2: With P0, a characterization set W is obtained
by forming a search tree from the input
alphabet to distinguish the state pairs in P0.
A3: From this W, some HSI set Di is selected for
each state i, such that the number of
distinguishable state pairs and the length of a
sequence are traded off.

33. The result of INRES responder
 The result P0 of the algorithm A1 is the set of all
possible state pairs.
 The characterization set W from the algorithm
A2 is {41}
 The HSI sets {{41}, {41}, {4}, {4}}

34. The algorithm A1
 The algorithm A1 is obtained by adapting the
FSM minimization algorithm give in
[ G. J. Holzmann. Design and Validation of
Computer Protocols, 1991 ]

35. The algorithm A2
 Forms the root as the current node to probe
 For each input word, a son is built if it can lead
to a better solution than other build nodes.
 If it is estimated that a subtree with a minimal
number of branches and with a minimal average
length that distinguishes a maximal number of
state pairs could be formed by probing an
unprobed son of ti.

36. The search tree in A2

37. The algorithm A3
 A sequence σk in W is selected such that
weighted sum of its length and the number of
the state pairs that it can not distinguish in the
left state pairs is minimal.
 For each remaining state pair (l, m) that can be
distinguished by σk, find a prefix of σk such that
(l, m) is distinguished.
 Then the prefix is put into the HSI sets Dl and
Dm.

38. W = {σ0, σ1,… σn-1}
σk :
Remain state pairs
σ:
distinguish ?
(l, m)
HSI Sets: {D0, D1, D2, .. Dl, …, Dm, …Dn-1}

39. 1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

40. 4. Experiments and Applications
State num. 10 50 100 150 200
Transition num. 100 2500 10000 23500 40000
CPU time (sec.) 0.11 2.96 40.87 197.06 760.89

41. 1. Introduction
2. Using the tool
1. An Example
2. Functions provided by the Tool
3. The FSM specification
3. Test derivation methods
1. Preambles
2. Postambles
3. Transition Cover
4. State Identification Sequences
4. Experiments and Applications
5. Conclusion

42. 5. Conclusion
 Proposed a heuristic solution to derive near-
optimal harmonized state identification sets.