An introduction to test scripting targeted at those who have not done this before. Tooling assumes MS Excel.

1. 1
Ian McDonald
Implementing Test Scripting
© 2006, 2007, 2010, 2013 Ian McDonald

2. 2
Contents
Purpose
Overview
Test Cases
Manual Test Scripts
Automated Test Scripts

3. 3
Purpose
 These slides were put in place over a period of
time, as I found myself often having to lead
teams on fast delivery projects, where testers
had no previous experience or training. It is
very much a quick guide and assumes that the
slides are supported by a skilled practitioner
leading the team. The approach assumes
basic level tooling. Obviously with specialist
tooling the approach will be adapted.
 There is also an assumption that other training
is also available to support test analysis to
create test cases and the use of specific tools.

4. 4
Overview
 This presentation is written to help those who
are new to test scripting, who may have had
little previous experience in this task.
 The slides address:
 The layout and management of Test Cases.
 Naming requirements for Test Scripts.
 Limitations imposed by Test Tooling.
 Creation and content of Test Scripts.
 Structuring of Automated Test Scripts.

5. 5
Test Cases
A Test Case is described as:
 “A set of inputs, execution, preconditions, and
expected outcomes developed for a particular
objective, such as to exercise a particular program
path or to verify compliance with a specific
requirement.” BS7925-1
 Tools like Classification Tree Editor (CTE) allow users
to generate Test Cases in terms of lists identifying
input parameters (or Test Vector sets). NOTE: A
separate presentation is available that deals with CTE.
 The Test Cases will therefore form a list of what needs
to be tested, against a Test Case will be hooked in
one or more Test Scripts which provide step by step
instructions in how to implement the Test Case.

6. 6
Test Case and Test Script
 A test case defines the parameters that are
the vector values which are input for a specific
set of tests. Defined with the input values are
corresponding output values that are expected
for the test outcome.
 A test script details the step by step actions
and observations to implement the input of test
vectors and observations.
 Often a test case may be implemented in a
number of different scripts. This might occur,
for example, if there are different input
methods.

7. 7
Creation of Test Cases
 Test cases are identified through systematic analysis, to identify
the test vectors to be used for testing.
 Techniques such as boundary analysis, equivalence partitioning,
state transition testing, etc can be found in BS7925-2. These
techniques can be applied to system as well as unit testing.
Note: Details of BS7925-1 and BS7925-2 can be found at:
http://www.testingstandards.co.uk/
 An additional technique is classification tree (CT), which is also
supported by the classification tree editor (CTE) tool.
Note: The CT approach and the CTE tool are described in a
separate presentation.

8. 8
Test Case Work Packages
 Following test analysis a series of work packages are
identified. These are the individual test cases, or test
vector sets.
 These test cases then need to be developed into a
series of test scripts.
 The test scripts outline the starting assumptions and
preparation for implementing a test, followed by step
by step clear repeatable actions and the observations
to make and what is expected to be observed for a
successful outcome.
 Later a test manager will want to run specific scripts
associated with test cases.

9. 9
Structuring Test Cases – A Practical Approach
 Test Cases are structured into well organised folders
to allow efficient management. During a test
programme, the Test manager will need to:
 Assign test cases for the creation of test scripts.
 Run scripts for specific test case sets for different software
deliveries and project milestone phases.
 Assign specific work packages to testers (so a tester will be
given a defined set of tests to run within a period of time).
 Prioritise testing for a defect fix, software delivery or
confidence test.
 Consequently Test Cases and Scripts need to be well
organised and the organisation well understood and
easily recognised by the team.

10. 10
Early Organisation of Test Cases
 Test Cases are usually organised under a
series of folders. The organisation may differ
from project to project. In structuring Test
Cases or Scripts the following will need to be
considered as a possible workable approach
for organising Tests:
 Requirements
 Business Use Cases
 Functional Grouping
 Equipment Functionality
 Data Type
 Other….

11. 11
Getting Started
 When creating Test Cases, it may be that the
Test Cases have been created with tooling
such as CTE and the test cases have been
created as a flat table or series of flat tables.
The test manager will want to group these for
assigning to testers to create Test Scripts.
 It is often easier to initially assign the test
cases to high level folders and then as the
Test Scripts are created, the Test Cases are
moved to a more appropriate folder structure
grouping.

12. 12
Setting Out Test Cases
 Test Cases can
be stored
initially at a high
level and as
sorted then
transferred to
lower level
folders.
Portal
Server Agent
Project ABC
Admin
User Set-Up
System User Functions
Positive Test
Negative Test

13. 13
Considerations for Priority and Regression
 Typically during testing a test manager will need to do the
following, which can be helped by good Test Case structure:
 Assign specific sets of Test Cases to specialist testers.
 Prioritise the creation of Test Cases by Positive testing.
 Positive tests check that the normal functional path is working for normal
inputs.
 In contrast a negative test will provide an illegal entry and will thus test
the resilience and error handling capability of the functionality. Often
negative tests are more efficient at finding less obvious defects and
specifically those defects that may cause a major system failure.
 Prioritise Test Cases for regression test sets.
 Prioritise Test Cases for automation.
 Choose Test Cases to test a defect fix.
 Choose a small subset of Test Cases to decide if a build is fit for
accepting into testing – eliminating bad build injection into test.
 Choose a subset of Test Cases for reuse in Acceptance Testing.
Hence the reason for structure needs to be understood by the
team.

14. 14
Dealing with Test Phases
 It is often simpler to
create a library of
Test Scripts, where
these are maintained
centrally and then
copy down Test
Scripts to appropriate
level folders for the
test phase:
Project ABC
Integration
Build1
FAT
UAT
SAT
OAT
COPY
Test
Sub-
Folders
Specialist test
management tooling
allows results and
scripts to be managed
without the need for
copying. However
some tools are better
than others.

15. 15
Limitations on Folder Structure
 Be aware that some test tools and older
versions of well established tooling, have
limitations on path length.
 The total number of characters for the Test
Case name (path) and Test Script (file) are
limited.
 If this happens to be the case then appropriate
abbreviations will need to be used that are well
understood by the team.

16. 16
Agile
 If using an Agile approach, then you will need
to consider how you add Test Cases and map
these to Agile Stories and at the same time
allow your administration of Regression
Tests.
 Ensure that Test Cases for Performance and
Security are added early. The content will
improve over the life span of the test delivery,
however they need to be created early on in
the test cycle.

17. 17
Warning
 Take great care to maintain a sustainable test script
library.
 There is always a danger that with a poorly defined
system, tests are not maintained and new and even
repeated tests are added. This is dangerous for a
project because:
 Older tests will eventually break, causing further delays to test
runs.
 There is a greater maintenance cost as more tests get added.
 The time to run a suite of tests increases, adding significant
costs to testing.
 Make certain the test team understand the need for
maintenance and make sure that tests for specific
functions can be quickly identified.

18. 18
Test Script Overview
 Test Scripts have 3 key elements.:
 Step Number – providing the sequence for Action (which includes Observation).
 Action – A unique step in a sequence of Action or Observation.
 The Expected Outcome from the result, against which a Pass or Fail is measured.
 Note: When using traditional Test Management Tools, further columns are added for
Pass/Fail and Comments. However if making do with tools such as Excel, there is a
need to add columns for this and have a way of referencing any attachments – such as
screen captures. So you will need to add a column for comments, defect ID cross
reference and for any hyperlink to screen captures, which will need to be stored in a
structured directory.
 With an excel solution, there also needs to be an approach for making sure the issue of
the test script is maintained and the results of [past test runs are mapped to a specific
issue of the script.
 Test script maintenance can also be achieved by each Excel workbook having a master
blank script, which is copied within the file and used for completion during a specific
run. Try to avoid the need to edit multiple scripts, so work maintenance is minimised.
Step Action Expected Result
1

19. 19
Setting Out Test Actions #1
 Test Script Actions need to be set out in a
logical order of events.
 Step 1 – will detail any test assumptions. Do
consider that often a test run may at times
vary. So if a test is for example about logging
in with a user name of 3 characters, then state
this early. This then avoids the situation of a
busy test team using a standard assigned user
name with more characters. Or avoiding the
automated tester writing a replacement
automated test script which misses the point of
the test.

20. 20
Setting Out Test Actions #2
 In setting out steps, we are ensuring that we
can carry out tests in a precise and ordered
approach that is repeatable. The developer
should be able to repeat the test step and
obtain the same result and understand exactly
what the failure was.
 Test Actions also include the action of
Observation. So for specific observations
ensure that these are included as single steps.
This then guides the automated tester to
ensure that specific observations need to be
captured.

21. 21
Setting Out Test Actions #3
Finally when all the script test actions
are complete, there is a final step which
will either:
Set the system to a known state ready for
another test OR
Direct the user to run a cleanup script.

22. 22
Level of Detail Required
 There is a fine balance in ensuring that there is sufficient
information that can be used to understand what action needs to
be taken and the amount of information that needs to be
maintained.
 If testing is likely to utilise an army of untrained testers, then more
detail will be required within a test script. This has to be
considered early on. Depending on the company this could
include: undergraduates, new graduates with no or little previous
experience, users, administration staff, etc. So if this is likely
then the scripting needs to be addressed early on in development
of the scripts.
 If code is likely to change significantly then you do not want to
create a large number of test scripts with lots of fine detail that will
take longer to maintain than it takes to change the code. So in
this instance test scripts will be general in description and be less
prone to the need of continuous maintenance. However the key
purpose of the test case and the choice of vectors needs to be
stated clearly.

23. 23
Test Script Referencing #1
 In Test Management Tooling, Test Scripts are often contained in
a flat table with a file reference. These files are then hooked into
Test Cases. This approach has a specific advantage that if a test
case is copied and repeated several times, they will always
reference the same batch of test scripts, that can be maintained
centrally. Changing one test script will update all instances of the
Test Script.
 Do ensure that a test run results are not impacted by changes in
the working Test Script. Any change in a Test Script should not
impact any recorded running history, where the previous version
was used. Not all Test Tools are helpful in this instance. If this
causes a problem you will need to ensure that any configuration
control deployed compensates for the flaw in the tooling.
 The reason for this is that if you needed to re-run an old version
of a test to check a result, or if development roll back code to a
previous build, you need to ne able to reference the appropriate
version of the test script. If you do not have control of this, then
you will be heading for problems.

24. 24
Test Script Referencing #2
 In creating Test Scripts, the Script name will
need to reflect a number of criteria:
 Identify any specific order that a series of scripts
has to be run in.
 Identify any priority (High, Medium, Low) that is
required for automation and regression testing.
 Identify, where necessary any version control
reference.
 Identify the functionality being tested. This can
include reference to positive or negative testing.

25. 25
When to Automate
 Automation creates a level of administration which can at times
be unsustainable. So in automating ensure:
 Code has stabilised and is unlikely to need automated tests to be
updated.
 Do not rely on record and play, where possible use coding to alter
data inputs. This means that scripts are far more easily adaptable
and maintainable.
 Take care with bit map comparisons. Different graphics cards have
in the past given rise to sufficient differences to cause failure, where
a pass is correct. Modern GUI automation tools allow the threshold
for bit map comparison to be adjusted to compensate. Also take
care not to compare a whole screen as routine. Positions may vary
slightly and so if looking for example a logo, it is possible in some
tools to highlight the image that is to appear on the screen and will
pass the test if the image appears anywhere on the screen, within
specific set boundaries.

26. 26
Cutting Duplication in Test Scripting
 This applies to both automated and manual testing. If there is a set-up
sequence say of 12 steps, then if you have to repeat that for 100 test
scripts that is 1,200 steps that you need to maintain. This is very
expensive.
 Instead consider setting up a preliminary set up script (say called
“preliminary_set_up_test24v1”. This would then have those 12 steps.
Any test that needed this would then have a step that says “first run
preliminary_set_up_test24” (note no version, however the date and issue
control will allow anyone to identify the version run). Any automated
scripting would then run the preliminary test before the extension.
However any Test Script name would need to indicate that a preliminary
test needs to be run first. This is then easier to check a regression suite
for the inclusion of any preliminary test.
 For Manual Testing this is easily implemented by the tester running the
repeat step, with a print out. While in the single repeat step will have a
failure that will correspond to one of many steps in the preliminary
preparation script, the results should have the ability to detail in which
step of the preliminary script the failure occurred. However in practice if
one had may repeated failures in a preliminary script, one would raise a
defect against the preliminary script and abandon testing on any script
that relied on that preliminary test being carried out.