it is about test automation methodologies which you should know before starting to automate your test cases.

1. Test Automation
What is test automation
Test automation is an activity that aims to test the most frequently used and least frequently
changed parts of an application automatically. This can be done by test automation tools by the help
of programming languages supported. Therefore, related test cases are written as a piece of code.
Thought test automation activity is criticized under testing activity, it is a software development
activity. Every change in the application should be reflected to the test automation scripts so, even
the cleverest test scripts are produced, and maintenance activity comes out as problem of test
automation since the application is a live object.
Test automation has an important role in software quality assurance. Because the needs of
the sector and trend towards to test automation is getting high, there are a large number of test
automation tools exist in the market, but the most famous tools can be listed as follows and more
can be found in the list below :
• QTP: Supported by HP, originally develop by Mercury and imported by HP. It has
many features, easily record and replay, fast and easily development, short learning
curve and powerful support by HP, however for only Windows and costly license
price.
• IBM Rational Robot: Developed by IBM, short learning curve for testing team and
gives options for more sophisticated scripts for test automation engineer, supports
most of the UI platforms, however for only Windows and costly license price.
• Selenium: Open source UI test automation framework, best of the selenium is that
you can write test cases with your favorite language for webpage testing. Although
Selenium is a Firefox plug-in, it can be used for any web browsers. There are lots of
blogs, websites and forum to gather help about new features.

2. Important points for test automation
Test automation is a live object this means that every change made for the item in the scope
of test automation should affect the test automation scripts. Keeping updated the scripts is crucial.
Therefore, test automation should be started after the test cases are mature and should be apply to
suitable test cases for test automation. This can reduce reworks and unnecessary scripting efforts.
Automation scripts should not cause a fail in the system since the aim of the test is to find
defects in the software in such a case finding the root of a failure is not feasible. Also in such
condition which the automation scripts find a bug or error, it should recover itself then reports the
issue and never stops until the full scripts runs. This can be explain by “transition handling” which
means that test automation is composed of many separate test cases and these test cases run in a
desired/fixed order one by one. Every test case starts then performs its job and then ends then the
following test case does its job but the starting and the end points for all the test cases are the same.
When a failure is catch the test case doesn’t do its job but goes to the end point which is the starting
point of the following test case. This can be illustrated by flower leaf model as shown below:
Another import point is that spending effort for the automation is beneficial or not. The idea
behind the automation is to perform many things on terminals without using human efforts instead
the human can perform test analysts or other test activities. However maintenance effort should be
eliminated because as we said automation is a live object. There should take a statistical study on the
aim of the automation and the gains after a successful automation is applied to the project. This is
named as automation’s return of investment (ROI), a ration of benefits to cost. A mathematical

3. formula can be written as and more about the calculation of ROI can be found here
http://www.dijohn-ic.com/test_automation_roi.pdf:
ROI should be greater than 100% otherwise test automation is not better than manual test
execution. ROI is calculated for only one execution of the test set but the idea on the automation is
reusing it for every iteration so if you have a frequently release candidate, you have to test your
application before release this means that you should run the automation more than one. If you run
the automation for 3 times effective ROI can be written as:
Graph shows the ROI with execution times:
What are the test automation methodologies
Since an aim to automate everything for every condition is a utopia, focusing on an
applicable automation target should be more realistic. It can be described by SMART which stands
for test automation should be Specific, Measurable, Achievable, Realistic, and Timely. For example,
40% of regression test cases with previously defined 10 users on 3 popular web browsers for an e-
commerce firm could be applicable test automation strategy. After you have decided a strategy, you
can apply a test methodology to practice your test automation. Test methodologies can be described
as follows:
• Record and Replay:
o Records every steps to create a test case
o Easily replay the existing test case
o It doesn’t have loop or if condition or other complicated features
• Functional Test Automation:

4. o Every test case is automated by a different test scripts
o Test scripts include test data
o No import/export data option
• Data-Driven Test Automation:
o In addition to functional test automation
o Test scripts can run by different data
o Data can be stored/read from an external storage/file
o Only data-sets are updated not the scripts
• Keyword-Driven Test Automation:
o Every case named by a keyword
o The important keywords are sequentially run as a separate test cases
o New test cases can be easily created not need a programming knowledge
o Data and scripts are separately located
o Error handling and Transition handling are difficult
o Creating complex test cases is difficult job
• Hybrid Test Automation:
o Combination of keyword-driven and data-driven automation techniques
o Exception handling and transition handling mechanism can be used
o Desired data can be used for different test cases
o Data and scripts are separately located
Example for keyword-driven testing, important keywords are sequentially run as separate test case
sets. Sample keywords:
open the the webpage named as open;
login the user information named as login;
check the product;
add to basket;
checkout the basket;
logout from the webpage;
close the browser;
Belonging to the keywords above some test case sets may be written as “open + login + logout”,
“open + login + check + logout” and so on.
Mesut Güneş – 22.02.2013
gunesmes@yahoo.com