Evolution of Test Automation

EVOLUTION OF
TEST AUTOMATION
A Cygnet Infotech Whitepaper

As technology gets more complex, quality expectations get more
exigent and development iterations get shorter. Software testing within
a limited budget and under time pressure has become a huge challenge.
Testing automation is the strategic investment for businesses to lower
costs while reducing the efforts to maintain the quality and time to
market.
Conventional automation testing is based on “scripting”, which
requires the testing team to have knowledge about native scripting
languages supported by automation tools. However, an extensive
usage and adoption of agile development has imposed many
challenges on the automation team. Issues like unclear project scope,
multiple iterations, sketchy documentation, early and frequent
automation needs and proactive stakeholder involvement has made it
more tedious in creating and maintaining automation scripts in sync
with the application under test. This has resulted into inadequate
testing and/or delayed releases.
Considering the current trends of application development
methodologies and delivery requirements, test automation using the
automation methodologies has raised several questions:
In addition to these questions, a few technical challenges like high
maintenance and technical skillset requirements for maintaining the test
scripts developed, need sensible answers.
Organizations are still struggling with their test automation initiative
either due to budget issues, poor skillset or lack of a suitable
automation strategy, says World Quality Report published by
Capgemini Group (Capgemini & Sogeti) and HP Software. A few of the
key discoveries of this report include:
 Almost 13% of the organizations surveyed indicate that they don’t
have dedicated budget for quality assurance team and they are still
struggling with estimating their testing efforts
 25% of the organizations surveyed indicate that their dedicated QA
teams run almost 81% to 100% of the tests manually
“Is it really practical and/or possible to use automated testing
in agile methodologies?”
“Is it possible to use traditional tools?”
“Should I opt for open source tools?
“What are the challenges in implementing test automation in
an agile environment?”
INTRODUCTION
A Cygnet Infotech Whitepaper: Code Less Test Automation

According to the “Risk Investment Test Automation?” by Konrad
Schlude in December 2008 edition of “Testing Experience”, almost
63% of the test automations fail to deliver the promises made by
automation. Such reports and failures have led the organizations to
search for more powerful and user-friendly alternatives of automation
testing.
Enter Code less Automation Testing! It is an innovative approach that
helps users automate their tests without even writing code or scripts
using programming languages. It addresses all the major issues of
current automation testing industry while improving the productivity &
reducing the costs. In this whitepaper, we will discuss about the
evolution of test automation, what Code less Test Automation is and
how it addresses the major issues discussed above.
EVOLUTION OF TEST AUTOMATION
The test automation process is a succession of generations, which have
become extensively advanced and sophisticated over a time. In order
to understand the advanced test automation, it is important to
understand the evolution of test automation approaches. The table
below showcases the evolution of test automation approaches:
Record &
Playback
Use/Reuse
of Functions
Data Driven
Scripts
Action
Keyword
Scripts
Code less
Automation
First
Generation
Second
Generation
Third
Generation
Fourth
Generation
Fifth
Generation
Figure 1: Evolution of
Test Automation Tools

The first generation automation tools could run unattended for a few
hours each and every day, which was the major benefit. Moreover, it
made automation of regression tests feasible. But unfortunately it was
deceitful. The scripts were inherently unstable: any minor change or
error in the application or the data caused the automation to break.
Even the playback timing was not as reliable as the automation runs
became quite slower with time.
Moreover, it was almost impossible to maintain the scripts as they
were not documented well. For example, the record script included
both the steps and the data embedded within the test, so that if a
hundred accounts were added as a part of the test, it led to the same
step being repeated hundreds of times. After that when changes or
alterations were made to the application, they too had to be repeated
hundreds of times. Look at the example below.
Some of these first generation tools were built as DOS batch files or
UNIX shell scripts.
FIRST GENERATION TEST AUTOMATION
Record and Playback
Figure 2: Record &
Playback Code

The second generation of the test automation tools brought a breath
of fresh air by bringing dozens of new features to the test tools. These
brought up similarities between the test tool IDE and developer’s IDE.
These tools offered a fully-grown scripting or even some support for
object oriented languages for the automation professionals. There were
tons of APIs available to simplify the very common window based
tasks like application launch, generating elementary test reports, file
operations and so on.
However, the script based approach still required huge investments in
terms of the amount of time spent in learning and gaining expertise in
the scripting language and the time required to build a suitable
framework that not only embraces the resource code but also simplifies
the data driving scripts.
This made automated testing similar to a software development
project as it demanded a whole new test process to create a new script.
It took a considerable amount of technical skills and support.
Moreover, managing the ever-rising number of automation scripts
became a great challenge to cope up with.
The Selenium IDE is a good example of a second generation tool.
SECOND GENERATION TEST AUTOMATION
Use / Reuse of
Functions
Compared with the earlier approaches, Data Driven frameworks
address the two major pain points: Test Coverage and Maintainability.
In this simple but effective approach, automation scripts were
separated from the input and output data from the software or
application under test.
In this approach, the test data is created separately and the test library
is comprised of the test scripts. Capture replay may or may not be used
as an input to the script-generating process but more important now
are the driver scripts.
THIRD GENERATION TEST AUTOMATION
Separation of Data &
Scripts

Figure 3: Separation of
Script and Test Data
Companies have to hire programmers or form automation teams who
were quite comfortable doing the programming for the driver scripts.
This meant externalizing the data into spreadsheets or text files and
adding logic to process the data, tackle timing synchronization, error
handling, recovery and result logging and reporting.
Each and every script had to be programmed and maintained by the
test specialists but could be used frequently with various kinds of data
sets to expand the test coverage. But testing is not all about just
entering data. It’s about simulating real life business scenarios to test
the application under test rigorously.
While this approach succeeded in empowering analysts to input test
data and reduce the length of code, it increased the complexity of the
code that needed to be developed and maintained. In the end,
companies realized that the test automation code is much like any
application software code: it is painful if not impossible for one
programmer to maintain the code written by another. Therefore,
scripted code had to be modified not only when the application
changed, but often rewritten when there was staff turnover.

The fourth generation further breaks down the test automation
approach in an advanced, structured and elegant approach. This
approach has reduced the cost and time of test design, automation and
execution by empowering an entire testing team to focus on what they
do best! Smart frameworks are used in this scenario to overcome the
inherent scripting issues of earlier generation tools by replacing
scripting for the major part. In addition, the maintenance required
because of the revisions in the test is greatly reduced.
Below is one example of a key/action word test, where the Test Case
column contains a list of keywords and the following columns provide
the data:
Figure 4: Keyword /
Action Driven Automation
FOURTH GENERATION TEST AUTOMATION
Action Keyword
Scripts
It is not the script now that directs the testing but it is the test itself. The
test data using keywords provides a sequence for the actions that need
to be carried out. When the automated test case runs, it will go
through the test data and call for related scripts specified by the specific
keyword. With this approach, the testers have total control over what
to do and in which manner. But the development of automation code
is still specific to the application under test.
In the example above, “add product” and “check product” are the
names of scripts that contain the detailed level code for adding a
product to the inventory application that is being tested.

Below is the example of how that key or action word might be
expressed:
Key/action word frameworks succeeded in enabling non-programmers
to design and execute automated test cases more easily. But it still
required some amount of custom code. Moreover, as the rate and the
amount at which the UI of the application under test undergoes
change, it will determine the increase in the maintenance of the scripts.
Thus, for large test automation projects, the need for finding ways to
reduce maintenance of the scripts has increased.
Even though the automation feasibility issues get resolved with more
robust object recognition technologies, the issue of poor ROI due to
the productivity problem is still a major headache. Some other
examples in this generation of tools include Selenium Web Driver and
HP QTP.
Figure 5: Script for the
Keyword Action
A Revolutionary Leap
Automation professionals have started looking for better and reliable
solutions to address all the issues in the previous generations of
automation approaches. Meanwhile, some automation experts have
noticed that Microsoft Visual Studio and Foundation classes have
brought a huge shift in productivity with which Windows GUI could be
created-instead of manual coding with the Win32 API. Developers can
now use the wizard to “Visually design” the GUI in a “studio”.
That’s where the concept of CODE LESS test automation came into
existence. Instead of conventional code editors, automation experts
now work in a graphical environment where they can easily compose
automated tests by choosing objects and actions from the drop down
menus. They can even visually create conditions, iterations and tons of
other stuff on the go! Let’s see what Code less test automation is all
about!

With the rapidly emerging trends in software quality and testing,
condition of the testing services providers, budgets and their
relationship with the business, many organizations are on the move to
modernize or modify their thought process. Code less test automation
is currently a “work in progress” in majority of the firms as most of
them are transitioning from the record and playback approach to the
Code less approach!
CODE LESS TEST AUTOMATION
automation is an approach to build
a test automation engine by
testing team to and
easily build automated test cases by simply
ordering but code assets to
make sure 100% test coverage is achieved
while increasing the test automation ROI.
Let’s start digging the statement...
Code less means no scripting/programming within the test automation
tool’s native language. But this doesn’t mean that there are no scripts
at all! It is inconceivable to think of automation without any piece of
software. Explicitly, this means that while automating the test cases,
you won’t need to program scripts for each and every test case. Now,
each and every stakeholder in the testing ecosystem can understand
and contribute towards the surety of the application under test.
Well-optimized means optimizing the existing test automation assets
like testers, automation tools, infrastructure and much more to
enhance the productivity. It involves operation optimization, financial
optimization and strategic optimization, which at the end maximizes
the test automation tool throughput and boosts up quality in sync with
the production.
Empowering means to elevate each and every stakeholder within the
testing ecosystem. Code less test automation is a layer that rides on the
top of the test automation tools to hide the complexities associated
with scripting from the stakeholders. It has become very easy for the
QC team to collaborate efficiently in order to build quality in the
product.

Quickly means to automate more number of test cases per unit of time.
With the Code less approach, automating test cases is like writing test
cases in MS Excel. No matter which test automation tool you are using,
with Code less approach, your testing team can easily and effectively
automate almost any kind of test cases up to 5 times faster than before.
Ready means nuggets of scripts already incorporated in the system,
which resides on the top of the test automation tool.
Reusable means it is possible to reuse the test specific components like
object repository and application components across the test
automation tools and technology.
Test automation has always been an attractive proposition as it reduces
dull and repetitive testing efforts and improves regression testing. But
whenever the product does change with each new release, the
automated test suit should also change, which is completely a waste of
time and money spent on creating the previous test automation.
Code less Test automation is going to provide what it promises:
reduced repetitive testing efforts and improved regression testing.
Going Code less will ensure that you leverage benefits from test
automation much faster than the script-based test automation. Here are
some solid benefits of code less automation.
Shorten the time to automate by almost 80%
The faster you automate, the faster your product will go to market.
With Code less test automation approach; your testing team can
automate giant test suites almost 5 times faster than before, which
means time to automate reduces almost by 80%. And this will
safeguard the true value of the product that undergoes alteration.
Visibility into test automation for the domain champions
It is one of the biggest benefits of going code less. Up till now, business
analysts and subject matter experts were unable to participate in the
test automation process as the coding world was like French for them.
But with this approach, they can easily establish the working of the
entire test automation suite or even an individual test case to make
sure the product is tested for its quality – not only for one or two
times, but with each new release or major update.
GO FOR CODE LESS AUTOMATION

Enable the testing team to do what they can do best
Going Code less will save loads of time of your testing team in creating
test automation. As there is no coding required, they can now spend
their time in achieving highest test coverage, which ensures that your
testing team can identify bugs at the very beginning of the product
development and help you deliver a high quality product at the speed
that you have never seen before.
Helps you save money on developing in-house scripting skills
Yes, that is now possible. Now no need to go for those lengthy and
boring training sessions on how to use the test automation tools! Code
less test automation will make sure that creating test automation is
only a matter of a few clicks.
While conventional test automation makes
you work hard, Code less test automation
will work hard for you!
So, if you really wish to revolutionize the test automation experience
within your existing testing team, then going Code less is one of the
best and only ways.
BENEFITS OF CODE LESS AUTOMATION
No more resource specific coding
Code less test automation makes the test automation tool almost
invisible by acting as mediator between the user and automation tool.
The end user can now create automation using the English language.
The Code less framework compiles this into the automation tool
compatible language. It frees the automation tool technology but
collaborates with it seamlessly.
Less time needed to automate
Going Code less empowers users with the ability to translate test cases
into test scripts, and that also in plain English language. This eliminates
the need of any kind of complicated programming and thus, you can
easily automate test cases faster with a significant time saving!
Rigorously tested and examined end product
As Code less framework compiles automated test cases into what the
test tool will understand on run time, there is no requirement to hard
code test cases to achieve the desired success.

Optimized investment
As Code less framework takes the responsibility to create automation,
the testers can now concentrate on increasing throughput – showcasing
the regression testing. The total number of regression cycles that earlier
automation tools were doing can now be achieved by less number of
testers. Even a manual tester can carry out automation, which reduces
the need for experts.
Zero complexity and reduced maintenance
Code less will help you cope up with the changes easily without having
a need to depend on any resources. It will also provide guidance on
critical dependencies of the change throughout the automation life
cycle, which at the end supports your decision at every stage to
improve the quality of your automation.
Highly automated code
“Creating automation that is not specific to the application under test
but UI object class specific” – this is the philosophy on which Code less
approach is built. Therefore, for an application with 500 textboxes in
its UI all actions are performed by a single script that is specific to the
object class of the test box. The more number of times that script is
used, the more it will tested.
More strategic bandwidth in hand
With Code less approach, you can easily:
 Create and architect test cases that can easily cope up with the
changes in application, complexity and technology change
 Spare more time in interpreting automation execution results
 Draw test cases towards the specific areas of change where there are
maximum chances to find bugs
 Utilize time to create a test suite to achieve 100% test coverage with
each successive release of the product
 Get more time to analyze and identify which is a more suitable and
better automation tool based on their features, reliability and
capability to tackle large test suites
In addition, all the components of a test automation script - the steps
in each keyword, the sequence of keywords, data per step, UI
definition and a lot more other requirements are available in a single
and easy to use interface – and that too without any programming.
Code less test automation approach enables updating the backend
automation engine on a regular basis by the Automation engineer to
support new operations or technologies taking place in the industry.

Some of the key features of a Code less framework, which aid in
automation creation are:
Moreover, such frameworks have a client-server based approach where
automation once created is readily available to a whole team.
Code less framework also provides great features for test execution or
re-execution, some of which are
ROI from Code less framework is also mind-blowing as it is productive
virtually from day one, as you don’t have to spend time at the
beginning to write the framework.

One example of a code less test automation tool is shown below. This
tool, TestingWhiz, captures the web object details during the recording
process and then automatically writes the back-end code. With an
intuitive test editor, even a novice tester can modify the scripts as the
application under test undergoes changes.

From the above discussion, it is clear that Code less automation has the
potential to be the next generation automation technology. It not only
addresses test automation’s productivity issue, but also does it in a way
that never forces you differentiate manual tester and automation
professional. However, it is also necessary to keep in mind that not all
the Code less framework or tools are truly Codeless. They may
provide many features or all the features in a limited manner. It is best
to evaluate the tool to find out whether it suits your test automation
needs.
CLOSING REMARKS
Cygnet Infotech is a global IT services and IT solutions provider with
clients in 24 countries. Software testing and QA is our forte and we
have provided various functional testing, performance testing and
automated testing services to clients in various industries.
Talk to us about QA, Testing and Test Automation
Tel: (India) +91-79-30487400 | (USA) +1-646-915-0021
Email: info@cygnet-infotech.com
Web: www.cygnet-infotech.com
All images in this whitepaper have been used for illustrativepurposes. Cygnet does not claim rights to any image except the logo.
ABOUT CYGNET INFOTECH

Evolution of Test Automation

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Evolution of Test Automation

Similaire à Evolution of Test Automation (20)

Dernier

Dernier (20)

Evolution of Test Automation