Test Automation is a strategic necessity, as the delivery cycles are shrinking and becoming repetitive. Also coupled is the need for ensuring success within Limited Budget and Stringent Deadlines. We would like to hear your comments on our Open source based Fitnesse, Watir and Ruby Based Test Automation System. sachink@harbingergroup.com

1. HSTW-3a01
A Harbinger Systems White Paper
Fitnesse, Watir and Ruby Based
Test Automation System

2. Copyright © Harbinger Systems. All Rights Reserved Page 2 of 8
www.harbinger-systems.com
Contents
Executive Summary.............................................................................. 3
Current Test Automation Technologies................................................ 3
Open Source Technologies..................................................................................................... 3
Proposed Test Automation System ...................................................... 4
System Architecture............................................................................................................... 5
Test Execution........................................................................................................................ 6
Compatibility.......................................................................................................................... 7
Benefits ............................................................................................... 8
Harbinger Systems
Your partner in technology innovation
About Harbinger Systems: Harbinger Systems (www.harbinger-systems.com) is a leading
provider of software engineering services to some of the world's best product companies.
This white paper is part of Harbinger’s Software Product Development practice. Harbinger
provides software product development and testing services to service the needs of
enterprise businesses as well as consumer products. Visit our Software Product
Development Practice page to learn more.
Visit our website to download or request our white papers on leading edge technologies and
trends.

3. Copyright © Harbinger Systems. All Rights Reserved Page 3 of 8
www.harbinger-systems.com
Executive Summary
Test Automation is a strategic necessity, as the delivery cycles are shrinking and becoming repetitive. In
this white paper, we propose and illustrate a complete test automation solution based on open source
technologies, Fitnesse, Ruby and Watir. This system is web based, and enables a diverse set of project
stakeholders to carry out automated testing from anywhere.
After a brief introduction to the available options in open source test automation technologies, we will
look at the building blocks of the solution, and finally the details of the proposed test automation
system and its application to the testing of Web based applications.
Current Test Automation Technologies
The current automation solutions are available as both proprietary and open source tools. While it is
generally thought that an end-to-end test automation can be achieved using proprietary tools like QTP,
Rational Robot etc. very easily, as the application undergoes revisions over time, many of these systems
begin to fall short on their promise. For one thing, maintenance of the test scripts becomes expensive.
Added to that, there are issues with them like licensing restrictions, support for scripting language,
pricing and dependencies for add-ons. Sooner or later, due to these restrictions, the expensive
automation ware rests on the shelf.
Open Source Technologies
Test Automation has been the buzzword in the open source forums. The Agile Development
methodologies insist on automated testing as a crucial component in the success of projects. This has
led to the development of several test frameworks for different stages of software development life
cycle. For the sake of this discussion, these frameworks can be broadly categorized into two categories:
Unit Test Frameworks and Systems Testing Frameworks.
Unit Test Frameworks
The most popular unit testing frameworks are JUnit or NUnit based on the application technology in
question. These frameworks are robust and they provide an engineering view of software testing.
The primary drawback with these frameworks is that these are too technical to be used by all project
stakeholders. Moreover, these frameworks perform white box testing over the classes in codebase, and
hence, do not provide sufficient information for the system level tests and application behavior.

4. Copyright © Harbinger Systems. All Rights Reserved Page 4 of 8
www.harbinger-systems.com
System Testing Frameworks
Selenium has traditionally been a well known open source framework for system testing. Selenium
executes tests within the browser. The selenium IDE also provides easy access to test cases.
In Selenium, however, JavaScript’s same origin policy is seen as the show stopper. To deal with this
problem, the framework was enhanced with a proxy server (Selenium RC) where the automation test
suite is to be deployed. The browser, where actual tests are being run, will then point to the proxy
server which will redirect the request to application server. While this configuration works, there are
two main issues:
The configuration is not exactly the same as the actual test environment in production.
The overall installation and deployment of this configuration is very heavy and time consuming.
Proposed Test Automation System
Harbinger Systems’ proposed test automation system is built on the following building blocks.
Fitnesse
An open source framework called Fit is used for integration testing. Fitnesse is a Wiki wrapper over the
Fit server, and thus provides a web interface to the test suite. The front end of Fitnesse is intuitive and
simple, and it enables customers, testers and programmers to compare what their software actually
does against what it should do. Very importantly, Fitnesse is thus seen as a software development
collaboration tool since all the stakeholders can collaboratively define the acceptance tests, run those
and see the results.
Test cases in Fitnesse are written in Excel-like tables on wiki pages. Fitnesse provides different types of
fixture libraries to perform data-driven or action-driven testing. It must be noted, however, that Fitnesse
was written to test the application code and not application UI. Hence, it requires interfacing code,
which can call the functions of the application testing class.
Ruby
Ruby is an open source object oriented programming language with a focus on simplicity and
productivity. It doesn’t run very fast, but it saves a lot of programming time. Further, Ruby code is very
concise and readable and is much simpler to maintain. These attributes make it a good programming
language to write the automation scripts in.

5. Copyright © Harbinger Systems. All Rights Reserved Page 5 of 8
www.harbinger-systems.com
Watir
Watir, pronounced water, stands for Web Application Testing in Ruby, and is an open source library of
functions used to automate browser based tests. Watir provides programming API for Ruby to drive
browser actions. While Watir traditionally supported Internet Explorer, it now also supports Firefox and
Safari, and enjoys a strong community support.
System Architecture
Figure 1: Schematic diagram of Harbinger's Proposed Test Automation System
The system is architected such that all tests pertaining to a module are packaged and saved in separate
folders. All testing related decisions are made by a class called Test Engine, which is the heart of the

6. Copyright © Harbinger Systems. All Rights Reserved Page 6 of 8
www.harbinger-systems.com
system, and is responsible for maintaining the context of testing. It is also a completely open source
solution, and hence, does not have licensing restrictions from its subcomponents.
In the proposed system, all the common navigational functionalities are abstracted in a separate class.
This is then sub-classed into module specific exploratory test functions. All the test wares are developed
in Ruby. A schematic diagram of this system is shown in Figure 1.
All artifacts of the automation system are deployed on the automation server machine. The system is
carefully architected after a long research on abstracting and organizing the test procedures into various
components. These components and their responsibilities are as follows:
Fixtures:
o Parses test data from Test cases in Wiki pages and to fetch them into test engine
o Compares the outcome of test execution with the expected behavior and to mark the test case
as pass or fail
Test Engine: Test Engine is the intelligent class of the system, which makes decisions during testing,
and keeps the test context of the system persistent over the next test case / scenario
Helper classes: Helper classes help the Test Engine in exploring the application in conformance to
the test execution steps described in the test case
Test Execution
The test automation server can be accessed from a browser, and the user can initiate testing just by
clicking the test button in the test scenario page of the wiki. Fitnesse would start executing tests as per
the procedure described in test cases and will report the outcome in the same test tables as shown in
Figure 2.

7. Copyright © Harbinger Systems. All Rights Reserved Page 7 of 8
www.harbinger-systems.com
Figure 2: Outcome of test execution
Table cells filled with green indicate success. When the test fails, the fixture class finds that
the actual state of the application is not the same as expected. The corresponding cell in the
test case is then marked in Red. The actual state of application is also shown in the same cell.
Compatibility
The testing system has been successfully tested on Web 2.0 applications with the following advanced
attributes:
Web elements in nested framesets
Ajax controls
Element identifiers created at runtime by Ext-JS

8. Copyright © Harbinger Systems. All Rights Reserved Page 8 of 8
www.harbinger-systems.com
Benefits
The proposed Test Automation System yields the following advantages:
Project stakeholders can access the test automation server via web
Test outcome clearly shows what the application is expected to do and what it actually does. Thus,
the system is an ideal collaboration tool for teams located geographically apart
As testing becomes very simple and can be executed by any team member, this system creates
ground for agile development cycles
Open source based automation solution keeps the system free from licensing restrictions
Modular design of test wares that is in conformance with modules of system testing designed by
manual test teams
Decision making business function classes have been isolated from the navigational details in
separate classes
The Test Engine component keeps context of test execution, and hence, can take appropriate
decisions in the next test scenario execution cycle
Learn More: Visit our website, http://www.harbinger-systems.com to download more white papers on
leading edge technologies and trends.
Sources
1 The Fitnesse Acceptance Testing Framework: http://fitnesse.org/
2 Using Fitnesse pages effectively: http://gojko.net/2008/10/21/using-fitnesse-pages-as-templates/
3 Architectures of Test Automation Systems - Cem Kaner: http://www.kaner.com/testarch.html
4 Getting Automated Testing Under Control – Hans Buwalda:
http://www.logigear.com/logi_media_dir/Documents/whitepapers/Getting_Automated_Testing_Under_Control.pdf
5 Watir: http://watir.com/
6 Using Watir for Test Automation: http://svn.openqa.org/fisheye/browse/~raw,r=267/watir/trunk/watir/doc/watir_user_guide.html
7 Firewatir – The Support of Watir for Firefox: http://angrez.blogspot.com/search/label/Firewatir
8 On Ruby Programming Language: http://www.ruby-lang.org/en/