The document discusses automated testing of web applications. It presents different approaches to web testing including GUI automation, HTML automation, and DOM automation. Selenium RC is introduced as a testing platform that uses DOM automation. Selenium RC allows tests to be run programmatically across browsers and platforms. It can be extended with custom commands and locators to better suit specific applications like ones built with the JavaScript framework qooxdoo.
4. Web Testing: Goals and Challenges
Goals:
create tests
run automatically, repetitively
regression testing
compare results
5. Web Testing: Goals and Challenges
Challenges:
GUI app testing
OS/browser/platform
JS engines/versions/levels, plug-ins
6. Web Testing: Approaches
GUI Automation
locating by coordinates
using generic OS mouse/keyboard events
suitable for any GUI app
HTML Automation
parsing HTML
evaluating HTML elements (forms, ...)
“batch browsers”
DOM Automation
DOM as presentation of application
DOM traversal (getElementByName, ...)
interacting with DOM tree (target.dispatchEvent,...)
13. Web Testing: Approaches (Rev.)
GUI Automation
constant complexity
platform-dependent
fragile
HTML Automation
insufficient content capture
DOM Automation
DOM tree represents application
sufficient content capture
14. Web Testing: DOM Automation
full access to app elements
full capture of dynamic content
issue: locating elements (reliably)
issue: non-interference with AUT
you can embed test code with AUT
issue: reliability of results
(i)frames
cross-frame interaction
22. Selenium and qooxdoo
crucial issues:
object identification
event generation
qooxdoo rendering: a world of <div>s
Selenium commands
Selenium locators
the need for (and feasibility of) qooxdoo specific extensions
24. user-extensions. js: locators
PageBot.prototype.locateElementByQx = function(qxLocator, inDocument, inWindow)
{
LOG.info("Locate Element by qooxdoo UserData Locator”);
var qxObject = this._findQxObjectInWindow(qxLocator, inWindow);
if (qxObject) {
return qxObject.getElement();
}
};
25. user-extensions.jss:: language bindings
new custom extensions:
support for qx locator
just a string passed as parameter of command
no change to client driver API
support for qxClick command
must be supported in language bindings
so client drivers can use it
Sample:
qxClick(“qx:foo/bar/baz”)
26. user-extensions.js: language bindings
Python:
●
def qxClick(self, locator, opts=None):
return self.do_command("qxClick", [locator,]) sel.qxClick(r'//div[text()="Form"]')
●
Java
●
public String qxClick(String locator) {
return this.commandProcessor.doCommand("qxClick",
new String[] {locator,}); } sel.qxClick('qxh=*/[@label="Tab"]/[@page]/*/[@label="Fi nd"]');
●
JavaScript using Rhino
●
uses Java infrastructure, so no additional overhead
27. A Client Driver Script
importClass(Packages.com.thoughtworks.selenium.QxS elenium);
var sel = new QxSelenium("localhost",4444,"*firefox",
"http://demo.qooxdoo.org");
sel.setSpeed("1000"); sel.start();
sel.open("http:/ demo.qooxdoo.org/0.7.3/showcase/"); Packages.java.lang.Thread.sleep(2000);
sel.qxClick('//div[text()="Tab"]'); sel.qxClick('qxh=*/[@label="Tab"]/[@page]/*/[@label=
"Find"]'); sel.stop();
28. Conclusion
Interacting with the DOM seems to be the best way to test Web applications
Selenium RC provides a distributed
environment to run tests on a broad range of platforms
Various language bindings exist, so tests can be written as stand-alone,
client-server programms in a preferred language
Custom commands and locators can be
added to the Selenium environment, to suite specific testing needs
29. Selenium RC: Automated Testing of Modern
Web Applications
Thank you!
Thomas Herchenröder, 1&1 Internet AG
30. Selenium RC: Automated Testing of Modern Web Applications
Thomas Herchenröder, 1&1 Internet AG
This talk is concerned with automated testing of Web applications. It looks at testing Web apps in
general, its goals and challenges; it will present Selenium and Selenium RC in particular as a
testing platform; and will then focus on adaptions made to Selenium to ease the effort to test apps
made with qooxdoo, a JavaScript framework.
32. Overview
Testing Web Apps
Selenium RC
Extending Selenium RC
Part I: An Introduction to testing Web applications and its approaches.
33. Web Testing: Goals and Challenges
Goals:
create tests
run automatically, repetitively
regression testing
compare results
●When we speak of creating automated tests for Web applications (“application” in the broadest
sense – from a static HTML page through CGI-driven applications to full-blown single-page
apps), what do we mean by the term 'test'?!
●All test start by retrieving a specific web page.
●The simplest test might then just look at its static contents, inspecting the page and checking for
specific properties (e.g. the correct title, presence of keywords or phrases, etc.).
●More advanced tests will want to look at the dynamic aspects of the web page, and therefore
need some means to interact with it. This encompasses detecting and following hyperlinks,
automatically filling in and submitting forms, or use other interactive elements. In all cases, the
outcome of such interaction (new page, change of current page content, ...) has to be checked
against some expected result.
●Depending on these requirements, test methodology and effort can vary a lot.
●Once you have created a test in a suitable form, its purpose is to be
run repetitively and, hopefully, automatically, i.e. without user intervention. Results need to be
verified, and probably captured, to be compared wit subsequent test runs.
●This allows for regression testing and test-driven development, to ensure existing functionality is
not broken while the web app is further developed.
34. Web Testing: Goals and Challenges
Challenges:
GUI app testing
OS/browser/platform
JS engines/versions/levels, plug-ins
●Every tester loves APIs! A clear defined interface that can be driven programmatically and in a
defined and reproducible way. Results can be compared, collected, logged, standardized, or put in
a database.
The scope of testing is clearly defined (list of API calls), and though parameter ranges and call
protocols (sequencing of calls) make things more complex, there often is hope for test coverage
(getting all – important – test cases).
●GUI apps evade from such a close grip. Although they often have a number of distinct
interaction elements (buttons, boxes, fields, ...), they rely on event handling that in most cases
defies protocols and a systematic way of testing. Events can be triggered in arbitrary order, and
might – or might not – lead to the same results.
●Moreover, any GUI can rely on one of the most versatile interpreters
available – the human brain. Pictures with colored dots are recognized as numbers, technically
unrelated geometrical elements are recognized as text, context is maintained across pages or parts
of a page, across time and across any change happening to a page. With this kind of semantic
processing, correlations can be established and information can be provided that is very hard to
capture by any known automaton. How can 'correctness' or its absence be detected in such a
context?
●To worsen things, in the realm of web applications the tester is confronted with the 'plague of
platforms': multiple OSs, multiple desktops, multiple browsers, multiple configurations, multiple
... To devise tests in this scattered landscape is hard, to devise tests that are
maintainable and provide reliable results in all situations is very hard.
35. Web Testing: Approaches
GUI Automation
locating by coordinates
using generic OS mouse/keyboard events
suitable for any GUI app
HTML Automation
parsing HTML
evaluating HTML elements (forms, ...)
“batch browsers”
DOM Automation
DOM as presentation of application
DOM traversal (getElementByName, ...)
interacting with DOM tree (target.dispatchEvent,...)
●There are mainly three approaches to tackle the problem of web testing:
●GUI Automation:
●
uses the least common denominator of GUIs: generic mouse and keyboard events.
uses OS-level I/O event generation.
doesn't care whether it is Web browser, Chat client, File browser, MP3 player, etc.
but is entirely platform-dependent, since it relies on OS-level APIs.
result verification is hard (comparing bitmaps?!)
fragile: depends on correct coordinates for mouse actions, keyboard layouts, themes in place,
current key bindings, ...
●
●
●
●
●
36. A Simple Example: GUI Automation
Click to add text
●Locate input field, push search button - GUI automation would certainly work
●But: what about result verification?!
37. A Simple Example: HTML Automation
Click to add text
●fair reflection of the rendered page:
●
opening “body” tag
navigational links (Images, Maps, ...), with embedded menu (“more”)
“form” element with input field, buttons
further information links
●
●
●
●functional “meat” is present: input field, buttons
38. A Simple Example: DOM Automation
Click to add text
●Interesting elements: HTML->Body->Center->Form->...->TD:
●
Input: hidden
Input: text field for query string Input: submit button
Input: I'm feeling lucky button
●
●
●
●uses name attribute for input fields ('q' for text field, 'btnG' for submit etc.)
●makes it easy to locate (getElementByName(), Xpath, ...)
●Conclusion:
●
good “semantic” representation of the page and its interactive elements (not layout dependend!)
easy and robust to drive through DOM manipulations
●
39. A Complex Example: GUI Automation
Click to add text
●Complex GUI:
●
buttons, tabs, panes
multiple embedded “windows”, independently movable check-/radio-buttons, pop-ups
●
●
●Tip: Try imagining this in HTML!
●GUI Automation:
●
easy, doesn't care about complexity problematic: exact locating (influence of screen resolution?)
problematic: re-arranging of elements (e.g. tabs)
●
●
●
●(Next Slide: corresp. HTML - “Prepare for disappointment!”)
40. A Complex Example: HTML Automation
Click to add text
●HTML: That's all there is to it! :-(
●nothing to work with
●all HTML-processing programms will fail
●Conclusion: completely unusable
41. A Complex Example: DOM Automation
Click to add text
●again, full access to page elements
●harder to relate DOM elements to visual result
●
DOM tree as “assembler”
●problematic: locating elements (“flood of divs”)
●remedy: “id”, “name” etc. attributes
●remedy: Xpath queries for “robust” features (unique labels, images etc.)
●
still not comfortable, but works
42. Web Testing: Approaches (Rev.)
GUI Automation
constant complexity
platform-dependent
fragile
HTML Automation
insufficient content capture
DOM Automation
DOM tree represents application
sufficient content capture
●GUI Automation:
●
use the least common denominator of GUIs: generic mouse and keyboard events.
Use OS-level I/O event generation.
Doesn't care whether it is Web browser, Chat client, File browser, MP3 player, etc.
But entirely platform-dependent, since it relies on OS-level APIs. Result verification is hard
(comparing bitmaps?!)
Fragile: Depends on correct coordinates for mouse actions, keyboard layouts, themes in place,
current key bindings, ...
Impossible to write one test that runs the same on Win/Linux/OSX/... in IE/FF/Safari/...
●
●
●
●
●
●
●HTML Automation:
●
just not feasible for modern applications
●DOM Automation:
●
provides everything needed (more next slide)
●
43. Web Testing: DOM Automation
full access to app elements
full capture of dynamic content
issue: locating elements (reliably)
issue: non-interference with AUT
you can embed test code with AUT
issue: reliability of results
(i)frames
cross-frame interaction
Overview of some features of the DOM automation approach:
●everything in the app is accessible through its DOM representation
●even across dynamic changes
●finding the element to interact with can be a challenge, esp. across various revisions of the same
app
●non-interference: if testing code changes DOM, the app is no longer the same; danger of
introducing subtle changes in behaviour
●
test code could be part of the app (and this is actually used!) will AUT behaviour be the same as
when driven by a real user? one solution: separation of AUT and testing logic in different frames
issue here: cross-frame interaction
●
●
●
44. Overview
Testing Web Apps
Selenium RC
Extending Selenium RC
Part II: An Introduction to Selenium and Selenium RC in particular.
45. Selenium: selenium.openqa.org
Selenium Core
Selenium IDE
Selenium on Rails
Selenium RC
Selenium Grid
open source, OpenQA hosted, ThoughtWorks
●Selenium Core:
●
JavaScript engine runs in browser
controls AUT in other frame exposes API
●
●
●
●Selenium IDE:
●
FF plug-in
directly communicates with Selenium Core in the browser record/replay features
great to get started (interactively)
●
●
●
●Selenium RC:
●
remote control
server component, proxy
allows test automation in various ways this is what we will focus on
has a capability called Selenium Grid, to drive multiple instances of the SRC server
●
●
●
●
●project frame: open source (Apache 2.0 lic.), OpenQA run by collaboration of a few companies
46. Selenium RC: Architecture
●Selenium RC server (blue)
●
starts browser instance with modified profile promotes itself as proxy for all requests injects
Selenium Core with startup page
all requests go through server, dispatches on URL
●
●
●
●browser instance (orange)
●
initial page installs Core, multi-(i)frame
Core maintains connection with server (comet-like, long polling) receives commands and
performs them in browser (load URL in AUT frame, click, type, check results, ...)
●
●
●web server hosting the AUT (brown)
●
SRC server appears as normal client
●client drivers (green)
●
multiple language bindings (Java, Python, Perl, ...) completely control SRC server
●
●
starting of browser loading of AUT
any (test) interaction ending of browser session
●
●
●
47. Selenium RC
overcome SOP restriction
drive tests programmatically
language bindings (Java, Python, Perl,
...)
access to OS features (timers, threads, screen shots, ...)
user-extensions.js
●SOP: same-origin-policy:
●
JS can only make XHR requests to domain it was loaded from browser only sees a single domain:
http://<domain_of_aut>/selenium- server/core/RemoteRunner.html?...
SRC server dispatches on URL path
●
●
●
even handles all requests to http://<domain_of_aut>/<aut_path> (due to proxy)
control ends when requesting a page in another domain
●
●user-extensions.js
●
main topic of part III
●Hint: some more 'hands-on' material before
48. Selenium RC: The Browser
Click to add text
●default appearance
●three parts:
●
AUT frame
Log frame (command history) Selenium frame
●
●
●
“show log” button, for message log important for debug messages
●
●Hint: next: log of interactive session to bring this up
49. Selenium RC: An Interactive Session
Starting the server:
java -jar selenium-server.jar -interactive
Interactive commands:
cmd=getNewBrowserSession&1=*firefox&2=http://d emo.qooxdoo.org
cmd=open&1=http://demo.qooxdoo.org/current/sho
wcase
cmd=qxClick&1=//div[text()="Tab"]
cmd=testComplete
quit
●Starting the server:
●
Selenium RC is a Java application (built atop of Jetty) So invoke Java, pass Selenium Server Jar as
argument
SRC takes arguments itself (-interactive for interactive shell, others for AUT window handling,
proxying, ... see SRC command
line reference)
●
●
●Interactive Commands:
●
First, create a new browser session for this test.
●
SRC can handle multiple sessions Yields SessionID
(which is mandatory as a parameter for later commands “...&ssessionId=4711”, but can be omitted
in single-session
mode)
is automatically kept in the Selenium object in client driver code
This will create a new browser instance running on the local machine (FF: creating profile, IE:
manipulating the Registry).
●
●
●
●
●
Then, open the AUT in this browser session.
●
50. This will load the web app in iframe/window, controlled by Selenium Core.
Perform various actions on the AUT, possibly checking outcomes (mouse clicks, typing, loading
pages, ...)
Finally, end the browser session. Eventually, end SRC server.
●
●
●
51. Overview
Testing Web Apps
Selenium RC
Extending Selenium RC
Part III: Extending Selenium RC through its user-extensions.js and language bindings.
52. Selenium and qooxdoo
crucial issues:
object identification
event generation
qooxdoo rendering: a world of <div>s
Selenium commands
Selenium locators
the need for (and feasibility of) qooxdoo specific extensions
●event generation:
●
Selenium: synthetic events on elements
Sel events on an element (think 'div') might not match qooxdoo's expected events on the element
'click' vs. 'mousedown', 'mouseup'
background:
●
●
●
●
qooxdoo establishes unified event model over AUT only top-most widget registers for all events
qooxdoo handles event capture and bubbling internally
●
●
●Selenium commands:
●
tell the browser what to do
●Selenium locators:
●
determine where to apply a command
●qooxdoo specific extensions:
●
exactly in these two areas new commands
new locators
●
53. ●
●Rest of talk: practical examples for these
54. user-extensions.js: commands
Selenium.prototype.doQxClick = function(locator, eventParams)
{
var element = this.page().findElement(locator); this.clickElementQx(element, eventParams);
};
●doQxClick:
●
naming convention for extension commands: “doXxYy” if command is “xxYy”
●locator: locator string from command
●eventParams: Selenium-provided eventParam object, filled from command or defaults (think:
mouse button, shift key, ...)
●findElement(): Sel function, returning DOM object reference
●clickElementQx(): custom method that triggers mouseover, mousedown, mouseup, ... events on
element (element.dispatchEvent(), element.fireEvent())
55. user-extensions.js: locators
PageBot.prototype.locateElementByQx = function(qxLocator, inDocument, inWindow)
{
LOG.info("Locate Element by qooxdoo UserData Locator”);
var qxObject = this._findQxObjectInWindow(qxLocator, inWindow);
if (qxObject) {
return qxObject.getElement();
}
};
●locateElementByQx:
●
naming conventions for locators: “locateElementByXxx” if locator is “xxx:”
●qxLocator: locator string from Selenium command, specific to locator “qx:”
●LOG: another Selenium interface object
●_findQxObjectInWindow():
●
custom method parses qxLocator
supports parent/child relation between widgets and “userData”
property of widgets
depends on qooxdoo object hierarchy: qx.ui.core.ClientDocument etc. returns JS object (not
DOM!)
●
●
●
●
●qxObject.getElement(): stand. qooxdoo object method to return corresp. DOM object
●
to satisfy Sel locator interface
56. user-extensions.js: language bindings
new custom extensions:
support for qx locator
just a string passed as parameter of command
no change to client driver API
support for qxClick command
must be supported in language bindings
so client drivers can use it
Sample:
qxClick(“qx:foo/bar/baz”)
●you want to support the extended features in your test scripts
●new locator: just use in locator strings
●new command: need to define in language-specific API, in order to use it in test scripts
●then use as method of Selenium test object
57. user-extensions.js: language bindings
Python:
def qxClick(self, locator, opts=None):
return self.do_command("qxClick", [locator,])
sel.qxClick(r'//div[text()="Form"]')
Java
public String qxClick(String locator) {
return this.commandProcessor.doCommand("qxClick",
new String[] {locator,}); }
sel.qxClick('qxh=*/[@label="Tab"]/[@page]/*/[@label="Fi nd"]');
JavaScript using Rhino
uses Java infrastructure, so no additional overhead
●here are two examples of doing this:
●Python:
●
the first item shows how to define the new API method the second how to use it in testing code
●
●Java:
●
define the new method use it in testing code
●
●
this one uses yet another locator that provides XPath-like syntax to traverse the qooxdoo object
hierarchy
●you can use JavaScript as implementation language for your tests,
atop of the Java language binding, through Rhino (www.mozilla.org/rhino/)
●Hint: complete JS example on next slide
58. A Client Driver Script
importClass(Packages.com.thoughtworks.selenium.QxS elenium);
var sel = new QxSelenium("localhost",4444,"*firefox",
"http://demo.qooxdoo.org");
sel.setSpeed("1000"); sel.start();
sel.open("http://demo.qooxdoo.org/0.7.3/showcase/");
Packages.java.lang.Thread.sleep(2000); sel.qxClick('//div[text()="Tab"]');
sel.qxClick('qxh=*/[@label="Tab"]/[@page]/*/[@label=
"Find"]'); sel.stop();
●importClass() Rhino built-in to load Java classes; QxSelenium is derived class from standard
Selenium class, to hold user extensions
●create interface object
●set properties
●start() method will launch browser
●open the AUT
●use a standard platform (Java) call (“waiting” is inherently difficult in browser-side JavaScript)
●invoke test commands with locators
●stop() brings down the browser
59. Conclusion
Interacting with the DOM seems to be the best way to test Web applications
Selenium RC provides a distributed environment to run tests on a broad range of platforms
Various language bindings exist, so tests can be written as stand-alone,
client-server programms in a preferred language
Custom commands and locators can be added to the Selenium environment, to suite specific
testing needs
60. No. 5/3, Second Floor and Third Floor,
BEML Layout,
Varathur Road,
Thubarahalli,
Kundalahalli Gate,
Bengaluru,
Karnataka 560066
