2. Overview
● Maintenance of automated tests
● Test script smell
● Common maintenance issues
● Conclusion
● Q&A
3. Automated testing in practice
total
cost /
effort
About 2/3 of all test 3
2
automation projects 1
fail sooner or later
– Brian LeSuer
(testing experience magazine,
December 2008, page 47)
time
1. manual testing
2. effective automated testing
3. less effective automated testing
4. Maintenance of automated tests
● The Record & Playback approach does not work
● Total maintenance depends on:
● Number of changes
● Maintenance sensitivity (does change affect test?)
● Maintainability (how much effort)
● For continuity of benefits, focus on:
● 3 R's: Tests must be easy to write, review and revise
● Maintainability and maintenance sensitivity
5. Test script smell (1)
● Test script smell is derived from 'code smell'
● Wikipedia:
● In computer programming, code smell is any
symptom in the source code of a program that
possibly indicates a deeper problem.
● Code refactoring is the process of changing a
computer program's internal structure without
modifying its external functional behavior or
existing functionality.
6. Test script smell (2)
● Test script smell is any symptom in an
automated test that possibly indicates a
deeper problem.
● Test refactoring is the process of changing
the internal structure of an automated test
without modifying its external functionality.
● Most test script issues increase maintenance
● Focus on the average tester!
7. Common maintenance issues
● Unreadable tests
● Long test cases
● Repeating sequences
● Interface details in test cases
● Tooling details in test cases
● Literal constants in test cases
Examples will use the freeware ETA Framework
8. A Selenium test (Java)
Selenium selenium = new DefaultSelenium (“localhost”, 4444, “*firefox”, “http://www.google.com”);
selenium.start ();
selenium.open (“/”);
if (selenium.getTitle ().indexOf (“Google”) < 0) {
reportError (“an error message”);
}
selenium.type (“q”, “the quick brown fox jumps over the lazy dog”);
selenium.click (“btnG”);
if (selenium.getTitle ().indexOf (“the quick brown fox jumps over the lazy dog”) < 0) {
reportError (“an error message”);
}
selenium.click (“link=YouTube”);
9. A keyword based test
new DefaultSelenium localhost 4444 *firefox http://www.google.com
start
open /
type
!
getTitle ().indexOf (“Google”)
q the quick brown fox jumps over the lazy dog
click btnG
getTitle ().indexOf (“the quick brown fox jumps over the lazy dog”)
click link=YouTube
10. Interface details removed
new DefaultSelenium localhost 4444 *firefox http://www.google.com
start
open /
getTitle ().indexOf (“Google”)
!
type ?queryField the quick brown fox jumps over the lazy dog
click ?searchButton
getTitle ().indexOf (“the quick brown fox jumps over the lazy dog”)
click link=YouTube
11. Tooling details removed
!
type URL
open browser firefox www.google.com
open /
type
!
check page title Google
?queryField the quick brown fox jumps over the lazy dog
click ?searchButton
!
check page title the quick brown fox jumps over the lazy dog
select link YouTube
12. Abstraction level increased
type URL
open browser firefox www.google.com
web page title
open page / Google
search string
search the quick brown fox jumps over the lazy dog
link text
select result YouTube
13. The automated test
Test case
Test case Test analyst
Scripted procedures
Scripted procedures Test analyst / Developer
Source code
Source code Developer
Application
Application
14. Conclusion
● Abstraction is the key to low maintenance
● For easy maintenance:
● Make tests readable (to your manager?)
● Make tests compact (high abstraction level)
● Remove interface + tooling details (implicit or config)
● Use symbolic constants
● Define test instructions top down
● Test automation is software engineering
● Example test engines: ETA Framework & FIT
16. Main sheet & configuration sheet
Main sheet: name
run sheet configuration
run sheet procedures
type URL
open browser firefox www.google.com
name
run sheet test cases
Config sheet: name value
define constant queryField q
define constant searchButton btnG
17. Procedures
name Param 1 Param 2
define procedure open page url pageTitle
url
open URL ?url
title
check page title ?pageTitle
end procedure
18. Procedures
name Param 1
define procedure search searchString
input field value
type ?queryField ?searchString
item name
click ?searchButton
Configuration sheet
title
check page title ?searchString
end procedure
19. The test (case)
identification description
begin test case google1 A Google test
web page title
open page / Google
search string
search the quick brown fox jumps over the lazy dog
link text
select result YouTube
end test case
close browser
