Introduction to TestNG and its Application for Test-Driven GUI Development

1. Introduction to TestNG
and its Application for
Test-Driven GUI Development
Zheng-Wen Shen
2007/05/14
1

2. References
1. "TestNG makes Java unit testing a breeze“, Filippo
Diotalevi, developerWorks, January 2005
2. "In pursuit of code quality: JUnit 4 vs. TestNG“, Andrew
Glover, developerWorks, Aug 2006
3. "In pursuit of code quality: Automate GUI testing with
TestNG-Abbot“, Andrew Glover, developerWorks, February
2007
4. Alex Ruiz, Yvonne Wang Price, "Test-Driven GUI
Development with TestNG and Abbot," IEEE Software,
vol. 24, no. 3, pp. 51-57, May/Jun, 2007.
2

3. Outline
1. Introduction
2. TestNG Quick Start
3. TestNG Features
4. TestNG-Abbot: Test-Driven GUI
Development
5. Summary
3

4. 1. Introduction 1/4
• JUnit: simple, pragmatic, and strict architecture
• Rules for using JUnit of unit testing
– Every single piece of code must be tested.
– Code must be tested in isolation (mock objects).
– Software must be easy testable (written with tests in
mind.)
4

5. 1. Introduction 2/4
Two opposed factions
KISS principle
The simplicity is necessary
Advanced features!
more flexibility!
more power!
5

6. 1. Introduction 3/4
Some peculiar features of JUnit
• The need to extend a TestCase class
• Impossible to pass parameters (setUp, tearDown)
• The execution model is a bit strange
– The test class is reinstantiated every time a test
method is executed.
• The management of different suites of tests in
complex projects can be very tricky.
6

7. 1. Introduction 4/4
1. TestNG: Testing, the Next Generation
• To cover all categories of tests:
unit, functional, end-to-end, integration, …
• JDK 5 Annotation
• Flexible test configuration
• Powerful execution model (no more TestSuite) Cedric Beust
2. TestNG-Abbot
• 0.3.2 Release, 2007/05/08
• Capture-Replay GUI Testing
• Test-Driven GUI Development
7

8. 2. TestNG Quick Start 1/3
A Test Class
1
The Lang Component provides a host
of helper utilities for the java.lang API,
2 3 notably String manipulation methods,
basic numerical methods, object
reflection, creation and serialization,
and System properties.
8

9. 2. TestNG Quick Start 2/3
Configuration XML
testing.xml Suite
Test 1
groups
classes
Test 2
groups
classes
…
9

10. 2. TestNG Quick Start 3/3
Test Results
HTML output
10

11. 3. TestNG Features
1. Configuration Methods
2. Test Group
3. Dependency Testing
4. Parametric Testing
5. Factories
6. Fail and Rerun
11

12. 3.1 Configuration Methods 1/1
Enter
• @Configuration beforeTestClass methods
1. beforeTestClass
2. afterTestClass beforeTestMethod methods
3. beforeTestMethod first test method
4. afterTestMethod
afterTestMethod methods
beforeTestMethod methods
second test method
afterTestMethod methods
…
afterTestClass methods
12
Exit

13. 3.2 Test Group 1/2
• Groupings of test methods
• Maximum flexibility doesn’t recompile anything
• Check-in tests
– Run before submit new code
– Fast, just make sure no basic functionality was broken
• Functional tests
– Cover all the functionalities of software
– Run at least once a day
Functional Test
Check-in tests
13

14. 3.2 Test Group 2/2
testing.xml
Functional Test
testMethod1
Check-in tests testMethod2
testMethod3
testMethod1
testMethod2
14

15. 3.3 Dependency Testing 1/2
• Invoke test methods in a certain order.
– A certain number of test methods have completed
and succeeded before running more test methods.
– Initialization methods to be test methods
• Hard dependencies
• Soft dependencies
15

16. 3.3 Dependency Testing 2/2
(Hard) Dependency on method
(Hard) Dependency on group
Soft Dependency
@Test {groups = {…} alwaysRun=true} 16

17. 3.4 Parametric Testing 1/3
• Use arbitrary number of parameters on test
method
• Instruct TestNG to pass the correct parameters
with the @Parameters annotation.
– From testng.xml
– From DataProviders
17

18. 3.4 Parametric Testing 2/3
From testing.xml
testing.xml
first-name = “Cedric”
18

19. 3.4 Parametric Testing 3/3
From DataProviders
Same class
Different classes
19

20. 3.5 Factories 1/2
• Create tests dynamically.
– Access a Web page several times with different
values
Test Method
20

21. 3.5 Factories 2/2
21

22. 3.6 Fail and Rerun 1/1
Delineates the failed tests
Run
TestNG
testing.xml testing_failed.xml
22

23. 4. TestNG-Abbot 1/4
Test-Driven GUI Development
• TestNG-Abbot (0.3.2 release, 2007/05/08)
• Intuitive fixture classes
– Logically linked to the code under test by name only.
– Tests do not rely on GUI components being
specifically located
– Tests can be authored early and withstand layout and
aesthetic changes during the development phase
23

24. 4. TestNG-Abbot 2/4
Application Under Test
1. The user enters name
and email address
2. The email address
must be valid
3. The system displays
an error message if
the required input is
missing or invalid
24

25. 4. TestNG-Abbot 3/4
The test class for testing Model
NewEmailAccountWizard View
Email Model
25

26. 4. TestNG-Abbot 4/4
The test class for testing View
1
2
3
4
5
26

27. 5. Summary
• TestNG
– annotations-based framework
– Make the whole testing process much more simple
and flexible
– Free to choose the testing strategy you prefer
• Practice TDD, even for GUIs
27