test automation strategies for agile development

1. Strategies for Agile Software Test Automation:
An Industrial Experience
Eliane Collins, Dr. Arilo Dias-Neto, Dr.
Vicente F. De Lucena Jr.
STA 2012 Nokia Technology Institute

2. Agenda
1. Introduction
2. Process and Tools
3. Software Testing Automation Strategies
4. Lessons Learned
5. Conclusion
Instituto Nokia de Tecnologia
1
Nokia Internal Use Only

3. Nokia Technlogy
Introduction Intitute
INdT – Nokia Technology Institute
Independent and Nonprofit institute
committed to conducting research and
development of technological solutions
through application development, new
technologies and concepts. Manaus
Recife
Brasília
Main Areas:
• Product Creation
• Apps and Services São Paulo
• Hardware and Manufacturing
• Software Open Source and User
Interface Instituto Nokia de Tecnologia
2

4. Introduction – Agile Testing
• Testing with a plan to learn about it,
let the customer information guide
the testing in line with agile values
working software which responding to
change .
• Tests to prevent defects.
• Proactive Testers.
• Test Automation is the key of
successfully agile development and
the core of agile testing.
• [Crispin, L.; Gregory, J. 2009]
Instituto Nokia de Tecnologia
3
Nokia Internal Use Only

5. Process and Tools
Scrum agile methodology
Scrum is an agile development process focusing on
teamwork;
Scrum ceremonies (sprint review, daily, retrospective and
planning meetings);
Test Process
Testers should plan and update test cases for sprint stories;
Automate test scripts when it is possible;
Execute regression tests (manual and automatic), report
defects found
Tools for Unit Testing: GoogleTest, Junit,
GUI test tools: Sikuli, Selenium RC and IDEand Fitnesse.
Instituto Nokia de Tecnologia
4
Nokia Internal Use Only

6. Software Testing Automation Strategies
Strategy 1: Testers Automating Unit and System Tests :
Development of a location API driver to be embedded in a
modem;
The project schedule was delayed and the developers
wanted to focus on developing features instead unit
testing;
Testers decided to perform the unit testing tasks using
GoogleTest tool according to Model V approach;
Functional tests for the desktop installer and configuration
screen were automated for MS Windows 7-x64 and MS
Windows 7-x86 using Sikuli tool ;
Instituto Nokia de Tecnologia
5
Nokia Internal Use Only

7. Software Testing Automation Strategies
Positive and Negative points from Retrospective meetings:
Positive Negative
• Poor knowledge transferring from
• Testers improved programming
testers to developers
skills
• System testing performed after
• Knowledge sharing on System
development
Architecture
• Separation between testing and
• Unit testing found important
development teams
defects
• Code improvements postponed
Instituto Nokia de Tecnologia
6
Nokia Internal Use Only

8. Software Testing Automation Strategies
Strategy 2: Developers Automating Unit Tests and Testers
Automating System Tests
Development of a web system to automate factory tasks on
counting material for production.
Developers coding new features, unit testing using TDD
approach and Continuous Integration (Hudson).
Testers automating UI Tests using Selenium, Load Tests and
Security using JMeter.
Instituto Nokia de Tecnologia
7
Nokia Internal Use Only

9. Software Testing Automation Strategies
Positive and Negative points from Retrospective meetings:
Positive Negative
• CI strategy improved testing • Poor knowledge transferring
automation • High effort to automate tests with record-
• High coverage of GUI testing and-play approach
automation Positive of integration testing
• Absence
• Security and load tests executed • Separation between testing and
early Negative
development teams
Instituto Nokia de Tecnologia
8
Nokia Internal Use Only

10. Software Testing Automation Strategies
Strategy 3: Developers Automating Unit Tests and Testers
Automating System Tests
Development of a server-side system to receive requests
from a mobile device, and send services and applications
back;
Testers started to collaborate reviewing test scenarios and
code by pair programming;
Developers and testers configured tools and programmed in
pair funcional test scripts using Fitnesse tool for API Tests;
Developers coded and wrote unit, integrations, and system
tests, reviewed test cases and executed all automated tests;
Instituto Nokia de Tecnologia
9
Nokia Internal Use Only

11. Software Testing Automation Strategies
Collaborative Strategy
Automation tasks are in the intersection and they can be executed
by everyone in the project team;
Instituto Nokia de Tecnologia
10
Nokia Internal Use Only

12. Software Testing Automation Strategies
Positive and Negative points from Retrospective meetings:
Positive Negative
• CI strategy improved testing
automation
• Security and load tests executed early • High initial effort to set the
• Intense team collaboration automation environment
• Knowledge transferring regarding the
tools • Rework when customer requires
• Testing responsibilities distribute for all changes in the system
team
• Team motivation to learn new solutions
Instituto Nokia de Tecnologia
11
Nokia Internal Use Only

13. Lessons Learned
Collaboration supports the Success of Agile Test Automation;
Automate Each Layer of Software When It Is Possible and Just for
Acceptance Tests in the Sprint;
Test Automation in the Agile Project Should Be Simple, Reusable
and Maintainable for Project Team;
Automate and Run Security Tests and Stress Tests Early Reduces
Risks and Rework;
Use Test Automation for Documentation and Information
Feedback ;
Instituto Nokia de Tecnologia
12
Nokia Internal Use Only

14. Conclusion
It is feasible to adapt agile practices and values to test automation.
The automation was a resource to document software, reduce
cost, and allocate tasks in smaller parts.
Next steps:
Evaluate the test automation collaborative strategy as
framework;
Evaluate the impact of team collaboration levels in the test
automation.
To extract the testing effort, number of detected failures pre
and post-delivery, and schedule adherence.
Instituto Nokia de Tecnologia
13
Nokia Internal Use Only

15. Thank you.
Questions?
Nokia Technology Institute