This document summarizes Archana Joshi's presentation on testing challenges in agile environments. It describes a program at Wipro to develop a new web application for wealth management using agile methods. The program involved multiple distributed teams and vendors. Key challenges included integrating with other systems, forming cross-functional teams, and meeting compliance requirements. The presentation outlines seven lessons learned: 1) involve testers early, 2) divide testing types, 3) work as a cohesive team, 4) accept frequent code changes, 5) establish readiness checks, 6) measure metrics, and 7) set up governance. Benefits included fewer defects, less testing effort, and improved team morale when adopting agile practices.

1. 3rd Asia Pacific Software Testing Conference
Testing in Agile
Easier said than done?
Archana Joshi
Wipro Consulting Services
archana.p.joshi@gmail.com

2. © 2009 Wipro Ltd - Confidential2 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential2
Who am I?
About Myself - Archana Joshi:
•Manager - Agile practice at Wipro Consulting Services
• Certified Scrum Practitioner with 9+ years in software development, management and consulting
• Involved as a scrum master, process consultant, agile coach on various teams
About Wipro Consulting Services:
• Wipro Consulting Services (WCS) is a division of Wipro Ltd (NYSE: WIT), a $5.5bn global enterprise.
• WCS has 1200+ consultants across the globe providing unique solutions to our clients worldwide.
• The Agile initiative started in 2004 and agile is now one of the mainstream processes at Wipro.
• WCS has a dedicated agile practice which consults teams and clients on agile implementation.

3. © 2009 Wipro Ltd - Confidential3 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential3
Program Landscape
Development program for a new web based application in wealth
management space
• Technology J2EE / Web 2.0 / Flex
• Involves integration with 3rd party components like – SFDC, SiteMinder,
LDAP AD, Akamai
Drivers for agile
• Respond to changing needs of the business
• Improve the relationship among the technology and business partners
• Gain competitive edge in dynamic financial market through reducing
time to market
Team
• Program divided across 3 agile tracks - based on functionality
• Each track was a cross-functional distributed team involving
•Developers (vendor 1 & customer IT)
•Testers (vendor2), BA (vendor3)
• User Experience personnel (vendor 4)
• Scrum Master (vendor1), Product Owner (customer business)
• Shared resources included business, legal, architecture, security and
compliance teams
Agile Awareness
• Excluding the scrum masters, entire team was new to agile
• Team had undergone 2 days class room training on agile

4. © 2009 Wipro Ltd - Confidential4 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential4
• New Product development
– Integration with other 3rd party applications
– Integration across multiple tracks
• Test Team belonged to different hierarchy within the
organization
– Forming cross-functional teams
– Work together as a cohesive team
• Compliance to financial and security regulations
– Testing to consider these requirements too
• Team new to agile way of working
– Documentation needed for everything to start work
– All manual testing, minimal automation
• Team distributed across locations
– Multiple location, time zone challenges
Challenges for testing

5. © 2009 Wipro Ltd - Confidential5 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential5
Overall Process and Release Cycle
Stabilization2 week Iterations of Development
with Testing
Production LiveInterim integration points
2 wks 1 wk
2 wks 2 wks
2 wks 1 wk 2 wks 1 wk
2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks
2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks
2 wks 1 wk 2 wks 1 wk 2 wks 2 wks
2 wks
2 wks
Track 1
Track 2
Track 3
Release Cycle
Initiation StabilizationIteration Release and
Deployment
Process

6. © 2009 Wipro Ltd - Confidential6 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential6
Key Lessons Learnt from the program
#1 Team Involvement - Sooner the better
#2 Divide and rule
#3 Work together as a team
#4 Code will change. Accept it.
#5 Readiness checks are necessary
#6 Do not forget to measure
#7 Establish governance mechanism at start

7. © 2009 Wipro Ltd - Confidential7 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential7
#1 Team Involvement - Sooner the better
• The test team was involved right from the initiation &
planning stage
• Test strategy formed during the initiation stage.
• It contained the high level approach on how
system shall be tested
• It also contained the responsibilities of the testers
within the team at different stages of the project
• For estimations and planning along with the business
and development teams, testers were also involved
• Early involvement helped testers in understanding
business perspective and dependencies better

8. © 2009 Wipro Ltd - Confidential8 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential8
#2 Divide and rule
• Instead of trying to fit all types of tests within an iteration,
division based on types was done
• Testing within an iteration
• Feedback provided by business, UX team & testers
• Unit testing & Integration testing (by developers)
• Functional testing (by testers with guidance from BA)
• Regression testing (by testers)
• Testing outside an iteration (stabilization)
• Performance testing
• Security (ethical hack) testing
• Final UAT & Regression testing
• Exploratory testing by business and legal teams

9. © 2009 Wipro Ltd - Confidential9 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential9
#3 Work together as a team
• Though there were multiple vendors spread across
different locations and time zones, the cross-functional
team worked as a cohesive team
• The scrum master had to be on a constantly look out
for any team behaviors which can result into group-ism
and arrest the behavior at the start
• Collaboration tools like net meeting, VCON, TCON
were extensively used
• Some of the key team members in the team – team
leads, sr. developers, testers, BA came together at same
physical location for planning and initial iteration of the
program

10. © 2009 Wipro Ltd - Confidential10 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential10
#4 Code will change.Accept it.
• Testers typically test on a code which is fairly stable
• When an iteration is 2 weeks long, there are code changes till
the last day of the iteration
• There were challenges in changing the mindset of the testers
that the code undergoing test is going to change frequently during
development of the iteration
• During iteration
• At end of the day, developers provided links to their code
from the development environment which the testers used
to provide feedback on.
• This was considered as feedback & not defects as the
iteration was work in progress
• At end of the iteration which ever feedback was not
addressed was entered as defects in the Quality Center by
the test team
• Test team was responsible for collating the feedback from
business and UX teams as well
C
h
a
n
g
e

11. © 2009 Wipro Ltd - Confidential11 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential11
#5 Readiness checks are necessary
• Agile does not mean that the checks or sign-off are no longer to be taken by the test team
• Within an iteration
• Review were done by the BA and business on the test cases and test plan to ensure
coverage
• “Done” ness criteria was established for tracking the iteration completeness.
• “Done” ness criteria also included the compliance and security checks
• Outside an iteration
• Readiness criteria to move from one environment (integration -> staging ->
production) was established by the test team
Demo Env. /
Dev. Env.
Integration
Env
Staging
Env.
Production
Env.
Test Environment during
stabilizationTest Environment during iterations

12. © 2009 Wipro Ltd - Confidential12 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential12
#6 Do not forget to measure
• Monitoring of metrics is important. Especially for a large program
• HP Quality Center was used as the test management tool
• Following metrics were captured
• Within an iteration
• Defects which are not addressed and carried forward in an iteration
• Test cases and scripts that were completed and on which environment
• Out side iteration
• Defect spread based on priority
• Test scripts % execution
• Test scripts % passing
747
61
58
3
4
685Total Pending Cases/Scripts
Total Pass
Total Fail
Total Percentage Passed
Total Scripts Blocked/ Descoped
Execution Status
Total Test Cases/Scripts Planned
Total Executed

13. © 2009 Wipro Ltd - Confidential13 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential13
• The teams – business, BA, test, developer belonged to different groups within the organization
• There was a need to establish a strong governance structure which was agreeable to various
stakeholders involved
• This was not finalized by the time the program started and hence there were issues seen during
the initial iterations
• The PMO was made aware of the process to be followed. They provided the by-in on the
process and artifacts that were produced as part of the agile process
• Learning from retrospective of each track was circulated at all levels thus ensuring
management support
#7 Establish governance mechanism at start
Level 1
Level 2
Level 3 Key stakeholders
from IT, business,
legal, security &
compliance, PMO
Scrum Master, IT
Program
Manager, IT
shared teams
Track – 1: Scrum
Master and Team
Track - 2: Scrum
Masters and
Team
Track - 3: Scrum
Masters and
Team
Weekly monitoring, Program risks,
Budget, Costs, Change Management
Weekly monitoring, Metrics,
Inter & Intra team dependencies
IT risks and mitigation for program
Daily tracking,
Project Quality,
Resource Capacity

14. © 2009 Wipro Ltd - Confidential14 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential14
Benefits
• Fewer number of defects found during final UAT
• Hardly any issues seen after the application went live
• Less number of testing resources required compared to
the traditional approach
• Better team morale – Team feeling agile is the way to go !
• Increased appreciation between business, developer &
test teams for their respective work

15. © 2009 Wipro Ltd - Confidential15 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential15
In Conclusion….
• Start with simple practices that are easy to adopt
• Implement a piece meal approach
• Advanced practices like extensive Automation, Test Driven
Development can come in later
• Allow the team time to adapt to the agile way of working
………..And you will realize instead of being
“Easier Said Than Done”
it will be “Easier Said And Done”

16. © 2009 Wipro Ltd - Confidential16 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential16
Thank you for your time !
Questions ?