This is a training that I conducted on Agile Testing. There is a lot of confusion around how Testing Professionals are impacted when they move from Traditional development models to Agile development models. This training attempted to clear some of the confusion around that.
2. Agile Testing
2
Agile testing is a software testing practice that
follows the principles of agile software
development (wikipedia)
Involves all members of a cross-functional
agile team, with special expertise contributed
by testers, to ensure delivering the business
value desired by the customer at frequent
intervals, working at a sustainable pace
12/10/2013
3. The role of Agile Testing
3
Agile teams do need testers
People
with strong testing skills
They might need QA in a different form/shape!
Your function is to support the business by
helping them understand the business and
acceptance criteria
Understand
risks!
12/10/2013
7. Everybody tests!
7
Everyone Tests
On traditional projects, independent testers are responsible
for all test activities.
In Agile, getting the testing done is the responsibility of the
whole team. Yes, testers execute tests. Developers do too.
The need to get all testing done in an iteration may
mean that the team simply cannot do as much in
each sprint as they originally thought.
If yes, then Agile has made visible the impedance mismatch
between test and dev that already existed.
The team was not going as fast as they thought. They
appeared to be going quickly because the developers were
going fast. But if the testing isn't done, then the features
aren't done, and the team just does not have the velocity they
think.
Goldratt's TOC says that the whole team can only
go as fast as the slowest part.
12/10/2013
To go faster, the team has to widen the throughput of the
9. 9
Case Study:
Test Planning for one release – in
2011
IR
Activities
Pre Release
Defect Validation
- Merges ~ all 6.2 HF1 -6.2HF6 = 150 defects
Planned Planned
Dev Build Start
End
Team
17-01-11 08-02-11
QA -Aryaans
Test Case Development
CHR CFT
IR1
Cal Total Reso
Days PDs urces Rate
3
16-02-11
4 15/day/person
10
2
QA
QA ,QA Aryaans
12
60
8
6
4
items
~1000
TCs
17-02-11 01-03-11
Regression Testing ~Tier3 (1000 testcases)-need to
skip incase of bandwidth issue
Test Case Development
6.3 CHR Testing [Estimated Cases ~ 1000]
Merge Defect Validation all 6.2 HF1 -6.2HF5 defects
QA -Aryaans
QA
QA
QA
3 10.5
3.5 100 tc/day/person
5
2
25
6
5
65
1
4
4 15/day/person
QA Aryaans
QA Aryaans
0.5 1.5
0.5 1.75
3
3.5 50/day/person
QA Aryaans
1.5
4.5
3
Only rejected defects and automation run critical fixes
QA Aryaans
1
1
1
Acceptance Scenario and Unautomated sanity Testing
Release Packaging
QA Aryaans
QA
1
2
TOTAL
IR2
01-03-11
Regression Testing ~6000 testcases (Tier1,Tier2 and
Tier3 ;these are non automated cases of the modules
related to the defects tagged for 6.3 on dhruva)
02-03-11 08-03-11 - 5 cal days -(2000 testcases)
4 QA Aryaans
10 Resources - 80 tc/day/person to fill in the
bandwidth for execution of 4000 testcases for 5 cal
days
Defect Validation
- Defect Validation - QA Initiator Closure ~ 50 +
- CHR Defects ~100
IR3
QA - Aryaans + 1 QA
10-03-11
18-03-11
13
11-03-11 16-03-11
Defect Validation - Merges ~75
Regression Testing(Unexecuted) ~ 100 testcases
Initiator Closure Defect Validation (CHR + Regression )
~75
IR4
5 100TC/day/person
1000
3 15 defects /day /person
21-03-11 23-03-11
7000
200
We still
had
regression
leaks!
2
2
2
1
203 pds
So, 60mm of DCUT needed 20-25mm (10mm for testing + 10 developers
for supporting all defects from testing) over a period of 6-8 weeks to make a
12/10/2013
release!
10. Execution Today
10
Team is delivering continuously...
Changes to scope can be taken anytime and delivered in 3-4 weeks
Testing is part of the Development process
Everyone tests! Development automates UTC; Testing automates ST
12/10/2013
13. Reduce test documentation
overhead
13
Lightweight Documentation: Instead of
writing verbose, comprehensive test
documentation, Agile testers:
Use reusable checklists to suggest tests
Focus on the essence of the test rather than the
incidental details
Use lightweight documentation styles/tools
Capturing test ideas in charters for Exploratory
Testing
Leverage documents for multiple purpose
Leverage One Test Artifact for Manual and
Automated Tests
Today, we invest in extensive, heavyweight step-bystep manual test scripts in Word or a test
management tool
Instead, capture expectations in a format supported
by automated test frameworks like FIT/Fitnesse.
The test could be executed manually
12/10/2013
More importantly that same test artifact becomes an
22. The testing pyramid...
22
Ideal State
In most environments
Manu
al
C
O
S
T
R
O
I
UI
(5%)
Services
(15%)
Unit Tests
(80%)
UI
Services
Unit
Tests
12/10/2013
23. But let us understand this in more
detail
23
12/10/2013
24. Impact of Agile Requirements
24
Agile testing must be iterative
Agile testers cannot rely on having complete
specifications
Agile testers must be flexible
The techniques exist to make this possible...
12/10/2013
26. 26
Let us discuss these test cycle
more...
Development Team Testing:
Testers are embedded in the development team, working side by side to build the
system
Focus of their testing efforts are often on confirmatory testing
Agile teams will take a whole team approach
Developer regression testing or better Test-Driven Development (TDD).
Parallel independent testing.
Continuous independent testing parallel to construction iterations throughout the
lifecycle.
Goal: find defects that got past the development team
Perform higher forms of testing such as system integration testing, security testing,
usability testing
Need significant testing skills, complex tools, and often complex pre-production testing
environments
10-15:1 ratio between people on the 2 teams
In larger organizations, one team can support several development teams
Release Testing
12/10/2013
30. ATDD
30
TDD at the requirement
level
Acceptance TC is a
expectation of the
customer
Write a single
acceptance test; make
code changes to pass it
Requirement spec (JIT)
If you do ATDD, you
don’t need to TDD
necessarily
Also, called BDD or
user-story driven
development
12/10/2013
33. Implications for Test Practioners
33
Become generalizing specialists
Be flexible.
Be prepared to work closely with developers.
Once again, be flexible.
Focus on value-added activities
and again... Be flexible
12/10/2013
Speed needs discipline – the fast car vs slow car analogy
Agile teams will take a whole team approachTesters are embedded in the development team, working side by side to build the systemFocus of their testing efforts are often on confirmatory testingdeveloper regression testing or better Test-Driven Development (TDD).
Business Facing: means functional test casesTechnology Facing: means technical test cases like for cross browser support. However, what browser should be dominant browser is still a business decision.Critique Product Testing: typically defect discovery processSupport Development Testing: testing that supports ability to make code changes (regression)
Become generalizing specialists. The implication of whole team testing is that most existing test professionals will need to be prepared to do more than just testing if they want to be involved with agile projects. Yes, the people on independent test teams would still focus solely on testing, but the need for people in this role is much less than the need for people with testing skills to be active members of agile delivery teams.Be flexible. Agile teams take an iterative and collaborative approach which embraces changing requirements. The implication is that gone are the days of having a detailed requirements speculation to work from, now anyone involved with testing must be flexible enough to test throughout the entire life cycle even when the requirements are changing.Be prepared to work closely with developers. The majority of the testing effort is performed by the agile delivery team itself, not by independent testers. Be flexible. This is worth repeating. ;-)Focus on value-added activities. I've lost track of the number of times I've heard test professionals lament that there's never enough time nor resources allocated to testing efforts. Yet, when I explore what these people want to do, I find that they want to wait to have detailed requirements speculations available to them, they want to develop documents describing their test strategies, they want to write detailed test plans, they want to write detailed defect reports, and yes, they even want to write and then run tests. No wonder they don't have enough time to get all this work done! The true value added activities which testers provide are finding and then communicating potential defects to the people responsible for fixing them. To do this they clearly need to create and run tests, all the other activities that I listed previously are ancillary at best to this effort. Waiting for requirements speculations isn't testing. Writing test strategies and plans aren't testing. Writing defect reports might be of value, but there are better ways to communicate information than writing documentation. Agile strategies focus on the value-added activities and minimize if not eliminate the bureaucratic waste which is systemic in many organizations following classical/traditional strategies.Be flexible. This is really important.