From the STP Online Summit "Agile Transitions"

2. Michael Larsen
Senior Tester, Sidereel.com
@mkltesthead
Producer & Commentator: TWiST Podcast
Speaker: PNSQC, CAST, STAREAST
Facilitator: Weekend Testing Americas
Contributor: How to Reduce the Cost of Software
Testing
Blog: TESTHEAD (http://mkltesthead.com)
Articles: ST/QA, StickyMinds, The Testing Planet, Tea Time
with Testers
Michael Larsen,
Senior Tester

3. About Me (cont.)
– 18 Years Software Testing Experience
– Mostly with Traditional Development
Approaches
– Usually Sole Tester for Projects or Companies
– Made the switch to Agile development in
January, 2011
– Close enough to the Pain to Remember It!

4. Traditional Test Workflow
Review requirements document
Review a software specification
Write out test cases based on software
spec
Create detailed scripted test procedures
Wait for a build to arrive so you can test
Execute your scripted tests
Re-run tests that were added to cover
bugs that were found in previous builds
Determine after enough testing or the
specific release date if software is ready
to ship

5. The Lone Tester in the Traditional World
– Finding issues in requirements and design
documents
– Verifying/validating software meets the spec
– Running tests at given milestone points to see if
the product break

6. Problems With Traditional Testing Approach
– Requirements are never complete
– Spec often misses the mark
– Product "meets the spec”, but isn't really what the
customer wants
– Separate development and test cycles
– Software thrown "over the wall" to be tested and
then thrown back to be reworked
– Waiting for the product to be viable again to test

7. Is There Another Way?!

8. What is Agile?
– Build system in smaller chunks
– Business requirements discussed along with product
development
– Build what is actually wanted
– Develop and test smaller bits of functionality more
frequently
– Delivering product that meets real business needs

9. The Agile Manifesto

10. The Agile Approach
• Small slices (Stories)
• Testing from the beginning of the
project
• The whole team responsible for
quality
• Test Driven Development used to
help design code
• Tests fail first, build code to pass
tests, refactor code to be more
efficient, integrate with other code
• Continuous build and integration
each time code is checked in

11. Is Your Team "agile"?
– is the team using a test-driven approach to
development?
– are stakeholders actively involved in the
development process?
– does the team produce high-quality software at
regular intervals?
– is the team highly collaborative and self-
organizing?
– is the team actively improving the process of
how they work?

12. A Typical Story
– Developed in
coordination with
the product owner
– Develop the story's
acceptance criteria
along with it

13. Test Driven Development
– Developers often
working in paired
teams
• Focus on Red,
Green, Refactor
• Repeat the process
if necessary
• When the story has
reached the state
of DONE, move on
to the next story

14. TDD is Not Testing
– ???
– TDD is a design process for development
– Helps to cover regression tests and unit tests
on all code segments
– No build goes out that is not "all green" (all
unit and regression tests pass)

15. The Whole Team is Responsible for Quality
– TDD
– Acceptance testing
– Automated testing throughout the entire cycle.
– Provide continuous feedback to the
development team

16. ???
Lots of
"Testing"...
But where are
the "Testers"?

17. Are Testers Needed?
– Some view Agile as a "post tester world”
– Testing is done by development, product
owners, and customers
– Is there a need for dedicated testers?

18. Testers Needed and Valued
– Testing is integrated at all levels
– Quality is seen as a team effort
– Traditional Testing role (Quality Police) not required
– Testing are being done at many different levels
– Model of dedicated tester doing all the testing has
changed

19. Agile Testing Requires Variety of Skills
– Domain knowledge about the system under test
– Technical competency to interact effectively with
development team

20. Testing in Thin Slices
– Testing focused on if story meets customer
requirements
– Exploration around stories to guide and provide
feedback for team
– Automating acceptance tests to run in the future
along with new features being developed

21. “Agile Testing is…”
• According to Bret Pettichord:
• “Headlights of the project –
where are you now? Where are you headed?”
• “Provide information to the team –
allowing the team to make informed decisions”
• “Find and inform team of bugs”
– A “bug” is anything that could bug a user –
testers don’t make the final call
• “Testers do not "assure" quality –
the team does (or doesn’t)”
• “Testing not a game of “gotcha” –
set goals, rather then focus on mistakes”

22. Challenges for Agile Testers
– Requirements and Acceptance Criteria specific to
single story
– Testing as early as possible
– Acceptance Tests developed before the software is
developed
– Automated tests run against code every time a
build is performed
– Regression testing requirements significantly
increased

23. Single Tester in an Agile Development Team
– Testers have to adapt to
a different expectation
and needs
– Plenty of room for
"testing" but we need
to broaden our toolkit

24. My Story: Sidereel

25. Sidereel: Pre-Tester
– Prior to 2011, no dedicated tester
– All developers actively use TDD
– 4 Stage deployment method put into place
• developer machines / demo / staging / production
– Each step had additional people looking at the
product
– Testing done at each level by product owners,
content developers, support staff and execs
– Worked well, but still plenty of issues discovered
after the fact

26. Sidereel: Post Tester
– In 2010, they decided that having a dedicated
tester would be a benefit
• Testers can provide many more directed efforts against
the code because of training and focus
• Product owners and other staff members can apply
some time to testing, but they have other jobs
• Testers can bring all of their time to testing an
application and continue to apply numerous
approaches

27. How Can We Use Testers?
Step Outside of the Obvious
• Tester verifies bug fixes
• Tester confirms user
stories
Good start, but there is a
lot more we can do!

28. Augment The Testing Already Happening
– TDD is meant to prevent defects
– TDD will help keep mistakes out, but not all of
them
– Testers operate from a different mindset
• Use Exploratory testing techniques
• Aspects development team might not have considered
• Look for missing story criteria

29. Acceptance Testing: Great Place for Testers
– Have testers implement Active and Automated
Acceptance Testing
– Not essential for testers to know how to code, but
helpful
– Developing acceptance test requirements and
potential automated tests at the start of the
iteration

30. Where to Look for Acceptance Tests
• Backlog
• Current user stories
• Current bugs

31. Automating Acceptance Tests
– Variety of Tools to
Help This Process
• Fitnesse
• Selenium/
WebDriver
• Capybara
• Cucumber
• Other testing
frameworks

32. Benefits
– Share results with the development team
– Help provide information about implementation
– Help the team get to DONE
• DONE from a development perspective
• DONE from a testing perspective

33. Disadvantages
– Passive "checking" as opposed to Active Testing
• Inquiring about the results we get
• Learning based on results and exploring new avenues
– Tests can quickly become stale
– Consistent upkeep required

34. Tester as Anthropologist
– Observe the
development team
– Look for feedback of
actual users
– Work with content
developers
– Examine workflows
– Discover the
underlying and
unspoken "culture”

35. Other Benefits
– Testers help with planning and future work
– Testers determine testability requirements
– Testers clarify stories and validate them early
– Testers can help identify when a story is "too big"

36. Advocate for the Customer
– Understand the customer needs
– Review and relate to the business needs
– Review and relate to development practices
– The tester can straddle all of these

37. Being The Lone Tester
– Requires communication
– Willingness to stretch
– Pair with programmers to help make automation a
reality
– Participate with planning and standups with
development team
– Become a domain expert about the customers
needs

38. Tips to Thrive as a Lone Tester
– Focus on Communication
– Learn the Expectations of Your Development Team
– Stop Thinking in Terms of Us vs. Them
– Learn How to Work with the Development Tools of Your
Organization
– Plan for Pairing Sessions with Developers and Domain
Experts
• Tester/Developer
• Tester/Support
• Tester/Designer
• Tester/Customer
– Cultivate Relationships with Mentors
– Develop Your Craft

39. Use Context-driven Principles
– The value of any practice depends on its context.
– There are good practices in context, but there are no best
– practices.
– People, working together, are the most important part of any
project’s context.
– Projects unfold over time in ways that are often not predictable.
– The product is a solution. If the problem isn’t solved, the product
doesn’t work.
– Good software testing is a challenging intellectual process.
– Only through judgment and skill, exercised cooperatively
throughout the entire project, are we able to do the right things at
the right times to effectively test our products.

40. Secret Memo for Lone Testers:
You're Not Really ALONE
– You have many allies
– Developers are actively
testing as they develop code
– Customer actively informs
you and guides you about
features
– Support team gives
feedback about pain points
– Designers give feedback
about usability and user
experience
– Every one of these people
does testing, it's not ALL ON
YOU!

41. References
• Ambler, Scott (2010). "Agile Testing and Quality Strategies: Discipline over
Rhetoric". (http://www.ambysoft.com/essays/agileTesting.html)
• Crispin, Lisa (2003). "XP Testing Without XP: Taking Advantage of Agile Testing
Practices" (http://www.methodsandtools.com/archive/archive.php?id=2)
• Hendrickson, Elizabeth (2008). "Driving Development with Tests: ATDD and TDD"
(http://testobsessed.com/wp-content/uploads/2011/04/atddexample.pdf)
• Larsen, Michael (2011). "The Challenges of the Lone Tester: Learn to Thrive"
(http://www.softwaretestpro.com/Item/5174/The-Challenges-of-the-Lone-Tester-
Learn-to-Thrive)
• Parkinson, Shane (2008). "Agile Methods and Software Testing"
(http://agiletesting.com.au/agile-methodology/agile-methods-and-software-
testing/)

42. Questions???