Software quality and testing for an agile and mobile world.

1. Quality in an Agile world
rethinking tools, processes, and teams
Presented by Jason Arbon
June 19, 2012

2. context on me
• co-author of "How Google Tests Software"
• uTest.com:
o Engineering Director
• Google:
o Chrome Browser, Chrome OS
o Google+ (Search)
• Social Search Startup
• Microsoft:
o Bing, BizTalk, WindowsCE
o ...and WinFS :)

3. Agile is the answer
Scary signals
• speed
• mobile
• appification + fragmentation

4. Agile Speed
Scary:
• No full test passes
• Less upfront design and planning
Happy:
• Rollback
• Fewer changes per build
• Quick feedback

5. Agile Speed

6. Agile Speed

7. Mobile
• Undeniable
• Don't miss this wave..

8. Appification + Fragmentation
Scary:
• Distribution
• User Voice
• Matrix
Happy:
• User Voice
• Data
• Quality is quantified

9. Fragmentation

10. Fragmentation

11. Appification + Fragmentation
Feedback from early adopters, dogfooders, crowd testers..
Enable feedback everywhere. its easy!

12. Appification + Fragmentation
leverage the crowd
• They have the device matrix
• They are the end users
• Spin up and down like AMZN
• No mangment 1:1's :)

13. Truly Agile Tools and Practices
• Only the quick will survive
• Beware if you think your software is too 'critical' or
'complicated' to adopt agile. Your software and practices
might actually be too complicated to compete.

14. Truly Agile Tools and Practices
Planning:
• Stop planning.
• If you have milestones that go out 30+ days, you aren't
agile and not responding to feedback and the market
"There is no need for a spreadsheet, database, or filing system. The
requests that really matter are the ones you will hear over and over
again ... your customers will be your memory ... If there is a request that
you keep forgetting, that's a sign that it isn’t very important. The really
important stuff doesn't go away."
--Jason Fried, David Heinemeier Hansson, Rework,
37Signals

15. Truly Agile Tools and Practices
Test Plans:
• Stop. No more documents
• Risk-based
• Continuous testing on whatever is the most risky
• Don't worry so much!
• Cut or hide red areas
Appification + Fragmentation

16. Truly Agile Tools and Practices
Regression Testing:
• Have lots of 'Unit+' test automation
• Delete as many as you can bear
• Low ROI...they usually pass :)
• Crowd and dogfooders find your bugs (calendar ex)
o Instrument your builds
o Feedback channels
Appification + Fragmentation

17. Agile, Continuous Testing at Scale

18. Truly Agile Tools and Practices
Bug Reporting:
• No more bug forms with Repro, Results, Expected!
• Capture all the environment + state data automagically

19. Truly Agile Tools and Practices
Testcase and Textual Bug databases:
• If you have them, you are still transitioning to Agile
• Exploratory testing
• Bugs should be fixed immediately

20. Truly Agile Tools and Practices
Team Structure and Process:
• Engineers own quality
• Daily standups and recurring meetings should be banned.
Just a sign that the team is too large or doesn't collaborate
well.
• re-Architect for smaller and agile teams. Services structure
(GOOG + AMZN).

21. Next gen continuous testing workflow

22. Final words
• World is getting scarrier and changing underneath us
• Agile used to be cool
• ...now it is necessary

23. evolve quickly

24. contact jasona@utest.com
Quality in an Agile World