Lessons Learned in Weekend Testing. Presentation for STAREAST 2012.

1. Delivering Quality: One
Weekend at a Time
Lessons Learned from Weekend Testing
Michael Larsen

2. Who am I?
Michael Larsen
• Senior Tester, SideReel.com (Rovi Corp.)
• Chair, Education SIG, Association for Software Testing
• Facilitator/Co-Founder, Weekend Testing Americas
• TESTHEAD (http://mkltesthead.com)
• Twitter: @mkltesthead

3. How do we learn about testing?
• Books
• Online Forums
• Web Sites
• Conferences
• Sometimes other testers
• (if we have access to them)

4. What if we could get help?
• From an organization that can:
• Practice the craft of testing
• Learn from mentors from all over the
world
• Tackle interesting problems
• Encourage both novices and experienced
testers to participate
• Be available for no cost

5. Weekend Testing
• It already exists
• - http://www.weekendtesting.com/

6. Weekend Testers
• Who are we?
• A loose confederation of testers from around
the world
• Anchored by facilitators that lead testing
sessions
• A library of past session experiences and
transcripts
• A model for short and focused testing
sessions/learning opportunities

7. Benefits
• No default “schools”
• Learn and practice in a fail-safe environment
• Have an opportunity to give back to the
community
• Learn through collaboration
• Share experiences
• Learn about new tools and techniques
• Share and receive peer feedback

8. .
"Software development advances
when the cool kids get bored”
-- Michael Feathers (@mfeathers, 02/04/2012).
• Same can be said for testers and testing.

9. History
• Formed in Bangalore, India
• Testers looking for a better way to learn and
practice
• Breaking free of scripted methods
• Looking to learn from and teach each other

10. History
• First session held August 15th, 2009
• (Indian Independence Day)
• Founders liked that symbolic link.
• Felt they were giving "freedom" to Indian
testers.

11. Growth
• Over 2000 members and participants since its inception.
• More than 135 sessions conducted.
• Chapters span the globe:
• India
• Europe
• Australia/New Zealand
• Americas
• United Kingdom (Weeknight Testing)

12. Observation
Learning Clearing Traps
Presentation /
Questioning
Group Discussion
Automation Investigation
Bug Advocacy Note Taking
Modeling Bug Hunting
Collaboration

13. Facilitating
Key role in all sessions is the facilitator:
• Determines when sessions held.
• Announcement of sessions
• main Weekend Testing site
• Mailing list
• Twitter
• Chooses the topic of the session and application to test
• Announce charter and mission
• Guide testing and discussions that follow.

14. Initiating a Session
For Weekend Testing Americas sessions:
• The Facilitator announces a session (Email, Twitter, Skype,
etc.)
• The Tester adds the SkypeID of the Weekend Testing facilitator
(“weekendtestersamericas”) to their contact list.
• Tester sends an email or Skype message to let us know they
want to participate.
• Facilitator adds participants to a group chat through Skype.

15. Directing A Session
• Each session has similar steps
• Introductions
• Announce Mission and Charter(s)
• Open testing
(pairing encouraged)
• Regroup and discussion
• Closing
• Sessions are time bound at two hours
• Active testing during the 1st hour
• Discussion/debrief during the 2nd
hour
• Facilitators provide an experience
report
• Discussion of session and objectives,
lessons learned
• Provide a full chat transcript

16. Challenges
• Limited time and focus
• Missions must be focused to
fit the time.
• Standalone sessions
• Rare for topics to span
multiple sessions
• Continuity is an issue
• Limits on certain topics
• Automation can only be
covered at a fundamental
level
• Advanced topics that require
prior knowledge

17. Lessons Learned
Lesson 1: Geography Matters
Session: WTA-01: Let’s Dance
Product: Stepmania (PC Based Dance/Rhythm game)
Issues:
• Application files were easily downloaded by American testers
• Testers in India not able to download the application at all during
the course of the session.
Discoveries:
• Different geography areas have different levels of Internet
capacity
• If downloading an application, announce well in advance of the
session

18. Lessons Learned
Lesson 2: The Value of Session Based Testing
Session: WTA-08: Quoth the Developer, Nevermore!
Product: ribbit.cc (Software Developer Quote Generator)
• Explore the product and plan testing missions – charters.
• Each pair or team split off and worked on their own testing charters
• Each group then broke down the areas further and created specific
charters in those individual areas.
• At the end of the session, all charters and missions combined into one
document.
Through this, the session attendees decided to optimize their reporting
directly into the session notes by creating hash-tags:
• Bugs used the hash-tag #bug,
• Issues used the hash-tag #issue
• Charters were added with the hash-tag #charter.
• Focus areas within the charter were added with the hash-tag #area.

19. Lessons Learned
Lesson 3: Let Divide and Conquer Be Your Ally
Session: WTA-04: Put it on the Board
Application: Corkboard.me
Goal:
“Employ domain testing techniques (“Divide and Conquer The Data”) on the sticky notes in
cork-board application to find bugs.”
As a reference, we provided the participants with the following documents:
www.testingeducation.org/a/DTD&C.pdf
http://www.kaner.com/pdfs/GoodTest.pdf
• Attendees split up into small groups and each took a chunk of the testing puzzle.
• Smaller pairs or teams were able to effectively “Divide and Conquer” the application.
Discoveries:
• By divvying up the testing into slices, the potential for effective coverage goes way up
• While not complete coverage, it gets the group closer to that goal.

20. Lessons Learned
Lesson 4: Make Sure the Testers Understand the Mission
Session WTA-03: Rapid Reporter, Mapping the Testing Tool
Application: Rapid Reporter
Goal:
The stated mission was “Produce either a list or a mind map of the features of
Rapid Reporter with the goal of guiding a future session of Weekend Testing“.
• The mission could be interpreted two ways:
• 1. Understand the features of Rapid Reporter so that we could test Rapid Reporter
more thoroughly (i.e. directly testing Rapid Reporter).
• 2. Understand the features of Rapid Reporter so that we could test another
application and utilize Rapid Reporter in the process.
Discoveries:
• Make sure that all testers understand clearly the intention of the mission.
• Ask questions to help clarify the mission.

21. Lessons Learned
Lesson 5: The Challenges of Understanding Cultural Norms
Session: EWT-39: Let’s Date
Application: Calendar Conversion Functionality
Goal:
The application must support Nepali dates in combination to the standard
Gregorian calendar.
• Question: What makes the Nepalese calendar different?
• Nepalese calendar based off of the Bikram Samwat Calendar (lunar-solar
calendar), which is decided by a committee every decade.
• Dates twenty or 30 years in the future are impossible to guarantee.
Dicoveries:
• Realization that there was no discernable repeatable pattern to the dates
• Requirements of any tests may be based on erroneous information
• Promise of quality might be made that could not be delivered upon.
• Reminder of the value of “testing the requirements”.

22. Lessons Learned
Lesson 6: Usability Testing and the Value of the “Persona”
Session: WTA-12: What Are You Watching?
Product: Sidereel.com
Goal:
Have a look at the aspects of “user experience”.
• Different interpretations of what feels right in an application and
what doesn't.
Discoveries:
• Trying to get into the mind of who might be the primary user of the
SideReel product,
• Developing “Persona” roles of different users.
• A middle aged man & a teen-aged girl will have different
expectations of what’s acceptable and what’s not.

23. Lessons Learned
Lesson 7: Clear Communication of Issues and the Value of a Close Read
Session: WTANZ-11: The Critical Thinking Skill of Close Reading
Application: Bug Database
Goal:
Help testers recognize the value of a close reading and applying critical
thinking skills to the process.
• You have two bugs that have been merged
• Which of the two bugs should have been used?
• Which should have been marked duplicate?
• Should a third bug have been written?
Discoveries:
• We have conditioned ourselves to focus on particular areas.
• We often neglect to do a close read of the real issue,
• Reading the same issue at different times, we can get more information
on each read through.

24. Conclusion
• Weekend Testing has the power of harnessing and utilizing the
minds and efforts of passionate testers all over the world.
• The methods and processes used in Weekend Testing can also
provide a jump-start to any organizations testing efforts.
• Teams can develop skills, transfer knowledge train newer
testers with greater confidence and have fun in the process.
• For those who are curious about this approach, by all means
consider joining a session of Weekend Testing.
• Consider joining one of the chapters in your area, or even
starting one of your own.

25. Questions???