How good is your testing? Do you know? And if you do, can you explain why? One way to measure the success of your testing is to compare the information you now have about your product, to the information you intended to gather when you started testing. That intention is what your test strategy should capture and communicate. That last word is important: communicate. Even the best testing in the world isn't worth anything if you can't communicate what you're going to do, what you've done, and why. In this presentation I will show how I think about, model and visualize a strategy that is useful and that lets me later assess how well testing is fulfilling its mission, whether it's for hands-on testing that I'm doing to do on that particular day, or if I'm managing a multi-site project with cross-functional agile teams and need to construct a high-level starting point for test planning for the coming 6 months. Furthermore, I will discuss how to divide the strategy work between different project members and how to make sure that the quality assessments and reporting can be made valuable and, above all else, actionable by the project manager and product owners. This talk is directed at all types of project members, managers and other stakeholders who works with product quality.

1. The Secret to Successful Testing
Johan Jonasson

2. Johan Jonasson
• Consultant, Speaker, Trainer & Coach
• From Linköping, Sweden
• Test professional since 2006
• Passionate about community building
• Engaged in non-profit organizations
focused on improving the testing craft

3. Successful Testing
What does it even mean to test successfully?

4. Recognizing Good Testing
An inquisitive mindset over a confirmatory one
Adaptable planning over rigid plans
Deliberate testing over following a script
Attention to opportunity cost over testing everything exhaustively
Focus on the testing mission over chasing your favorite type of bug
Tell the story of your testing over relying on potentially vacuous metrics

5. The Secret to Successful Testing
• Have a strategy that supports your mission
• Always be ready for new discoveries
• Surround yourself with good people
• Make sure your strategy doesn’t fail

6. Strategy
• ”Immature strategy is the cause of grief.”
– Miyamoto Musashi
• Mindset: Intent on learning
• What – Why – How

7. Test Strategy
Product
Model
Information
goal
Risk
Test idea
Test idea
Charters
Scenarios
Role-based
Automated
tests
Etc...
Impediment
Information
goal
Task
Question
Assumption
Need
TeamTest Lead
(Product Owner)
Tester

8. Becoming a Super Hero Tester
• When you don't know what to do, please ask someone
• When something is not right, please speak up
• When your boss is being an idiot, please tell them (nicely)
• When you notice you won’t be able to finish what you've promised on time,
please let your stakeholders know, while there’s still time to mitigate, re-
negotiate scope, delegate or even just prepare for the effects of a delay
• Speaking up is (should) always preferable to keeping quiet

9. Why We Still Fail (sometimes)
• We are irrational
• Dan Ariely’s Behavioural Economics experiments
• We often react intuitively or emotion-driven
• Thinking Fast & Slow
• The Elephant & The Rider
• Drive – Autonomy, Mastery, Purpose
• Somatic marker hypothesis
• We are creatures of habit
• Change is scary, routines create a sense of security
• Change might not be fast, and it isn’t easy, but with
time and effort, almost any habit can be reframed.

10. Be Aware of Your Biases
• We’re all biased
• Tendencies to act in an irrational way
• Can lead to poor decision making
• Knowing them, and recognizing which
ones you yourself are most affected by,
can improve your chances of making
more rational decisions

11. Autonomy
PurposeMastery
Embrace Emotions
• Direct the Rider
• Motivate the Elephant
• Shape the Path

12. The Power of Habit
• The brain is constantly looking to
save effort
• By chunking actions into routines
(habits), the brain saves energy
• Cravings reinforce habits
• Insert new routine in the
cue/reward to change a habit

13. Keystone Habit
• Keystone habits start a process that, over time, changes everything
• Change one or a few key things to work as a lever
• Wrap new habits in between familiar ones to help adoption
• Small wins -> new structures -> contagious cultural change
• Test Impact as a Keystone Habit of testing

14. Test Impact as a Keystone Habit
• != Test Impact Analysis (TIA)1
• Why impact and not risk?
• Consider, not just code, but many
different angles and models
• Let the team use their own language
• Start small, sandwich between other
analyses already being performed
1 The Rise of Test Impact Analysis: https://goo.gl/KKiqft
Capability
Reliability
Usability
Security
Scalability
Performance
Installability
Compatibility
Supportability
Testability
Localizability
Maintainability
Portability
Customers
Information
Developer relations
Test team
Equipment & tools
Schedule
Test items
Deliverables
Quality AspectsProject Aspects
Heuristic Test Strategy: http://www.satisfice.com/tools/htsm.pdf

15. Closing thoughts
• Multi-level strategy
• Explicit responsibilities
• Never stay silent, and embrace your inner super hero tester
• Know your biases
• Direct the Rider, Motivate the Elephant, Shape the path
• Sandwiched Keystone Habits

16. Suggested Reading

17. Questions?
Feel free to connect online for life time support on this material
About.me: https://about.me/johanjonasson
Web: http://www.johanjonasson.com
LinkedIn: http://linkedin.com/in/johanjonasson
Twitter: https://twitter.com/johanjonasson