From your own experience it might not come as a surprise that most of today’s testing is unhelpful, filled with unnecessary paper work and folkloric activities. For some reason testing work often does not seem to be very helpful in projects. That is definitely a problem. If you are a tester, your manager might ask you for metrics that don’t make sense to you. And since you are a smart person, you have probably once in a while gamed the system. All that is certainly damaging to the industry. What can you do? This session brings you insight into Complexity Thinking with Dave Snowden’s Cynefin model and ties that to your job as a software tester. It offers you a way to look at software testing from a complexity thinking standpoint of view and gives you tools to argue your case if you are exposed to dysfunctional project settings. In addition to that, we will have some fun with idiotic metrics and to lighten up the serious topic we’ll engage in hilariously entertaining real life examples of bad metrics. To round it up, we’ll propose more meaningful alternatives.
3. Who Am I?
Who – Problem – The Model –Application – Metrics – Action – End
4. llari Henrik Aegerter
Managing Director – House of Test
VP of Marketing – Association for Software Testing
Beer Brewer
@ilarihenrik
www.houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End
5. Disclaimer:
Keep in mind that I am a
*%§!$ Consultant
Question everything I say!
Who – Problem – The Model –Application – Metrics – Action – End
6. What is the problem?
Who – Problem – The Model –Application – Metrics – Action – End
7. We are applying methods in
software testing we wouldn’t in
comparable other situations
Who – Problem – The Model –Application – Metrics – Action – End
8. Who – Problem – The Model –Application – Metrics – Action – End
10. Confusion between
Complicated vs. Complex
Complicated:
Degree to which something is difficult to
understand
Complex:
Degree to which Cause and Effect
are deterministic
Who – Problem – The Model –Application – Metrics – Action – End
11. Who – Problem – The Model –Application – Metrics – Action – End
13. How to apply Cynefin in
Testing
Who – Problem – The Model –Application – Metrics – Action – End
14. When confronted with a
testing task, ask yourself
how to act
Who – Problem – The Model –Application – Metrics – Action – End
15. Obvious
Sense - Categorize – Respond
World Expample:
Traffic Light
Testing Example:
Checking GUI Elements
Who – Problem – The Model –Application – Metrics – Action – End
16. Complicated
Sense - Analyze – Respond
World Expample:
Building a Space Ship
Testing Example:
Analysis of non-trivial
calculations
Who – Problem – The Model –Application – Metrics – Action – End
17. Complex
Probe - Sense – Respond
World Expample:
A roundabout
Testing Example:
Almost everything in testing,
Unknown unknowns, Exploratory
Testing
Who – Problem – The Model –Application – Metrics – Action – End
18. Safe to Fail Experiments
Who – Problem – The Model –Application – Metrics – Action – End
19. Chaos
Act - Sense – Respond
World Expample:
Kids
Testing Example:
A project in disarray
Who – Problem – The Model –Application – Metrics – Action – End
20. Some words about metrics
Who – Problem – The Model –Application – Metrics – Action – End
21. Cem Kaner on Metrics and
Measurements
http://kaner.com/pdfs/PracticalApproachToSoftwareMetrics.pdf
Who – Problem – The Model –Application – Metrics – Action – End
22. A General Problem with Metrics
Question is about Quality -
Measurements are Quantitative
Hence your measurement is a
surrogate one
Who – Problem – The Model –Application – Metrics – Action – End
24. Who – Problem – The Model –Application – Metrics – Action – End
25. Examples of useless Metrics
- Number of Test Cases
- Number of Bugs found
- Degree (in %) of Automation
Example: Percentage of manual TC
”transformed” into Automation
→ coverage is always according to a model,
not the software itself
- ““““Test Progress” i.e. “number of “test
cases” “executed””””
Who – Problem – The Model –Application – Metrics – Action – End
26. Who – Problem – The Model –Application – Metrics – Action – End
27. Who – Problem – The Model –Application – Metrics – Action – End
28. Who – Problem – The Model –Application – Metrics – Action – End
29. Who – Problem – The Model –Application – Metrics – Action – End
30. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics
Who – Problem – The Model –Application – Metrics – Action – End
31. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month
Who – Problem – The Model –Application – Metrics – Action – End
32. Here’s a useful metric:
Number of managers fired for
asking their employees to
follow idiotic metrics, per
month, per tester
Who – Problem – The Model –Application – Metrics – Action – End
33. Call to Action
Who – Problem – The Model –Application – Metrics – Action – End
34. Study Cynefin to get a
better understanding of
complexity
Who – Problem – The Model –Application – Metrics – Action – End
35. Keep in mind that most of
testing belongs in the
complex area
Who – Problem – The Model –Application – Metrics – Action – End
36. Push back on invalid
demands
Who – Problem – The Model –Application – Metrics – Action – End
37. Good Luck on Your
Journey!
Who – Problem – The Model –Application – Metrics – Action – End
38. And Now Your Questions
Thanks!
@ilarihenrik
www.houseoftest.ch
ilari.aegerter@houseoftest.ch
Who – Problem – The Model –Application – Metrics – Action – End