The software testing business is in the grip of a commoditization trend in which enterprises routinely flip flop between vendors—vendors who are engaged in a race to the bottom on price. This trend introduces perverse incentives for service providers, undervalues skill, and places excessive emphasis on processes, tools, and methods. The result is a dumbing down of testing and the creation of testing services that are little more than placebos. Using examples drawn from three recent projects in the banking industry, Iain McCowatt explores the dynamics of commoditization and introduces a quality model that can be used for framing the value of testing services. As a testing vendor, learn how to pursue a differentiation strategy, shifting the emphasis of the testing conversation from cost to value; as a customer of testing, learn how to make informed decisions about the value of what you are buying; as a tester, learn how to buck the trend and find professional growth.

1. Changing the Testing Conversation
from Cost to Value
Why testing has become a commodity, and what you can do about it
© Can Stock Photo Inc. / frenta
Iain McCowatt
http://exploringuncertainty.com
iain@exploringuncertainty.com
@imccowatt
imccowatt

2. Introduction
• What this presentation is
• What this presentation is not

3. Project #1

4. Project #1
•
•
•
•
•
An opportunity
Staffing decisions
Things start to unravel
Micromanagement, master/slave
Focus on process, KPIs galore
• And ultimately…

5. Project #1
GAME OVER
• This wasn’t the first testing vendor
• Nor were they the last
• There will be more

6. Commoditization: Introduction
• Investopedia:
– When a product becomes indistinguishable from
others like it and consumers buy on price alone, it
becomes a commodity

7. Commoditization: Where?
• More common in the enterprise, particularly
in organizations with global buying power
• Endemic in the consulting industry
• May be less common amongst software
vendors

8. Commoditization: Origins
• How to run fast: put one foot in front of the
other, quickly
• Who here could win Olympic gold?

9. Commoditization: Origins
• A lot of people see testing like this:
Test Plan
Write scripts
Execute Cycle 1
Execute Cycle 2
Execute Cycle 3
Sign off

10. Commoditization: Dynamics
Demand for
skilled testers
Testing
failures
Collaboration
Perceived
as low skill
Perceived
as commodity
Supply of
cheap testing
Supply of
skilled testers
Negative effect
Choice of effect
Demand for
cheap testing

11. Commoditization: Dynamics
Vendor A competes
on value
Vendor A competes
on price
Vendor B competes
on value
Both stand to win
long term
relationships
Customers get better
testing
Vendor B competes
on price
Vendor A: loses
market share
Vendor B: grows
market share
Vendor A: grows
market share
Vendor B: loses
market share
Race to the bottom:
short term revenue,
short term
relationships
Everyone loses

12. Commoditization: Consequences
•
•
•
•
•
•
Master/Slave, chilling effects on collaboration
Information starvation
Fungibility and economic incentive to juniorize
Focus on control, process and method
Demand for skill suppressed
…and ultimately the projects suffer

13. Project #2

14. Project #2
•
•
•
•
•
What’s the problem?
Your competition can do it cheaper
No budget!
If IBM said it, it must be true
Let’s give it a try
• That was a leap of faith, but…

15. Project #2
A Happy Client
• A second year, a second release
• A growing portfolio

16. Spot the Difference
• This was not a commodity testing service.
• But what was different?
http://freear.org.uk/nick

17. Sources of Differentiation
• People
– Passion
– Skill
– Creativity
• Relationships and trust
• New ideas, new technologies
• Location

18. NOT Sources of Differentiation
•
•
•
•
•
•
Process
Methodology, branded or otherwise
Templates and other “accelerators”
Tools and technology, unless new and unique
ISO, CMMI etc.
Just about anything easily replicated or at
home in buzzword bingo

19. Project #3

20. Project #3
•
•
•
•
•
Passion speaks volumes
The managed service misnomer
Early conversations, aspects of quality
Where’s the code?
Tests as experiments
• Just getting started, but…

21. Project #3
The Future Looks Bright
• Taking a very different approach predecessor
• Building on a solid relationships
• And pricing? Part of the conversation, but not
defining it

22. Reframing the Conversation
• We focus on software quality but rarely
discuss the quality of testing
• Perhaps a quality model might help?
• This model formed part of the conversation on
Project #3…

23. A Model of Testing Quality

24. What can you do?

25. Customer
• Observe vendor behavior: are they interested
in you and your problem?
• Do they understand your problem?
• Do they care?
• Do they have any ideas?
• Do they have the skills to deliver?

26. Vendor
• Dare to be different
• Sack the salesmen, consult
• Be prepared to say “No”:
– If they want it cheap, let them go somewhere else
– Don’t follow anyone off a ledge
• Education, education, education
• Shared values => relationship => business
worth doing

27. Tester
•
•
•
•
Empowerment starts with YOU
Don’t wait to be given a learning agenda
Learn how to learn
Learn anything and everything that might help
you
• Select, remix, invent

28. A Final Word
• Commodity testing isn’t going anywhere: both
demand and supply will remain
• Some of us (vendors) are successfully
exploiting this to build differentiated services
• This shows there is a demand for something
better
• Skilled testers can find fulfilling roles

29. Questions?