Quality Assurance is a common source of confusion in agile development. What is ‘quality’ and how do we ‘assure’ it? There’s more to it than just putting testers on the team. How do we know that we’re even building the right product? How do we avoid building up technical debt? When is it right to sacrifice quality? What is the right level of test coverage? In this presentation, Henrik looks at some common problems, patterns and misconceptions about agile quality assurance

1. Author
Father
Agile & Lean coach
www.crisp.se
Consultant
Henrik Kniberg
henrik.kniberg@crisp.se
@HenrikKniberg
How do you know that your
product works?
Colombo Agile Conf, June 2014

2. Henrik Kniberg
Todo
Doing
Done
?

3. Todo
Doing
Code
committed
Henrik Kniberg
Where’s my
product?
User
I’m done!
Developer

4. Henrik Kniberg
Code
committed
Tested
Where’s my
product?
User
We’re Done!
Developer

5. Henrik Kniberg
In
production
We’re Done!
Wait... is anyone
actually using it?
Code
committed
100% completed requirements
No reported defects
Tested

6. Henrik Kniberg
People
actually
use it
NOW we’re
surely done!
Aren’t we?
100% completed requirements
No reported defects
In
production
Code
committed
Tested

7. Henrik Kniberg
http://www.youtube.com/watch?v=FWTtcOQjOXI

8. Henrik Kniberg
People
actually
use it
100% completed requirements
No reported defects
In
production
Code
committed
Tested

9. Henrik Kniberg
People
actually
use it
100% completed requirements
No reported defects
In
production
Code
committed
Tested
Solves the
user’s
problem

10. Henrik Kniberg
Value of
your solution

11. Henrik Kniberg
Value of
your solution
Value of old
solution

12. Value of old
solution
Relative gain
Value of
your solution
Henrik Kniberg

13. Henrik Kniberg
Relative
Gain
Relative
Pain/Cost

14. Henrik Kniberg
People
actually
use it
In
production
Code
committed
Tested
Solves the
user’s
problem
... in a way
that is
better than
before
Relative
Gain
Relative
Pain/Cost

15. Henrik Kniberg
Relative
Gain
Relative
Pain/Cost
People
actually
use it
In
production
Code
committed
Tested
Solves the
user’s
problem
... in a way
that is
better than
before
... and
recommend it
to others
People use it
willingly

16. All products start with a Great Idea!
Henrik Kniberg

17. Risk
Henrik Kniberg
Business risk
Social risk
Cost & schedule risk
Technical risk

18. Henrik Kniberg
Google
Buzz
Google
Wave
Google
Answers

19. Assume you are building the WRONG thing!
• List your hypotheses
• Build an MVP to validate/invalidate it
• ”Minimum Viable Product” – the fastest & cheapest thing
you can build to field-test your hypothesis
Henrik Kniberg
Suppose we’re
building the wrong
thing....
How can we find out
as quickly as
possible?

20. Example:
Henrik Kniberg
2. Our product will
solve the problem
1. File sync is a
problem for people
Hypotheses
Demo video
3. People will want
to use our product

21. Henrik Kniberg

22. Dropbox MVP
Henrik Kniberg

23. Henrik Kniberg
Drew
Dropbox MVP
It totally blew us
away.
Our beta waiting list went from
5,000 people to 75,000
people literally overnight.
It drove hundreds of thousands of
people to the website.

24. Paper Prototyping = Lo-tech MVP
Credit: Ariel Waldman, on Interaction Design/ Rachel Ilan
http://chiefdisruptionofficer.com/helpful-rapid-prototyping-methods-and-tools-to-bring-digital-ideas-to-life-fast/

25. Pirate metrics
Henrik Kniberg
Aquisition
Activation
Retention
Referral
Revenue
AARRR!
Do people
come?
Do they use the
product?
Do they come
back?
Do they pay?
Do they recommend
it to others?

26. RISK
Big Bang = Big Risk
Henrik Kniberg
Cumulative
Value

27. Big Projects usually fail. Regardless of process.
Henrik Kniberg
< $1 million
> $1 million
”The secret to project success is to strongly
recommend and enforce limits on size
and complexity.”
”These two factors trump all other factors.”

28. Agile = Iterative + Incremental
Henrik Kniberg
Don’t try to get it all right
from the beginning
Don’t build it all at once
cost
value
cost
value
RISK

29. Release must be REALLY easy!
Henrik Kniberg
Req
Code
Test
Release!
Release = Drama!
Release = Routine
Release
often
Release is
small
Release
seldom
Release
is big
Release
is hard
Release is
simple

30. Fastest learner wins!
Delivery frequency = Speed of learning
Henrik Kniberg
Feedback,
Requests,
Data
Demos,
Releases
Development team
Stakeholders, Users

31. Improving the Value Curve
Henrik Kniberg
Big Bang
Big increments
Small increments
Highest value first
Value
Effort
RISK

32. Henrik Kniberg
1
1
2
3
2
3
4
5
4
Not like this....
Like this!

33. Henrik Kniberg
Maximize Value, not Output
High Output
High Value

34. Henrik Kniberg
MVP
Don’t focus on
maximizing
speed
Focus on
Minimizing
distance

35. Henrik Kniberg
Analyze data
Narrative &
Prototypes &
Metrics
Build MVP
Deploy
Tweak
“Radio you
can save!”
A/B stats
Idea/Problem
“Follow your
favorite artist”

36. Henrik Kniberg
User
Feedback
Product

37. Deliver, measure, adjust
continuously
How do you know that your product works?
Henrik Kniberg
ensure
1. Understand the
problem
2. Iterate until
you’ve solved it
Who are the
stakeholders?
What need do they have,
that we want to solve?
How will we know
when we’ve solved
it?
How will we know if we’re
moving in the right direction?
Minimize the distance to MVP