Quick tips on how to improve your Lean experiments. Learn faster. Make better product decisions.

1. The Dos and Don’ts of
Hypothesis Testing
{
@ttorres
Teresa Torres
www.ProductTalk.org
!
!
!
!

2. Product & UX are going through a
mindset shift.
@ttorres

3. Old way:
Build 10 things, find out only 3 worked.
@ttorres

4. New way:
Run 10 experiments, build the 3 that worked.
@ttorres

5. You can waste thousands of dollars, hundreds of
hours, and countless sprints running
experiments that don’t
matter.
@ttorres

6. How to get the most from your experiments.
@ttorres

7. Know what you want to learn.
@ttorres

8. Don’t throw spaghetti at the wall and see
what sticks.
@ttorres

9. Understand what level you are testing.
@ttorres

10. Start with a testable hypothesis.
@ttorres

11. A good hypothesis can be supported or
refuted by an experiment.
@ttorres

12. The 5 components of a good hypothesis.
@ttorres

13. Have a clear reason for why you think the
change will drive the desired impact.
@ttorres

14. Choose the right participants.
@ttorres

15. Draw a line in the sand.
@ttorres

16. Start with your baseline.
@ttorres

17. Look at comparable changes.
@ttorres

18. Assess how much impact makes the
investment worth it.
@ttorres

19. Don’t stop your test at the wrong time.
@ttorres

20. Don’t test too many variations.
@ttorres

21. Mix-and-match methods to get the
answers you need.
@ttorres

22. Be mindful of the conclusions you draw.
@ttorres

23. Know what you want to learn.
• Don’t test everything.
• Understand which level you are testing.
Start with a testable hypothesis.
• Include the 5 components.
Have a clear reason why the change will have the desired impact.
Choose the right participants.
Draw a line in the sand.
• Start with your baseline.
• Look at comparable changes.
• Assess how much impact makes the investment worth it
Don’t stop your test at the wrong time.
Don’t test too many variations.
Mix-and-match methods to get the answers you need.
Be mindful of the conclusions you draw.
@ttorres

24. Let’s keep the
conversation going.
ProductTalk.org
@ttorres
Teresa Torres
Product Consultant & Coach
!
!
{