Lean how to maximize your learning by running effective experiments.

1. Learning What Works
Lean Startup Machine San Francisco

2. Zach Nies
@zachnies

4. At Rally, we believe empowered
people who actually want to
come to work are essential for
solving today’s big, complex
problems.

5. We inspire continuous
innovation and collaboration in
our customers, helping them
empower their people and
deliver a factor of four increase
in value to their business.

6. We do this through our world-
class software, coaching and
community services.

8. Social
MISSION

9. A
founda-on
for
the
mobilize
ci-zen
engineers
around
the
globe
with
our
employees,
customers
and
partners

11. Why I care
about startups

13. Font Sampler
http://media.kelbymediagroup.com/layersmagazine/images/columns/artoftype/may06/glyph-set.jpg http://en.wikipedia.org/wiki/File:Caslon-schriftmusterblatt.jpeg

15. 2,500,000
Customers
Revolutionized
an industry

18. h;p://www.flickr.com/photos/isaacmao/380257543

19. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

20. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

21. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

22. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

23. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

24. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

25. Market research
Product definition
Product requirements
Detailed plan
Alpha release
Beta release
Final release
FIRST CONTACT WITH CUSTOMERS

26. HOW MANY GUESSES?

27. FIRST CONTACT WITH
POTENTIAL CUSTOMERS

30. h;p://www.flickr.com/photos/oregondot/4132135156

31. Create a plan to
predict the future
h;p://www.flickr.com/photos/isaacmao/380257543

32. Why doesn’t
this work?

33. Obscure the guesses
with details

34. Discover the future
Vision
&
Empathy
Culture
Business
Model
Frame Frame Frame Frame
Build & Ship
Build & Ship
Build & Ship
Build & Ship
Learn
Learn
Learn
Learn
Test & Measure Test & Measure Test & Measure Test & Measure

35. Guesses
Empathize & Frame
Learn & Share
Build & Ship
Test & Measure
Knowledge

36. Design
Thinker
Scientist

38. h;p://www.richard-­‐feynman.net/gallery.htm

39. What big guesses?

40. First guess:
We understand the
problem a customer
wants solved

41. Next guess:
Our product solves
the customer’s
problem

42. Next guess:
Our business model
can be profitable

43. Next guess:
The founders can:
sell our offer
or acquire users

44. Next guess:
Understand the flow of
money and influence

45. Next guess:
Our draft sales plan
will repeatedly close
customers
or activate users

46. Next guess:
We can position
our company

47. Turn guesses
into knowledge

48. TAKE EACH MAJOR GUESS

49. h;p://www.youtube.com/watch?v=Iyjs3zeoSrM

50. Guesses
Knowledge

51. Empathize & Frame
Learn & Share
Build & Ship
Test & Measure

52. Empathy and insight

56. Building empathy
Connecting through conversation
Ask why
Discover their pain
Reveal their hopes and dreams
Dig for emotional stories

58. Framing the problem
Synthesize your conversation(s)
Needs (with verbs)
Insights (underlying drives/themes)
Point of View

64. Running experiments

65. How do I run
meaningful
experiments?

67. Experiment Name:
Background: What do you want to learn and why?
Owner:
Mentor: Date:
Measures
What will you measure to invalidate your hypothesis?
What will you measure to indicate the experiment is safe to run?
What will you measure to indicate you should amplify the experiment?
Measures can be Qualitative and Quantitative.
Frame the Experiment: What is your Problem Statement?
Write the Problem Statement from the Define worksheet here. What pain or problem is being experienced?
[Customer Segment] needs a way to [describe job to be done], (because|but|surprisingly) [describe insight]. Experiment Backlog
Stack ranked list of actions needed to run the experiment.
Hypothesis to Test
[Specific repeatable action] will create [expected result].
Is this hypothesis falsifiable?
Experiment Details
Describe the experiment you plan to run and how you are going to attempt to falsify your hypothesis.
Experiment Results and Learnings
Describe what you learned from the experiment? Did you invalidate your hypothesis or does it live on?
Safety: How is the experiment safe to run and how will you recover? Next Steps: Given what you learned, what’s next?
Describe how the experiment is safe to run.
Describe how you will recover from running the experiment upon completion or if you discover it isn’t safe to run.
I-2

68. Experiment Name:
Background: What do you want to learn and why?
Owner:
Mentor: Date:
Measures
What will you measure to invalidate your hypothesis?
What will you measure to indicate the experiment is safe to run?
What will you measure to indicate you should amplify the experiment?
Measures can be Qualitative and Quantitative.
Frame the Experiment: What is your Problem Statement?
Write the Problem Statement from the Define worksheet here. What pain or problem is being experienced?
[Customer Segment] needs a way to [describe job to be done], (because|but|surprisingly) [describe insight]. Experiment Backlog
Stack ranked list of actions needed to run the experiment.
Hypothesis to Test
[Specific repeatable action] will create [expected result].
Is this hypothesis falsifiable?
Experiment Details
Describe the experiment you plan to run and how you are going to attempt to falsify your hypothesis.
Experiment Results and Learnings
Describe what you learned from the experiment? Did you invalidate your hyp
Safety: How is the experiment safe to run and how will you recover? Next Steps: Given what you learned, what’s next?
Describe how the experiment is safe to run.
Describe how you will recover from running the experiment upon completion or if you discover it isn’t safe to run.

69. Experiment Name:
o learn and why?
Owner:
Mentor: Date:
Measures
What will you measure to invalidate your hypothesis?
What will you measure to indicate the experiment is safe to run?
What will you measure to indicate you should amplify the experiment?
Measures can be Qualitative and Quantitative.
our Problem Statement?
efine worksheet here. What pain or problem is being experienced?
cribe job to be done], (because|but|surprisingly) [describe insight]. Experiment Backlog
Stack ranked list of actions needed to run the experiment.
xpected result].
and how you are going to attempt to falsify your hypothesis.
Experiment Results and Learnings
Describe what you learned from the experiment? Did you invalidate your hypothesis or does it live on?
fe to run and how will you recover? Next Steps: Given what you learned, what’s next?
un.
ng the experiment upon completion or if you discover it isn’t safe to run.

71. “Get out of
the building!”
Steve Blank

72. “Meaning
emerges through
interaction”
Dave Snowden

73. What could go wrong
with this?
Empathize & Frame
Learn & Share
Build & Ship
Test & Measure

77. Why & Vision
h;p://www.flickr.com/photos/klearchos/4541072707

78. The good news

79. Big
Companies
h;p://www.flickr.com/photos/ironnickel/6076087519

80. Startups
h;p://www.flickr.com/photos/elcapitan/2387917709

81. Apply new
disciplines
to your startup

82. Become disciplined
explorers…

83. Don’t squander
your time

84. building
the wrong business

85. Do what you love

86. Love what you do

88. Books that informed this talk

89. People who informed this talk
David Eric Ash
Snowden Ries Maurya
George
Kembel
Geoffrey Steve
Moore Blank

90. h;p://www.flickr.com/photos/fla;op341/224597838

91. @zachnies
#LeanSF