Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Designing to Learn
Creating Effective MVP Experiments
Melissa Perri
@lissijean
Consultant
Lean Product Manager
UX Designer
Teacher
Twitter addict (@lissijean)
Blogger (melissaperri.com)
Hi! I’m Melissa.
@lissijean
• What is an MVP (and why should we care)?
• Setting Up MVP Experiments
• Types of MVPs and How to Choose
• Ite...
@lissijean
Why MVP?
@lissijean
“Match your taste with
the exceptional finds of
industry insiders.”
@lissijean
@lissijean
My way is faster.
This isn’t what
I needed.
I don’t know how to
use it.
The information I want to
see isn’t on here.
@liss...
@lissijean
@lissijean
@lissijean
Validate
Before
Building
@lissijean
Build
Measure
Learn
@lissijean
Iterate your way to
Product/Market Fit
your users
@lissijean
MVPMinimum Viable Product
@lissijean
MVPSmallest amount of work you can do
to test your idea.
@lissijean
@lissijean
@lissijean
@lissijean
@lissijean
LEARN
@lissijean
MVP = Minimum Feature Set
Agile Methodology MVPs
@lissijean
Focus on problem
not features.
@lissijean
MYTH
@lissijean
Crappy products
Crappy products everywhere
@lissijean
Good MVPs offer a great
experience to the customer.
@lissijean
@lissijean
MVP
=
Experiment
@lissijean
Experiment Process
Set Up
!
Design
!
Run
!
Feedback
!
Iterate
@lissijean
@lissijean
Setting up MVP
Experiments
PART I
@lissijean
1. Define Customer & Problem
2. Investigate assumptions
3. Design a test
4. Measure customer behavior
5. Evaluat...
@lissijean
@lissijean
Define
Define
@lissijean
Customers want twitter-like
messages in their feed.
@lissijean
Customers want twitter-like
messages in their feed.
Not a problem.
@lissijean
Real Problem
Customers don’t think the
curators are authentic.
@lissijean
Investigate
@lissijean
Assumptions
Shoppers care
that curators
are authentic.
Shoppers will
engage more
with real
messages on
site.
Sh...
@lissijean
Riskiest Assumption
Shoppers care
that curators
are authentic.
Shoppers will
engage more
with curator
messages ...
@lissijean
Design a Test
@lissijean
Setting up a test
1. What do we want to learn?
2. How do we measure that?
3. What’s the least we can build to l...
@lissijean
Will shoppers engage more
with curator messages?
What do we want to learn?
@lissijean
Engagement
Clicks, purchases, added to cart
How do we measure that?
@lissijean
What is the least we can build to learn?
Wizard of Oz
Looks real to the customer,
completely manual on the back...
@lissijean
@lissijean
@lissijean
Measure
@lissijean
How do we measure
engagement?
@lissijean
Clicks on
product.
Purchases. Add to cart.
@lissijean
What do we consider
successful?
@lissijean
Minimum Success Criteria
What is the minimum number we need to
hit to consider this validated?
@lissijean
Time Salary# Peoplex x
Cost to build completely
@lissijean
Minimum
Success Criteria
Improvement needed
to justify cost to build.
@lissijean
25% increase in
engagement in A/B test.
@lissijean
@lissijean
Evaluate
@lissijean
@lissijean
0% change over other
products.
@lissijean
INVALIDATED
@lissijean
But why?
@lissijean
Why didn’t users engage?
What do we want to learn?
@lissijean
User Testing Sessions
watch them use it.
How do we find that out?
@lissijean
Emails.
@lissijean
Pivot
@lissijean
strategic change in direction
based on learning.
Pivot
@lissijean
Riskiest Assumption
Shoppers will
engage more
with curator
messages.
@lissijean
Will shoppers engage more
with curator messages
through email?
What do we want to learn?
@lissijean
Engagement
Opens, Click throughs, purchases.
How do we measure that?
@lissijean
What is the least we can build to learn?
@lissijean
How do we determine success?
30% increase in engagement over other emails.
@lissijean
Success!
2x Open Rate
3x Purchases
@lissijean
Now, we automate.
Now, we invest.
@lissijean
workshop one:
Setting up MVP Experiments
@lissijean
@lissijean
We’ve recently received a flurry of
complaints from Macy’s shoppers.
@lissijean
“Every time I come into Macy’s, I spend HOURS I
don’t have and can never seem to find what I’m
looking for. I kn...
@lissijean
“I needed a blue tie to go with my suit for a friend’s
wedding, so I went to find it but there were too
many opt...
@lissijean
Who is the customer?
@lissijean
“OpenSky shoppers who
visit the site at least once
a month, but rarely buy.”
@lissijean
What is their problem?
@lissijean
“Do not think the
curators are authentic.”
@lissijean
10 minutes
In your teams, write down your customer
and their problem for Macy’s.
@lissijean
Shoppers who are purchasing
something specific out of
necessity, not browsing.
Customer Hypothesis
@lissijean
Cannot find what they
need efficiently.
Problem Hypothesis
@lissijean
What are we assuming about
the customer and the problem?
@lissijean
Riskiest Assumption
What are we the most uncertain about?
@lissijean
10 minutes
In your teams, write down your
assumptions and pick the riskiest one.
@lissijean
Assumptions
Shoppers are
overwhelmed
by the amount
of options.
Shoppers do
not want to
browse.
Shoppers
want he...
@lissijean
Assumptions
Shoppers are
overwhelmed
by the amount
of options.
Shoppers do
not want to
browse.
Shoppers
want he...
@lissijean
Now, we design a test for
that riskiest assumption.
@lissijean
Types of MVPs
PART II
@lissijean
WHAT SHOULD YOU
ALWAYS DO FIRST?
@lissijean
Interviews
Get out of the building and talk to people.
@lissijean
• First thing you should do.
• Talk to at least 10 people.
• Verify the problem and the customer hypothesis.
• ...
@lissijean
Concierge
Delivering a service manually.
Delivering a service manually.
Concierge
@lissijean
@lissijean
@lissijean
The Good The Bad
• Very close to the
customer, easy to learn.
• Can adjust easily based on
feedback.
• Very eas...
@lissijean
• If you are testing a service.
• Trying to determine what customers will respond to.
• Testing and refining wit...
@lissijean
Wizard of Oz
Looks real, but manual on the back end.
@lissijean
@lissijean
Subscription
@lissijean
@lissijean
@lissijean
The Good The Bad
• Looks exactly real to
customers, so very
accurate response to if
you would build it.
• Need ...
@lissijean
• Running a test with a large audience.
• Trying to sell a feature or service before it is built,
but pretend i...
@lissijean
Landing Page
A pitch of your product, to gauge idea reception.
@lissijean
@lissijean
@lissijean
The Good The Bad
• Just a pitch, don't need to
refine the whole idea
• Can launch in a matter of
minutes
• MOST ...
@lissijean
• Looking for initial customers to test.
• Trying to gauge how much interest there is
in the idea.
• Great way ...
@lissijean
Videos
Demo what your product can do through hacking
it together for a video.
@lissijean
@lissijean
The Good The Bad
• Can demonstrate
complicated things.
• Show a prototype before
producing them all.
• More com...
@lissijean
• If your product does something very easily.
• When your product sounds complicated.
• When people need to see...
@lissijean
What if I’m not solving
a dire need?
@lissijean
workshop two:
Design your test.
@lissijean
You have already
interviewed your customer,
they have that problem.
@lissijean
Now how are you going
to solve that problem?
@lissijean
Choose an MVP strategy
and plan out a test.
30 minutes
• What you are trying to learn?
• Explain every little d...
@lissijean
Resources
Access to 2 Macy’s stores
Access to Macy’s online page
!
3 Store Interns $30,000 salary each
10 Sales...
@lissijean
Every time you start with
a minimum feature set,
a kitten dies.
@lissijean
Responding to
Constraints &
Feedback
PART III
@lissijean
The goal of an MVP is
to learn.
@lissijean
Customer Feedback
@lissijean
@lissijean
@lissijean
Constraints
@lissijean
Restricts medium.
Restricts resources.
@lissijean
workshop three:
Alter your test based on constraints.
@lissijean
You learned that people only have
this problem online.
How do you make an online MVP?
@lissijean
You learned that people only have this
problem in store.
How do you make an in person MVP?
@lissijean
How would you change your MVP
experiment based on these new
constraints?
5 minutes
@lissijean
MVPs in Agile Teams
PART IV
@lissijean
Gather Requirements
Spec
features
Design Interfaces
Developm
ent
QA Process
Release
Waterfall
@lissijean
Gather Requirements
Spec
features
Design Interfaces
Developm
ent
QA Process
Release
Product Manager UX Designer...
@lissijean
Agile
@lissijean
Individuals and Interactions
over
Processes and Tools
@lissijean
Agile Sprints
Short, goal oriented sprints formed around a
problem running Build, Measure, and Learn.
@lissijean
Build
Measure
Learn
@lissijean
Product Manager
UX Designer Developer
Small, cross-functional teams
@lissijean
@lissijean
monday
tuesday
wednesday
thursday
friday
Reflect & Define
Specify
Build & Refine
Build & Refine
Customer Feedback
S...
@lissijean
Problem
Exploration
MVP
Experiments
Feature
Development
@lissijean
MVPs are only a test.
@lissijean
“My solution is too obvious, if I just build it
people will use it.”
@lissijean
Customers will only use your
features if it fills a need.
Customers will only use your
features if it fills a nee...
@lissijean
“Lean is just for startups, my
company is too grown up.”
@lissijean
Lean applies to all companies.
@lissijean
Melissa Perri
!
Want an MVP Guide?
melissa@produxlabs.com
!
produxlabs.com
melissaperri.com
!
!
@lissijean
Prochain SlideShare
Chargement dans…5
×

Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

34 581 vues

Publié le

Designing MVP workshop for Lean UX NYC 2014.

Publié dans : Design, Technologie, Business
  • Login to see the comments

Designing To Learn: Creating Effective MVP Experiments - Lean UX NYC 2014

  1. 1. Designing to Learn Creating Effective MVP Experiments Melissa Perri @lissijean
  2. 2. Consultant Lean Product Manager UX Designer Teacher Twitter addict (@lissijean) Blogger (melissaperri.com) Hi! I’m Melissa.
  3. 3. @lissijean • What is an MVP (and why should we care)? • Setting Up MVP Experiments • Types of MVPs and How to Choose • Iterating on MVPs • Agile + MVP: How does it work together? Agenda
  4. 4. @lissijean Why MVP?
  5. 5. @lissijean “Match your taste with the exceptional finds of industry insiders.”
  6. 6. @lissijean
  7. 7. @lissijean
  8. 8. My way is faster. This isn’t what I needed. I don’t know how to use it. The information I want to see isn’t on here. @lissijean
  9. 9. @lissijean
  10. 10. @lissijean
  11. 11. @lissijean Validate Before Building
  12. 12. @lissijean Build Measure Learn
  13. 13. @lissijean Iterate your way to Product/Market Fit your users
  14. 14. @lissijean MVPMinimum Viable Product
  15. 15. @lissijean MVPSmallest amount of work you can do to test your idea.
  16. 16. @lissijean
  17. 17. @lissijean
  18. 18. @lissijean
  19. 19. @lissijean
  20. 20. @lissijean LEARN
  21. 21. @lissijean MVP = Minimum Feature Set Agile Methodology MVPs
  22. 22. @lissijean Focus on problem not features.
  23. 23. @lissijean MYTH
  24. 24. @lissijean Crappy products Crappy products everywhere
  25. 25. @lissijean Good MVPs offer a great experience to the customer.
  26. 26. @lissijean
  27. 27. @lissijean MVP = Experiment
  28. 28. @lissijean Experiment Process Set Up ! Design ! Run ! Feedback ! Iterate
  29. 29. @lissijean
  30. 30. @lissijean Setting up MVP Experiments PART I
  31. 31. @lissijean 1. Define Customer & Problem 2. Investigate assumptions 3. Design a test 4. Measure customer behavior 5. Evaluate success 6. Pivot or persevere Process
  32. 32. @lissijean
  33. 33. @lissijean Define Define
  34. 34. @lissijean Customers want twitter-like messages in their feed.
  35. 35. @lissijean Customers want twitter-like messages in their feed. Not a problem.
  36. 36. @lissijean Real Problem Customers don’t think the curators are authentic.
  37. 37. @lissijean Investigate
  38. 38. @lissijean Assumptions Shoppers care that curators are authentic. Shoppers will engage more with real messages on site. Shoppers think tweets are authentic.
  39. 39. @lissijean Riskiest Assumption Shoppers care that curators are authentic. Shoppers will engage more with curator messages on site. Shoppers think tweets are authentic.
  40. 40. @lissijean Design a Test
  41. 41. @lissijean Setting up a test 1. What do we want to learn? 2. How do we measure that? 3. What’s the least we can build to learn?
  42. 42. @lissijean Will shoppers engage more with curator messages? What do we want to learn?
  43. 43. @lissijean Engagement Clicks, purchases, added to cart How do we measure that?
  44. 44. @lissijean What is the least we can build to learn? Wizard of Oz Looks real to the customer, completely manual on the backend.
  45. 45. @lissijean
  46. 46. @lissijean
  47. 47. @lissijean Measure
  48. 48. @lissijean How do we measure engagement?
  49. 49. @lissijean Clicks on product. Purchases. Add to cart.
  50. 50. @lissijean What do we consider successful?
  51. 51. @lissijean Minimum Success Criteria What is the minimum number we need to hit to consider this validated?
  52. 52. @lissijean Time Salary# Peoplex x Cost to build completely
  53. 53. @lissijean Minimum Success Criteria Improvement needed to justify cost to build.
  54. 54. @lissijean 25% increase in engagement in A/B test.
  55. 55. @lissijean
  56. 56. @lissijean Evaluate
  57. 57. @lissijean
  58. 58. @lissijean 0% change over other products.
  59. 59. @lissijean INVALIDATED
  60. 60. @lissijean But why?
  61. 61. @lissijean Why didn’t users engage? What do we want to learn?
  62. 62. @lissijean User Testing Sessions watch them use it. How do we find that out?
  63. 63. @lissijean Emails.
  64. 64. @lissijean Pivot
  65. 65. @lissijean strategic change in direction based on learning. Pivot
  66. 66. @lissijean Riskiest Assumption Shoppers will engage more with curator messages.
  67. 67. @lissijean Will shoppers engage more with curator messages through email? What do we want to learn?
  68. 68. @lissijean Engagement Opens, Click throughs, purchases. How do we measure that?
  69. 69. @lissijean What is the least we can build to learn?
  70. 70. @lissijean How do we determine success? 30% increase in engagement over other emails.
  71. 71. @lissijean Success! 2x Open Rate 3x Purchases
  72. 72. @lissijean Now, we automate. Now, we invest.
  73. 73. @lissijean workshop one: Setting up MVP Experiments
  74. 74. @lissijean
  75. 75. @lissijean We’ve recently received a flurry of complaints from Macy’s shoppers.
  76. 76. @lissijean “Every time I come into Macy’s, I spend HOURS I don’t have and can never seem to find what I’m looking for. I know exactly what type of item I want, but options are all over the place. It’s daunting.”
  77. 77. @lissijean “I needed a blue tie to go with my suit for a friend’s wedding, so I went to find it but there were too many options. Where do I even start? I don’t even know how to choose.”
  78. 78. @lissijean Who is the customer?
  79. 79. @lissijean “OpenSky shoppers who visit the site at least once a month, but rarely buy.”
  80. 80. @lissijean What is their problem?
  81. 81. @lissijean “Do not think the curators are authentic.”
  82. 82. @lissijean 10 minutes In your teams, write down your customer and their problem for Macy’s.
  83. 83. @lissijean Shoppers who are purchasing something specific out of necessity, not browsing. Customer Hypothesis
  84. 84. @lissijean Cannot find what they need efficiently. Problem Hypothesis
  85. 85. @lissijean What are we assuming about the customer and the problem?
  86. 86. @lissijean Riskiest Assumption What are we the most uncertain about?
  87. 87. @lissijean 10 minutes In your teams, write down your assumptions and pick the riskiest one.
  88. 88. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  89. 89. @lissijean Assumptions Shoppers are overwhelmed by the amount of options. Shoppers do not want to browse. Shoppers want help finding items.
  90. 90. @lissijean Now, we design a test for that riskiest assumption.
  91. 91. @lissijean Types of MVPs PART II
  92. 92. @lissijean WHAT SHOULD YOU ALWAYS DO FIRST?
  93. 93. @lissijean Interviews Get out of the building and talk to people.
  94. 94. @lissijean • First thing you should do. • Talk to at least 10 people. • Verify the problem and the customer hypothesis. • Verify as many assumptions as you can. When to do Interviews
  95. 95. @lissijean Concierge Delivering a service manually. Delivering a service manually. Concierge
  96. 96. @lissijean
  97. 97. @lissijean
  98. 98. @lissijean The Good The Bad • Very close to the customer, easy to learn. • Can adjust easily based on feedback. • Very easy and cheap to get started. • Not scalable. • Time consuming to run. Concierge
  99. 99. @lissijean • If you are testing a service. • Trying to determine what customers will respond to. • Testing and refining with a small number of people. When to use Concierge.
  100. 100. @lissijean Wizard of Oz Looks real, but manual on the back end.
  101. 101. @lissijean
  102. 102. @lissijean Subscription
  103. 103. @lissijean
  104. 104. @lissijean
  105. 105. @lissijean The Good The Bad • Looks exactly real to customers, so very accurate response to if you would build it. • Need to code to get started. • Can’t tell you why customers don’t use it. Wizard of Oz
  106. 106. @lissijean • Running a test with a large audience. • Trying to sell a feature or service before it is built, but pretend it is already there. • If you are concerned about sacrificing brand. When to use Wizard of Oz.
  107. 107. @lissijean Landing Page A pitch of your product, to gauge idea reception.
  108. 108. @lissijean
  109. 109. @lissijean
  110. 110. @lissijean The Good The Bad • Just a pitch, don't need to refine the whole idea • Can launch in a matter of minutes • MOST PEOPLE FORGET THEY NEED TO DIRECT PEOPLE HERE THROUGH MARKETING. Landing Page
  111. 111. @lissijean • Looking for initial customers to test. • Trying to gauge how much interest there is in the idea. • Great way to pitch to a large audience. When to use Landing Page.
  112. 112. @lissijean Videos Demo what your product can do through hacking it together for a video.
  113. 113. @lissijean
  114. 114. @lissijean The Good The Bad • Can demonstrate complicated things. • Show a prototype before producing them all. • More complicated to put together. • Needs to be a certain amount done before you can make the video. Videos
  115. 115. @lissijean • If your product does something very easily. • When your product sounds complicated. • When people need to see it to believe it. When to use Videos.
  116. 116. @lissijean What if I’m not solving a dire need?
  117. 117. @lissijean workshop two: Design your test.
  118. 118. @lissijean You have already interviewed your customer, they have that problem.
  119. 119. @lissijean Now how are you going to solve that problem?
  120. 120. @lissijean Choose an MVP strategy and plan out a test. 30 minutes • What you are trying to learn? • Explain every little detail of the test. • What are you measuring? • Minimum success criteria.
  121. 121. @lissijean Resources Access to 2 Macy’s stores Access to Macy’s online page ! 3 Store Interns $30,000 salary each 10 Sales Associates In Store $40,000 salary each 5 Developers $100,000 salary each 1 Designer $100,000 salary Yourself $100,000 salary
  122. 122. @lissijean Every time you start with a minimum feature set, a kitten dies.
  123. 123. @lissijean Responding to Constraints & Feedback PART III
  124. 124. @lissijean The goal of an MVP is to learn.
  125. 125. @lissijean Customer Feedback
  126. 126. @lissijean
  127. 127. @lissijean
  128. 128. @lissijean Constraints
  129. 129. @lissijean Restricts medium. Restricts resources.
  130. 130. @lissijean workshop three: Alter your test based on constraints.
  131. 131. @lissijean You learned that people only have this problem online. How do you make an online MVP?
  132. 132. @lissijean You learned that people only have this problem in store. How do you make an in person MVP?
  133. 133. @lissijean How would you change your MVP experiment based on these new constraints? 5 minutes
  134. 134. @lissijean MVPs in Agile Teams PART IV
  135. 135. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Waterfall
  136. 136. @lissijean Gather Requirements Spec features Design Interfaces Developm ent QA Process Release Product Manager UX Designer Developer
  137. 137. @lissijean Agile
  138. 138. @lissijean Individuals and Interactions over Processes and Tools
  139. 139. @lissijean Agile Sprints Short, goal oriented sprints formed around a problem running Build, Measure, and Learn.
  140. 140. @lissijean Build Measure Learn
  141. 141. @lissijean Product Manager UX Designer Developer Small, cross-functional teams
  142. 142. @lissijean
  143. 143. @lissijean monday tuesday wednesday thursday friday Reflect & Define Specify Build & Refine Build & Refine Customer Feedback Sprint Structure
  144. 144. @lissijean Problem Exploration MVP Experiments Feature Development
  145. 145. @lissijean MVPs are only a test.
  146. 146. @lissijean “My solution is too obvious, if I just build it people will use it.”
  147. 147. @lissijean Customers will only use your features if it fills a need. Customers will only use your features if it fills a need.
  148. 148. @lissijean “Lean is just for startups, my company is too grown up.”
  149. 149. @lissijean Lean applies to all companies.
  150. 150. @lissijean Melissa Perri ! Want an MVP Guide? melissa@produxlabs.com ! produxlabs.com melissaperri.com ! ! @lissijean

×