Honest Experimentation by Jonathan Bertfield
•Télécharger en tant que PPTX, PDF•
0 j'aime•721 vues
Honest Experimentation by Jonathan Bertfield @Agile Israel 2019
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Honest Experimentation by Jonathan Bertfield
Similaire à Honest Experimentation by Jonathan Bertfield (20)
Plus de AgileSparks
Plus de AgileSparks (20)
Dernier
Dernier (20)
Honest Experimentation by Jonathan Bertfield
- 5. courage to be transparent
- 8. lean startup execution reality
- 9. experiment principles SMALL BATCHES MOVING FROM DOUBT TOWARDS CERTAINTY
- 11. experiment calculus LIKELIH OOD OF SUCCE SS QUANTITY OF LEARNING QUALITY OF LEARNING X=
- 12. what we get from experiments
- 13. from hypothesis to experiment MEAS URE BUIL D LEAR N What do we need to build to generate the data we can measure What data will allow us to measure this learning What do we need to learn to validate or invalidate our riskiest assumption
- 14. what is a hypothesis? “If then” statement that helps design tests for an assumption Clarifies your current understanding of what uncertainty you seek to resolve Is specific in the action, timing and value / amount of impact
- 15. Assumption vs. Hypothesis “I believe that better photos of the listings will improve the visibility and desirability of those listings when compared to those with owner-generated photos” ——————————————————— "If we take professional photos for listings, those listings will get 2-3 times more business than others in the first 60 days of the experiment.” Assumption Hypothesis
- 16. experiment structure ▸The hypothesis we believe to be true ▸What we will do to test the hypothesis ▸What will we measure to validate or invalidate the hypothesis ▸What threshold for success will we look to
- 17. experiment structure ▸We believe that urban vehicle owners in India place a high value on saving time when refueling and will sign up for auto payment so they can refuel quicker. ▸We will set up an express lane for scooters / motorbikes at 1 filling station for 1 day to simulate an auto payment sign up and payment process ▸We will measure how many riders participate in the sign up process and how long it takes to convert each rider ▸We will be successful if we can convince 25% of riders to participate and take an average of less than 3 minutes to convert riders
- 18. concierge mvp
- 19. concierge mvp
- 20. mvp case study CONDUIT CASE STUDY US DEPT OF DEFENSE
- 21. Problem interview - Card Sorting The team sends more relevant products. The team finds products from fewer sites. The team spends less time reformatting. The team sends more timely products. The team spends more time interacting with briefers. The team spends less time on daily product dissemination, more time on liaison work The team is more involved in daily product dissemination. The team targets product dissemination by topic area. The team misses fewer products in "the funnel." The team sends more relevant products. The team finds products from fewer sites. The team spends less time reformatting. The team sends more timely products. The team spends more time interacting with briefers. The team spends less time on daily product dissemination, more time on liaison work The team is more involved in daily product dissemination. The team targets product dissemination by topic area. The team misses fewer products in "the funnel." our hypotheses… the users… What do our users value most?
- 25. experiment structure ▸We believe that a wide community of DoD analysts will value the ability to quickly synthesize content into briefing ready reports ▸We will run an publicity campaign on internal DoD web channels to drive traffic to a landing page where we will capture interest ▸We will measure views and signups for pilot ▸Success will be 2000 views and 20 sign ups within 1 week
- 26. Landing page + explainer video
- 28. 3D modelling
- 29. value pitch
- 30. Thank you JONATHANBERTFIELD JONATHAN@MYSPINNAKER.CO WWW.MYSPINNAKER.CO bit.ly/leadinginnovationIs Join us tomorrow in Herzilya for a full day workshop on leading innovation
Notes de l'éditeur
- Good morning everyone. My name is Sonja Kresojevic, I am an executive, an entrepreneur (co-founder of NY/London based strategy and innovation consultancy, speaker, writer, feminist, liberal, digital nomad with three passports, mom to two amazing kids. 6 countries, 10 cities, 30 apartments, 2 masters, 3 citizenships, 20 years of global corporate
- a lot of these ideas come together in thinking about risk. How do we minimize risk, how do we maximize value of the least amount of investment. the longer a product is kept behind closed doors striving for perfection, the higher the risk that we are exposed to in the event that we failed to understand the market, the user, the pricing, the messaging.
- a lot of these ideas come together in thinking about risk. How do we minimize risk, how do we maximize value of the least amount of investment. the longer a product is kept behind closed doors striving for perfection, the higher the risk that we are exposed to in the event that we failed to understand the market, the user, the pricing, the messaging. Question for room : who has shipped something that customers didn’t want. Only we found that out after shipping
- conversely, every time we run a small bet, let an experiment out into the wild, test an assumption, we de risk the product and increase the likelihood that what ends up reaching the mass market is in fact road tested and has a much higher chance of winning whatever fight it was intended to target.
- Uncertainty is key here because it means we can transfer practices from other domains into this context. Where there are too many unknown unknowns we have to operate differently than where we are faced with mostly known knowns some known unknowns . there it is effectively a question of execution - How do we do this thing. In startup land the first order question is should we do this thing.
- Ask: So what is a hypothesis? Well, If assumptions are our guesses, then hypotheses are what we use to test for “truth”. You are clarifying your understanding of the uncertainty you are hoping to resolve. The hypothesis needs to be specific in the action, timing and value/amount of impact…without these elements it will be difficult to measure the success of your test… Say: I’d like to show you a video from Airbnb, discussing an experiment they ran to test an assumption…
- Say: So, this is how Airbnb’s Assumption compares to their hypothesis, which helped them define their experiment and obsevre the behavioral change of their customers. Say: And this is also how you can begin to think of your experiments as a vehicle by which you can test your assumptions.