1.
Welcome to Startup Testing Previewby The Testing Ground

2.
About me
•Founder of The Testing Ground and Co- Founder of Jumpdesk
•President of the Association of Lean Startups
•Organiser of Singapore Lean Startup Circle
•Mentor at Lean Startup Machine, Startup Leadership Programme, The Scape, Ideasinc, Startup@Singapore, Youth Entrepreneurship Symposium…
•Engineering degree, MBA…and just graduated from NUS law! Woohoo!

3.
Introduction to Lean Startup

4.
Definition of a Lean Startup
Validated Learning as the measure of progress
Customer Development
(discover unknown problem and solution)
Lean Product Development e.g. agile dev, design thinking, lean engineering
(create unknown product)
+

5.
From sequential activities …
…to an iterative process

6.
Why Lean Startup?
•Developed by Eric Riesalong with Steve Blank in response to all the waste that comes from failed startups.
•They realized that startups are taking too long developing their products only to launch them with no customers wanting to buy.

7.
Lean Startup

8.
A temporary organisation built to search for the answers to what makes a repeatable and scalable business model before running out of resources

9.
Atemporaryorganisation built to search for the answers to what makes a repeatable and scalable business model before running out of resources

10.
LeanStartup

11.
Lean= No Waste

12.
No Waste= figure out the right thing to build as quickly as possible

13.
No Waste= Figure out the right thing to build as quicklyas possible

14.
Waste= Not Figuring OutWaste= Building Wrong ThingWaste= Too Slow

15.
Figure Out= Validated Learning
No waste: figure out the right thing to build
as quickly as possible

16.
Lean Startup Concept: Build-Measure-Learn
No waste: figure out the right thing to build
as quickly as possible

17.
No waste: figure out the right thing to build
as quickly as possible

18.
No waste: figure out the right thing to build
as quickly as possible
= Experiment

19.
No waste: figure out the right thing to build
as quickly as possible
Image courtesy William Harris
Lean Startup is about making entrepreneurship into a management science

20.
No waste: figure out the right thing to build
as quickly as possible
Validated
Learning

21.
No waste: figure out the right thing to build
as quickly as possible
The Testing Board helps to keep track of your experiments

22.
As quickly as possible= speed
No waste: figure out the right thing to build
as quickly as possible

23.
No waste: figure out the right thing to build
as quicklyas possible
Traditional Development

24.
No waste: figure out the right thing to build
as quicklyas possible
Lean Startup

25.
Why?
You have more tries
Photo Credit: Roger Smith via Compfightcc 2.0

26.
Lean Startup Concept: Get out of the Building
No waste: figure out the right thing to build
as quickly as possible

27.
“There are no facts inside your building, so get outside
”
-Steve Blank
Photo Credit: http://www.troll.me/images2/steve-blank/keep-calm-and-get-out-of-the-building.jpg

28.
Testing Board
Level 101 introduction to conducting Lean Startup Experiments

29.
State your hypotheses and assumptions
Build your Experiment
Learn
The Testing Board helps you structure your experiments

30.
State your hypotheses and assumptions
Build your Experiment
Learn
Get rid of your biases:
1) State your hypotheses (guesses) and your assumptions (beliefs)
2) Forces you to take the view that your startup will fail

31.
State your hypotheses and assumptions
Build your Experiment
Learn
Provides Focus and Clarity
1)Know what you are building to test and how you are going to do it
2)Know what signal the customers must give you

32.
State your hypotheses and assumptions
Build your Experiment
Learn
Gives you the basis for the changes you are going to make to your business model

33.
Your potential customer:
Your Early-vangelistsaka Cookie Monster:

34.
You can’t reach out to everyone
•So break up a large customer segment into smaller ones
•“Characters living on Sesame Street”
•Big Bird
•Elmo
•Cookie Monster!!

35.
WHAT ARE PROBLEMS?
•Specific to the Customers
•“People have difficulty with transportation” is not specific
•“Motorists don’t know when is the best time to avoid a jam” is a problem

36.
WHAT ARE PROBLEMS?
•A problem is when a customer has:
•Difficulty doing a task
•Difficulty having a desireor needmet
•…in a particular situation

37.
TYPES OF PROBLEMS
Doing a Task
Gettinga Need or Desire Met
Sellingsomething
in a hurry
Highest Price
whenbroke
Findinga Date
when you are bald
Marryingthe right spouse
when very few choices are left
Buying a car
forthe first time
Status recognition
whenpeople are comparing

38.
18 TO 22 YEAR OLD UNIVERSITY STUDENTS
DIFFICULTY FINDING FOOD AT NIGHT FOR A PARTY
These are only your guesses! SO DON’T ARGUE WITH EACH OTHER

39.
Assumptions
•What do you believe is true so that your customer or problem hypothesis is true?

40.
Assumptions
•Two ways to figuring out assumptions:
•Explaining the hypothesis
•Failing the hypothesis

41.
Assumptions
Explain the Hypothesis

42.
Assumptions
We believe [CUSTOMER]
Has this [PROBLEM
Because
[reason 1]
[reason 2]
[reason 3]

43.
Assumptions
We believe [CUSTOMER]
Has this [PROBLEM
Because
[reason 1] = Assumption
[reason 2] = Assumption
[reason 3] = Assumption

44.
Assumptions
Force your startup to fail

45.
Assumptions
•My startup helping busy working womenwith their difficulty in finding cheap healthy food will fail because

46.
Assumptions
•My startup helping busy working womenwith their difficulty in finding cheap healthy food will fail because
•They don’t care about their health
•Someone else is cooking food for them
•They already find it easy to use google

47.
Assumptions
What would killmy startup idea
Therefore I am assuming that
They don’t care about their health
They care about their health
Someone else is cooking food for them
Noone else is cooking food for them
They already find it easy to use google
They don’t find googlingfor cheap food easy

48.
Listing Assumptions
•VERY IMPORTANT
•Forces you to face your cognitive biases (remember the blind man in a hotel?)

49.
Riskiest Assumption
Very Uncertain
Very Certain
Big Impact If Wrong
Small Impact If Wrong

50.
18 TO 22 YEAR OLD UNIVERSITY STUDENTS
DIFFICULTY FINDING FOOD AT NIGHT FOR A PARTY
THIS IS WHAT YOU BELIEVE IS TRUE ABOUT YOUR BELIEF
STUDENTS ORGANISE PARTIES AT THE LAST MINUTE
NO FOOD PLACES ON CAMPUS

51.
Minimum Viable Interaction
•What is the smallest interaction you can have with the customer to validate your riskiest assumption?
For now, it’s just a simple face to face interview

52.
Customer Acquisition Strategy
•How are you going to get the customers to carry out the MVI?
•E.g.
•Facebook invitation
•Networking events
•Referrals
For now, it’s just a simple “get out of the building” to talk to people off the streets

53.
Experiment Steps
•Forces you to state what you need to do and to time box it
•Creates
•Focus
•Urgency
•Common understanding between team members
For now, just state
1)Prepare interview questions (15 minutes)
2)Get out of the building for 2 hours
3)Find customers at 5 star hotels. 2 customers/30 minutes
4)Team member 1 go to Hotel A. Member 2 go to Hotel B ….

54.
Success Criteria
•What kind of signal are you expecting so that you have confidence to proceed?
Example:
I expect that 8 out of 10 university students have difficulty finding food at night for their last minute party because there are no food places on campus opened at night
Would you put your money down based on that signal?

55.
Problem Interviews

56.
Interviews are not surveys

57.
Interviews are not sales pitches

58.
This is what we want

59.
It’s about you learning from the customer the real truth…

60.
…even when it hurts to hear it

61.
It’s about letting the customers talk to you

62.
…and you listening

63.
http://www.wikihow.com/React-to-an-Ugly-Baby

64.
Big Steps
•Build
•Hypotheses
•Form your Questions
•Finding prospects
•Measure
•Carry out interviews
•Learn
•Consolidate learning and patterns

65.
let’s work now on your questions

66.
Testing a problem: The Mum Test (coined by Rob Fitzpatrick)
Can you ask questions such that even your mother won’t be able to lie to you?

67.
Carrying out The Mum Test
Talk about their life instead of your idea
•Otherwise Mum will always listen to you talk

68.
Carrying out The Mum Test
Ask about specifics in the past instead of generics or opinions about the future
•Mum will always say what you want to hear or cover up discouraging words with generic words
•“Tell me about the last time the problem happened”
•“What did you do?”

69.
Carrying out The Mum Test
Talk lessand listen more
“Mum, that’s interesting…tell me more…”

70.
Things to look out for
•Too many closed-end questions
•Only use them for qualifying.
•Start questions with What, Who, Why, When and How

71.
Things to look out for
•Avoid bad data by anchoring Fluff
•Fluff
•Generic claims (I usually, I always, I never)
•Future-tense promises (I would, I will)
•Dealing with Fluff
•Ask them to bring you to specifics in the past
•When it last happened and how they solved it.
•Avoid “would you ever” questions

72.
Do interviews in your team
•One asks questions. The others writes down answers and think about how to improve
•Change to another team member for each new customer

73.
Learning from the Experiment

74.
Result and Decision
•Result = What did you actually get?
•12/16 have the problem
•Decision:
•Persevere: Pass success criterion. Move to next stage
•Pivot: Did not pass success criterion. Repeat with a change in strategy
•Iterate: Data not enough. Repeat the experiment with more data points.

75.
Pattern

76.
Insight

77.
Insight

78.
Learning
•What other customer insights did you get?
•Alternatives
•Behaviours
•Competitors
•Mentality
•Bigger problems to solve

79.
All about Pivoting

80.
Building the Right thing = the thing customers want and will pay for
No waste: figure out the right thing to build
as quickly as possible

81.
But what do you do when you learn that the problem you wanted to solve isnot a problem?
No waste: figure out the right thing to build
as quickly as possible

82.
Lean Startup Concept : Pivoting
No waste: figure out the right thing to build
as quickly as possible

83.
Pivoting= A change in your business model without a change in visionbased on validated learning
No waste: figure out the right thing to build
as quickly as possible

84.
Group Action
For Changing The World
Group Action
For Buying Cheaper Stuff
Pivot

85.
The key to pivoting is thus your vision

86.
Product = Vision

87.
Product= Vision

88.
Big Vision
Problem 1
Product 1
Product 2
Problem 2
Product 3
Product 4
Problem 3
Product 5

89.
Types of Pivots

90.
Zoom-In Pivot
A single feature become the whole product
E.g. From Electric Car to just Electric Car Batteries

91.
Zoom-Out Pivot
A whole product becomes a single feature of a much larger product
E.g. Galaxy Note –WalcomDigitizer

92.
Customer Segment Pivot
Attracted real customers…but not the original ones

93.
Customer Problem/Need Pivot
Original problem not big enough but another problem is bigger for that customer segment

94.
Put Customer and Problem Together
How you are going to solve the problem

95.
Solution versus Product

96.
A is howyou are going to solvethe key problem the customer is facing

97.
A is what you are using to solvethe key problem the customer is facing

98.
A focuses on the customers and about the benefitsthey receive

99.
A focuses on the features which customers use to get the benefits

100.
A can have be made up of many products

101.
A may not be a solution

102.
A can be described with just wordsfor customers to give you feedback

103.
A is hard to describe and is better experienced before the customer can give you feedback

104.
Problem: Commuters spend a lot of time waiting for buses
Solution: Information about bus arrival time
Products:
1)Bus arrival guide
2)SMS about bus arrival
3)App with real time information based on GPS

105.
Problem: Commuters spend a lot of time waiting for buses
Solution: Share a cab service
Products:
1)Online booking
2)Call a telephone operator
3)Booking app

106.
Problem: Commuters spend a lot of time waiting for buses
Solution: Tell you when to leave work to avoid crowd
Products:
1)Whatsappgroup
2)Blog
3)Phonecall

107.
So it’s more important to think about the solution first.
Then you build a Minimum Viable Product to test the solution with the Customers

108.
Building a Successful MVP

109.
What is an MVP?
“The minimum amount of effort you have to do to complete exactly one turn of the Build-Measure-Learn feedback loop.”
-Eric Ries

110.
In Simpler Terms
The MVP is the fastest way to achieve learning about the customer with the least effort

111.
Pre-selling MVP
Exchange of product for some form of currency: time, money, information or work.
Tests the Riskiest Assumption associated with your Solution Hypothesis.

112.
Product Pitch

113.
One easy way to do
the pre-selling MVP…
Landing Pages
Determine if the problem resonates enough with the customer that they will give up currency

114.
Landing Page

115.
Offline Landing Page

116.
Storyboard of problem areas

117.
Pre-selling MVI –What Do You Want to Get?
* Cash $$$
* Letter of Intent
* Email Addresses
* Pay With a Tweet
* Taking a Meeting
* Time

118.
CURRENCY
PERSONAL INFORMATION
-Email Address
-Mobile Number
WORK
-Effort to make an appointment to meet me
-Time spent for the meeting

119.
131

120.
Lean Startup is a like a straight ruler

121.
Measure of Success
Time
Failure without Lean
Failure with Lean

122.
Measure of Success
Time
Small Failures
But Learning
Quickly
= Changes in Strategy without Change in Vision
= Pivoting
Finding Product-Market Fit
Getting to Success
Small Failures
But Still Learning
Quickly

123.
But beware, Lean Startup can be a bit confusing
Strategy Level
Tactical Level
Product Level

124.
oFor Seed Stage Startups
o20 –Day Programme to develop your customer development skills
oIncludes the use of a space at Orchard Road
oHelps you answer three important questions at the end:
Is the problem worth solving?
Does any one want your solution?
Do you want to do it?

125.
oGet the focus
oComes with a 150 page guidebook and Startup Testing Boards
o6 hours of face to face workshops together with other startups in the region
oMentoring with lead mentors and participant mentors
oSave money
oGet access to Powtoons
oYour own urlshortener
oLanding page creators
oComes with the use of TTG
oJoin a community
oMember of Association of Lean Startups
oAlumni of TTG

126.
Join us atwww.thetestingground.asia