Ton Wesseling / @tonw Keynote - My Daily Job as a growth hacker / conversion rate optmizers
1. ONLINE D IALOGU E
G u i d i n g y o u t o d i g i t a l m a t u r i t y
To n W e s s e l i n g – G r o w t h H a c k e r
# e l i t e c a m p E s t o n i a 2 0 1 4 / 0 6 / 1 4
My D aily Job
42. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
But first: statistics - history
ü 4 out of 10 tests were successful
ü With a 90% significance level
ü Purchase cycle = 3 days
ü Thousands room bookings per week
43. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Power & Significance
ü 10 wrong winners every 100 tests
ü Only +-65% of these winners were uncovered
ü 40 winners = 30 real winners, so
ü 45 winning ideas, 55 losing ideas
ü Lucky: 75% of the winners is real
44. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Cookie deletion & cross device
ü 90% significance is ok
ü We can stay at this low power level
ü 2 weeks max test-time in this case (to
reduce sample pollution)
ü Only test 1 type of device at the same time
48. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Learn
ü If a new functionality should be build
ü What message really impacts behavior
ü What we should feed the bandit
Always Re-test before you decide on these 2:
52. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
No room available
§ 15% sees a no room available page
§ Conversion rate of them is 50% lower
§ Low number of conversions per week
§ 2 weeks testing – low power warning
§ Pontentially high impact
54. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Test protocol
ü Put the control live a week before test
ü Always use a “copy control” as control
ü After the test period – new traffic to
control – for again 1 week
(should be: 1 purchase cycle minimum)
60. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Scarcity data
§ 30% sees rooms with scarcity
§ Conversion rate of them is the same
§ A lot of conversions per week
§ 2 weeks testing – pretty OK power
66. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
The results
ü Yes, significant result again
ü The other 2 messages are also significant
ü But : the combination is even significant
better than the scarcity version itself!
67. Email: ton@onlinedialogue.com
onlinedialogue.com
Twitter: @TonW #elitecamp
Actions
ü Build scarcity in the CMS system
ü Re-test the combination of messaging
ü Keep an eye on customer life time value
development from new guests through scarcity
82. ONLINE D IALOGU E
G u i d i n g y o u t o d i g i t a l m a t u r i t y
To n W e s s e l i n g – G r o w t h H a c k e r
# e l i t e c a m p E s t o n i a 2 0 1 4 / 0 6 / 1 4
My D aily Job