1.
SUBTITLE BELOW
Experimenteren & personaliseren
binnen (grote) webshops

2.
Agenda
14.30 – 14.40 Welkom
14.40 – 15.00 Tom van den Berg | Conversie Manager @ Online Dialogue
15.00 – 15.30 Niels Reijmer | Lead Data Analyst @ Bijenkorf
Pauze
16.00 – 16.30 Ellen Tonkens | Conversion Specialist @ wehkamp
16.30 – 17.00 Ask them anything
17.00 – 18.30 Borrel

3.
Validation in every organization

4.
Digital made validation possible

5.
Validation to the max?

6.
Validation to the max?
> 2.000 experiments
per day
> 7.000 experiments
7 years ago
> 2.000 experiments
live now

7.
“Our success at Amazon
is a function of
how many experiments
we do per year, per month, per
week, per day…”
Jeff Bezos, CEO Amazon

8.
More tests = more growth

9.
3 Validation rules:
1. OEC
2. Effectiveness over Efficiency
3. Quantity over quality

10.
3 Validation rules:
1. OEC
2. Effectiveness over Efficiency
3. Quantity over quality

11.
First agree on your OEC

12.
3 Validation rules:
1. OEC
2. Effectiveness over Efficiency
3. Quantity over quality

13.
Effectiveness
Over efficiency

14.
But…
$

15.
Even better…
ROI

16.
ROI
Return
Investments
= ____________________

17.
ITOptimization

18.
ITOptimization
Validation

19.
Useless to spend scarce resources onHurts our bussiness Brings value
Significantly positiveInconclusiveSignificantly negative
Validation

20.
Hurts Doesn't make a difference Improves
0% 25% 50% 75% 100%
25%60%15%
Typically, most optimization are ineffective

21.
Typically, most optimization are ineffective
Hurts Doesn't make a difference Improves
0% 25% 50% 75% 100%
Brings value
B
A
A B
A
B
B
A
25%60%15%
25% - 15% = 10%

22.
Validation boosts effectiveness!
Hurts Doesn't make a difference Improves
0% 25% 50% 75% 100%
regret
Brings value
regret

23.
3 Validation rules:
1. OEC
2. Effectiveness over Efficiency
3. Quantity over quality

24.
What is easier? Double experiments in:
1. Quantity
2. Quality

25.
What is easier? Double experiments in:
1. Quantity
2. Quality

26.
But… there is a maximum
What’s your
Bandwidth?

27.
test velocity
Validation is about Velocity!
Prevent Regret:
Run as much experiments as possible

28.
Validation is about Velocity!

29.
Validation = Culture

30.
Validation is about reducing investments!
ROI
Increase Improvements
Lower Investments
= ____________________
What’s your
value / hour spend?

31.
test velocity
You think & talk
too much!

32.
Too much discussion
All new experiments

33.
Velocity over Value
$$
Resources per validation
Costs of resources
Value per validation
Value per hour spend validating

34.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

35.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

36.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

37.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

38.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

39.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

40.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

41.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

42.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M

43.
Example Scenario 1 2
Uren per week 200 200
Uren per experiment 40 25
# experimenten p/w 5 8
# experimenten p/y 260 416
% winnaars 25% 20%
# winnaars 65 83
Value per winner €100K €100K
Revenu €6,5M €8,3M
Extra revenue
€1,8M
€1,8M

44.
Niet lullen, maar toetsen!