1. Was Ebay Right About Paid Search?
Richard Fergie
@richardfergie
fergie@eanalytica.com
2. Two Types of Task
1. Changing the shape of the curve
2. Positioning on the curve
3. Incrementality and the Wider Ecosystem
How do we correctly value PPC when we know that it
influences other channels and is influenced by other
channels?
4. Attribution Modelling
I do not know how to tell if an attribution model is correct
30%
First Touch
35%
First Touch
30%
Last Touch
40%
Last Touch
40%
Split evenly
between all
touches
25%
Split evenly
between all
touches
5. The Nuclear Option
Entirely switching off a channel is one way to see what the
true impact is
1.You might lose your job
2.Seasonality can make it difficult to compare before/after
6. The MIRV Option
Rather than nuking all of the account, nuke parts of the
account
If you are clever about which parts of the account you turn off
you can minimise losses and still get high value information
7. What Doesn't Work
Pausing keywords or campaigns just means Google will
match queries elsewhere in the account
Unless you are very switched on with negative keywords
8. An easier alternative?
Use the paid/organic report if your only concern is the
interaction between AdWords and Google domain searches
Issues with the accuracy of organic data pulled from
webmaster tools
Why don't Google show ads on all the queries?
9. Time slicing
1. Turn adverts off for short periods of time
2. Pick similar time slots to compare:
a. 1400 – 1500; no ads
b. 1500 – 1600; ads
3. Flip this around the next week to make doubly sure
10. Time slicing
Long buying cycles can make this tricky
Unless you are willing to take the risk of not running ads for a
longer period of time
11. Summary So Far
We can figure out the true value of a channel by turning bits
of it off and comparing results
By being clever with which bits are turned off the reliability of
conclusions can be increased and the risk of missing out on
sales reduced
12. Ebay – Geo slicing
1. Divide the target area into regions
2. Group the regions into threes – the regions within each
triple should be as similar as possible
3. Randomly select one region out of each triple to be in the
test group
4. Stop activity in the test group and see what happens
5. The two controls in each region make this like an AAB test
13. Ebay – Geo slicing
The only thing that is difficult about this is the grouping into
regions
Ebay used Nielsen areas
1. Expensive
2. Based on average of whole population
3. Potential issues around accuracy of geo-targeting in the
UK
14. Grouping UK Towns & Cities
1. Ignore London
2. Look at monthly performance by town for the last X
months
3. Normalise for population size
4. Discard small towns where you will be unable to detect a
change
5. Dendrogram
16. Grouping UK Towns & Cities
Increase the size of the groups to control risk
Groups of 3 = 1/3 traffic lost
Groups of 5 = 1/5 traffic lost
Groups of 10 = 1/10 traffic lost
Not all groups will be valid – so don't go to far!
17. Interpreting Results
Compare drop with what you think PPC is worth
Most extreme results I've seen:
10% undervalued
15% overvalued
“Normal” seems to be within 5%
YMMV
18. Final Summary
1. Think about positioning on the curve as well as changing
the shape
2. Don't be afraid – being correct is important
3. It is possible to control risk and avoid the nuclear option
4. This might be easier than arguing with the display team
about which attribution model to use
