Marketing has become extremely data-driven. I don't love it, but we need to go with it. That means finding more and better ways to track attribution across channels. This slide deck explains some basic techniques - Pearson Correlation and holdout testing - that you can use to connect channel performance to business KPIs.

1. BECAUSE MATH
DIPPING YOUR TOE IN THE WATER W/ CORRELATION
Ian Lurie
@portentint
ian@portent.com
#PORTENTU

2. portent.com
portent.co/bccorrel

3. HOW THIS WILL GO
Disclaimers
Use cases & despair
Principles
2 ways to test
We all collapse

4. THE LINK BUNDLE
portent.co/bccorrel

5. DON’T BELIEVE A WORD I SAY

6. MY MATH QUALIFICATIONS
A degree in history
A JD
C- in Calculus
No statistics training at all

7. I’VE MADE MISTAKES

8. LUCKY FOR YOU, I’VE HAD
SOME GREAT TEACHERS

9. aNNIE CUSHING
kevin hillstrom
pete meyers
avinash kaushik
john caples

10. WHAT I WON’T DISCUSS
learn the rules
before you use
the tools
GOOGLE ANALYTICS
R LANGUAGE
TABLEAU SOFTWARE
OTHER FANCY STUFF
LOVE IT ALL. BUT let's
STICK TO TOOLS WE
ALL KNOW AND HAVE,
NO MATTER what

11. HOW DO WE KNOW IF
THIS IS WORKING? WE
CAN'T TRACK IT.
THOSE ARE CALL
CENTER LEADS.
INTERNET MARKETING
HAD NOTHING TO DO
WITH IT.
WE’VE ALL KNOWN DESPAIR
I WANT TO STOP THE
CONTENT CAMPAIGN.
IT'S NOT EARNING ANY
MONEY.
I'M AFRAID THE WEB
IS JUST
CANNIBALIZING
DIRECT MAIL.
DON'T BUY PPC FOR OUR
NAME. WE ALREADY RANK
#1. IT'S A WASTE OF
SOCIAL MEDIA
MONEY.
DOESN'T CONVERT.

12. WE’VE ALL KNOWN DESPAIR
gaaaah i can't take it.
attribution?!!!!
hahahahahah i'll attribute
my sanity to finding a job
at a bicycle shop
bwahahahahahahaha

13. CORRELATION IS YOUR FRIEND
Car Speed
Car top speed vs. age of American Male
180
160
140
120
100
80
60
40
20
0
CLEAR
CORRELATION
30
35
40
45
50
Age
55
60
65
70

14. CORRELATION IS YOUR FRIEND
WORST ENEMY
Car Speed
Car top speed vs. age of American Male
180
160
140
120
100
80
60
40
20
0
BUT… WHY?
30
35
40
45
50
Age
55
60
65
70

15. CORRELATION IS YOUR FRIEND
WORST ENEMY
MIDDLE-AGED AMERICAN MEN ARE
COMPENSATING FOR SOMETHING
OR
ALL AMERICAN MEN LIKE TOYS, AND
CAN AFFORD THEM AT MIDDLE AGE

16. LATE-NIGHT SNACKING
MAKES YOU FAT
WRONG
Late night snacks vs. weight gain
50
Weight gain (lbs)
40
30
20
10
0
0
2
4
6
Snacks/week
8
10

17. WHY IS IT WRONG?
THERE IS NO CAUSAL LINK.
OR, AS STATISTICIANS LIKE
TO SAY…

18. correlation does
not equal causation
...but i wouldn't
worry about that too
much.

19. (WE LOVE YOU, MATT)

20. WE CAN MAKE THIS WORK BY
USING EXCEL/GDOCS
USING OUR KNOWLEDGE
GETTING A BIT GEEKY
TESTING ASSUMPTIONS

21. EXAMPLE 1
our e-mail list? make
it a low priority. it
doesn't sell much
anyway.

22. A HOLDOUT TEST!!!
TEST ONE CHANNEL’S
IMPACT ON OTHERS

23. A HOLDOUT TEST!!!
SEGMENT YOUR AUDIENCE
5%/5%/90%

24. A HOLDOUT TEST!!!
5% GETS THE USUAL

25. A HOLDOUT TEST!!!
5% GETS NOTHING FROM
THE CHANNEL BEING
TESTED

26. A HOLDOUT TEST!!!
THE RESULT GIVES YOU
INSIGHT INTO CROSSCHANNEL IMPACT

27. OVERALL RESULT
(all channels)
Average revenue
value generated
from all channels
of a single
address
List Size
Total sales
Revenue/address
Total
400,000
$250,000.00
$0.63
Held out
20,000
$5,000.00
$0.25
E-mailed
20,000
$12,000.00
$0.60
Revenue from all
channels for each
segment

28. OVERALL RESULT
(all channels)
E-mail caused
these customers
to buy more, no
matter where
they came from.
Organic Search
PPC
Social
E-mail
Total
$0.20
$1.25
$0.11
$0.63
Held out
$0.17
$0.90
$0.02
$0.25
E-mailed
$0.21
$1.24
$0.13
$0.60
Ah HA! Held out segment
generated less value from
other channels, too.

29. OVERALL RESULT
(all channels)
Now you
estimate impact
by channel.
Organic Search
PPC
Social
E-mail
Total
$0.20
$1.25
$0.11
$0.63
Held out
$0.17
$0.90
$0.02
$0.25
E-mailed
$0.21
$1.24
$0.13
$0.60
E-mail appears to boost PPC
revenue/customer 39%

30. EXAMPLE 2
SOCIAL MEDIA SUCKS.
LET'S STOP. FOREVER.

31. UH-OH
THIS IS ABOUT EXISTING &
NEW CUSTOMER ACQUISITION.

32. UH-OH
HOW DO WE TRACK LIFT IN
NEW CUSTOMER ACQUISITION?!

33. OPTION 1
TURN OFF ONE
CHANNEL. MEASURE
CHANGE IN THE OTHERS.

34. GENERALLY UNPOPULAR
SHIIIIIIIIIIIII

35. OPTION 2
BOOST SPEND ON ONE
CHANNEL. MEASURE
IMPACT ON OTHERS.

36. ALSO UNPOPULAR
PLUS, LOTS OF NOISE.
YOU PROBABLY CAN’T
BOOST SPEND 300%.
SHIIIIIIIIIIIII

37. OPTION 3
IAN’S SEAT-OF-THE-PANTS
CORRELATION METHOD

38. USE AT YOUR OWN
RISK. MAY CAUSE

39. WHAT YOU NEED
A LOT OF DATA
CLEAN DATA
ANALYTICS THAT WORK
INTERNAL SALES DATA
(NOT JUST WEB)
AN OPEN MIND

40. WHAT YOU NEED
COMMON SENSE

41. STEP 1: GET YOUR DATA
YOU MUST HAVE
OVERALL SALES!!!!

42. STEP 2: IMPORT IT

43. STEP 3: CREATE SCATTERPLOT
THIS TESTS WHETHER
YOU HAVE A CHANCE

44. SELECT 2 COLUMNS
CMD+SHIFT OR CTRL+SHIFT

45. WHICH COLUMNS?
HERE, WE’RE CHECKING
FOR CONNECTIONS
BETWEEN SOCIAL MEDIA
ACTIVITY AND REVENUE,
SO I’M STARTING WITH ONE
SOCIAL MEDIA METRIC AND
OVERALL REVENUE

46. CLICK

47. CHECK THIS BOX

48. IN GENERAL
STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION
R SQUARED CLOSER TO 1 =TIGHTER CORRELATION

49. BUT REMEMBER…
I WAS A HISTORY MAJOR.

50. HMMMM
Overall Rev
30,000.00
Revenue (USD)
25,000.00
20,000.00
R² = 0.40636
15,000.00
LOW R SQUARED= LOW
10,000.00
CORRELATION = LOWHMMM. MIGHT BE A
CHANCES THESE ARECONNECTION
5,000.00
CONNECTED
-
1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000
Visits from Social Media

51. HMMMM
Revenue (USD)
PPC Rev
$20,000.00
$18,000.00
$16,000.00
$14,000.00
$12,000.00
R² = 0.81774
$10,000.00
$8,000.00
LOW R SQUARED= LOW
CORRELATION = LOWHMMM. MIGHT BE A
$6,000.00
CHANCES THESE ARECONNECTION
$4,000.00
CONNECTED
$2,000.00
$1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000
Visits from Social Media

52. HMMMM
Overall Rev
30,000.00
R² = 0.30871
REVENUE
25,000.00
20,000.00
15,000.00
10,000.00
WEAKER RELATIONSHIP
BETWEEN OVERALL REVENUE
AND SHARES
5,000.00
-
20
40
60
80
SHARES
100
120
140

53. HMMMM
REEVNUE (USD)
E-mail Rev vs. Social Unique Visits
$10,000.00
$9,000.00
$8,000.00
$7,000.00
$6,000.00
$5,000.00
$4,000.00
$3,000.00
$2,000.00
$1,000.00
$-
SOCIAL MAY
CANNABILIZE E-MAIL
R² = 0.04969
-
500
1,000
1,500
SOCIAL UNIQUE VISITS
2,000
2,500

54. STEP 4: USE CORRELATION
NUMBERS, TO
QUANTIFY THE PLOTS

55. TYPE THIS FORMULA
INTO A CELL:
=CORREL(RANGE,
RANGE2)

56. IN MY EXAMPLE:
=CORREL(D2:D363,
K2:K363)

57. CLOSER TO 1 = STRONG
CORRELATION

58. MY RESULT:

59. MY RESULT:
DANG

60. MY RESULT:
HOLY @)(*!@#

61. correlation does
not equal causation

62. HMMM. A GOOD POINT.

63. STEP 5: APPLY COMMON SENSE
REMEMBER OUR
SCATTERPLOT OF
SOCIAL SHARES VS.
OVERALL REVENUE?

64. IT’S A PRETTY GOOD ‘FIT’
Overall Rev
30,000.00
R² = 0.30871
REVENUE
25,000.00
20,000.00
REMEMBER OUR
15,000.00
SCATTERPLOT OF
10,000.00
SOCIAL UNIQUES VS.
5,000.00
OVERALL REVENUE?
-
20
40
60
80
SHARES
100
120
140

65. NOT THE STRONGEST,
BUT SOMETHING’S GOING
ON THERE.

66. (NO, THE NUMBERS
DON’T MATCH UP. JUST
SAMPLE DATA. PLUS:
HISTORY MAJOR)

67. MORE COMMON SENSE
KNOWING WHAT I’VE DONE IN
SOCIAL MEDIA RECENTLY
HELPS, TOO. WE HAVEN’T
CHANGED A THING. BUT STILL,
THIS CORRELATION.

68. MORE COMMON SENSE
WE RAN A PROMO, THOUGH,
VIA E-MAIL. THAT MIGHT
CREATE ‘NOISE.’

69. MORE COMMON SENSE
SO WE DO THE MATH

70. MORE COMMON SENSE
THAT MAY MEAN THE E-MAIL
PROMO LOOSENED THE
RELATIONSHIP BETWEEN EMAIL AND SOCIAL. SO WE RUN
ANOTHER CORRELATION
EXCLUDING THE DAYS OF THE
PROMO.

71. MORE COMMON SENSE
THERE’S PROBABLY
CANNIBALIZATION GOING ON.

72. THE RESULT
YOU CAN AT LEAST DEMONSTRATE
THERE’S A STRONG CONNECTION
BETWEEN REVENUE AND SOCIAL,
EVEN IF SOCIAL DOESN’T
DIRECTLY GENERATE THAT
REVENUE.

73. NEXT STEP?
DO A HOLDOUT TEST (COUGH) –
STOP POSTING FOR 2 WEEKS.
TRY TO REALLY SPIKE SOCIAL
MEDIA ACTIVITY AND SEE HOW
THAT IMPACTS OVERALL REVENUE.

74. CAUTION
DON’T TRY TO ‘FIT’ THE DATA TO
YOUR EXPECTATIONS
IF COMMON SENSE <> THE DATA,
GO WITH COMMON SENSE UNTIL
PROVEN OTHERWISE

75. CAUTION
REALLY LEARN THIS STUFF

76. QUESTIONS?
IAN@PORTENT.COM
@PORTENTINT
WWW.PORTENT.COM
portent.co/bccorrel

77. NEXT MONTH
MICHAEL WEIGAND
NEXT-LEVEL SEGMENTATION
DIVIDE & CONQUER
FEBRUARY 27 TH
11 AM PACIFIC