Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
BECAUSE MATH

DIPPING YOUR TOE IN THE WATER W/ CORRELATION
Ian Lurie
@portentint
ian@portent.com

#PORTENTU
portent.com

portent.co/bccorrel
HOW THIS WILL GO
Disclaimers
Use cases & despair
Principles
2 ways to test
We all collapse
THE LINK BUNDLE
portent.co/bccorrel
DON’T BELIEVE A WORD I SAY
MY MATH QUALIFICATIONS
A degree in history
A JD
C- in Calculus
No statistics training at all
I’VE MADE MISTAKES
LUCKY FOR YOU, I’VE HAD
SOME GREAT TEACHERS
aNNIE CUSHING
kevin hillstrom

pete meyers

avinash kaushik

john caples
WHAT I WON’T DISCUSS
learn the rules
before you use
the tools

GOOGLE ANALYTICS
R LANGUAGE
TABLEAU SOFTWARE
OTHER FANCY ST...
HOW DO WE KNOW IF
THIS IS WORKING? WE
CAN'T TRACK IT.

THOSE ARE CALL
CENTER LEADS.
INTERNET MARKETING
HAD NOTHING TO DO
W...
WE’VE ALL KNOWN DESPAIR

gaaaah i can't take it.
attribution?!!!!
hahahahahah i'll attribute
my sanity to finding a job
at...
CORRELATION IS YOUR FRIEND

Car Speed

Car top speed vs. age of American Male
180
160
140
120
100
80
60
40
20
0

CLEAR
COR...
CORRELATION IS YOUR FRIEND
WORST ENEMY

Car Speed

Car top speed vs. age of American Male
180
160
140
120
100
80
60
40
20
...
CORRELATION IS YOUR FRIEND
WORST ENEMY
MIDDLE-AGED AMERICAN MEN ARE
COMPENSATING FOR SOMETHING
OR
ALL AMERICAN MEN LIKE TO...
LATE-NIGHT SNACKING
MAKES YOU FAT
WRONG
Late night snacks vs. weight gain
50
Weight gain (lbs)

40
30
20
10
0
0

2

4
6
Sn...
WHY IS IT WRONG?
THERE IS NO CAUSAL LINK.
OR, AS STATISTICIANS LIKE
TO SAY…
correlation does
not equal causation

...but i wouldn't
worry about that too
much.
(WE LOVE YOU, MATT)
WE CAN MAKE THIS WORK BY
USING EXCEL/GDOCS
USING OUR KNOWLEDGE
GETTING A BIT GEEKY
TESTING ASSUMPTIONS
EXAMPLE 1

our e-mail list? make
it a low priority. it
doesn't sell much
anyway.
A HOLDOUT TEST!!!
TEST ONE CHANNEL’S
IMPACT ON OTHERS
A HOLDOUT TEST!!!
SEGMENT YOUR AUDIENCE
5%/5%/90%
A HOLDOUT TEST!!!
5% GETS THE USUAL
A HOLDOUT TEST!!!
5% GETS NOTHING FROM
THE CHANNEL BEING
TESTED
A HOLDOUT TEST!!!
THE RESULT GIVES YOU
INSIGHT INTO CROSSCHANNEL IMPACT
OVERALL RESULT
(all channels)

Average revenue
value generated
from all channels
of a single
address

List Size

Total sal...
OVERALL RESULT
(all channels)

E-mail caused
these customers
to buy more, no
matter where
they came from.

Organic Search
...
OVERALL RESULT
(all channels)

Now you
estimate impact
by channel.

Organic Search

PPC

Social

E-mail

Total

$0.20

$1....
EXAMPLE 2

SOCIAL MEDIA SUCKS.
LET'S STOP. FOREVER.
UH-OH
THIS IS ABOUT EXISTING &
NEW CUSTOMER ACQUISITION.
UH-OH
HOW DO WE TRACK LIFT IN
NEW CUSTOMER ACQUISITION?!
OPTION 1
TURN OFF ONE
CHANNEL. MEASURE
CHANGE IN THE OTHERS.
GENERALLY UNPOPULAR

SHIIIIIIIIIIIII
OPTION 2
BOOST SPEND ON ONE
CHANNEL. MEASURE
IMPACT ON OTHERS.
ALSO UNPOPULAR
PLUS, LOTS OF NOISE.
YOU PROBABLY CAN’T
BOOST SPEND 300%.

SHIIIIIIIIIIIII
OPTION 3
IAN’S SEAT-OF-THE-PANTS
CORRELATION METHOD
USE AT YOUR OWN
RISK. MAY CAUSE
WHAT YOU NEED
A LOT OF DATA
CLEAN DATA
ANALYTICS THAT WORK
INTERNAL SALES DATA
(NOT JUST WEB)
AN OPEN MIND
WHAT YOU NEED

COMMON SENSE
STEP 1: GET YOUR DATA
YOU MUST HAVE
OVERALL SALES!!!!
STEP 2: IMPORT IT
STEP 3: CREATE SCATTERPLOT
THIS TESTS WHETHER
YOU HAVE A CHANCE
SELECT 2 COLUMNS
CMD+SHIFT OR CTRL+SHIFT
WHICH COLUMNS?
HERE, WE’RE CHECKING
FOR CONNECTIONS
BETWEEN SOCIAL MEDIA
ACTIVITY AND REVENUE,
SO I’M STARTING WITH ONE
SO...
CLICK
CHECK THIS BOX
IN GENERAL
STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION
R SQUARED CLOSER TO 1 =TIGHTER CORRELATION
BUT REMEMBER…
I WAS A HISTORY MAJOR.
HMMMM
Overall Rev
30,000.00

Revenue (USD)

25,000.00
20,000.00
R² = 0.40636

15,000.00

LOW R SQUARED= LOW
10,000.00
CORR...
HMMMM

Revenue (USD)

PPC Rev
$20,000.00
$18,000.00
$16,000.00
$14,000.00
$12,000.00
R² = 0.81774
$10,000.00
$8,000.00
LOW...
HMMMM
Overall Rev
30,000.00

R² = 0.30871

REVENUE

25,000.00
20,000.00
15,000.00
10,000.00

WEAKER RELATIONSHIP
BETWEEN O...
HMMMM

REEVNUE (USD)

E-mail Rev vs. Social Unique Visits
$10,000.00
$9,000.00
$8,000.00
$7,000.00
$6,000.00
$5,000.00
$4,...
STEP 4: USE CORRELATION
NUMBERS, TO
QUANTIFY THE PLOTS
TYPE THIS FORMULA
INTO A CELL:
=CORREL(RANGE,
RANGE2)
IN MY EXAMPLE:
=CORREL(D2:D363,
K2:K363)
CLOSER TO 1 = STRONG
CORRELATION
MY RESULT:
MY RESULT:

DANG
MY RESULT:

HOLY @)(*!@#
correlation does
not equal causation
HMMM. A GOOD POINT.
STEP 5: APPLY COMMON SENSE
REMEMBER OUR
SCATTERPLOT OF
SOCIAL SHARES VS.
OVERALL REVENUE?
IT’S A PRETTY GOOD ‘FIT’
Overall Rev
30,000.00

R² = 0.30871

REVENUE

25,000.00
20,000.00

REMEMBER OUR
15,000.00
SCATTER...
NOT THE STRONGEST,
BUT SOMETHING’S GOING
ON THERE.
(NO, THE NUMBERS
DON’T MATCH UP. JUST
SAMPLE DATA. PLUS:
HISTORY MAJOR)
MORE COMMON SENSE
KNOWING WHAT I’VE DONE IN
SOCIAL MEDIA RECENTLY
HELPS, TOO. WE HAVEN’T
CHANGED A THING. BUT STILL,
THIS ...
MORE COMMON SENSE
WE RAN A PROMO, THOUGH,
VIA E-MAIL. THAT MIGHT
CREATE ‘NOISE.’
MORE COMMON SENSE
SO WE DO THE MATH
MORE COMMON SENSE
THAT MAY MEAN THE E-MAIL
PROMO LOOSENED THE
RELATIONSHIP BETWEEN EMAIL AND SOCIAL. SO WE RUN
ANOTHER COR...
MORE COMMON SENSE
THERE’S PROBABLY
CANNIBALIZATION GOING ON.
THE RESULT
YOU CAN AT LEAST DEMONSTRATE
THERE’S A STRONG CONNECTION
BETWEEN REVENUE AND SOCIAL,
EVEN IF SOCIAL DOESN’T
DIR...
NEXT STEP?
DO A HOLDOUT TEST (COUGH) –
STOP POSTING FOR 2 WEEKS.
TRY TO REALLY SPIKE SOCIAL
MEDIA ACTIVITY AND SEE HOW
THA...
CAUTION
DON’T TRY TO ‘FIT’ THE DATA TO
YOUR EXPECTATIONS
IF COMMON SENSE <> THE DATA,
GO WITH COMMON SENSE UNTIL
PROVEN OT...
CAUTION
REALLY LEARN THIS STUFF
QUESTIONS?
IAN@PORTENT.COM
@PORTENTINT
WWW.PORTENT.COM
portent.co/bccorrel
NEXT MONTH
MICHAEL WEIGAND
NEXT-LEVEL SEGMENTATION
DIVIDE & CONQUER
FEBRUARY 27 TH
11 AM PACIFIC
Prochain SlideShare
Chargement dans…5
×

Attribution-fu: Using Correlation Data to Track Marketing Attribution

16 269 vues

Publié le

Marketing has become extremely data-driven. I don't love it, but we need to go with it. That means finding more and better ways to track attribution across channels. This slide deck explains some basic techniques - Pearson Correlation and holdout testing - that you can use to connect channel performance to business KPIs.

Publié dans : Marketing, Business, Technologie
  • Soyez le premier à commenter

Attribution-fu: Using Correlation Data to Track Marketing Attribution

  1. BECAUSE MATH DIPPING YOUR TOE IN THE WATER W/ CORRELATION Ian Lurie @portentint ian@portent.com #PORTENTU
  2. portent.com portent.co/bccorrel
  3. HOW THIS WILL GO Disclaimers Use cases & despair Principles 2 ways to test We all collapse
  4. THE LINK BUNDLE portent.co/bccorrel
  5. DON’T BELIEVE A WORD I SAY
  6. MY MATH QUALIFICATIONS A degree in history A JD C- in Calculus No statistics training at all
  7. I’VE MADE MISTAKES
  8. LUCKY FOR YOU, I’VE HAD SOME GREAT TEACHERS
  9. aNNIE CUSHING kevin hillstrom pete meyers avinash kaushik john caples
  10. WHAT I WON’T DISCUSS learn the rules before you use the tools GOOGLE ANALYTICS R LANGUAGE TABLEAU SOFTWARE OTHER FANCY STUFF LOVE IT ALL. BUT let's STICK TO TOOLS WE ALL KNOW AND HAVE, NO MATTER what
  11. HOW DO WE KNOW IF THIS IS WORKING? WE CAN'T TRACK IT. THOSE ARE CALL CENTER LEADS. INTERNET MARKETING HAD NOTHING TO DO WITH IT. WE’VE ALL KNOWN DESPAIR I WANT TO STOP THE CONTENT CAMPAIGN. IT'S NOT EARNING ANY MONEY. I'M AFRAID THE WEB IS JUST CANNIBALIZING DIRECT MAIL. DON'T BUY PPC FOR OUR NAME. WE ALREADY RANK #1. IT'S A WASTE OF SOCIAL MEDIA MONEY. DOESN'T CONVERT.
  12. WE’VE ALL KNOWN DESPAIR gaaaah i can't take it. attribution?!!!! hahahahahah i'll attribute my sanity to finding a job at a bicycle shop bwahahahahahahaha
  13. CORRELATION IS YOUR FRIEND Car Speed Car top speed vs. age of American Male 180 160 140 120 100 80 60 40 20 0 CLEAR CORRELATION 30 35 40 45 50 Age 55 60 65 70
  14. CORRELATION IS YOUR FRIEND WORST ENEMY Car Speed Car top speed vs. age of American Male 180 160 140 120 100 80 60 40 20 0 BUT… WHY? 30 35 40 45 50 Age 55 60 65 70
  15. CORRELATION IS YOUR FRIEND WORST ENEMY MIDDLE-AGED AMERICAN MEN ARE COMPENSATING FOR SOMETHING OR ALL AMERICAN MEN LIKE TOYS, AND CAN AFFORD THEM AT MIDDLE AGE
  16. LATE-NIGHT SNACKING MAKES YOU FAT WRONG Late night snacks vs. weight gain 50 Weight gain (lbs) 40 30 20 10 0 0 2 4 6 Snacks/week 8 10
  17. WHY IS IT WRONG? THERE IS NO CAUSAL LINK. OR, AS STATISTICIANS LIKE TO SAY…
  18. correlation does not equal causation ...but i wouldn't worry about that too much.
  19. (WE LOVE YOU, MATT)
  20. WE CAN MAKE THIS WORK BY USING EXCEL/GDOCS USING OUR KNOWLEDGE GETTING A BIT GEEKY TESTING ASSUMPTIONS
  21. EXAMPLE 1 our e-mail list? make it a low priority. it doesn't sell much anyway.
  22. A HOLDOUT TEST!!! TEST ONE CHANNEL’S IMPACT ON OTHERS
  23. A HOLDOUT TEST!!! SEGMENT YOUR AUDIENCE 5%/5%/90%
  24. A HOLDOUT TEST!!! 5% GETS THE USUAL
  25. A HOLDOUT TEST!!! 5% GETS NOTHING FROM THE CHANNEL BEING TESTED
  26. A HOLDOUT TEST!!! THE RESULT GIVES YOU INSIGHT INTO CROSSCHANNEL IMPACT
  27. OVERALL RESULT (all channels) Average revenue value generated from all channels of a single address List Size Total sales Revenue/address Total 400,000 $250,000.00 $0.63 Held out 20,000 $5,000.00 $0.25 E-mailed 20,000 $12,000.00 $0.60 Revenue from all channels for each segment
  28. OVERALL RESULT (all channels) E-mail caused these customers to buy more, no matter where they came from. Organic Search PPC Social E-mail Total $0.20 $1.25 $0.11 $0.63 Held out $0.17 $0.90 $0.02 $0.25 E-mailed $0.21 $1.24 $0.13 $0.60 Ah HA! Held out segment generated less value from other channels, too.
  29. OVERALL RESULT (all channels) Now you estimate impact by channel. Organic Search PPC Social E-mail Total $0.20 $1.25 $0.11 $0.63 Held out $0.17 $0.90 $0.02 $0.25 E-mailed $0.21 $1.24 $0.13 $0.60 E-mail appears to boost PPC revenue/customer 39%
  30. EXAMPLE 2 SOCIAL MEDIA SUCKS. LET'S STOP. FOREVER.
  31. UH-OH THIS IS ABOUT EXISTING & NEW CUSTOMER ACQUISITION.
  32. UH-OH HOW DO WE TRACK LIFT IN NEW CUSTOMER ACQUISITION?!
  33. OPTION 1 TURN OFF ONE CHANNEL. MEASURE CHANGE IN THE OTHERS.
  34. GENERALLY UNPOPULAR SHIIIIIIIIIIIII
  35. OPTION 2 BOOST SPEND ON ONE CHANNEL. MEASURE IMPACT ON OTHERS.
  36. ALSO UNPOPULAR PLUS, LOTS OF NOISE. YOU PROBABLY CAN’T BOOST SPEND 300%. SHIIIIIIIIIIIII
  37. OPTION 3 IAN’S SEAT-OF-THE-PANTS CORRELATION METHOD
  38. USE AT YOUR OWN RISK. MAY CAUSE
  39. WHAT YOU NEED A LOT OF DATA CLEAN DATA ANALYTICS THAT WORK INTERNAL SALES DATA (NOT JUST WEB) AN OPEN MIND
  40. WHAT YOU NEED COMMON SENSE
  41. STEP 1: GET YOUR DATA YOU MUST HAVE OVERALL SALES!!!!
  42. STEP 2: IMPORT IT
  43. STEP 3: CREATE SCATTERPLOT THIS TESTS WHETHER YOU HAVE A CHANCE
  44. SELECT 2 COLUMNS CMD+SHIFT OR CTRL+SHIFT
  45. WHICH COLUMNS? HERE, WE’RE CHECKING FOR CONNECTIONS BETWEEN SOCIAL MEDIA ACTIVITY AND REVENUE, SO I’M STARTING WITH ONE SOCIAL MEDIA METRIC AND OVERALL REVENUE
  46. CLICK
  47. CHECK THIS BOX
  48. IN GENERAL STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION R SQUARED CLOSER TO 1 =TIGHTER CORRELATION
  49. BUT REMEMBER… I WAS A HISTORY MAJOR.
  50. HMMMM Overall Rev 30,000.00 Revenue (USD) 25,000.00 20,000.00 R² = 0.40636 15,000.00 LOW R SQUARED= LOW 10,000.00 CORRELATION = LOWHMMM. MIGHT BE A CHANCES THESE ARECONNECTION 5,000.00 CONNECTED - 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Visits from Social Media
  51. HMMMM Revenue (USD) PPC Rev $20,000.00 $18,000.00 $16,000.00 $14,000.00 $12,000.00 R² = 0.81774 $10,000.00 $8,000.00 LOW R SQUARED= LOW CORRELATION = LOWHMMM. MIGHT BE A $6,000.00 CHANCES THESE ARECONNECTION $4,000.00 CONNECTED $2,000.00 $1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Visits from Social Media
  52. HMMMM Overall Rev 30,000.00 R² = 0.30871 REVENUE 25,000.00 20,000.00 15,000.00 10,000.00 WEAKER RELATIONSHIP BETWEEN OVERALL REVENUE AND SHARES 5,000.00 - 20 40 60 80 SHARES 100 120 140
  53. HMMMM REEVNUE (USD) E-mail Rev vs. Social Unique Visits $10,000.00 $9,000.00 $8,000.00 $7,000.00 $6,000.00 $5,000.00 $4,000.00 $3,000.00 $2,000.00 $1,000.00 $- SOCIAL MAY CANNABILIZE E-MAIL R² = 0.04969 - 500 1,000 1,500 SOCIAL UNIQUE VISITS 2,000 2,500
  54. STEP 4: USE CORRELATION NUMBERS, TO QUANTIFY THE PLOTS
  55. TYPE THIS FORMULA INTO A CELL: =CORREL(RANGE, RANGE2)
  56. IN MY EXAMPLE: =CORREL(D2:D363, K2:K363)
  57. CLOSER TO 1 = STRONG CORRELATION
  58. MY RESULT:
  59. MY RESULT: DANG
  60. MY RESULT: HOLY @)(*!@#
  61. correlation does not equal causation
  62. HMMM. A GOOD POINT.
  63. STEP 5: APPLY COMMON SENSE REMEMBER OUR SCATTERPLOT OF SOCIAL SHARES VS. OVERALL REVENUE?
  64. IT’S A PRETTY GOOD ‘FIT’ Overall Rev 30,000.00 R² = 0.30871 REVENUE 25,000.00 20,000.00 REMEMBER OUR 15,000.00 SCATTERPLOT OF 10,000.00 SOCIAL UNIQUES VS. 5,000.00 OVERALL REVENUE? - 20 40 60 80 SHARES 100 120 140
  65. NOT THE STRONGEST, BUT SOMETHING’S GOING ON THERE.
  66. (NO, THE NUMBERS DON’T MATCH UP. JUST SAMPLE DATA. PLUS: HISTORY MAJOR)
  67. MORE COMMON SENSE KNOWING WHAT I’VE DONE IN SOCIAL MEDIA RECENTLY HELPS, TOO. WE HAVEN’T CHANGED A THING. BUT STILL, THIS CORRELATION.
  68. MORE COMMON SENSE WE RAN A PROMO, THOUGH, VIA E-MAIL. THAT MIGHT CREATE ‘NOISE.’
  69. MORE COMMON SENSE SO WE DO THE MATH
  70. MORE COMMON SENSE THAT MAY MEAN THE E-MAIL PROMO LOOSENED THE RELATIONSHIP BETWEEN EMAIL AND SOCIAL. SO WE RUN ANOTHER CORRELATION EXCLUDING THE DAYS OF THE PROMO.
  71. MORE COMMON SENSE THERE’S PROBABLY CANNIBALIZATION GOING ON.
  72. THE RESULT YOU CAN AT LEAST DEMONSTRATE THERE’S A STRONG CONNECTION BETWEEN REVENUE AND SOCIAL, EVEN IF SOCIAL DOESN’T DIRECTLY GENERATE THAT REVENUE.
  73. NEXT STEP? DO A HOLDOUT TEST (COUGH) – STOP POSTING FOR 2 WEEKS. TRY TO REALLY SPIKE SOCIAL MEDIA ACTIVITY AND SEE HOW THAT IMPACTS OVERALL REVENUE.
  74. CAUTION DON’T TRY TO ‘FIT’ THE DATA TO YOUR EXPECTATIONS IF COMMON SENSE <> THE DATA, GO WITH COMMON SENSE UNTIL PROVEN OTHERWISE
  75. CAUTION REALLY LEARN THIS STUFF
  76. QUESTIONS? IAN@PORTENT.COM @PORTENTINT WWW.PORTENT.COM portent.co/bccorrel
  77. NEXT MONTH MICHAEL WEIGAND NEXT-LEVEL SEGMENTATION DIVIDE & CONQUER FEBRUARY 27 TH 11 AM PACIFIC

×