Andy Young — Data-Driven Growth: Analytics Tools and Tips for Marketers in 2016 (Turing Festival 2016)
1. 📈 🚀
Andy Young, 500 Startups / Freelance
Turing Festival 2016
Analytics tools and tips for marketers in 2016
Data-driven Growth
2. Andy Young - @andyy - http://bit.ly/analytics-turing
Hi, I’m Andy!
@andyy
#fb
Photo by Dan Taylor - dan@heisenbergmedia.com
3. Andy Young - @andyy - http://bit.ly/analytics-turing
Agenda
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
4. Andy Young - @andyy - http://bit.ly/analytics-turing 🤔
Why analytics?
5. Andy Young - @andyy - http://bit.ly/analytics-turing
Don’t start with the data
6. Andy Young - @andyy - http://bit.ly/analytics-turing
Common Analytics FAILS
• Drowning in data
• Not identifying
key questions to answer
• Not starting with
clear hypotheses
7. Andy Young - @andyy - http://bit.ly/analytics-turing
Common Analytics FAILS
• Drowning in data
• Not identifying
key questions to answer
• Not starting with
clear hypotheses
8. Andy Young - @andyy - http://bit.ly/analytics-turing
We need to know how we’re doing
9. Andy Young - @andyy - http://bit.ly/analytics-turing
• Overall business/product
• Individual experiments, campaigns, channels, customer segments
We need to know how we’re doing
10. Andy Young - @andyy - http://bit.ly/analytics-turing
• Overall business/product
• Individual experiments, campaigns, channels, customer segments
We need to know how we’re doing
• What is working?
• What is not?
• Where to focus for improvement?
11. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics Pros..
• Start with a hypothesis
• Identify, collect + analyse necessary relevant data
• Conclude & action
• Iterate, revise
12. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics for Growth
📈
13. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth?
•More users
•More revenue
14. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics techniques
• Early stage:
- innovation accounting (Lean Startup)
- cohort analysis
16. Andy Young - @andyy - http://bit.ly/analytics-turing
Early stage questions
17. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we getting traction?
18. Andy Young - @andyy - http://bit.ly/analytics-turing
“Traction is the
rate
at which
monetizable value
is extracted
from customers”
19. Andy Young - @andyy - http://bit.ly/analytics-turing
Innovation
accounting?
20. Andy Young - @andyy - http://bit.ly/analytics-turing
Cohort analysis?
21. Andy Young - @andyy - http://bit.ly/analytics-turing
22. Andy Young - @andyy - http://bit.ly/analytics-turing
23. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth stage questions
24. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we growing?
25. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
26. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we growing?
27. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
28. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
👌👉 http://bit.ly/social-capital-growth-accounting
29. Andy Young - @andyy - http://bit.ly/analytics-turing
Funnel performance?
30. Andy Young - @andyy - http://bit.ly/analytics-turing
As marketers
we need to understand
the full funnel
31. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth stage
Which channels bring
the most customers?
Which channels bring
the best customers?
32. Andy Young - @andyy - http://bit.ly/analytics-turing
How much do we want to
pay per click?
33. Andy Young - @andyy - http://bit.ly/analytics-turing
How much do we want to
pay per click?
..as much as possible?!
Outspend competitors
Activate new channels
34. Andy Young - @andyy - http://bit.ly/analytics-turing
Customer segmentation
& channel performance
For different groups of customers:
• Where do they come from?
• How do they behave?
35. Andy Young - @andyy - http://bit.ly/analytics-turing
Customer segmentation
Users acquired via different channels
will have different behaviours
Different cohorts will have
different experiences of your product
Different users will have been exposed to
different A/B tests
36. Andy Young - @andyy - http://bit.ly/analytics-turing
These are all properties of your users
UTM tags: source, medium, campaign, terms
Landing page
Signup time
A/B test buckets
Referrer
Viral source
Customer segmentation
37. Andy Young - @andyy - http://bit.ly/analytics-turing
Demystifying
direct traffic?
38. Andy Young - @andyy - http://bit.ly/analytics-turing
Demystifying
direct traffic?
39. Andy Young - @andyy - http://bit.ly/analytics-turing
Tag all the things!
Aim: map all Direct + Referral traffic
to a specific source/medium
Do this for Email, Social,
Offline(!)..
Tagging guide:
- bit.ly/tagging-guide
- bit.ly/utm-reference
40. Andy Young - @andyy - http://bit.ly/analytics-turing
Landing
page:
often a big
clue to source
41. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device?
Offline channels?
42. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device?
• Common pattern:
Research on mobile/tablet,
purchase on tablet/desktop
• Challenge: attribution on mobile
• We need: unique identifying information
E.g. email, mobile/cell #
43. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device attribution?
https://www.twilio.com/blog/2012/11/increase-app-downloads-with-mobile-app-distribution-powered-by-twilio.html
44. Andy Young - @andyy - http://bit.ly/analytics-turing
Filling the
attribution
blanks..
https://twitter.com/cannonjw/status/432586695490670592
45. Andy Young - @andyy - http://bit.ly/analytics-turing
Filling the
attribution
blanks..
https://twitter.com/cannonjw/status/432586695490670592
46. Andy Young - @andyy - http://bit.ly/analytics-turing
Geographic A/B tests
https://econsultancy.com/blog/64204-12-inspiring-marketing-campaigns-from-google/
47. Andy Young - @andyy - http://bit.ly/analytics-turing
Brand vs. non-brand terms
for organic + paid search channels
48. Andy Young - @andyy - http://bit.ly/analytics-turing
Channel Groupings
49. Andy Young - @andyy - http://bit.ly/analytics-turing
Attribution models?
• First touch? Last touch? Multi-touch?
50. Andy Young - @andyy - http://bit.ly/analytics-turing
Attribution models?
• First touch? Last touch? Multi-touch?
• First touch:
grow our audience reach
• Subsequent touches:
grow our conversion rate %
51. Andy Young - @andyy - http://bit.ly/analytics-turing
Most campaigns FAIL
52. Andy Young - @andyy - http://bit.ly/analytics-turing
Most campaigns fail
..but most every failed campaign will contain
partial successes
• Plan in advance: how will we know
at which point in the user journey
a campaign failed?
• Do things that don’t scale
53. Andy Young - @andyy - http://bit.ly/analytics-turing
Pulling this all together
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
54. Andy Young - @andyy - http://bit.ly/analytics-turing
What to track?
• Discovery / Acquisition
• Activation
• Engagement
• Conversion / Purchase
• Retention
• Referral
55. Andy Young - @andyy - http://bit.ly/analytics-turing
So many platforms
56. Andy Young - @andyy - http://bit.ly/analytics-turing
So many platforms
¯_( )_/¯
57. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality
do you need?
58. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
59. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
60. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
61. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
62. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
63. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
64. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
65. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
66. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
67. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
• Platforms
Google Firebase, AWS Mobile Analytics
68. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
• Platforms
Google Firebase, AWS Mobile Analytics
😱
69. Andy Young - @andyy - http://bit.ly/analytics-turing
How to pick?
• What functionality do you need?
• Who will be using it? Devs, data scientists, product folks, marketing people?
• How do you want to use it? Analysis, reporting, dashboards..
• Which platforms? Web, mobile, server?
• Which other tools/data do you need to integrate with?
• What data volumes? Data points, events per second, data points per month.
• Budget?
70. Andy Young - @andyy - http://bit.ly/analytics-turing
How to pick?
Things to be wary of:
• Data lock-in, future portability
• Building your own
71. Andy Young - @andyy - http://bit.ly/analytics-turing
Tools are not a panacea
72. Andy Young - @andyy - http://bit.ly/analytics-turing
Biggest single factor?
73. Andy Young - @andyy - http://bit.ly/analytics-turing
Biggest single factor?
Quality of your
tracking implementation
- Coverage/depth
- Accuracy/lack of bugs
74. Andy Young - @andyy - andy@apexa.co.uk
Use a tag manager
75. Andy Young - @andyy - andy@apexa.co.uk
Data collection: Segment.com
76. Andy Young - @andyy - http://bit.ly/analytics-turing
👉 http://sitehound.co
Free code!
77. Andy Young - @andyy - http://bit.ly/analytics-turing
Dashboards?
78. Andy Young - @andyy - http://bit.ly/analytics-turing
What makes a
good dashboard?
79. Andy Young - @andyy - http://bit.ly/analytics-turing
👉 http://bit.ly/kpi-sheet
80. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth model spreadsheet
• Review weekly - zoom out vs. zoom in
• Share with whole team
• Use to predict and prioritise
• What to look at daily vs. weekly vs. monthly
👉 http://bit.ly/kpi-sheet
81. Andy Young - @andyy - http://bit.ly/analytics-turing
Metrics are people too
Every metric should be measured in
unique people
or % conversion
82. Andy Young - @andyy - http://bit.ly/analytics-turing
Joining the dots
Plugging all these tools and platforms together
83. Andy Young - @andyy - http://bit.ly/analytics-turing
Initially, manual is OK!
84. Andy Young - @andyy - http://bit.ly/analytics-turing
Initially, manual is OK!
• Forcing function to ensure we:
• learn what works
• understand the data
• need the data
• Paste weekly into a Google Sheet/Excel
85. Andy Young - @andyy - http://bit.ly/analytics-turing
Automate over time
to reduce effort
86. Andy Young - @andyy - http://bit.ly/analytics-turing
Plumbing for automation
• Zapier, Tray.io
• Supermetrics, Blockspring for Google Sheets/Excel
• Segment.com
• Own database/data warehousing - AWS RDS/Redshift
• Custom queries - Tableau, Periscope etc
87. Andy Young - @andyy - http://bit.ly/analytics-turing
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
88. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers
never add up
89. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of
(Analytics)
Grief
90. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
91. Andy Young - @andyy - http://bit.ly/analytics-turing
FB ad clicks
vs.
GA visits
92. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
93. Andy Young - @andyy - http://bit.ly/analytics-turing
Mixpanel vs. own database
94. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
95. Andy Young - @andyy - andy@apexa.co.uk
Measure
Twice:
Compare
data across
different tools
https://jeffpeachey.com/2010/12/02/measure-twice-cut-once-woodcut-for-sale/
96. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
• Depression
97. Andy Young - @andyy - http://bit.ly/analytics-turing
Where did
it all go
wrong?
98. Andy Young - @andyy - http://bit.ly/analytics-turing
Where did
it all go
wrong?
• The data is bad
• Our definitions are wrong
99. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers never add up
• Absolute truths
• Signups, Transactional data, UGC, data from our
own DB
• Lossy/noisy measurements
• Client-side tracking: GA, Mixpanel/Amplitude etc
• Nuanced definitions
• Uniques vs. totals, funnel conversion rates
100. Andy Young - @andyy - http://bit.ly/analytics-turing
Even if the numbers do add up,
they are still often misleading
101. Andy Young - @andyy - http://bit.ly/analytics-turing
102. Andy Young - @andyy - http://bit.ly/analytics-turing
Data discrepancies
Google Analytics vs. Mixpanel
• Definition of unique user
- cookie rules
• Ad/tag blockers
103. Andy Young - @andyy - http://bit.ly/analytics-turing
Data discrepancies
• Page did not finish loading
- e.g. tag manager, or user gave up & hit
back button
• Cross-browser JS bugs
104. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
• Depression
• Acceptance
105. Andy Young - @andyy - http://bit.ly/analytics-turing
3-5% discrepancies between
platforms?
Roll with it.
106. Andy Young - @andyy - http://bit.ly/analytics-turing
Large (>5%) difference?
Debug/analyse to figure out why
107. Andy Young - @andyy - http://bit.ly/analytics-turing
Takeaways?
108. Andy Young - @andyy - http://bit.ly/analytics-turing
Secrets to successful analytics
• Start with your questions / hypotheses
• Be precise about what to measure, and why
• Plan in advance, simplify
• Automate required work for weekly/monthly
calculations
• Document all analysis with simple bullet-points
109. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers tell us What
They don’t tell us Why
110. Thanks, & good luck!
@andyy // andy@apexa.co.uk
Slides: bit.ly/analytics-turing
111. Andy Young - @andyy - http://bit.ly/analytics-turing
References & further reading
• My more in-depth analytics presentations: Slideshare, YouTube
• KPI google sheet
• SiteHound website tracking javascript library
• UTM Tagging Guide
• Lean Analytics
• Social Capital blog posts: - Accounting for User Growth