How do I analytics? A practical guide for pragmatic startups.
1. Andy Young // @andyy // andy@500.co
How do I analytics?
a practical guide for
pragmatic startups
2. Andy Young // @andyy // andy@500.co
Hi, I’m Andy!
@andyy
3. Andy Young // @andyy // andy@500.co
Wait what, why?
Analytics?
4. Andy Young // @andyy // andy@500.co
What do we measure, and why?
Vanity metrics
Revenue metrics
Conversion rate metrics
Pirate metrics..
We need to know how we’re doing.
5. Andy Young // @andyy // andy@500.co
If you're not keeping score
there's no point playing the game -
you'll never know if you're
winning or not
- @distrodom
6. Andy Young // @andyy // andy@500.co
Today’s tools make it
super-easy to track
things
Google Analytics
Mixpanel
KissMetrics
Localytics
Branch Metrics..
7. Andy Young // @andyy // andy@500.co
Today’s tools make it
super-easy to track
things
BUT they also make it really easy to
- become overwhelmed with data
- focus on the wrong things
8. Andy Young // @andyy // andy@500.co
Typical analytics challenges/pitfalls
Drowning in too much data
Failure to select + focus on the top metrics that
matter
Not tracking the data you need to answer key
questions
9. Andy Young // @andyy // andy@500.co
Why analytics?
1. How are we doing?
- are KPIs on the right track?
2. What are the results of our experiments?
- so we can learn
3. What’s happening right now?
- did something great or terrible just happen?
10. Andy Young // @andyy // andy@500.co
A. How are we doing?
11. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
12. Andy Young // @andyy // andy@500.co
Identify top-level KPI
it’s hard.
13. Andy Young // @andyy // andy@500.co
Identify top-level KPI
if you pick the wrong KPIs, you're screwed.
If you pick KPIs and then ignore them, you're
screwed.
If you pick and monitor KPIs diligently, but don't
assess everything you and your whole team does
on the basis of whether your tasks are the most
effective way to grow your KPIs, you're screwed.
14. Andy Young // @andyy // andy@500.co
Identify top-level KPI
How?
15. Andy Young // @andyy // andy@500.co
Identify top-level KPI
Keep it simple!
The good news: there’s probably a
pre-determined answer
for
what drives your business
16. Andy Young // @andyy // andy@500.co
Identify top-level KPI
There’s probably a pre-determined
answer for what drives your business
Spoiler: ultimately it’s $$$
17. Andy Young // @andyy // andy@500.co
How’re we doing?
There’s probably a pre-determined
answer for what drives your business
Commerce: # sales
Subscription / SaaS: # subscribers
Marketplace: # transactions
18. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
19. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
2. Next, add nuance
20. Andy Young // @andyy // andy@500.co
How’re we doing?
Nuance behind your top level KPI
E.g. for commerce: # sales
Nuance: average sale $; # customers
21. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
2. Add nuance
3. Add drivers
22. Andy Young // @andyy // andy@500.co
How’re we doing?
Drivers behind your top level KPI
E.g. for marketplaces: # transactions
Drivers: # suppliers, # customers
23. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
2. Add nuance
3. Add drivers
4. Add funnel for these drivers
24. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
2. Add nuance
3. Add drivers
4. Add funnel for these drivers
End up with AARRR
25. Andy Young // @andyy // andy@500.co
How’re we doing?
1. Identify top-level KPI
2. Add nuance
3. Add drivers
4. Add funnel for these drivers
Put in a spreadsheet
(Template Google Sheet: http://bit.ly/kpi-sheet)
26. Andy Young // @andyy // andy@500.co
How’re we doing?
http://bit.ly/kpi-sheet
27. Andy Young // @andyy // andy@500.co
How’re we doing?
Put in a spreadsheet
- key KPI at the top, nuance and drivers below,
finally the detailed funnel below for reference
- columns for weekly numbers, w/w growth
- review weekly
- share with whole team
28. Andy Young // @andyy // andy@500.co
Weekly/Monthly reporting
% week-on-week or month-on-month growth
in your one metric that matters
30. Andy Young // @andyy // andy@500.co
Tracking events
Events vs. Properties vs. People
Events: something happened
Properties: something about what just happened
People: connect events to particular users
- people can also have properties
31. Andy Young // @andyy // andy@500.co
Track events from where?
Client/app vs. server
Tracking events
32. Andy Young // @andyy // andy@500.co
Tracking events
Tip #1: Choose easy-to-read and meaningful
event names
Short!
Pick a convention; stick to it
Omit superfluous words
“user_viewed_homepage”
“Viewed homepage”
33. Andy Young // @andyy // andy@500.co
Tip #2: Track each user based on a distinct ID
Don’t use email address -
use autogenerated user_id from your own DB
Use aliasing to connect up events tracked
pre/post signup
Tracking events
34. Andy Young // @andyy // andy@500.co
Tip #3: Annotate your users with source data
referrer; utm tags; install tracking via AppsFlyer
1. Track a signup event
2. Add as user properties
3. Potentially also as properties to key events
Tracking events
35. Andy Young // @andyy // andy@500.co
Tip #4: (Mixpanel specific) - People vs. Events
Mixpanel won’t let you query for users
who did particular events
So, our options:
- Do this using your own DB
- Annotate your users (People) with properties
for each key event
Tracking events
36. Andy Young // @andyy // andy@500.co
Tip #5: Ecommerce/revenue tracking
Mixpanel/AppBoy etc have
native support for tracking revenue
Annotate your Purchase events with revenue data
using the relevant properties for each platform
Tracking events
37. Andy Young // @andyy // andy@500.co
Tip #6: Use a development project for testing
Tracking events
38. Andy Young // @andyy // andy@500.co
Tracking the funnel
Start with the pirate metrics AARRR
Top of funnel: acquisition; signups/installs
Mid funnel: post-install events; engagement;
retention
Bottom of funnel: purchase / monetisation.
39. Andy Young // @andyy // andy@500.co
Tracking the funnel
Looking at each stage (AARRR) in aggregate
is a good start
but it will only get you so far
the “truth” is much more nuanced
40. Andy Young // @andyy // andy@500.co
Tracking the funnel
Users acquired via different channels
will have different behaviours
Different cohorts will have
different experiences of your product
Different users will have been exposed to
different A/B tests
41. Andy Young // @andyy // andy@500.co
Tracking the funnel
Key: these are all properties of your users
UTM tags: source, medium, campaign, terms
Landing page
Signup time
A/B test buckets
Referrer
Viral source
42. Andy Young // @andyy // andy@500.co
Tracking the funnel
Annotate your users in your database/analytics
system with these attributes
UTM tags: source, medium, campaign, terms
Landing page
Signup time
A/B test buckets
Referrer
Viral source
43. Andy Young // @andyy // andy@500.co
Other key metrics
CAC, LTV, churn
44. Andy Young // @andyy // andy@500.co
Other key metrics
Customer Acquisition Cost (CAC)
how much you spend (on average) to acquire a
customer
Lifetime Value (LTV)
How much revenue $$ an average customer
brings you in all time
45. Andy Young // @andyy // andy@500.co
If your
LTV
is greater than your
CAC
then you’re in business
46. Andy Young // @andyy // andy@500.co
If your
LTV
is greater than 3x your
CAC
then you’re in a good business
47. Andy Young // @andyy // andy@500.co
CAC & LTV: nuances
Payback period: time to recoup CAC
Magnitude of your numbers
e.g. enterprise vs. social
48. Andy Young // @andyy // andy@500.co
Calculating CAC
Simple approach: total spend / total signups
“50% of the money I spend on advertising
is wasted - the problem is
I don't know which half”
- John Wanamaker
Eventual goal: calculate CAC per channel
49. Andy Young // @andyy // andy@500.co
Calculating LTV
Problem!
You don’t have a lifetime of data
We don't measure LTV - we estimate it
Extrapolate revenue curve over time
50. Andy Young // @andyy // andy@500.co
Analysing your data
54. Andy Young // @andyy // andy@500.co
How not to do Metrics
Outdated information
Just 1 view of your data
Manual calculations
Bad metrics lead you astray
55. Andy Young // @andyy // andy@500.co
Doing metrics right
Graphs
Automated
Realtime
56. Andy Young // @andyy // andy@500.co
Cohort analysis?
64. Andy Young // @andyy // andy@500.co
Use your existing database
Users
Learn SQL! It's not hard
Just need a slave database for analytics
- “read replica” - i.e. a live copy
65. Andy Young // @andyy // andy@500.co
Use your existing data
Users
SELECT COUNT(*) FROM users
66. Andy Young // @andyy // andy@500.co
Use your existing data
Users
SELECT COUNT(*) FROM users
WHERE created > ‘2013-07-01’
AND created < ‘2013-08-01’
67. Andy Young // @andyy // andy@500.co
Use your existing data
SELECT COUNT(*) FROM users
LEFT JOIN sales USING (user_id)
WHERE users.created > ‘2013-07-01’
AND users.created < ‘2013-08-01’
AND sales.date < DATE_ADD(users.created, 1 MONTH)
68. Andy Young // @andyy // andy@500.co
1. Automate running queries (every hour!)
2. Store the results in a simple database
3. Create a page to graph the results
(HighCharts..)
Roll your own