Workshop Girls in Tech #6 by Matteo Panfilo, Matteo Scarabelli e Umberto de Feo Cosa significa individuare Key Performance Indicators e saper misurare le metriche per business legati a internet ed al digitale? Cosa tener presente per effettuare delle stime (anche ai fini del business plan), quali obiettivi e quali strumenti utilizzare per ottimizzare i propri ritorni (e “tassi di conversione” per i business online)? Come e con quali strumenti interagire con i propri clienti?

1. KPIs & metrics
(slides by Matteo Panfilo)
Presenters
scarabelli.matteo3@gmail.com
@mattepv
umbertodefeo@yahoo.it
@umbertodefeo
GiT Workshop - January 2014

2. About us
What I’ve done until 2013
I studied economics in Pavia & management in France
I wrote a thesis on Angel Investing
I attended the AIFI Class for VC professionals in 2011
I worked over two years @ Italian Angels for Growth
(11 investments in the period = >4mln)
I worked one year @ Fashionis (e-commerce startup)
What I'm doing since the beginning of 2014
I co-founded @Fast4ward
I keep on promoting local startup ecosystems @ Activators Pavia
What I’ve done until 2013
I studied International Business and Finance at LUISS
I wrote a thesis on the investment opportunities in
the green energy sector in South East Asia
I lived and worked in Kiev, London, Rome, Kuala Lumpur and Milan
I attended the AIFI Class for VC professionals in 2012
I worked @ Italian Angels for Growth
I worked @ Zernike META Ventures
What I'm doing since the beginning of 2014
I started working on the international activities @META Group
1

3. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
2

4. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
3

5. 1. Different businesses, different metrics
Airbnb
Number of house with HD pics
Friendfinder
Viral factor (K)
Blogo
Pages per user
6sicuro.it
Privalia
Generated quotes (lead)
Cost per shoting
4

6. Hundreds of different metrics
PV
UV
CPM
BR
CTR
CPR
CPL
CPA
CPC
LTV
CR
Churn
Page Views
Unique Visitors
Cost per Thousand
Bounce Rate
Click Trough Rate
Cost Per Registration
Cost Per Lead
Cost Per Acquisition (or
Cost per Action)
Cost Per Click
Lifetime Value
Conversion Rate
Churn Rate
5

7. Lean Analytics
http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/
6

8. Lean Analytics (alcuni esempi)
http://www.startuplessonslearned.com/2013/03/lean-analytics.html
http://www.slideshare.net/Leananalytics
http://www.kaushik.net/avinash/lean-analytics-cycle-metrics-hypothesis-experiment-act/
7

9. Scalability (basic formulas)
CPA
<
LTV
“LTV is the projected profit that a
customer will generate during his
lifetime”
CPC
%CR
Dipende fondamentalmente
da: soddisfazione dei clienti,
CRM, biz model, etc
General CPA = $ Adv cost / # Customers acquired
CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired
LTV formulas
•LT Revenues: # Purchases x $ Avg. basket
•LT Revenues: $ Avg. basket / % Churn rate
•LT Gross Profit: # Purchases x $ Avg. basket x % gross profit margin
•LT Gross Profit: $ Avg. basket / % Churn rate x % gross profit margin
•LT DCF: ($ Avg. basket margin x Life Expectancy) x (Customer retention rate / 1 + discount
rate – Customer retention rate)
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
8
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/

10. Scalability (basic formulas)
CPA
CPC
<
LTV
Small numbers,
huge impact:
CPC = 0,4€
%CR = 2,5%
CPA = 10€
%CR
CPC = 0,4€
%CR = 4%
CPA = 16€
CPA = Cost Per Acquisition (or Cost per Action), CPC = Cost per Click, LTV = Lifetime Value
LTV Starbucks: http://blog.kissmetrics.com/wp-content/uploads/2011/08/calculating-ltv.pdf
9
LTV Avinash Kaushik: http://www.kaushik.net/avinash/analytics-tip-calculate-ltv-customer-lifetime-value/

11. Scalability, make it happen!
Note: CAC = CPA
Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
10

12. Scalability, how you can do it?
Note: CAC = CPA
Source: David Skok - http://www.forentrepreneurs.com/startup-killer/
11

13. Conversion
CPA =
CPC
%CR
Define what are you looking to measure!
i.e.: define who you consider a customer; define the difference between
CPL and CPA (if there is)
What’s the meaning of conversion rate?
What’s the meaning of funnel?
General CPA = $ Adv cost / # Customers acquired
CPA based on CPM campaign: (CPM x Impressions /1000)/ # Customer acquired
12

14. The Funnel .. <offline mode>
1. A lot of people see the adv
wall
2. Some people recognize it
walking in front of a
sunglasses shop
3. Some of them decide to go
in and try them
1. One of them want to buy it
13

15. The Funnel .. <online mode - ie. ecommerce>
1. I paid to have some users on my
website
100
50
50%
20
40%
2. They arrive on the landing page
3. They leave their email
1. They put one product in the basket
2. They start the checkout process
5
40%
3
38%
1%
33%
1
3. They complete all the payment
info
4. They pay
14

16. The Funnel .. <online mode - ie. ecommerce>
Some drivers:
Paid users
Landing page
•
•
Pictures
Copy/text button
50
Leave the email
•
•
•
Info required
Password
Email confirm
20
Product on the basket
•
•
Add button
Related product
5
Checkout process
•
•
•
Info required
Shipping days
Delivery cost
3
Payment info
•
•
Payment gateway
Invoice/legal info
1
They pay
•
•
CC circuit
PIN request
100
15

17. Why measure the funnel
You need to identify the bottleneck and solve it!
Examples:
- Pictures
- Checkout process
- Payment gateway
16

18. The Funnel .. (flash sales ecommerce)
Measure your CPL/CPA!
Unique users
Users that complete
the form
Registered users
Users that
arrive till
checkout
Paying
users
1%-0,5%
The CPA initially refers only to
the traffic resulting from paid
campaigns is the main KPI to
be monitored in an
ecommerce / SaaS, but in
the medium term it should
be monitored along with the
metrics of the traffic that
came from other channels
(email, facebook page, etc.
)
17

19. Customer journey
Awareness
Consideration
Conversion
Loyalty
Desire
Information
Choiche
Users
Customers
Source: IAB Seminar 2013, Ilaria Zampori Managing Director QUISMA, Individual Conversion Attribution
http://www.iabseminar.it/wp-content/uploads/2013/02/9_QUISMA.pdf
18

20. Customer journey
Awareness
Consideration
Conversion
Loyalty
Desire
Information
Choiche
instruments
instruments
instruments
Source: IAB Seminar 2013, Marco Caradonna Simple Agency Dynamic Customer Journey
http://www.slideshare.net/mcarado/4-iab-seminar-performance-2013-simple-agency
19

21. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis, measure customer behaviour
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
20

22. Cohort Analysis referral source analysis
“A cohort is a group of people who share a common
characteristic over a certain period of time”
Customer spending by referral source
Direct
Facebook
Google
Blog
21
Source: www.cohortanalysis.com/

23. Cohort Analysis time-period analysis
How much revenue do the users that I’ve acquired on the second months
generate?
How many users that I’ve acquired on the second month will I probably still
have on the ninth month?
Time period
1
1
2
3
4
5
6
7
8
9
10
11
12
513
410
328
263
210
168
134
108
86
69
55
44
923
739
591
473
378
303
242
194
155
124
99
1.258
1.007
805
644
515
412
330
264
211
169
1.537
1.230
984
787
630
504
403
322
258
1.777
1.421
1.137
910
728
582
466
373
1.987
1.590
1.272
1.017
814
651
521
2.178
1.742
1.394
1.115
892
714
2.355
1.884
1.507
1.206
965
2.524
2.020
1.616
1.292
2.689
2.151
1.721
2.852
2.282
2
3
4
5
Time-period cohorts
6
7
8
9
10
11
12
paying users
3.017
513
1.334
2.325
3.398
4.495
5.583
6.644
7.671
8.661
9.618
10.547
11.454
22
Source: andrewchenblog.com

24. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate, do not throw the job done
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
23

25. Churn Rate
Your churn rate is the amount of customers or subscribers who cut ties
with your service or company during given time period.
These customers have “churned”
Recurring revenues model
# Subscribers lost
# Starting subscribers
= % Churn rate
Traditional Ecommerce
# Customers who has
not made a
purchase in 90 days*
= % Churn rate
# Starting active
customers
* it depend on
business model
and offer
24
Source: www.churn-rate.com

26. Optimize..
1.Metrics & KPIs
2.Funnel
3.Cohort analysis
4.Churn rate
Analyse
Iterate
Learning
process
Consolidate
Change
Learn
25

27. Why you need to optimize..
• You have 0-8 seconds to make a compelling headline and
landing page. After 8 seconds, the majority of visitors leave.
• Approximately 96% of visitors that come to your website are
not ready to buy.
• The more landing pages you have, the more leads you are
likely to get.
• Product videos can increase purchases of the product by
144%.
• A 1 second delay in your site speed can result in a 7%
reduction in conversions.
• A/B testing is becoming the preferred method that has
brought a lot of the companies the most success.
Source: http://blog.kissmetrics.com/what-converting-websites-do/
26

28. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test, what do customers prefer?
6. Source channels, email/DEM
7. CRM
Conclusion
27

29. A/B test…
• Elaborate 2 different versions (text, colors, font, layout, etc)
• Propose versions to the target sample (with the same content)
• Measure conversion rate
• Profile users based on conversion rates (male/female, addicted, etc)
• (Visualize where the users click on the pages)
The same process could be applicable to web pages, email/DEM,
games dynamics, etc!
28

30. Headline..
Original
+30%
+15%
Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
29

31. Layout..
Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
30

32. Layout..
Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
31

33. Layout..
Is not always a good
idea to write more…
Fonte: http://37signals.com/svn/posts/1525-writing-decisions-headline-tests-on-the-highrise-signup-page
32

34. A/B test are easy but..
Consider a site that has 10,000 visitors per month and has a 5 percent conversion
rate. The table below shows how long it will take to run a “conclusive” test (95
percent confidence) based on how much the change impacts conversion rate.
Source: http://techcrunch.com/2013/01/12/current-conversion-rate-and-desiredconfidence-interval-will-help-you-avoid-analysis-paralysis-stop-running-stupid-tests/
33

35. Other tips..
• Badging
up to +50%
• Message consistency
up to +100%
• International website
up to +100%
• Email sign-up
up to +1.000%
Fonte: http://www.slideshare.net/Emerce/emerce-performance-bryan-eisenberg
34

36. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
35

37. Email & DEM
Email could become your first traffic source, improve the LTV of your
customers, decrease your customer acquisition cost !
1. Choose & Optimize email form
2. Choose & Optimize password request
3. Choose the confirmation process
4. Choose emails’ arguments and style
5. Choose & Optimize email subjects
6. Metrics, metrics, metrics:
• Open rate/Unique OR
• Click rate
• Click-through-rate
• Delivery time
• List growth
• Unsubscribe rate (or churn rate)
• Spam complaint rate
Drivers:
 Subject, time
 Content
 Layout
 All of the above
 Too many
 Frequency & content
 Too many
36

38. Email & DEM
37
Source: http://retail-analytics.quora.com/Email-Open-Click-Through-Analytics-from-over-80-Billion-Emails

39. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
38

40. The true cost of bad support
Source: https://www.helpscout.net/blog/bad-customer-service/
39

41. Interact with customers…
People now expect the same—if not better— level of
service from online storefronts and service channels as
they do from a visit to a retail store or a phone
conversation with an agent.”
Source: “ E mail Customer Service in North American Small and Medium Businesses” by
BenchmarkPortal, 2005 and https://www.helpscout.net/blog/bad-customer-service/
40

42. Interact with customers…
Source: Communication Preference, Live Chat Effectiveness, A Survey of Internet Shoppers 2012
41

43. Interact with customers…
83% of small and medium-sized
businesses responded with
inaccurate or incomplete answers.
41% of consumers expect an e-mail
response within six hours.
Only 36% of retailers responded that quickly.
Source: “Email Customer Service in North American Small and Medium Businesses” by BenchmarkPortal, 2005,
42
Source: Forrester Research Inc., 2008

44. Interact with customers…
“Customer experience quality could
result in a swing of $184 million for a
large Internet retailer.”
Source: Forrester Research Inc., 2008
43

45. The power of great video…
Video: http://www.youtube.com/watch?feature=player_embedded&v=ZUG9qYTJMsI
44

46. Agenda
1. Metrics & KPIs
2. Funnel
3. Cohort analysis
4. Churn rate
5. A/B test
6. Source channels, email/DEM
7. CRM
Conclusion
45

47. Recap & useful tools
1.Metrics, measure and understand your KPIs
2.Funnel, know the steps to improve your conversion
3.Cohort analysis, analyse customer behaviour
4.Churn rate, do not throw the job done
5.A/B test, what do customers prefer?
6.Source channels, email/DEM still a very interesting
channel
7.CRM, interact and learn from your customers
-
Mixpanel
Jirafe free
Woopra
Kissmetrics
RJ metric
Geckoboard
46

48. My 2 cents
Non pensate che fare una startup sia un gioco.
Cercate di creare valore, non fate le cose perché vanno di moda
e non sviluppate l’ennesima app per condividere le foto.
Non pensate a raccogliere soldi ma pensate a creare un prodotto
e ad avere una visione per i successivi 10 anni.
Studiate e documentatevi tanto prima di prendere ogni decisione,
c’è un’enorme quantità di informazioni online, basta saperle
cercare.
“I believe in the power of a well-operated, sophisticated organization
that generates social and economic value. My career goal is to found
and lead this type of organization.”
Hansoo Lee
47

49. Must read/view:
• Steve blank tools (www.steveblank.com/tools)
• Avinash Kaushik (www.kaushik.net)
• Quora (follow Matthew Caroll)
• Slideshare (xerox, 500startups, etsy)
• Techcrunch
• Angel.co
• Italian Startup Scene
• Indigeni Digitali
• Startupnumbers.com ;)
48

50. Must read/view:
• Blog etsy (http://mcfunley.com/)
• Blog Fabrice Grinda
• Blog fab.com
• Blog google
• Blog a16z.com
• Foundat.io/n
• ecorner.stanford.edu
• avc.com
• Angel.co
49

51. Primo esempio: wikli
+70%
Spesa tot: 200€
Strumenti utilizzati: opimizely
Utenti attuali >6.000

52. Secondo esempio..

53. Secondo esempio…iterazione

54. Secondo esempio…iterazione

55. Terzo esempio…non solo web

56. Quarto esempio (fashion subscription commerce)
Conversion rates:
From inception October 30th until December 31th
• Unique visitors to registered free members: 27%
Benchmark successful players: circa 22%
• Free members to paying subscribers: 2,6% in 2 mesi
Benchmark successful players: circa 3% in un anno
Absolute metrics:
Form inception November 5th until December 30th
•26k unique visitors
•7k registered members
•200 subscription sales
•7k $ revenues
Traffic sources:
•Direct: 59%
•Facebook 23%
•Daily Deal Sites 10%
•Others 9%

57. Always in test..
“We put it in a feature that we can
build in a week. It's a ghetto build. We
AB test it. We flow test it. We put it out
to 1%, 10% of our users. We build a
data warehouse with the testing
platform. So, we're running several
hundred tests at any given time for
every one of our games. And no single
user ever is involved in more than one
test. So, we love testing. When we see
that it moves our metrics in a
considerable way, that's when we take
it to be a full feature roll out and then
we do the full 2.0.
Capitalizzazione 3,6 $bn --- Ricavi 1,22 $bn --- 12% ricavi da facebook
Video (da vedere!): http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313

58. Always in test..
“We don't want to go down those paths until we've test them
with our users. And what's amazing and this is a feature that you
will all have available to you as you enter this third Internet
business is that you're going to run a service and you are going
to every week be able to test things with your users and
something that I never have available to me in previous
companies. It won't be build it for three months and hope and
pray. It's test it now. And you can test it with - you don't need to
be a big company - you can test it by buying Google AdWords
or Facebook Self Service ads. You can put these links out. You
could spend 50 bucks a day and know most of what we know.
So, it's an awesome way...”
Mark Pincus – Founder & CEO @ Zynga
Sources: http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2313