9. 25-Jun-14
• When we estimate, we are making
assumptions on the true population
mean.
• If you don’t have every piece of
information (e.g. from every person), you
are estimating.
• Sometimes you are confident that your
estimate is good, others not.
9
Confidence in estimations
We usually estimate things. Our confidence varies.
11. 25-Jun-14
• “I am xx% confident that the true population
mean falls between yy and zz.”
• Measurements reduce uncertainty.
11
Confidence intervals
Measurements aim to narrow a confidence interval.
12. 25-Jun-14
• Monte Carlo
• Basic concept:
RETURNSOCIAL = [n(conversion traffic referred from social)
* av(value sale) *
P(social actually drove them there)] +
[n(customer acq through social) *
av(value of socially referred cust)]
Estimates
n(sales) == 2,000 av(val sale) = $110 (sd 25)
P(social drive) = 0.1– 0.7 n(cust acq) = 400-1200
Av(value social ref cust) = $340 (sd 90)
12
Simulation is important
Simulation helps model uncertainty & its impact
13. 25-Jun-14
• Midpoint estimate: $338,000
• Monte Carlo estimate: $359,590 (n=1k)
• Similar, yes. But very few numbers
estimated!
• Often used to work out
P(ROI>0)…
13
Monte Carlo Estimates
Monte Carlo simulates across many variable values
15. 25-Jun-14
1.Why are we conducting an ROI evaluation?
2.What are the costs and benefits of doing so?
3.Do we understand our social media strategy and
uses?
4.Do we have the required analysis and data
collection skills?
5.Are we aware of the SMROI principles and
framework?
15
Fast Five
Ask these questions before you begin
p5
16. 25-Jun-14
1. Understand the objectives
2. Involve stakeholders
3. Understand the platforms
4. Identify which metrics and KPIs change
5. Be selective in what is valued
6. Understand the valuation process
7. Make reasonable assumptions
8. Do not over-claim or over-fit
9. Be transparent
10.Verify the result
16
10 key SMROI principles
Following these principles will increase the SMROI integrity
p5
17. 25-Jun-14
• Business Strategy Avenue
Evaluating social media based on the costs and revenues of
executing business objectives
• User Lifetime Value
Evaluating social media based on the users connected to your social
media accounts
• User Network Analysis
Evaluating social media based on the user network and amplification
model
• Opportunity Cost Evaluation
Evaluating social media based on available alternatives
17
Remember the models
There are four main models to evaluate SMROI
p5
18. 25-Jun-14
Agenda
18
1
• What is SMROI?
2
• Why should you evaluate SMROI?
3
• Why isn’t it more common?
4
• How can you calculate SMROI?
5
• How does Dialogue Consulting do it?
20. 25-Jun-14
• Ratio between costs and revenues
• 100% ROI == (?success?)
– $1 expenditure, $2 made in revenues
20
SMROI?
SMROI is simply ROI applied to a social media context
p17
21. 25-Jun-14
Common examples include:
• Sales
• Advocacy
– Word of mouth
– User generated content
• Leads
• Donations
21
Revenues
There are many sources of social media revenues
22. 25-Jun-14
Common examples include:
• Personnel
• Social media budgets e.g. Facebook Ads
• Time
• Hardware
• Software
• Other resources
22
Costs
Social media costs are more easily identified
25. 25-Jun-14
• Understanding use
• Justify expenditure
• Gain insights into Fans and Followers
• Identify where value is generated
• Identify cost reduction opportunities
• Modify and optimise social media use
25
The SMROI rationale
Return-on-investment evaluations bring many benefits
p10
26. 25-Jun-14
26
Quantifying use
SMROI helps identify where social media generates
value
• Financial tangibility –
is the return monetary
(eg sales) or non-
monetary (eg brand
equity)?
• Time – is the return
made in the long or
short term?
p8
27. 25-Jun-14
The Social Media Exchange
27
adapted from Larson & Watson,
2011
Modelling use identifies value sources and
opportunities
p20
28. 25-Jun-14
Some examples:
• Modelling optimum marketing mix with social
media
• Determining best customer service delivery
• Identify most efficient HR recruitment methods
• Utilise most effective communication channels
28
Applications of the SMROI evaluation
SMROI findings can be used in multiple ways
30. 25-Jun-14
• No ‘stock’ formula or method
• Many different social media uses
• Difficult to link and quantify revenue sources
• Need to understand the process
• Requires financial and statistical knowledge
• Not considered important to evaluate
30
Barriers to SMROI evaluation
Tangibility, skill, and knowledge barriers prevent evaluation
“too hard”
“unnecessary”
p11
32. 25-Jun-14
• Model inputs
• Appropriateness and exhaustiveness
of the model
• Use of Proxies
32
1. Misunderstanding
Occurs before the SMROI is calculated
p39
33. 25-Jun-14
• Over-valuation
• Combined platform valuations
• Metrics
• Future value of the platform
33
2. Miscalculation
Occurs during the SMROI process
p40
34. 25-Jun-14
• Comparing ROIs
• Application to different platforms
• Accounting for risk
• Changes to platforms and users
34
3. Misinterpretation
Occurs after the SMROI is evaluated
p41
36. 25-Jun-14
The Dialogue Consulting Process
1
Context
2
Model Calculation
4
Evaluation
5
Reporting
36
There are five main stages to determining SMROI
p29
37. 25-Jun-14
37
It’s impossible to measure something you don’t understand
1 Context
A problem well stated is a
problem half solved.
- Charles Kettering
38. 25-Jun-14
1 Context
38
Understanding your social media use is crucial
Assessment
•Why and
how is social
media being
used?
Hypothesis
•What is the
SMROI
evaluating?
Cost-benefit
•How
extensive
should it be?
p29
39. 25-Jun-14
39
Use Assessment
Identifying social media use has two perspectives
Business
Perspective
What are your
goals, objectives
and strategies for
social media?
User
Perspective
How are Fans and
Followers engaging
with your
presences?
p29
40. 25-Jun-14
• Decide what to calculate
• Different approaches require different processes
– extensiveness
– models
– data
– reporting
• Helps clarify SMROI objectives
• Determine the value and costs of evaluation
40
Determine hypothesis
Establishing hypotheses shapes the SMROI process
p30
41. 25-Jun-14
41
Hypothesis examples
“Customer service delivery on social media generates a positive ROI”
(…therefore we should use it more and ask for more resources)
“Social media marketing isn’t generating positive ROI”
(…therefore we need to identify why, and modify our use)
“If we used social media, it would yield a positive ROI”
(…if so, we should implement a presence)
44. 25-Jun-14
• Model choices: which models can be used?
• Fit to SMROI purpose: does it apply to what is
being measured?
• Available data: do we have the right
information?
• Impact: does the data accurately reflect social
media use?
44
2 Model Identification
Four main factors determine appropriate model
selection
p31
45. 25-Jun-14
2.1 Models
45
There are four main way of quantifying social media
Business strategy avenue User lifetime value
Network Analysis Opportunity Cost
p17
46. 25-Jun-14
• Common Examples
– Customer Service
– Marketing
– HR
• Value and cost drivers depend on strategy
– Savings
– Revenues (sales etc)
– Time
– Resources
46
2.1.1 Business Strategy Avenue
Social media value comes from executing strategies online
p20
47. 25-Jun-14
• Users are ‘assets’
• Assumed to generate future revenue
– Sales, advocacy etc
• Compare revenue to cost of acquisition and retention
47
2.1.2 User Lifetime Value
Fans and Followers have value for businesses
p23
48. 25-Jun-14
48
2.1.3 Network Analysis
Social media can be valued by the connectedness of
users
• Users connect to businesses and
each other
• Users may share your content or
mention you online
• Word of mouth and advocacy yields
value
• Need to model user connectedness
• Weight users by value
• Approximates value of network
p25
49. 25-Jun-14
• Social media isn’t the only way!
• Examines the value of the next best option
• Value a strategy
– Marketing or customer service on social media?
• Value a platform
– Customer service on Twitter or by phone?
• Value of social media
– Quantify losses of not having a presence
49
2.1.4 Opportunity Cost
Social media can be valued by its best equivalent alternative
p27
50. 25-Jun-14
• What data is needed?
• How to collect it?
– Surveys and questionnaires
– Observation and tracking
• Reaching users
• Representative samples
50
2.3 Data Collection
Ensure the right data is sourced for the model and
evaluation
p45
51. 25-Jun-14
• Non generalizable findings
• Biased samples
• Non-responsiveness
• Badly phrased questions
• Unusable data
• Ethics
51
Issues with data collection
These main considerations impact on data collection
p47
52. 25-Jun-14
• Determines how much of an outcome relates to
social media
• Outcomes indicators are metrics which explain
costs and revenues
• Errors arise from assuming your social media
activity accounts for 100% of outcome indicators
52
2.4 Impact
Determine how costs and revenue factors relate to social
media
p32
53. 25-Jun-14
• Alternatives to social media activity which may better explain
outcomes
• Compare these to determine impact
53
Benchmark Indicators
Comparing against indicators reduces attribution
errors
p34
54. 25-Jun-14
• Deadweight: the outcome may have occurred
regardless
• Displacement: increase in social media activity may
decrease other activities
• Attribution: other factors may be contributing to the
outcome
• Drop-off: the outcome may reduce over time
54
Determining Impact
Four factors explain the difference between indicators
p32
55. 25-Jun-14
3 Calculation
• Build the model: does it reflect your social
media use?
• Calculate the ROI figure: what is the return?
• Apply a sensitivity analysis: what are the
most influential factors?
• Forecast changes: how might the ROI change
in the future?
55
Determining SMROI should include sensitivity and
forecasting
p35
56. 25-Jun-14
4 Evaluation
• Compare ROI values: how did social media
perform?
• Improvements: how can social media be used
better?
56
Implementing findings can optimise social media use
p35
57. 25-Jun-14
• How does investment in social media
perform against traditional channels?
• Modify social media use accordingly
57
Compare the ROI figures
Determine how social media and other investments
compare
p36
58. 25-Jun-14
• How to gain more positive ROI?
• Remember the ratio!
• Examine current processes
• What could generate higher return?
• Where can cost savings be achieved?
58
Improve the SMROI figure
Examine current activity to increase ROI
p36
59. 25-Jun-14
5 Reporting
• Format: Who is it for?
• Process: How was it calculated?
• Rationale: What outcome is intended?
– Increased budget
– More resources and personnel
– Greater social media scope
• Assurance: is it accurate?
59
SMROI findings must be communicated appropriately
p37
60. 25-Jun-14
• Who conducted the SMROI
• Why the SMROI was calculated
• Stakeholder involvement
• SMROI evaluation methodology
• Rationale of proxies, assumptions, estimates or benchmarks used
• Explanations and demonstrations of calculations
• Review of the sensitivity analysis and the impact of changing
variables
• Recommendations based on findings
60
What to include in an SMROI report
These key areas should be covered in your report
p37
61. 25-Jun-14
How do we do it?
A case study…5
61
Move to part 2 of your workbook!
62. 25-Jun-14
• Train provider for large city
• Two presences
• 12,000 Facebook Fans
• 33,000 Twitter Followers
• Frequent social media
posting
• Dedicated social media
team (two employees)
62
Connectro Case Study
Dialogue Consulting completed an ROI evaluation
p4
63. 25-Jun-14
1. Consultation with Connectro
2. Determine what they wanted to evaluate
– Customer service
– Marketing
3. Examine how they use social media
4. Data collection and analysis
5. Calculation and evaluation
6. Findings and recommendations
7. Reporting
63
Process
Application of the SMROI procedure to Connectro
p4
64. 25-Jun-14
• Justify expenditure
• Identify potential value from social media
• Quantify current revenues and costs
• Optimise use of social media
• Identify opportunities for social media use
• Focus on marketing and customer service
strategies
64
Objectives
Connectro and Dialogue identified key objectives
p7
65. 25-Jun-14
• Provide service updates
• Other Connectro and
train usage information
65
Information channel
Connectro keeps users updated via social media
p6
66. 25-Jun-14
• Encourage train usage
• Promote Connectro services
• Frequent social media
campaigns
• Some paid promotion
66
Marketing Strategy
Connectro uses social media to promote their services
p6
67. 25-Jun-14
• Respond to user inquiries
• Focus on quick resolution
• Guaranteed response
• Gain feedback from users
67
Customer Service
Connectro uses social media as a customer service channel
p6
68. 25-Jun-14
Overall Marketing Customer Service
Costs
• Time and
resources
• Software
Paid promotion
Value of lost use
due to customer
dissatisfaction
Revenue
s
• Efficiency
and cost
savings
Value of
increased use
due to
campaigns
Value of increased
use due to user
satisfaction
68
Cost and Revenue Identification
Dialogue identified cash flows related to social media
use
p11
69. 25-Jun-14
• Positive figures
• Every $1 gained
$1.34 return
69
SMROI Results
Dialogue found Connectro had an overall ROI of 34%
p12
72. 25-Jun-14
72
Identify the value drivers
Significant cost and value drivers provide
opportunities
Greater utilisation
and increase
efficiency of social
media
Both
Savings/Resources
Focus campaigns to
encourage behaviour
change
Improve user satisfaction
Marketing
Increased train use
Customer service
User satisfaction
p15
73. 25-Jun-14
• Increase and focus on customer service satisfaction
– Implement proper training and processes
• Transfer additional customer service delivery to social media
– Scale the customer service team and investment as required
• Focus campaigns to encourage behaviour change
– Reduce paid marketing on other channels
– Apply demographic targeting strategies
– Better utilise specific platform features
– Research and target high value customer segments
– Increase user awareness of their social media capabilities
73
Improving Connectro’s SMROI
Apply SMROI findings to optimise social media
strategy
p15
74. 25-Jun-14
• Focused on key stakeholders
• How to best communicate findings
• How to implement recommendations
• Provided assurance
74
Reporting
Dialogue produced tailored SMROI reports for
Connectro
p16
80. 25-Jun-14
80
Recommendation 5
To be valid and accurate, ROI models
should remain fluid to reflect changes
in social media platforms, uses and
users.
Want to answer these questions before you begin
Know rationale for SMROI (the ROI of SMROI – costs and benefits)
Know how you use SM, to then identify what to measure, and therefore what data you need
May discover you are doing more on social than you think
High value examples
Four overview types detailing how extensive the roi should be
Look at hypothesis and ‘value’ of ROI and match with costs
Other benefits:
Eg NPS
Identify most influential users, determine most valuable customers/online users (ie for better targeting)
Savings as compared to “traditional” and non social
Dialogue provided actionable recommendations to Connectro
