Cision Webinar: State of Social Media in PR: Everything You Always Wanted to ...
Social Media Measurement 101 + Day of aTONEment
1. SOCIAL MEDIA
MEASUREMENT 101
A Day of aTONEment
MRA ANNUAL CONFERENCE
JUNE 6, 2011
Alan Chumley
Senior Vice President
@alanchumley / @CARMA_Tweets
@ l h l @CARMA T t
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
2. ALAN CHUMLEY, SVP
• 14 years in the industry
• Recovering PR practitioner: ING, Bell Canada
R i PR titi ING B ll C d
• Former Director of Measurement @ Hill & Knowlton
• Masters in Communications & Culture:
• Research foci:
• media effects & uses
• reception analysis
reception analysis
• audience studies
• University‐level adjunct instructor of PR measurement
• LinkedIn early 2006
Li k dI l 2006
• Blogging / Tweeting since early 2007
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 2
@alanchumley / #mra_ac
3. ABOUT CARMA
• Traditional & social
• Media monitoring & analysis
M di it i & l i
• Largely human‐based….on an automation journey
• In biz since 1984
• Head office here in DC
• 14 offices in 11 countries
14 offices in 11 countries
• Clients:
• PR f lk i bi
PR folk in big organizations
i ti
• Market research folks in big organizations are starting to emerge
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 3
@alanchumley / #mra_ac
4. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
IN-ROOM STRETCH (5 mins)
PART 2 (30 mins): TONE ALONE
i )
PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
5. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
6. ISN’T
• Not a Case Study
• Not a mention of Old Spice in sight
• Not a specific ‘how to’ / no one-size-fits all
• Doesn’t focus on a singular or few social
media platforms—looks at all holistically
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
7. ISN’T IS
• Not a Case Study • An emerging / experimental philosophy /
approach / filter / lens through which to look
at measuring social media
• Not a mention of Old Spice in sight
• A high level connected / systems way of
• Not a specific ‘how to’ / no one-size-fits all approaching the problem
• Doesn’t focus on a singular or few social • About communities not an audience
b d
media platforms—looks at all holistically
• Beyond simple counts and sentiment
• More sociologist than publicist
• Influence potential
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
8. Every hour more than 500,000 new
blog + micro-blog posts, status updates
micro blog posts
and comments are posted
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
9. MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES
Why is this the case?
Gap between leap of
faith true believers and
C-suite’s need for
metrics to validate the
time/$?
$
Source: Harvard Business Review: the New Conversation: Taking Social Media from Talk to Action
Surveyed 2100 Harvard Biz Review subscribers
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 9
@alanchumley / #mra_ac
10. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down
Command & Control
of the Message is Gone
• Transparency demanded
• Dialogue vs. Monologue
• Many-to-many not one-to-
many
• Conversation & community
NOT (strictly) coverage
• About relationships
• Relationship trust is co-
created by the
organization and its
communities of interest via
conversation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
11. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down There is no Passive
Command & Control Ready-to-Consume Audience’
of the Message is Gone
• Transparency demanded • Prosumers not just consumers
• Dialogue vs. Monologue • Actively engaged in creating and
re-creating conversations and
g
• Many-to-many not one-to- communities
many
• Challenge in tracking &
• Conversation & community measuring something as
NOT (strictly) coverage nebulous as a community
• About relationships
• Relationship trust is co-
created by the
organization and its
communities of interest via
conversation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
12. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down There is no Passive Disciplinary Lines
Command & Control Ready-to-Consumer ‘Audience’ are Eroding
of the Message is Gone
• PA, PR, Marketing, CRM,
• Transparency demanded • Prosumers not just consumers Customer Service etc.
• Dialogue vs. Monologue • Actively engaged in creating and • Customer touch points
re-creating conversations and
g
• Many-to-many not one-to- communities • Integrated cross-discipline
many re: strategy
• Challenge in tracking &
• Conversation & community measuring something as • Integrate cross-discipline
NOT (strictly) coverage nebulous as a community re: measurement
• About relationships
• Relationship trust is co-
created by the
organization and its
communities of interest via
conversation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
13. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY
Top Down There is no Passive Disciplinary Lines
Command & Control Ready-to-Consumer ‘Audience’ are Eroding
of the Message is Gone
• PA, PR, Marketing, CRM,
• Transparency demanded • Prosumers not just consumers Customer Service etc.
• Dialogue vs. Monologue • Actively engaged in creating and • Customer touch points
re-creating conversations and
g
• Many-to-many not one-to- communities • Integrated cross-discipline
many re: strategy
• Challenge in tracking &
• Conversation & community measuring something as • Integrate cross-discipline
NOT (strictly) coverage nebulous as a community re: measurement
• About relationships
• Relationship trust is co-
C-Suite
C Suite is Intrigued but Anxious / Want Data to:
created by the
• Show why and how they should…
organization and its
• Demonstrate the value of having done so…
communities of interest via
conversation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
14. SIMPLE STATUS QUO
Counts:
Quick Quantity counts:
•Fans, Likes Follows, Etc.
•Fans Likes, Follows Etc
•Tantamount to reach in the offline or 1.0 world
•Reach does not equal Influence
Quality:
•Quick automated read on sentiment: but concerns re: accuracy
Rankings:
•Visibility, popularity, traffic, link love
Visibility,
•Technorati for blogs
•NOT RECOMMENDED: Twitrratr, Klout,
•BETTER: Twendz, Traackr, Twittalyzer, Twinfluence
•They rarely qualify people or content (the message)
The rarel q alif
•The message AND the medium are the measure
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 14
@alanchumley / #mra_ac
15. LOT’S OF TOOLS OUT THERE
BEWARE OF THE SILOs AVOID USING (ONLY) THE NEW SHINEY FREEBIES
•Isolated / silo’d approach
•No one tool on it’s own is sufficient—not even one paid tool
it s sufficient not
•Have to connect the dots with a more relational, systems approach
Would you like your job performance
(read: raise/bonus) to be based only on a
1615 M Street, NW; Suite 750
Klout Score, for example?
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
16. REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE
& NOT ALL CONVERSATIONS ARE RELEVANT
Off topic / Irrelevant
Relevant
R l t Relevant “ ”
R l t “-”
Relevant “+”
Neutral
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 16
@alanchumley / #mra_ac
17. REMEMBER: IT’S SOCIAL MEDIA
•It’s not coverage
•It’s not static
It s
•There isn’t an ‘audience’ of consumers
•Prosumers not just consumers:
•simultaneously receiving, interpreting, co-opting,
re-appropriating and creating content
•It’s more many-to-many di l
It’ t dialogue th one-to-many monologue
than t l
•So, traditional off line ways of measuring ‘coverage’ don’t cut it
•They don’t adequately account for the multi-dimensionality and dynamic
nature of a conversation and a community of interest in a network
•We need to think bigger
gg
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 17
@alanchumley / #mra_ac
18. THINK BIGGER / INTEGRATE / CORRELATE
•Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all that Way
•Think about all customer‐facing touch points
•Understand PR / Marketing / Business Metrics
g p
•Integrate Cross‐Discipline
•Report on the Critical Few
•Focus on /correlate with Outcomes
•Close the loop / connect the dots
Source: W b Sh d i k Measurement & Strategy Practice
S Weber-Shandwick M t St t P ti Source: Oliver Blanchard, S i l M di ROI
S Oli Bl h d Social Media
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 18
@alanchumley / #mra_ac
19. THE MEASUREMENT MASHUP
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 19
@alanchumley / #mra_ac
20. THINK BIGGER & BROADER
MOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS
Systems or N t
S t Networked
k d
Thinking:
Interdependent
relationships between
p
those in a network
Social Capital:
Connections within and
between those in a social
network as a source of
civic engagement among
a community leading to
1615 M Street, NW; Suite 750
cohesion; that has ‘value’
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 20
@alanchumley / #mra_ac
21. FOCUSING ON CONTENT ANALYSIS AND NETWORK ANALYSIS
AS A WAY OF MEASURING INFLUENCE POTENTIAL
Systems or N t
S t Networked
k d
Thinking:
Interdependent
relationships between
p
those in a network
Social Capital:
Connections within and
between those in a social
network as a source of
civic engagement among
a community leading to
1615 M Street, NW; Suite 750
cohesion; that has ‘value’
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 21
@alanchumley / #mra_ac
22. The 7Cs THINKING MORE LIKE A SOCIOLOGIST THAN PUBLICIST
Counting What some call search, syndication and site; stuff we can count like traffic, popularity, fans, followers,
retweets, etc.
Content Analysis that is. Using content analysis, ideally a hybrid of some automation for volume and speed and
y g y , y y p
human analysts for context, nuance and depth to look for how your brand, messaging, topics, issues,
stakeholder voices are portrayed; the inclusiveness, accuracy and tone of the conversations.
Conversations And conversationships recognizing that social media is not strictly another communications channel
through which to get word out or reach and influence. It is more about genuinely engaged dialogue
than a simple corporate monologue; not only a vertical, one to many conversation between an
than a simple corporate monologue; not only a vertical, one‐to‐many conversation between an
organization and stakeholders but a lateral, many‐to‐many conversation among stakeholders.
Cohesion The extent to which folks are agreeing with you, your position, your brand essences. The extent to
which they are agreeing with each other and coalescing around and advocating a core theme, idea, or
some call to action either in support or opposition to your organization s position.
some call to action either in support or opposition to your organization’s position
Community The extent to which a core group of people with common interests are gathering and growing.
Measuring the change in community size, and volume and nature of chatter over time
Connectedness A look at how inter‐connected the highly engaged/key influencers/advocates are and how centrally
located they are in both the network and dialogue. (See Appendix for the 5Ps of influence)
Conversion Stay tuned. More to follow.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 22
@alanchumley / #mra_ac
23. THE 5Ps OF INFLUENCE
Popular Visible, vocal, has a substantial following, reach. In‐bound links, trackbacks, subscribers,
bookmarks, followers, friends, views, listens, saves, downloads, etc.
Polarized in tone Neutrality does little to drive influence way or the other. A clearly positive or
negative view will polarize readers/followers and is more likely to drive cohesion and
mobilize advocates and have those advocates coalesce around a core theme, idea, or call
to action.
Prolific / Relevant / Raw author contribution and # of on‐topic, related posts
Frequent
Prominent / Are they an idea starter or spreader; source or spider? They may be prolific but are they
prominent? Are they highly inter related inter connected and centrally located in the
Are they highly inter‐related, inter‐connected, and centrally located in the
Authoritative network? How engaged is this person’s following in a dialogue? How much dialogue is
there and what is its nature? Here we need to recognize, though, that authority is
contextual and topical. One might be an authority on PR measurement but not on 18th
century Russian literature.
Promoter / Loyal
/ l How many of th f ll
H f the followers/commentators active contributors advocating, endorsing,
/ t t ti t ib t d ti d i
advancing (or the opposite) your position? Are they adding links, tags. Is the nature of the
Advocate language they are using inter‐connective, expanded, clarifying, reinterpreting? RTs, digs,
fans, votes, buzz‐ups, up/downloads, shares, likes, invites, favorites, embeds. (More active
than the metrics in popularity)
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 23
@alanchumley / #mra_ac
24. VARIETY OF OBJECTIVES & CAMPAIGN TYPES (NON-EXHAUSTIVE)
Objectives
Campaign Types
• Visibility
• Push (get bloggers to review / write)
• Visits
• Pull (drive visitors on‐line)
• Volume
• CSR
• Velocity
• Product Launch
Product Launch
• Awareness
• Product Engagement
• Message delivery
• Marketing
• Link love
• Blogger Outreach
• Dialogue vs. Monologue
Dialogue vs Monologue
• Advocacy
• Generate dialogue / comments
• Issues Management
• Feedback
• Community building
• Using, sharing, pass‐a‐long
• Customer relations
Customer relations
• Embedding of interactive content
• Thought leadership
• Diggs, Delicious tags, stumbles, etc.
• Executive profiling
• Votes
• Etc., etc., etc.
• Etc., etc., etc.
Etc., etc., etc.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 24
@alanchumley / #mra_ac
25. SOME THINKING ON AN APPROACH
AN INDEX IS ONLY PART OF THE ANSWER
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 25
@alanchumley / #mra_ac
26. COMPONENTS OF AN INDEX ?
Popularity + Relevance + Authority + Presence + Engagement + Advocacy/Loyalty + Cascade
x (+/‐) Sentiment
=
Influence Potential
• Based on an assumption that the components above are drivers of on-line influence, potential action
• They can be measured individually, but it’s important to measure them collectively as they are inter-
related
• The index is intended as a measure of potential to influence not actual real influence that is another
influence—that
research project all together.
• All components are important (but not equal) drivers of potential influence on-line
p p ( q ) p
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 26
@alanchumley / #mra_ac
27. INDICATORS DEFINED
• Cascade High High •Authority
• Citation analysis / ripple index authority authority •Of both poster and commentators
• Network analysis low AND •Idea starters or spreaders?
relevance relevance •Degree of inter connectedness / inter relatedness
•Degree of inter‐connectedness / inter‐relatedness
• Volume & velocity of message spread across the
V l & l i f d h
network •Connected but respected?
• Inter‐connectedness / inter‐relationships among: •How central is the influencer to the network?
Low High •Following: follower
• all authority relevance
• Only the influentials AND low
•RTs/1000 followers
relevance authority
• Advocacy (active)
y( )
• Are social media users advancing our cause? • Relevance
Rele ance
• % containing links • On topic?
• % active contributors • Author contribution: how often / prolific?
• Tags added • Topical / textual / temporary
• Engagement (more coming)
Engagement (more coming) • Sentiment (more coming)
S ti t( i )
• Dialogue? 2‐way? How much? How good?
• How many commentators? Repeats? Nature of • Multi‐level, multi‐author tone
the language • Post‐level, theme level, sentence level
• Network analysis: inter‐connected/related • Gaps in tone in originating content and resulting
discussion
• Entity Presence
Entity Presence
• Present? Prominent? Frequent? • Popularity (passive)
• # of + mentions
• Competitive share of original post • ‘Reach’ ?
• Competitive share of resulting discussion • Voice of the consumer stuff we can count
• Attribution recognition • Links, votes, shares, recommends, subscribers,
registrations, tweets, RTs, followers, fans, friends,
diggs, shares, likes, fav s, upvotes, buzzups,
diggs shares likes fav’s upvotes buzzups
embeds, tags, wish list saves, forwards, invites,
etc.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 27
@alanchumley / @CARMA_Tweets
28. NETWORK ANALYSIS
Source: K. Ognyanova, UCLA, 2010
How are these changing over time?
Might be part of many systems / networks?
Linking networks to networks
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / @CARMA_Tweets
29. (A QUICK ASIDE ON) SENTIMENT AS MARKET RESEARCH PROXY / PREDICTOR
Think here about social media monitoring and
analysis as:
•Real time situational awareness
•Listening at the point of need
•Listening close to the moment of influence
A real-time digital form of or proxy for market
research and CRM:
Mining conversations to tease out how people
‘feel’ about products/services/issues/causes
and perhaps even:
1) their intended (reported) behavior and what
motivates them to buy, (as cited in the
conversations), and
2) customer satisfaction
3)their actual behavior tracked via CRM 2.0 and
Web 2.0 analytics
y
1615 M Street, NW; Suite 750 Source: Crimson Hexagon
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 29
@alanchumley / #mra_ac
30. BUT REMEMBER THAT OBJECTIVES DRIVE METRICS
Objectives
Campaign Types
• Visibility
• Push (get bloggers to review / write)
• Visits
• Pull (drive visitors on‐line)
• Volume
• CSR
• Velocity
• Product Launch
Product Launch
• Awareness
• Product Engagement
• Message delivery
• Marketing
• Link love
• Blogger Outreach
• Dialogue vs. Monologue
Dialogue vs Monologue
• Advocacy
• Generate dialogue / comments
• Issues Management
• Feedback
• Community building
• Using, sharing, pass‐a‐long
• Customer relations
Customer relations
• Embedding of interactive content
• Thought leadership
• Diggs, Delicious tags, stumbles, etc.
• Executive profiling
• Votes
• Etc., etc., etc.
• Etc., etc., etc.
Etc., etc., etc.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 30
@alanchumley / #mra_ac
31. IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 31
@alanchumley / #mra_ac
32. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 32
@alanchumley / #mra_ac
33. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 33
@alanchumley / #mra_ac
34. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 34
@alanchumley / #mra_ac
35. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 35
@alanchumley / #mra_ac
36. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 36
@alanchumley / #mra_ac
37. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 37
@alanchumley / #mra_ac
38. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 38
@alanchumley / #mra_ac
39. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 39
@alanchumley / #mra_ac
40. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 40
@alanchumley / #mra_ac
41. AN INDEX WITH MOVABLE INDICATORS
FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES
Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment
Presence
Score 10 10 10 10 10 10 10 10
Weight 1 2 3 4 5 6 7
(@ d f lt)
default)
Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential
• Each of the above indicators can be adjusted to account for different types of campaigns & contexts
• The weights would change. The scores don’t.
• For an organization, new to social media, perhaps they might only want to establish an on‐line presence
• So we’d dial or slide the weights of all but the popularity indicator down to zero
Has a significantly scaled back / rudimentary version of this.
This is inspired by but significantly expands on R6 thinking.
p y g y p g
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 41
@alanchumley / #mra_ac
42. WHAT DOES THE INDEX INDICATE?
• It’s a relative measure of the quality of social media coverage:
– One post relative to another and over time
– Many posts relative to a number of other similar clusters of postings
• It is a measure of both output and outtake, but not outcome
– Output: how much and how good?
– Outtake: how engaged, inter‐connected, inter‐related, volume & velocity of network /
community spread
– Outtake: advocacy
– it does not claim to be a measure of outcome or impact
d l b f
• It’s a proxy for on‐line influence; it’s about potential to influence
– to measure actual influence, we’ve have to survey
• In a diagnostic, benchmarking sense the index can help inform an organization s social media
In a diagnostic benchmarking sense the index can help inform an organization’s social media
engagement strategy if tracked over time
• It’s tactical on the fly for mid‐campaign course correction and looking back over the long term
• It’s strategic looking forward: helps shape communications recommendations
• This data can be correlated with tangible outcomes—CARMA Connect.
• This data can be plugged into a market mix model to show causality…isolate for PR’s unique
contribution to the marketing mix.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 42
@alanchumley / #mra_ac
43. SO WHAT?
WHAT ABOUT
CONVERSION & ROI?
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
44. WE’RE BACK TO THE PYRAMID THE TOP THIS TIME
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 44
@alanchumley / #mra_ac
45. OUTCOMES: KINDS OF CONVERSION
SETTING THE STAGE FOR CORRELATION
• Conversion to a tangible like web traffic,
sales, donations, awareness, opinion,
Micro (steps toward destination) Macro (Destination) usage, brand preference, likelihood to
Usually an intermediate means to an end The end try/buy/recommend.
Non‐transactional (Engagement) Transactional
• Could also mean conversion toward any
Non‐financial Financial measurable marketing communication or
public affairs, issues, advocacy-based
Get to x clicks in a site, spend x time on
, p Sale, donation,
, ,
objective.
objective
site, download, lead generated, watch a (paid) subscription
video, created account, contacted us etc. etc.
• Here we could / should also be looking at
the extent to which the quantity and q
q y quality
y
of social media activity correlates with
these outputs.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 45
@alanchumley / #mra_ac
46. 2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITE
Where were they? What drove them to your site?
What did they do once on your site? What are they ‘worth’ to you?
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 46
@alanchumley / #mra_ac
47. SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICS
Export Data to Your Web Analytics
•#of posts by date
•# of posts with key message included
# of posts with key message included
•# of post with brand mentioned X or more times
•# posts X average favorability by date
•# posts X average favorability X reach by date
p g y y
Or
Integrate web analytics into
your social media
monitoring platform
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 47
@alanchumley / #mra_ac
48. LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. •No indication of
web traffic on a graph relationship other
than lines that
might trend
together
•Could be a fluke
Source: Olivier Blanchard, Social Media ROI
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 48
@alanchumley / #mra_ac
49. LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. •No indication of
web traffic on a graph relationship other
than lines that
might trend
together
•Could be a fluke
Correlation •“R score” •Showing there is
S t t
•Some stats some relationship
l ti hi
•You can do this in beyond coincidence
excel
•Pearson’s Product •Not ‘proving’
Moment media exclusively
responsible for sales
Source: Olivier Blanchard, Social Media ROI
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 49
@alanchumley / #mra_ac
50. LEVELS OF LINKING OUTPUT TO OUTCOMES
Level of Linkage Notes
Just graph ‘er up Plot # of articles vs. •No indication of
web traffic on a graph relationship other
than lines that
might trend
together
•Could be a fluke
Correlation •“R score” •Showing there is
S t t
•Some stats some relationship
l ti hi
•You can do this in beyond coincidence
excel
•Pearson’s Product •Not ‘proving’
Moment media exclusively
responsible for sales
Source: Olivier Blanchard, Social Media ROI
Causality •Heavy duty stats •Isolates
•You so can’t do this in (
(statistically
y
excel ‘proves’) for PR’s
•Sophisticated unique contribution
modeling to the
•Market Mix Modeling communications
•Multivariate mix
Regression Analysis
Regression Analysis
•Ton ‘o data, time, $
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 50
@alanchumley / #mra_ac
51. TECH INDUSTRY EXAMPLE
Correlating opinion with
CARMA IQ (favorability +
reach)
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 51
@alanchumley / #mra_ac
52. AUTO INDUSTRY EXAMPLE
Monthly Sales Results vs. Four Month CARMA Favorability:
Monthly Sales to Four Previous Months’ Favorability
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 52
@alanchumley / #mra_ac
53. Toyota’s Heat Map:
CARMA Media IQ Metrics by Various Time Frames Correlated to
Monthly Sales Results
y
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 53
@alanchumley / #mra_ac
54. UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs
ROI roi
MBA PR
Financial Non‐financial
Non financial
Macro Micro (not all conversion events are ROI)
A mathematical equation Usually conceptual only; an equation in words only; can’t actually
that works
that works be calculated; something the industry tosses around too liberally
be calculated; something the industry tosses around too liberally
A business metric All‐too‐often a media or audience metric
Direct, clear A loose proxy or an intermediate metric (return on expectation,
return on objective, return on engagement)
return on objective return on engagement)
Source: Olivier Blanchard, Social Media ROI
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 54
@alanchumley / #mra_ac
55. THOSE USING TXT ANALYTICS ARE MEASURING ‘ROI’ NOT ‘roi’
Source: Text Analytics 2009: U
S T t A l ti 2009 User P Perspectives on S l ti
ti Solutions and P id
d Providers
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
56. SOCIAL MEDIA ROI CALCULATOR(S) THE GOOD & THE BAD
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
57. SOCIAL MEDIA BOOK BAG
• Social Media Metrics:
• Jim Sterne, (@jimsterne)
• Social Media ROI:
• Olivier Blanchard (@thebrandbuilder)
• Social Media Analytics: Effective Tools for Building, Interpreting & Using Metrics:
• Marshal Sponder (@webmetricsguru)
• Measure What Matters:
M Wh M
• Katie Paine (@kdpaine)
• Using Web Analytics to Measure Impact of Earned Online Media…:
g y p
• Seth Duncan (www.instituteforpr.org)
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 57
@alanchumley / #mra_ac
58. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT
IN-ROOM STRETCH (5 mins)
PART 2 (30 mins): TONE ALONE
PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
59. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Humans or Machines or Both?
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 59
@alanchumley / #mra_ac
60. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Why Automation?
•Humans or Machines or Both?
•Situating Automated S
S Sentiment Analysis
•Defining Automated Sentiment Analysis
•Output
•Taxonomies etc.
•Automated Vendors
•Complications
p
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 60
@alanchumley / #mra_ac
61. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT
•Sentiment Analysis
•Why is it important/ why should we care?
•Sentiment as a Proxy / Predictor
•Applications
•Why Automation?
•Humans or Machines or Both?
•Situating Automated S
S Sentiment Analysis
•Defining Automated Sentiment Analysis
•Output
•Human vs. Automated
vs
•Taxonomies etc.
•Human-based vendors
•Automated Vendors
•Human-Based Sentiment Analysis
•Complications
p
•Favorability not (just) Sentiment
•Humans supplemented by automation
Linking outputs to outcomes revisited
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 61
@alanchumley / #mra_ac
62. I AM NOT IAM
A COMPUTATIONAL LINGUIST
A SEMANTICIAN
COMPUTER ENGINEER
COMPUTER SCIENTIST
LOGICIAN
PSYCHOLINGUIST
BUT…CHECK
BUT CHECK OUT SETH GRIMES :
GRIMES’:
Text Analytics 2009: User Perspectives on Solutions and Providers
A number of excellent presentations on LinkedIn / Slide Share
I’ve borrowed liberally from both for this Part 2
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 62
@alanchumley / #mra_ac
63. I AM NOT IAM
A COMPUTATIONAL LINGUIST
A RECOVERING PR FLACK
CO G C
A SEMANTICIAN
PR MEASUREMENT MOUTHPIECE
COMPUTER ENGINEER
WITH AN INTEREST IN MONITORING +
COMPUTER SCIENTIST
MINING ONLINE CONVERSATIONS
LOGICIAN
PSYCHOLINGUIST
BUT…CHECK
BUT CHECK OUT SETH GRIMES :
GRIMES’: Sentiment through the eyes
Text Analytics 2009: User Perspectives on Solutions and Providers
A number of excellent presentations on LinkedIn / Slide Share of a simpleton
I’ve borrowed liberally from both for this Part 2
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 63
@alanchumley / #mra_ac
64. IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 64
@alanchumley / #mra_ac
65. WHY IS IT IMPORTANT? WHY SHOULD WE CARE?
Questions for biz & gov’t.:
•What are people saying?
Wh t l i ?
•What’s trending?
•How has opinion about x changed?
•Does it d e c a e to c a e
oes differ channel o channel
•How has sentiment correlated with
news / marketing / events etc.?
•Can we link opinions to transactions?
Computers are trading stock
based on automated sentiment
analysis of online content
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 65
@alanchumley / #mra_ac
66. SENTIMENT ANALYSIS AS MARKET RESEARCH PROXY / PREDICTOR
Think here about social media monitoring and
analysis as:
•Real time situational awareness
•Listening at the point of need
•Listening close to the moment of influence
A real-time digital form of or proxy for market
research and CRM:
Mining conversations to tease out how people
‘feel’ about products/services/issues/causes
feel
and perhaps even:
1) their intended (reported) behavior and what
motivates them to buy, (as cited in the
conversations), and
ti ) d
2) their actual behavior tracked via CRM 2.0
and Web 2.0 analytics
1615 M Street, NW; Suite 750 Source: Crimson Hexagon
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 66
@alanchumley / #mra_ac
67. APPLICATIONS
Call Centre Notes?
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 67
@alanchumley / #mra_ac
68. SITUATING SENTIMENT ANALYSIS AUTOMATED
Text analytics:
•Applies linguistic +/or statistical
Artificial Intelligence techniques to extract concepts and
patterns
Text A l ti
T t Analytics •Transforms unstructured information into
data for analysis
•Unlocks meaning and relationships in
large volumes of information
Sentiment Analysis
•Adds semantic understanding of:
•Sentiment
•Opinions
Opinions
•Attitudes
•Entities
•Concepts
•Facts
Facts
1615 M Street, NW; Suite 750
•Abstract attributes: “expensive,”
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
“comfortable”
68
@alanchumley / #mra_ac
69. AUTOMATED SENTIMENT ANALYSIS DEFINED
Sentiment analysis or opinion mining refers to the application of:
•Natural language processing
Linguistics Comp Science
•Computation linguistics
p g
•Text analytics…
…to identify and extract subjective information in content. Statistics
Natural Language Processing
• Grounded in statistical machine learning
• Much overlap with…
Computational Linguistics
Computational Linguistics
• Statistical or rule‐based modeling of language
Text Analytics
• Turning text into data
• Lexical analysis: word frequencies
• Entity recognition
• Pattern recognition
• Tagging/ annotation
Tagging/ annotation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com
@alanchumley / #mra_ac
70. EXTRACTING ATTITUDES TURNING ATTITUDES INTO DATA
Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a
writer with respect to some topic or the overall tonality of a document.
The attitude may be his or her:
• judgment or evaluation
• affective state (emotional state of the author when writing), or
• the intended emotional communication (emotional effect the author wishes to
have on the reader).
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 70
@alanchumley / #mra_ac
71. HUMANS OR MACHINES OR BOTH?
ALGORITHM AL GORITHM
Can it be over ridden?
Does it learn?
THE ALGORITHMICS
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 71
@alanchumley / #mra_ac
72. WHY AUTOMATION?
Time Spent
Analyzing Content
y g
w/out automation
Time Spent on
Ti S
Insights &
Implications
w/ automation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 72
@alanchumley / #mra_ac
73. WHY AUTOMATION?
Time Spent
Analyzing Content
y g
w/out automation
Time Spent on
Ti S
Insights &
Implications
w/ automation
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 73
@alanchumley / #mra_ac
74. THE OUTPUT
Entity E t ti
E tit Extraction
Issues / Themes /
Messages
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 74
@alanchumley / #mra_ac
75. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
• Words/phrases that
PRE‐SET are generally good or
are generally good or
bad (Hate. Love)
GENERAL • Caution!
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 75
@alanchumley / #mra_ac
76. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
• Words/phrases that are
PRE SET
PRE‐SET generally good or bad
generally good or bad
(Hate. Love)
GENERAL • Caution!
• Financial / market news
PRE SET
PRE‐SET • Mergers & Acquisitions
• Competition
SPECIFIC • Better
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 76
@alanchumley / #mra_ac
77. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
• Words/phrases that are
generally good or bad
generally good or bad
PRE‐SET GENERAL (Hate. Love)
• Caution!
• Financial / market news
PRE‐SET
PRE SET • Mergers & Acquisitions
SPECIFIC • Competition
• Better
CUSTOM
• Client‐specific context
CLIENT/PROJECT • Best
SPECIFIC
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 77
@alanchumley / #mra_ac
78. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES
• Words/phrases that are
generally good or bad
generally good or bad
PRE‐SET GENERAL (Hate. Love)
• Caution!
• Financial / market news
PRE‐SET
PRE SET • Mergers & Acquisitions
SPECIFIC • Competition
• Better
CUSTOM • Client‐specific context
CLIENT/PROJECT • Best
• Multiple cartridges
SPECIFIC
But! Still needs some human
intervention / validation /
feedback so it can ‘learn’ and
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
improve
carma.com 78
@alanchumley / #mra_ac
79. AUTOMATED TEXT ANALYTICS VENDORS
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 79
@alanchumley / #mra_ac
80. 1st GENERATION vs. 2nd GENERATION AUTOMATED SENTIMENT
Conceptual Search-Based
Key Word (only) Deeper linguistic level
p g
Based
Operates on a semantic level
Explicit Understands sentence grammar:
References
•Subject
Counting up the •Predicate
•Object
“+” & “-” •Who?
•Action
•Idea
•…and the relationships between these
Most studies show that key •…must contain these terms but in a particular
word matching is wrong relationship to one another
more than 50% of the time
Opinions, claims, arguments, facts, actions, concepts,
messages
Non-implicit: theme / spirit / i t t
N i li it th i it intent
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 80
@alanchumley / #mra_ac
81. COMPLICATIONS & PROBLEMS w/ 1ST GEN AUTOMATED APPROACHES
•Sentiment may be of interest at multiple levels:
•Document
•Several Documents
•Statement / sentence
Statement
•Entity
•Topic or Theme
•Concept
•Language is chaotic:
•Jargon, slang, (slanguage)
•Colloquialism
•Forum-specific acronyms
•ambiguity,
•Anaphora (it, them)
Anaphora
•Irony, sarcasm
•Misspellings
•Poor grammar
•Variations in expression
•Synonyms
•Polysemy
•Context is key:
•Meaning can change depending on context.
•Need to look not only at the word but the connective tissues surrounding the word.
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 81
@alanchumley / #mra_ac
82. CONTEXT IS KEY
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 82
@alanchumley / #mra_ac
83. ANALYSING THE “CONNECTIVE TISSUE” OF A SENTENCE
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 83
@alanchumley / #mra_ac
84. HUMAN VS. AUTOMATION BOTH IS BEST
•Most sentiment analysis algorithms rely on
HUMANS
humans using simple terms to express our sentiment AUTOMATION FOR
RELEVANCE,
FOR VOLUME
about a product or service. We don’t. & SPEED
DEPTH,
CONTEXT,
NUANCE
•Cultural f t
C lt l factors, li
linguistic nuances and diff i
i ti d differing
contexts make it extremely difficult to automate turning
Bridge the Gap with
a string of written text into a simple p or con
g p pro Human analysis
Human analysis
of a sample
sentiment.
•The fact that humans often disagree on the sentiment
of text illustrates how big a task it is for computers to
get this right.
right
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 84
@alanchumley / #mra_ac
85. HUMAN (or hybrid)-BASED VENDORS
1615 M Street, NW; Suite 750
Washington, DC 20036 USA Tel 1.202.842.1818
carma.com 85
@alanchumley / #mra_ac