During its first year of incubation, Upwell successfully pioneered the development of new methodologies in social monitoring, demonstrated success in elevating the ocean conversation above the baseline, earned praise for its non-branded approach to campaigning from social media thought leaders and attracted additional philanthropic interest in expanding the project beyond the intent of the pilot phase across a range of environmental issues. We are grateful for the Waitt Foundation’s significant initial investment, which provided the vision and commitment to launch this entrepreneurial initiative and are appreciative of other funding we have received for the project. This is the final report of Upwell’s pilot phase, completed in February 2013.

1. By Rachel Weidinger, Rachel Dearborn, Matt Fitzgerald,
Saray Dugas, Kieran Mulvaney and Britt Bravo
http:/
/upwell.us
Twitter: @upwell_us
Upwell Pilot Report
http:/
/creativecommons.org/licenses/by-sa/3.0/

2. Table of Contents
I: Executive Summary Page 3
II: Introduction Page 15
III: Theory of Change and Context for Our Work Page 16
IV: Process and Methodology
Methods: Big Listening Page 25
Methods: Campaigning Page 50
V: Metrics of Impact
Attention Impacts and Graphs Page 75
Ocean Evangelist Capacity Impacts Page 96
VI: Insights
Comparative Ocean Conversation
Page 114
Analytics
Insights: Big Listening Page 138
Campaigning, Collaboration and Powerful
Page 145
Amplifiers
Network Map: Ocean Evangelists and Ocean
Page 161
Voices Online
2

3. Executive Summary
The ocean is in crisis, plagued by overfishing, habitat loss, and acidification, among other issues.
While the ocean serves as the engine for our climate and plays a central role in the global food
system, it still fails to register for many as a relevant and primary issue. It is, quite literally, out of
sight and out of mind. The virtual invisibility of the ocean in public discourse is a major obstacle
for the ocean conservation community to adopt and implement solution-based policies.
The key to Upwell’s success—and thus, the success of the ocean conservation community—is
not to blast new, shiny information into the interwebs, but rather to nurture and bridge virtual
and real-life distributed, diverse networks, and to leverage the combined reach and power of
those networks of communicators to participate in and amplify the best content and campaigns.
In inventing a new kind of collaboration, we’ve provided the tools and the space, and relied on
the ever-growing community of ocean communicators to work together to make change.
Upwell’s array of goals—to utilize the immediacy of online communications, experiment with
ways to increase the reach of valuable content, empower and foster a broader network of ocean
communicators, and enrich our understanding of the conversational ecosystem surrounding
ocean topics—coalesced our broader vision of “conditioning the climate for change.” We
believe that by getting more people talking about ocean issues and raising the baseline of
conversation, broader audiences will be more likely to take action, change behavior, and push
for policy change that will have positive effects for our oceans.
Our primary metric for understanding the conversations analyzed in this report is what we refer
to as a “social mention” (or “social item”). Upwell defines a social mention as the text inclusion
of a monitored keyword in a post on social media platforms like Twitter, Facebook, blogs,
mainstream news with RSS feeds, forums/boards, YouTube or Pinterest. Social mentions are
online acts of self-expression in which individuals, organizations and other entities invest (at
least) a small amount of social capital.
Upwell employs Big Listening in order to understand the volume and character of online
conversations about ocean issues. Big Listening is the art and practice of tracking topical online
conversations over time—listening to what “the internet,” writ large, is talking about. When
combined with data-informed campaigning, Big Listening provides a methodology for
increasing both the frequency and volume of online conversation around a particular issue. The
basic idea is to identify pockets of real-time or historical conversation, wherever they may be,
and then to use that information to make the conversation bigger. Big Listening is distinguished
from traditional social media monitoring by its scale, fluidity, focus on issue or cause monitoring,
and expanded access to historical data.
3

4. In our work to date, the team at Upwell has come to believe that there are three measurable
characteristics of the online ocean conversation. We are increasingly attentive to:
1. Constant level of conversational volume (Baseline),
2. Notable outliers in increased volume (spikes), and
3. Density of conversational hotspots (spike frequency).
Upwell practices Big Listening on English-language conversations in the following eight topic
areas: Overfishing, Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans
(whales and dolphins), Sharks, Tuna, Gulf of Mexico and Ocean Acidification. For each topic,
both real-time and historical data provide essential context for understanding the volume,
evolution and characteristics of the overall conversation.
Each topic we monitor is characterized and defined by a set of search terms (including
exclusions) that we refine on an ongoing basis. While we recognize the limitations of “keyword
groups,” such as their reliance on text-based results and the absence of contextual awareness,
they do provide a powerful tool for analyzing online attention. The development and active
refinement of keyword groups is at the heart of Big Listening methodology.
At the time of writing, our current Baseline (v3.1) is the average of the lowest 20% of social
mention values for a topic on a given day of the week. In addition to the Baseline, we track
significant increases in online attention for a particular topic, or spikes. When you graph those
social mentions, you can actually see that burst of attention ‘spike’ the graph—hence the name.
Upwell defines a spike as occurring when the social mention volume for a given day meets or
exceeds one standard deviation from the mean of all recorded values for that same day of the
week.
Upwell’s campaigning model is informed by Big Listening data and combines a few additional
key elements. Our campaigns are attention campaigns, focused on raising attention to ocean
issues. They are minimum viable campaigns, operating on short time-frames and focused on
rapid delivery of content, continuous learning and iteration. They are run and amplified across a
distributed network, rather than being housed on and amplified by way of our own platforms.
What we do with attention campaigns is drive more attention to existing content and actions
that are not on our properties. They’re not associated with our brand. We use this loosely held
connection, tying into the momentum of the news cycle and being strategically opportunistic in
the pursuit of creating spikes in attention.
Through our minimum viable campaigns, we employ ongoing, iterative, continuous delivery of
content, resisting our urges toward perfection and providing irreverent, timely, contextual
content to audiences immediately instead of strategizing for six months or a year. We focus on
4

5. the quickest, dirtiest thing we can get out the door that we think will have a measurable effect
on a conversation.
By applying both these models, Upwell has crafted a new way of campaigning that is easily
delivered, measured, and adapts to the ever-changing sea of conversation. In summary, through
our campaigns, Upwell:
• Surfs existing conversations in order to increase and expand attention.
• Measures social mentions (rather than policy outcomes, petition signatures, or public
opinion) to evaluate the success of our campaign efforts.
• Delivers, measures, and learns from campaigns on a short time cycle, embedding lessons
and insights immediately. We sacrifice perfection.
• Collaborates with a network of ocean stakeholders and curating a diverse set of existing
ocean content, rather than building on our own brand and creating our own content. Our
campaigns are not aligned with Upwell program priorities or policy goals, but instead
amplify attention to the priorities and goals of those in our network.
• Running our campaigns across a distributed network of ocean communicators, rather than
relying on our own platforms as information hubs.
Rather than collect a large set of official MOU’s and partner logos to put up on our website, we
built a loosely held, distributed network. We’ve reached out to nodes of people who control the
communications channels that reach lots of supporters and followers who are interested in
ocean issues. We’ve been scrappy and ruthless about who we put into that distributed network,
trying to make it diverse and ensure the reach is big.
These are the values that guide Upwell in building and strengthening our distributed network:
• Trust: we share only science-based content, ensuring that other science-based
institutions know that the content we share is trustworthy.
• Transparency: we share our campaign and big listening data with our network, so they
can apply our lessons in their own work.
• Brand-agnostic: we work as willingly with Greenpeace as we do with Deep Sea News, as
we do with the Facebook page “I Fucking Love Science.” We will share an organization or
individual’s content or campaign, as long as it promotes ocean conservation goals and fits
our curation criteria (detailed below). Often, promoting content from an array of brands
means releasing control of the message.
• Issue-agnostic: We aren’t only focusing on overfishing, through GMO salmon or catch
shares, to cultivate the network. We amplify any ocean campaign or content as long as it
fits our curation criteria, raising attention for the crisis the ocean faces.
5

6. • Personal: We build relationships with humans, not organizations. The liveliest online
conversations happen between people, not institutions. We model the authentic behavior
of the internet.
• Generous: We provide small bits of advice and feedback to help our network do better. If
their work will get more people talking about the ocean online, it fits with our mission.
Our Big Listening practice helps us understand the volume and character of ocean
conversations, individually and in relation to one another. It also helps us to strategically choose
where to invest attention. Knowing the scale of conversations—for instance, that the sharks
conversation regularly spikes to over 40,000 social mentions in a day (and often much higher),
whereas the marine protected areas/marine reserves conversation sits at about 50 per day—
helps us right-size our expectations for attention, identify pockets of audiences ripe for
engagement, and time our campaigning efforts to capitalize on the regular ebb and flow of
conversation.
We curate things to amplify that meet these criteria:
• Good science
• Socially shareable
• Conservation impact
• Building social capital
• New influencers
• Topical
• Spikeability
• Under amplified
Once we’ve identified an opportunity, choosing a tool for dissemination is only part of the battle.
We often research, curate, and create in order to provide the most shareable content. There’s no
exact science to what we do—our methods are mostly informed by years of experience
campaigning in social media. However, a few scenarios, outlined below, highlight the most
common ways we approach attention campaigning.
• Scenario 1: The science and the message is good, but the content isn’t shareable.
• Scenario 2: There’s conversation beyond the ocean community. Can we tap into it?
• Scenario 3: Team Ocean isn’t coordinated. Can we create more message redundancy?
• Scenario 4: The Upwell network doesn’t have direct access to Big Listening data. Can we
provide insights to make their campaigns more effective?
6

7. Over time, we’ve seen the number of social mentions generated from each attention campaign
grow, concurrent with the growth of our distributed network. This is the proof in the pudding.
As we continue to expand Team Ocean and encourage networked sharing, the number of social
mentions about the ocean will increase, and ultimately increase the baselines of ocean
conversations.
Both the Sustainable Seafood and Overfishing conversations have substantially changed
since the founding of Upwell. Both distinct conversations have seen significant increases in
spike volume, spike frequency, and ratio of average daily social mentions to the average
baseline.
Sustainable Seafood
1400 1400
1200 1200
1000 1000
800 800
600 600
400 400
200 200
0 0
Oct-11 Nov-11 Dec-11 Jan-12 Oct-12 Nov-12 Dec-12 Jan-13
Baseline Spike Threshold High Spike Threshold Sustainable Seafood Baseline Spike Threshold High Spike Threshold Sustainable Seafood
Side-by-side comparison for Winter 2011 (left) and Winter 2012 (right) showing social mentions by
day for Upwell’s Sustainable Seafood keyword group, as compared to the baseline, spike threshold
and high spike threshold (Winter 2011: 10/17/2011 - 1/31/12; Winter 2012: 10/1/2012 - 1/29/13)
In Winter 2011 (above left), when Upwell began Big Listening in Sustainable Seafood, social
mention volume was an average of 423 mentions per day. By Winter 2012 (above right),
Sustainable Seafood social mention volume is up 29.9%. Spike frequency in the Sustainable
Seafood conversation increased 265%. Those spikes were not just occurring more often, they
were also getting bigger. Large volume spikes, those meeting Upwell’s high spike threshold, saw
a 475% increase.
7

8. Overfishing
14000 14000
12000 12000
10000 10000
8000 8000
6000 6000
4000 4000
2000 2000
0 0
Oct. 17, 2011 Nov. 17, 2011 Dec. 17, 2011 Jan. 17, 2012 Oct. 1, 2012 Nov. 1, 2012 Dec. 1, 2012 Jan. 1, 2013
Baseline Spike Threshold High Spike Threshold Overfishing
Baseline Spike Threshold High Spike Threshold Overfishing
Side-by-side comparison for Winter 2011 (left) and Winter 2012 (right) showing social mentions by
day for Upwell’s Overfishing keyword group, as compared to the baseline, spike threshold and high
spike threshold (Winter 2011: 10/17/2011 - 1/31/12; Winter 2012: 10/1/2012 - 1/29/13)
In Winter 2011 (above left), when Upwell began Big Listening in Overfishing, social mention
volume was an average of 423 mentions per day. By Winter 2012 (above right), Overfishing
social mention volume is up 71%. Overfishing spike frequency increased 784%. Those spikes
were not just occurring more often, they were also getting bigger. Large volume spikes, those
meeting Upwell’s high spike threshold, also saw a similar 475% increase.
Annotated campaign graphs are included in this report, and illustrate more specifically where
and how Upwell intervened in these two conversations.
The Overfishing Conversation The Sustainable Seafood Conversation Winter 2012
Winter 2012 Upwell Campaign and Social Mention Spikes Oct 2012- Jan 2013 !
Upwell Campaign and Social Mention Spikes Oct 2012- Jan 2013 !
Gangnam Style,
CA MPAs,
Fish Tornado
1400
14000 NU-20
NU-24
Vote4
1200 Ocean
Video
12000 Antarctic Ocean (day 10)
& I Oyster NY
NU-22 Cuomo
Pacific Bluefin Oysters NY
the 96.4% 1000 Big Blue
10000 NMS 40th &
NYT Trawling
FAD Safeway NU-21 Blogs
NU-24
NU-5 NU-19
800 NU-23
8000 Vote4the
Ocean
How to Kill
a Great White
Seamounts Cuomo
JAWS vs & Rooftops Oysters NY
Sinatra
Costa Rica Big Blue 600
6000 Fin Ban Blogs
4000 400
2000 200
Antarctic Antartic
(day 1 of 15) (day 15)
0 0
Oct-12 Nov-12 Dec-12 Jan-13 Oct-12 Nov-12 Dec-12 Jan-13
Baseline Spike Threshold
Mean +1.0 STDEV Sustainable Seafood
SS
Baseline Spike Threshold
Mean +1 STDEV Overfishing
OF
The Tide Report, Upwell’s blog and social media channels, topic-specific webinars, plus staff
speaking engagements, guest blog posts and project consulting have provided channels for
delivering shareable content, and practical training and tools to a diverse audience of time-
8

9. starved ocean activists. According to our February 2013 survey, through these tools and
opportunities, Upwell has helped the community:
• Receive content that they wouldn’t come across through their usual channels
• Stay up-to-date on the hottest ocean news
• Save time by providing content that they could amplify to their community
• Made them feel like they’re part of a community
• Helped them balance humor with serious issues in their communications
Which ocean topics have the most Baseline
volume?
3000
90000
2500
80000
70000
2000
60000
50000 1500
40000
30000 1000
20000
10000 500
0
Sun Mon Tue Wed Thu Fri Sat
0
Sun Mon Tue Wed Thu Fri Sat
MPAs Ocean Acidification Sustainable Seafood
Sun Mon Tue Wed Thu Fri Sat
Overfishing Gulf of Mexico Tuna
Sharks Cetaceans Ocean
Social mention Baselines for Upwell’s primary ocean topics
Perhaps not surprisingly, when we look at overall levels of conversational Baselines, the generic
“oceans” conversation is orders of magnitude larger than the conversations for its constituent
components. While to some extent this is the result of so many conversations being conducted
under the “oceans” banner, the word “ocean” is itself so widely used that, without proper
filtering, those other uses can distort the apparent size of the discussion. The next two largest of
our topics, Cetaceans and Sharks, also demonstrate comparatively high Baselines when assessed
against the others. We can see substantial differences among our lowest-volume topics. MPAs
has the lowest Baseline, Ocean Acidification and Sustainable Seafood are basically tied for
second-lowest (each exceeds the for certain days of the week), and Overfishing comes in at
9

10. about five times higher than them.
Collectively, overfishing represents a grab bag of ocean brands. The Overfishing conversation
brings together species such as sharks, tuna, salmon and lesser known but equally important
fish, with wonkish report subjects such as fisheries management and lackluster international
conferences. The topic encompasses a relatively broad conversational area, and one that has
historically churned out quarterly bursts of dire news.
Overfishing has about five times the Baseline volume of Sustainable Seafood, and roughly two-
thirds that of our next biggest topic, the Gulf of Mexico. The comparison with Sustainable
Seafood is particularly interesting because the two topics are obviously intricately connected—
the difference is how people talk about them. Whereas sustainable seafood suffers from a
fragmented and cloudy brand identity (what is sustainable seafood, anyway?), overfishing has
charismatic ocean species such as sharks and bluefin who are in clear and present danger.
Danger is catnip to the internet. The Overfishing conversation actually benefits, from an
attention point of view, from the ongoing damage that we are doing to our oceans and fisheries.
Bad news spikes high and fast online and then it goes away. Intriguingly, the spikes within
Overfishing have been occurring more frequently as Upwell has been monitoring (and
campaigning on) the topic. Overfishing is becoming more spikey and the spikes are increasing in
volume.
The Sustainable Seafood conversation is low-volume with low-level spikes, even while the
concept is becoming increasingly well-established in consumer minds. For comparison, Marine
Protected Areas has a lower baseline than Sustainable Seafood but occasionally spikes higher
than the Sustainable Seafood max. Ocean Acidification displays the same characteristic. And
despite their obvious connections, the volume of the Sustainable Seafood conversation is only
one fifth of that of the Overfishing conversation. Good news for fisheries and consumers, it turns
out, is not as attention-generating as bad news.
The overall brand of Sustainable Seafood is fragmented, awkward and wonky. People simply do
not talk about the sustainable seafood that they ate last night, or, crucially, not in those terms.
The food service industry has recognized this: one trade publication forecasts growing demand
for sustainable seafood even as it pointed out that consumers prefer the term “wild”—which
obviously means something very different. Furthermore, “sustainable seafood” itself is not a
term well-suited for short-form platforms like Twitter—it takes too many characters and is hard
to use in a sentence that doesn’t read as dry. Taken as a whole, the fragmentation of the
Sustainable Seafood conversation means that it is more difficult to accurately capture it
accurately with keywords, and that a low volume doesn’t necessarily mean people aren’t talking.
Unlike Overfishing, which has regular media hooks through connections to Shark Week, dire
report releases and celebrity activists, the Sustainable Seafood conversation doesn’t generally
translate into spikes from live events and or big news stories. Where we do see spikes occur they
10

11. are usually based in one of three elements: well-known brands promoting their sustainable
offerings (Safeway, McDonalds), fraud, or a bridge campaign (many of them attributable to
Upwell). One other notable burst of attention can be expected from the Sustainable Seafood
Summit—although the resulting content hasn’t been particularly shareable.
A comfort with complexity is necessary to forecast weather. Big Listening, similarly, requires
significant human skill and intuition to, first, develop robust conversational descriptors
(keywords) and then, second, to use the resulting information to identify opportunities for a
campaign to spike a given conversation. Upwell has intentionally cross-trained campaign and
listening roles so that this integration between listening and intervention is as efficient as
possible. This comes not from any computer readout but from regular, hands-on practice.
‘Weather’ forecasting of the social web is a nascent practice. Regular Big Listening to a given
conversation is essential for building an analyst’s awareness of the conversational dynamics at
play. It is most efficient to listen on an ongoing basis. Presence in the conversation is the
difference between watching a baseball game and reconstructing it through the box score.
The structure of Upwell intentionally underpins the process for doing Big Listening. Each
member of Upwell draws on a variety of tools and practices—some shared, some personalized—
to generate immediately actionable insight into each day’s online events. We supplement our
personal suite of tools and practices with shared Upwell systems (such as Radian6).
Personal Listening Systems [human and machine-assisted]
+ Shared Listening Systems [machine-assisted and human-network-assisted]
+ Morning Meeting [humans in conversation]
= Big Listening
While the context for Big Listening is constantly shifting, we believe that current trends point to
some likely future developments. These include:
• New firehoses
• Divergent functions
• Smarter robots
• Privacy fights
• Buyer beware
• Social science catches up to social media
• More visuals
• Spike marketplaces
• More upwellings
11

12. Emergent best practices for online campaigning from the Upwell pilot include:
• You can’t predict what will go viral.
• Timeliness and a hook are still really important, but the half-life of news online is shorter
than it used to be. Pay attention to ROI on campaigns.
• Bridge conversations, movements and communities to make your message go farther.
• Identify opportunities based on Big Listening.
• Use simple messaging.
• Think about the whole viewing and sharing experience.
• Narrow in.
• Be poised for rapid response.
• Pair content with asks, but balance asks across a spectrum of engagement.
• Celebrate victories.
• Normalize obscure issues or complex ideas with iconic imagery, cultural anchors, or tribe
signifiers.
• Define your goals and metrics based on what is actually measurable.
• Revive old stuff.
• Videos: shorter, prettier, more pithy.
• Memes: don’t try to make them from scratch.
• Celebrity promotion: not a silver bullet.
Collaboration in communications is hard, and can be expensive. Emergent best practices for
Collaboration, the Distributed Network Way from the Upwell pilot include:
• Provide brand neutral content.
• Embrace the larger ecosystem of communicators.
• Be open to ad hoc partnerships.
• Share other organizations’ and people’s content.
• Find unique high-touch activities to cultivate personal relationships.
• In difficult times, be human.
In running rapid attention campaigns, and focusing primarily on social platforms as the medium
for our ocean famous-making, Upwell has developed a few best practices that can be applied to
other small, nimble online teams.
• Develop systems to capture insights.
12

13. • Encourage a flat structure.
• Keep the campaigning team small, but not too small.
• Keep time for developing creative assets to a minimum.
• Run lots of little campaigns, and extend the ones that work.
• Lean on the personal interests, strengths and networks of your team members.
• Recognize and admit your weaknesses.
Top insights and best practices for amplifying attention to ocean issues in general, as well as
some that are specific to those communicating about overfishing, sustainable seafood, and
marine protected areas from the Upwell pilot include:
For ocean communications:
• The ocean is out of sight and out of mind.
• We assumed there would be a lot of great ocean content. We were wrong about the ‘great’
part.
• Plan social media outreach in advance of scientific report releases.
• Lower your science hackles.
• Cross-promote social content via collaborative outlets.
• Anthropomorphize ocean creatures.
• Don’t let beautiful ocean pictures do all the talking.
Sustainable Seafood:
• Scary stories get attention.
• “It’s complicated” is a bad relationship status and a bad brand.
• The actual practice of eating sustainable seafood continues to be challenging, and news
coverage is not making it appear easier.
• Focus on specific products, brands and species rather than the overall sustainable seafood
issue.
• Recipes and fluff pieces don’t generate social mentions.
Overfishing:
• Focus on actions that are doable and close to home.
• Sensational stories make headlines.
• Sharks are the quarterback of overfishing, and Shark Week is the Super Bowl of online
ocean conversations. Don’t sleep on Shark Week.
13

14. Marine Protected Areas:
• The MPA conversation is tiny in comparison to other ocean conversations.
• Our MPA vocabulary is fragmented, awkward and wonky.
• Share successes.
• Emphasize individual connection to MPAs as public commons to create support.
This is the final report of Upwell’s pilot phase, completed in February 2012. In it, the founding
team of Upwell documents new methodologies for conversation analysis, the shape of key
ocean conversations, the impacts of our campaign efforts, and emerging best practices for a new
era of online communications. We do so in service of the larger marine conservation sector, and
with the hope that what we have learned in our short effort will speed all our collective efforts.
The ocean is our client.
14

15. Introduction
Ocean Conservancy and the Waitt Foundation collaboratively developed the Upwell project in
2010. The project’s goal was to increase the volume of the conversation about the ocean to
enhance awareness and support for ocean issues among mass audiences. Ocean Conservancy
initially envisioned an 18-24 month pilot phase for the project, with the incubation stage
concluding in the summer of 2013. We conducted a national search for the project’s leadership,
hiring Rachel Weidinger, and launched the fully staffed program in early 2012. During our pilot
phase, the Upwell team has enjoyed the contributions of a great number of excellent crew
members, including Kieran Mulvaney, Ray Dearborn, Matt Fitzgerald, Saray Dugas, Britt Bravo,
Lara Franklin, Aaron Muszalski and Kevin Zelnio, and interns Christine Danner, Paulina Dao,
Liana Wong, and Kaori Ogawa. We’re grateful for the support of the Waitt Foundation, an
anonymous donor, the Gordon & Betty Moore Foundation, and our colleagues at the Ocean
Conservancy including Janis Jones, Melissa Ehrenreich, Shannon Crownover, Amelia Montjoy,
Julia Roberson and George Leonard. Vicki Spruill has been an important mentor for the project.
This experimental pilot project charted new territory to engage a larger and more diverse
audience in the ocean conversation and to elevate the ocean while not elevating any particular
organization or perspective. We have done this by quantifying the level of the ocean
conversation across a range of topics and measuring the impact of engagement on the issue, a
first for the strategic ocean communications initiatives. The Waitt Foundation served as Ocean
Conservancy’s lead partner to help shape the direction, finance the use of new cutting-edge
technological tools to actively monitor online conversations, and develop aggressive rapid
response campaigns to reach and mobilize new audiences to care about ocean content. At the
behest of the Foundation, Upwell focused primarily on elevating the online conversations about
overfishing and sustainable seafood during this incubation period to test the efficacy of this
innovative approach. We have had other forward-thinking funders join us in support of this
project over the past two years.
During its first year of incubation, Upwell successfully pioneered the development of new
methodologies in social monitoring, demonstrated success in elevating the ocean conversation
above the baseline, earned praise for its non-branded approach to campaigning from social
media thought leaders and attracted additional philanthropic interest in expanding the project
beyond the intent of the pilot phase across a range of environmental issues. We are grateful for
the Waitt Foundation’s significant initial investment, which provided the vision and
commitment to launch this entrepreneurial initiative and are appreciative of other funding we
have received for the project.
This is the final report of Upwell’s pilot phase, completed in February 2012.
15

16. 16

17. Theory of Change and
Context for Our Work
The ocean is in crisis, plagued by overfishing, habitat loss, and acidification, among other issues.
While the ocean serves as the engine for our climate and plays a central role in the global food
system, it still fails to register for many as a relevant and primary issue. It is, quite literally, out of
sight and out of mind. The virtual invisibility of the ocean in public discourse is a major obstacle
for the ocean conservation community to adopting and implementing solution-based policies.
The genesis of Upwell was rooted in a need, identified and articulated by Ted Waitt jointly with
Vikki Spruill of Ocean Conservancy in late 2010, to build an “ocean war room” whose core focus
was to increase attention to ocean issues among new and mainstream audiences. The effort
would be a non-branded communications effort that would utilize new and traditional media to
build a fast, aggressive and agile strategic communications platform to increase attention to
ocean conservation issues in real-time.
Although the specific look, feel and direction had yet to be determined, some key elements
articulated in the earliest stages remain true, over two years later:
• It would be an informational effort, one that would simultaneously seek to raise the
volume of key ocean issues while elevating them above the growing cacophony of
background noise on the Internet and elsewhere.
• The effort would be, in a sense, ‘unbranded.’ It would not act as a competing entity in
ocean conservation, but would instead highlight the work in conservation and science
already being done by others. As was expressed often in internal deliberative
conversations in the project’s earliest stages, “people within the community should
ideally be fully aware we exist, and that we are a resource to be utilized; but ideally,
people in the street will never know of our existence, but simply be more aware of the
ocean than they were before.” For that reason, the project was initially referred to
internally as “Ocean Underground.”
• It needed to be fast, ready to respond to and amplify developments and news at a
moment’s notice.
17

18. Testing the Waters
With the loose ideas of direction (detailed above) in mind, we reached out to members of the
community to gauge their levels of interest, and understand where we could be the most
effective. We spoke with many different players in the diverse ecosystem of ocean
communications: researcher/bloggers (e.g., John Bruno of the University of North Carolina),
social media experts at NGOs with a particular focus on online mobilization (e.g., Ben Kroetz at
Greenpeace), scientist-communicators (e.g., Nancy Knowlton, Jeremy Jackson and Steve
Palumbi) and many more.
We shared with them these still-nascent goals and philosophy:
We aim to raise the volume of ocean messaging, by utilizing the huge variety of outlets now
available to maintain a constant drumbeat of news and information.
We will be doing more than tweeting, blogging, and linking to every ocean story we come
across. We will also be providing context, emphasizing issues and topics of particular import
and helping ensure an understanding of the way they link to each other. In this way, we
hope to raise the volume without merely contributing to the overall background noise.
We will not be competing for the limelight. We will not be competing for funding. We do not
intend to be another “brand” in the public eye. We shall we be a resource, a means of
highlighting, synthesizing and contextualizing ocean issues in a way that brings further
attention to those issues and to those who are researching and campaigning on them.
We will seek to operate on a multitude of levels. We will work with researchers,
organizations and others to produce original content for publication online or in print. We
will aim to work with existing outlets and to create our own. We will offer to work with, and
provide content for, outlets that need it and also to disseminate content for those who lack
such networks. We will be content aggregators, synthesizers, distributors, creators, and
magnifiers. [...]
We hope that once we become established, we’ll swiftly settle into a pattern, in which it will
become routine for members of the community to provide us with content that we can
aggregate, magnify and synthesize, and that as a result ocean issues will have no problem
being heard in the cacophony of the Internet.
Initial feedback was incredibly positive. Members of the community agreed that there was a
need for deeper understanding of the currents of ocean conversation, and a need for a new way
to harness the energy and content and funnel it into more effective campaigning.
18

19. A Social Focus
We decided early on during this scoping process that social media was going to be our playing
field. Campaigns in the digital age—at all levels—require the rapid communication and personal
connections that social media cultivates. We would focus on unique metrics to evaluate our
success in penetrating and motivating social networks to spread ocean content within their peer
networks to both boost the volume of online conversations focused on the ocean and to
broaden the conversation beyond the choir. We would also respect and leverage the power of
traditional media by helping to connect social content in ways that would create mainstream
media attention or extend shelf life. We would strive to make scientific research accessible to
popular audiences online and identify relevance to new social and mainstream audiences. As we
articulated in early documents:
“As an overarching goal, we have been tasked to increase the volume of ocean news,
information and issues. ...Where Ocean Underground can carve out a unique and invaluable
role is, as noted at the top of this section, cutting through that noise, highlighting the news
that is important, discarding that which is less so, and providing a greater context for that
news.”
The articulation of this goal responded to what Eli Pariser—formerly of MoveOn.org and now
founder and CEO of Upworthy—calls the “Filter Bubble”: an increased personalization of the
internet. Search engines, social media platforms and online news outlets apply highly
sophisticated algorithms to analyze our internet behaviors and customize our experience,
necessarily limiting our exposure to new ideas that might be critical to progressive social
discourse. We wanted to explore ways to circumvent and manipulate filter bubbles to broaden
the ocean conversation beyond the choir.
We also wanted to experiment with ways to modernize conservation communications, aligning
them with the language and speed of the internet. By monitoring spiking attention and online
conversation in real-time, we would quickly intervene in a conversation and inject ocean
content into popular conversations. By using humor, wit, hard-hitting facts and education and
recontextualization, we would create highly shareable content tailored to spread through social
spaces.
The Need for a Big Team Ocean
In our initial conversations, we learned quickly that many people in the ocean world felt they
operated in silos, focused on one topic, region, or issue. Competition in the marine conservation
space is real—“blue” organizations get only a small fraction of environmental funding. Indeed,
several of those with whom we spoke showed particular interest in and enthusiasm for our
19

20. mission when they fully appreciated that we would not be a part of that competition and would
be trying to find ways to make their work more effective. Our philosophy at Upwell is that we’re
part of a big Team Ocean that includes marine conservation organizations, marine scientists and
ocean activists. Our competition is Justin Bieber, not each other. While Team Ocean is anything
but small, most activists, researchers and free agents fly the ocean flag far below their own,
specialized flag on the pole. It was clear that we had to invent a new way to collaborate.
A primary way that we’d cut through the noise and make valuable content reach broader
audiences was to foster a bigger and more diverse network of ocean communicators. In our
early analyses of ocean conversations, we saw what nonprofit social media expert and trainer
Beth Kanter calls the rise of “free agents,” and a shift away from traditional nonprofit “fortress”
communications. Fortress institutions, Kanter asserts, “work hard to keep their communities
and constituents at a distance, pushing out messages and dictating strategy rather than listening
or building relationships.”
Welcome to the Fortress. Now please go away. Photo by Stuck in Customs
Increasingly, environmental NGOs—the fortresses—were not driving conversation. Free agents—
bloggers at Deep Sea News, managers of Facebook pages like I Fucking Love Science, and social
media savvy public figures like George Takei, to name a few—were generating conversation by
sharing irreverent content and engaging their followers in a more personal way. Citizen-led
efforts utilizing social media, like the Arab Spring and the Occupy movement, are examples of
the power of decentralized communication.
We wanted to bridge and engage both these communities, helping organizations to chip away at
their fortress walls, and connecting free agents with the deep, science-based content that
20

21. organizations and academics could provide. Some organizations outside the ocean sector are
beginning to experiment with this. Greenpeace elevates examples of people-powered
organizing through its Digital Mobilisation Lab, and in December 2012, MoveOn.org embarked on
what they call a “bottom-up revolution,” empowering its supporters to determine what issues
and campaigns MoveOn throws its weight behind. Aside from polling members on issues to
work on, MoveOn also has a platform that allows members to upload or share content that can
bubble up into campaigns.
The key to our success—and thus, the success of the ocean conservation community—would
not be to blast new, shiny information into the interwebs, but rather to nurture and bridge
virtual and real-life distributed, diverse networks, and to leverage the combined reach and
power of those networks of communicators to participate in and amplify the best content and
campaigns. In inventing a new kind of collaboration, we’d provide the tools and the space, and
rely on the ever-growing community of ocean communicators to work together to make
change.
The Very Large Array in New Mexico harnesses a network of radio telescopes to increase its
listening power. Upwell does the same. But with social media networks rather than radio
telescopes.
21

22. We Shall Have Bigger Ears and Eyes
Into the Internet
If Upwell was going to operate on behalf of Team Ocean, we were going to need a way to
identify its members and assess our collective efforts. We needed a big picture perspective on
the ocean online. How many people were on our team? What were they talking about? Were we
getting our butt kicked, attention-wise, by I Can Haz Cheezburger? We needed to understand the
volume and character of online conversations about the ocean.
LOLrus: playing for both Team Ocean and Team Cheezburger [source]
Upwell recognized that the broadcast model of communications was insufficient for a
networked world in which attention and engagement are the primary currency. But we also
recognized that if online attention was a currency, the ocean was basically broke. Despite
widespread love for the actual thing, the ocean as represented online was a shadow of itself. Go
to the beach and the ocean was captivating. Go on Facebook and it was hard to find at all.
We began to develop what would become Big Listening, a methodology and philosophy, of
listening to dynamically evolving online conversations writ large. Our primary lens for assessing
success would be whether or not our shared purpose succeeded, not whether our organization
did. We would eschew the brand constraints that had crippled the ocean’s institutional voices
online and we would speak fluent internet. We would do everything we could to make the
ocean more famous on the internet, and we would use Big Listening to measure our progress.
22

23. Step change vs. Incremental Change [source]
Upwell entered this challenge looking for step change—a massive, discontinuous leap forward—
because the ocean needs a win that really matters. Since those initial developments, Big
Listening has gone from an abstract concept to a replicable, demonstrated methodology. Our big
window on Team Ocean has also had the fortuitous effect of developing new campaigning
techniques for which the ocean sector now has a competitive advantage. That advantage won’t
last forever.
Conditioning the Climate for
Change
As Kari Marie Norgaard notes in her book Living in Denial1 :
Before an issue can make it into a council meeting, onto picket signs, into the framing of a
local news story, or into a newspaper editorial, somebody has to start talking about it. ...
Conversation is the site for exchange of information and ideas, for human contact, and for
the building of community.
Upwell’s array of goals—to utilize the immediacy of online communications, experiment with
ways to increase the reach of valuable content, empower and foster a broader network of ocean
communicators, and enrich our understanding of the conversational ecosystem surrounding
ocean topics coalesced into our broader vision of “conditioning the climate for change.” We
believe that by getting more people talking about ocean issues and raising the baseline of
conversation, broader audiences will be more likely to take action, change behavior, and push
for policy change that will have positive effects for our oceans.
1 Norgaard, K. M. Living in Denial. 2011, Kindle Edition, location 809.
23

24. Metrics: Social Mentions
Our primary metric for understanding the conversations analyzed in this report is what we refer
to as a “social mention” (or “social item”). Upwell defines a social mention as the text inclusion
of a monitored keyword in a post on a social media platform like Twitter, Facebook, a blog,
mainstream news with an RSS feed, a forum/board, YouTube or Pinterest. Social mentions are
online acts of self-expression in which individuals, organizations and other entities invest (at
least) a small amount of social capital.
Social mentions have more in common with the
Other Metrics
Social Mentions metric of media hits than they do with the more
(not social mentions)
common, older PR and marketing metric of
Tweets and retweets Impressions impressions. Upwell focuses on counting and
analyzing social mentions (rather than
Mainstream news
articles with RSS Views impressions or online mentions) because we
feeds and comments believe that the number of people who choose
to take an action to create or share content is a
Posts, shares and better indicator of engagement than the number
comments on Clicks
of people who have simply seen (or could have
Facebook
seen) that content.
Likes / Loves / Favs
Blog posts and
[Facebook, Tumblr, It is worth noting that, while it is theoretically
comments
Twitter] possible to accurately count every single social
Re-blogs on tumblr mention on a topic, Upwell’s Big Listening
methodology focuses on characterizing
Forum or board posts conversations just thoroughly enough to
campaign successfully within them.
Furthermore, Upwell believes that social mentions are a
better leading indicator of willingness to take action for the What About “Likes”?
oceans than other communications metrics. This is because Likes, loves, and faves (different
social mentions represent actions, the choice of an terminology for different social
individual to risk a small amount of social capital by media platforms) are in a middle
associating their online identity with a piece of online ground. While they are not social
content. In aggregate, the volume of social mentions not mentions (as people are not
only represents the amount of attention being paid to a creating new content), they are
topic, but a forecast of potential campaign success. also not as passive as views or
impressions. While likes, loves,
The strength of a community, by our standards, is measured and faves are not counted by
not by its size, but rather by its engagement level. For Radian6, Upwell does measure
example, if one tweet has 12,000 impressions (the number them, when possible. However,
of people who follow the account that posted the tweet), for the purposes of this report we
we count the tweet the same way that we would count a have omitted these metrics since
tweet with 200 impressions. If a person or organization is they constitute only minimal
network-oriented, it would follow that their content would public engagement and can
require laborious, resource-
lead to more retweets, replies and/or mentions. If a tweet
intensive manual calculation.
goes out to 12,000 followers but gets zero retweets, it is less
of an indicator of willingness to take action than a tweet
that goes out to 200 followers and gets 10 retweets.
24

25. Methods: Big Listening
Introduction
Upwell employs Big Listening in order to understand the volume and character of online
conversations about ocean issues.
In our work to date, the team at Upwell has come to believe that there are three measurable
characteristics of the online ocean conversation. We are increasingly attentive to:
1. Constant level of conversational volume (Baseline),
2. Notable outliers in increased volume (spikes), and
3. Density of conversational hotspots (spike frequency).
Over time, we want to come to understand the role of all three as they contribute to
conditioning the climate for change. This section details the current state and maturity of
Upwell’s Big Listening practices, including our Baseline methodology and spike quantification
methodology.
What is Big Listening?
“Take things as they are. Punch when you have to punch. Kick when you have to kick.”
- Bruce Lee
As more and more of our conversation moves
online, the potential of big data to help advocacy
organizations understand the environment for their
works also increases. To seize this opportunity fully
requires setting aside preconceptions and engaging
with the world as it is, right now, not as it was
assumed to be nine months prior in a grant proposal.
In the words of Bruce Lee, you have to “take things
as they are,” and use immediate insights to inform
your actions.
Bruce Lee, strategic opportunist
Since November 2011, Upwell has been monitoring
the online ocean conversation on a daily basis to identify opportunities to use our distributed
network for online campaigning. We listen to eight primary ocean topics: Overfishing,
Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans (whales and dolphins),
25

26. Sharks, Tuna, Gulf of Mexico and Ocean Acidification. Our method of conversational analysis has
been called Big Listening, first by Micah Sifry of Personal Democracy Forum, and later by
nonprofit social media expert Beth Kanter.
Typically, Upwell has used Big Listening to inform campaigns that are then implemented across
our distributed network of evangelists, influencers and social media managers in order to spread
the marine conservation conversation beyond the "ocean sector" (beyond ocean conservation
organizations and marine scientists). During and after our campaigns, we use the same Big
Listening methodology to measure how many social mentions (e.g., tweets, Facebook posts,
blog posts) happened in real time.
Big Listening is the art and practice of tracking topical online conversations over time—
listening to what “the internet,” writ large, is talking about. When combined with data-informed
campaigning, Big Listening provides a methodology for increasing both the frequency and
volume of online conversation around a particular issue. The basic idea is to identify pockets of
real-time or historical conversation, wherever they may be, and then to use that information to
make the conversation grow bigger. Big Listening is distinguished from traditional social media
monitoring by its scale, fluidity, focus on issue or cause monitoring, and expanded access to
historical data.
We are not alone in innovating in online conversation. Big corporations, their brands and the
military are all attempting to make sense of the new networked landscape, whether we realize
it’s there or not. From Target predicting a teenage girl was pregnant through her purchase
pattern, or the Obama campaign stitching together millions of voter records with proprietary
consumer datasets, this cloud-based, networked, indexable world is here to stay.
At Nestle, the largest food company in the
world, they have gotten the message. After
reaching the peak (or perhaps the trough) of
social media mismanagement during a
Greenpeace campaign that targeted the use of
palm oil in Kit Kats, the company dramatically
ramped up their online listening through their
Digital Acceleration Team. As profiled in a
recent Reuters story, the team operates out of
a social media war room. Radian6 widgets
gleam on wall-mounted flat screen monitors
as employees fight for the reputation of,
A view inside Nestle’s Digital Acceleration Team among other Nestle products, the plastic
headquarters [original source] water bottle industry.2
2 http:/
/uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026
26

27. Our war room is a little different. Instead of monitoring a corporate brand or a product, we
monitor the brand of the ocean, focusing on sustainable seafood and overfishing. Whereas
Nestle’s listening topics are comparatively static and focused on their company properties, ours
flow and evolve with the dynamic cause or movement-based conversations that we monitor.
This distinction in listening, between the static product and brand conversations typified by
Nestle, and the shape-shifting, dynamic ocean conversations that Upwell follows, is significant,
and a key distinction of Big Listening as we define and practice it. We should note that when
Nestle D.A.T. members monitor the plastic water bottle conversation as a whole, rather than
their company’s share of it, they are practicing something more similar to what we do at Upwell.
Baseline Methodology
What is a Baseline?
Upwell practices Big Listening on English-language conversations in the following eight topic
areas: Overfishing, Sustainable Seafood, Marine Protected Areas (MPAs), Oceans, Cetaceans
(whales and dolphins), Sharks, Tuna, Gulf of Mexico and Ocean Acidification. For each topic,
both real-time and historical data provide essential context for understanding the volume,
evolution and characteristics of the overall conversation.
Each topic we monitor is characterized and defined by a set of search terms (including
exclusions) that we refine on an ongoing basis. While we recognize the limitations of “keyword
groups,” such as their reliance on text-based results and the absence of contextual awareness,
they do provide a powerful tool for analyzing online attention. The development and active
refinement of keyword groups is at the heart of Big Listening methodology.
We use Big Listening in order to:
• identify and target high-value items for campaign purposes,
• compare the relative size of different ocean sub issues (e.g. sharks vs. whales), and
• measure the impact of our campaigns.
Since Upwell is a campaign agency (among other things), we needed a way to characterize these
conversations as they exist, absent our interventions. Enter the Baseline. Baselines help us to
anchor campaign performance targets in measures of past conversational volume. We set goals
informed by the Baseline (as well as by spikes), and then campaign to meet and exceed those
targets.
Upwell informally defines a conversation’s Baseline at the point below which the daily volume
doesn’t drop. It can be thought of as a floor (although it is often quite high—in the tens of
thousands for a conversation like Cetaceans) or as the number of social mentions performed
27

28. each day by the topic’s diehard conversationalists. If
everyone else left the party, the Baseline would still be
there, dancing by itself.
The Baseline: Up Close and
Personal
Upwell’s Baseline methodology has evolved to capture
the highly dynamic conversations we watch, especially
cyclical variations by day of the week. These cyclical
variations often result from usage and posting patterns.
For example, people tend to talk on the internet when
Robyn: euro popstar, solitary dancer,
they’re at work. Over the course of our pilot phase,
human Baseline metaphor
Upwell has used three different version of Baseline
methodology to better measure the dynamic online
conversation space:
• Baseline v1.0: The lowest level of daily social mentions for a given conversation, for a given
period (implemented using Upwell topical keyword groups) [in use through late August
2012]
• Baseline v2.0: The median daily social mentions for a given conversation/keyword group
for a given period [in use through mid-November 2012]
• Baseline v3.0: The average of the lowest 10% of social mention values for a topic on a given
day of the week [in use through early January 2013]
• Baseline v3.1: The average of the lowest 20% of social mention values for a topic on a
given day of the week [currently in use]
Our Baseline quantification methodology was created with input from leaders in the field
including: K. D. Payne, Chairman & founder of Salience/ KDPaine & Partners, and co-author of
the recently released Measuring The Networked Nonprofit; leading nonprofit technologist and
Packard Fellow, Beth Kanter; and a senior educational policy analyst for a leading national
measurement/ social statistics firm contracted by the Gates Foundation, the Broad Foundation
and the U.S. Department of Education.
To calculate the Baseline for a particular topic we begin by compiling all available social
mention data for the period since we started monitoring it (mid-October 2011 or later, depending
on the topic). We then disaggregate the data by the day of the week in order to deal with cyclical
variations in post volume and compare Mondays to Mondays, and Sundays to Sundays. Once
that’s done we calculate the average (mean) of the lowest 20% of values for each of the seven
days. Taken together, those day-of-the-week values are what we refer to as the Baseline.
28

29. We selected the mean to establish a specific value for each day of the week for three reasons.
First, the mean is the starting point for calculating standard deviations used in our spike
quantification methodology. Second, given the small size of most ocean conversations, the
mean is the most typically consistent and available measurement when analyzing the
conversation on a by day-of-the-week basis.
Baseline Social Mentions by Day-of-Week for Upwell’s Sustainable Seafood keyword group
(10/17/11 - 1/29/13).
These daily Baseline values are then graphed against social mention data over time. The graph
below shows the result.
29

30. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood
Baseline, June 1, 2012 - August 1, 2012.
Context and Challenges for Baseline
Quantification
As seen in the overfishing and sustainable seafood conversations, day-of-the-week periodicity is
highly evident in social mention volume, largely driven by mention increases during U.S.
working hours across Eastern to Pacific time zones, and with substantial drop offs on weekends.
Using the current Baseline as a reference for setting campaign goals removes the disincentives
present in previous Baseline quantifications to campaign on lower volume days, and gives a
more accurate picture of success on all campaign days.
Our campaign efforts, along with unexpected conversational developments (e.g., the release of a
new report or a natural disaster) require us to add new terms to the relevant keyword group so
that our work, or online mentions of a relevant, unanticipated event, is captured in our search
terms. In the inevitable cases where we find noise in the results returned by particular terms (old
and new), we respond by tightening or removing those terms from the keyword group.
30

31. Since keyword groups have keywords added and subtracted on an ongoing basis, there are some
inherent challenges. How should Baseline calculations account for these changes? How and
when should Baselines be refreshed? Should campaign targets be refreshed retrospectively?
How should we treat a spike that loses or gains volume with a refreshed keyword group?
Moving forward, Upwell will continue to drive innovations in our Baseline methodology and in
the integration of that methodology into our campaign process.
Finding the Baseline
Developing a Baseline for a topic or conversation requires an iterative process of definition,
testing and measurement. And once you’ve got it, you have to refresh it to account for
conversational evolutions. It’s like a marriage in that way, or so we’re told.
Measuring something requires you to define it. The trick in measuring a conversation is that
conversations change over time as participants engage in dialogue. As conversations
dynamically evolve over time, so too do the methods of expression (i.e. terminology, imagery,
metaphors, etc.), the composition of participants, and the accompanying platforms. Much like
species evolution, these changes are not always in a direction that we perceive to be fruitful.
Evolution can lead to progress just as it can lead to dead-ends or fragmentation. Because of this,
no conversational “listening” can ever be exhaustive (as some elements of the conversation will
always be overlooked), nor can it be perfectly accurate (as noise will always creep in). This is
why we continue to add and subtract terms to our Baseline keyword groups. Ongoing
monitoring and modification of a Baseline keyword group is the most effective way to keep that
keyword group refreshed and accurate. Since designing a keyword is as much about selective
addition as selective omission, our ability to effectively evolve keywords and keyword groups is
informed just as much by our listening practice with Radian6 as it is by our personal listening,
professional network and subject matter expertise. No robot can do this, although some are
trying, and interns probably can’t do it either.
In “baselining” a conversation, Upwell begins by developing a conceptual framework for the
topic in question. For the purposes of explanation, let’s imagine our topic is marine debris. We
would begin our Baseline development process by outlining the conceptual and temporal
boundaries of analysis for marine debris. The temporal aspect is important because a keyword
group developed for one time period may lose significant accuracy (and utility) when applied to
another period. For the conceptual outlines we often make use of a mind map such as the one
for Oceans shown below.
31

32. Upwell mind map for Oceans keywords, January 2012
For marine debris the concept map would include items such as marine trash, the pacific gyre,
marine plastics, great pacific garbage patch, seaplex (minus exclusions for the botanical
shampoo of the same name), albatross AND plastic. The concept map would also include people,
campaigns, expeditions and organizations such as Miriam Goldstein (a marine debris expert),
The Trash Free Seas Alliance, the Plastiki, and Seaplex.
The concept map becomes a design artifact for further conversations about the conversation.
Although we sometimes shortcut this process in the interests of time, we refine the map
through a series of discussions and email exchanges with subject matter experts and
knowledgeable people in the industry or industries at play.
Once we have a solid map of the conversation, we turn the map into a series of keywords.
Keywords are textual search terms, much like something you might google. A keyword for
Upwell, for example, would be “Upwell.” It might also be a distinctive phrase (or fragment
thereof) such as our tagline, “the ocean is our client.” A keyword, in this way, can actually be a
number of elements (such as multiple words in a phrase) despite its singular form. This fact will
become more important as we discuss more of those elements. A collection of keywords is
called a keyword group, and search results for keyword groups are the foundational output of
Big Listening.
A couple wrinkles make the construction of keywords and keyword groups significantly more
challenging than one might expect. The first is noise. When you type something into a Google
search bar and click on a result, you are deploying a potent combination of Google’s massive
32

33. computational power, billions of dollars in said company’s algorithmic investments, and the
concentrated smarts of your own interpretive brainpower. The last element is particularly
important. Whereas Google displays dozens, if not thousands of search results for you to choose
from—and then asks you to filter those results—the keyword queries we construct for Big
Listening must be built so as to filter out as much noise as possible. To return to friend-of-
Upwell Miriam Goldstein, mentioned above for her marine debris expertise, a well-constructed
keyword group for that subject would probably not include her name as a standalone keyword—
the reason being that she talks about other, non-marine-debris subjects as well. Entering
“Miriam Goldstein” as a keyword nets you any mention of her full name, whether that takes the
shape of a blog post about ocean trash or a friend’s tweet referencing her attendance at a
particularly yummy brunch meeting. What you leave out of a keyword or keyword group is as
important as what you put in.
Pruning out extraneous results through proper keyword construction brings us to the second
wrinkle: exclusions. Exclusions are also textual search terms, but their purpose is to filter out
results that match their terms. Exclusions can be tied to specific keywords or to entire keyword
groups. A well-constructed (or scoped) exclusion can be the difference between finding online
mentions of sharks, the creatures, or finding online mentions of the San Jose Sharks, the hockey
team (creatures of a different sort). Exclusions can filter out things beyond keywords (such as
entire categories of website domains, particular geographies of origin, or what a computer
determines to be a particular language) using a variety of tools used in Big Listening—source
filters, to give one example.
Another wrinkle in keyword construction is proximity. Proximity is not available in every tool
that might be applied in a Big Listening process but it is present in Radian6, our tool of choice at
the moment. Proximity is a modifier that can be applied to two or more words in a keyword, say
“marine” and “debris.” Proximity, denoted by “~”, tells the tool/service how close a set of words
must be in order to return a match. Closeness basically means: how many other words come in
between? If we were to set proximity to zero for “marine debris,” Radian6 would return only
items that include that exact phrase. If we set proximity to three, for example, we might get
results such as “marine layer clotted with debris.” That distinction becomes increasingly
important when Radian6 scrapes long forum discussions, news articles or blog posts in which a
topic might be mentioned in an extremely peripheral manner. Proximity provides another tool
to scope a given keyword and focus the results in a particular way.
Keyword development feeds into an ongoing measurement process of Scope / Test / Adapt /
Share. The cycle is presented below.
33

34. SCOPE
Initial investigation
• Outline the conceptual and temporal boundaries of analysis for the topic
• In consultation with subject-matter experts and other stakeholders, create a seed list of
topics, subtopics, potential online influencers, and known online sources, events, and
campaigns
TEST
Generating preliminary keywords
• Use the seed list to develop initial keyword inputs for online search and social media
monitoring services
• Develop a more detailed set of keywords
• Verify keyword accuracy and relevance using Radian6 to graph and spot-check search
results, adding exclusion terms to filter extraneous results/noise, or various degrees of
proximity to widen the net
34

35. ADAPT
Refining keyword groups
• Share keyword lists with key informants (subject experts, foundation staff, campaigners
etc.) and incorporate feedback (e.g., additional terms, scope adjustments)
• Repeat steps 3, 4 and 5 with updated keywords
Cultivating and maintaining keywords
• Campaign or otherwise monitor keyword group results on an ongoing basis
• Update Baseline keyword groups with new inclusions and exclusions based on current
events, campaigns and other developments (while always testing for the introduction of
noise, per step 5)
• On an as-needed basis, generally after at least three months of listening, share Baseline
keyword groups with subject matter experts and other groups to gather feedback and
potential improvements
SHARE
Exporting and preparing data
• On a monthly, quarterly or to-order basis, export Big Listening data based on the most
current keyword groups
• Recalculate Baseline values
• Graph and annotate charts with spike identifications
Packaging and distributing insights
• Create reports, blog posts and other types of synthesis for external audiences
Improving the methodology
• Gather feedback and process what we’ve discovered
• Iterate our overall set of procedures
A crucial detail of the final stage of this process is the fact that exporting the data freezes it in
time. To offer a contemporary example, all of the data in this report was current as of the end of
January 2013, and then it was frozen in a spreadsheet. It’s important to remember that the
conversations we monitor continue to change, even as we’re measuring and reporting on them.
Because the exported data is a snapshot of results for a particular conversation’s keyword group,
35

36. as it existed at a particular time (of export), from a particular tool (or combination of tools), the
resulting values cannot and should not be separated from the keyword group that produced
them. Furthermore, due to the item volume returned by some of the larger keyword groups,
exporting data will sometimes produce variations in measurements for the same hour, day or
time period. This variation is due to the tools we use and is generally extremely small given the
scale of the topics we’re monitoring. These two factors combine to reinforce our belief that Big
Listening data can only be fully interpreted if the underlying keywords are available—anything
less is a black box.
Spike Quantification
What is a Spike?
A spike is a significant increase in online attention for a particular topic. When you graph those
social mentions, you can actually see that burst of attention ‘spike’ the graph—hence the name.
We have been observing spikes in the wild, so to speak, since the beginning of Upwell. It’s a
concept that is at least somewhat familiar to anyone who has ever described a video as “viral,”
or checked out the list of the most shared articles on the New York Times website. A lot of
people sharing one thing over a short time creates a spike. In the world of Big Listening, that one
thing they share can actually be a large number of different things on the same topic, but the
general point remains the same. Surges in attention create spikes. So how do you measure one?
Let’s revisit that graph of the Sustainable Seafood keyword group that we looked at earlier.
36

37. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood
Baseline, June 1, 2012 - August 1, 2012.
It seems pretty clear that there are two spikes in this time period. One appears on June 8, the
other on June 16. But what about the other days? How far above the Baseline does social
mention volume have to be in order to qualify as a spike? We set out to find a way to compare
spikes that would answer the question.
Before we dive in, it’s important to note that social mention volume for a given day is a
construct. We decided to use a day as the operating unit of time both because the tools we have
available to us use that temporal distinction, and because a day as a unit of measurement is
widely understood. That is not to say that one couldn’t decide to measure spikes by the hour, by
the minute, or by some other amount of time. We made a conscious decision to build our initial
definition of a spike around the day, but infinite other options exist as well.
A second caveat is that focusing on spikes may obscure what is actually making up the long tail
of post volume. Upwell talks about, and quantifies, much of this activity as the Baseline, but
there may be other small-to-medium bursts of attention that last more than a day and
consequently don’t visually ‘spike’ a graph in the same way (think of a multi-day increase in
37

38. attention as a hump or a mesa, rather than the taller, more angular spike). Spikes look good on
charts, and they help push conversations into the wider internet, but they are not the whole
story of an online topic. We long for a day when tools for Big Listening allow us to view topic
volume graphs like geologists look at cross-sections of rocks—that day is not here yet.
With those caveats out of the way we can return to our earlier question: what is a spike?
Remember from our discussion of Baseline quantification that Upwell’s analysis is designed to
inform a set of interventionist activities. We:
• identify and target high-value items to campaign on;
• compare the relative size of different ocean sub issues (e.g. sharks vs. whales); and
• measure the impact of our campaigns.
Spike quantification informs our campaigning and provides one measure of results. We’re not
interested in just contributing to the noise around a given ocean topic, we actually want to help
a signal to emerge. Spikes are those signals. Evaluating opportunities to campaign becomes a
much more concrete activity when you know exactly how many social mentions are needed to
break through the regular volume of conversation.
After examining historical social mention volume for our Sustainable Seafood and Overfishing
keyword groups, we calculated a variety of statistical thresholds for the exported data and
compared the results to our measured campaign and spike data. As discussed earlier, Upwell’s
Baseline calculations are derived from the insight that our primary ocean topics each
demonstrate a weekly periodicity. Similarly, in calculating potential thresholds for what
constitutes a spike, we started with that same insight and then calculated various multiples of
standard deviation above the average (mean) value for that day of the week. Because standard
deviation measures how spread out the values within a data set are, using it to measure a
particular value’s variation from the “normal” value of that data set is a good way to test for a
spike. Spikes are visible because they’re outliers, and that’s what the standard deviation
threshold(s) tests.
38

39. Day-of-the-week values for the Sustainable Seafood Baseline, along with the Sustainable Seafood
mean, and mean +1x, +1.5x and +2x standard deviations (10/17/11 - 1/29/13). [Source]
As seen above, the standard deviation thresholds are higher than both the Baseline and the
mean. Graphing those thresholds against our campaign and event records revealed that the one
standard deviation threshold was the most accurate representation of what we were observing
on a day-to-day basis.
39

40. Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood
Baseline vs. ‘Mean + 1 Standard Deviation’ Spike Threshold (June 1, 2012 - August 1, 2012)
Upwell defines a spike as occurring when the social mention volume for a given day meets
or exceeds one standard deviation from the mean of all recorded values for that same day of
the week.
While a critic might accuse us of working backwards to find the threshold that gives the best fit,
we would actually agree. Sustainable Seafood and Overfishing are the topics that we know the
best—because we’ve monitored them and campaigned on them with the most focus—and we
were looking for a metric that would have practical implications for attention campaigns. As
mentioned before, we remain open to other spike quantification approaches but this one is our
preferred option, given what we know right now.
40

41. What Does Spike Quantification Tell Us?
Upwell’s spike quantification methodology is in alpha, so to speak, and going forward we will
look to improve it. The possibilities for more comparative measures of success are numerous.
One thing is certain, however: applying a spike quantification lens to our work is illuminating.
Spike comparison beta methodology?
The graphs on the following pages show our first Winter in 2011 and most recent Winter in 2012
working in the Overfishing and Sustainable Seafood conversations. Both one standard deviation
and two standard deviation threshold lines are included for reference.
The comparison in time periods for both conversations is dramatic. There is a noticeable
increase in spike frequency (the number of spikes), spike volume or “spikiness” (see: the number
of spikes exceeding two standard deviations), and in the overall volume of conversation in the
time period as measured against the Baseline. To be blunt: this is what success looks like.
41

42. Sustainable Seafood: Winter 2011
Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to the
Sustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and two
standard deviations above the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume:
45,255 social mentions over 107 days. Average volume / day: 423 social mentions.
42

43. Sustainable Seafood: Winter 2012
Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to the
Sustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and two
standard deviations above the day-of-the-week mean (10/1/2012 - 1/29/2013). Total post volume:
66,456 social mentions over 121 days. Average volume / day: 549 social mentions.
43

44. Overfishing: Winter 2011
Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishing
Baseline, as well as to spike thresholds of one standard deviation and two standard deviations
above the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume: 211,799 social mentions
over 107 days. Average volume / day: 1,979 social mentions.
44

45. Overfishing: Winter 2012
Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishing
baseline, as well as spike thresholds of one standard deviation and two standard deviations above
the day-of-the-week mean (10/1/2012 - 1/29/13). Total post volume: 409,692 social mentions over
121 days. Average volume / day: 3,386 social mentions.
45

46. Keyword Sets
The following search terms are Upwell Radian6 keyword sets for Upwell’s primary campaign
topics—Sustainable Seafood and Overfishing—as of the writing of this report, along with a brief
description of what each keyword group is designed to capture. As online content and context
continually changes, keyword groups should ideally be monitored and refined on an ongoing
basis as well. A keyword that returns noise-free results for one period of time may be filled with
unrelated results for another. Upwell’s keyword groups were designed for the time periods
specified in each description.
Fishing and Seafood: Sustainable Seafood
Primary Keyword Group: Sustainable Seafood
Earliest Data: 10/17/2011
Keywords: "#seafoodsummit", "#ss12hk", "@leodicaprio" AND "@upwell_us", "@seafoodwatch",
"@upwell_us" AND "vote4stuff", "alaska salmon", "alaskan salmon", "aquaculture dialogs",
"aquaculture dialogues", "aquaculture stewardship council", "barton seaver", "big listerner",
"bitly.com/wppomr", "bittman" AND "tuna" AND "safeway", "cannibal endtimes lobster"~6,
"cannibalistic lobsters" AND "end times", "cannibalistic lobsters overfishing"~20, "casson trenor",
"catch limits", "catch shares", "chefs collaborative" AND "seafood", "cruel new fact of crustacean
life" AND "lobster cannibalism", "davidsuzukifdn lobsters into cannibals"~9, "dungeness crab",
"environmentally responsible seafood", "f.a.d.-free" AND "tuna", "f.a.d.-free tuna comes to
safeway", "fad-free" AND "tuna", "fao" AND "fisheries", "fishing quotas", "fishphone", "fishwatch",
"food and agriculture organization" AND "seafood", "friend of the sea", "green chefs blue ocean",
"how safeway ended up selling cheap, responsibly-caught store brand tuna", "http:/ /
bittman.blogs.nytimes.com/2012/10/02/fad-free-tuna-comes-to-safeway-affordably/", "http:/ /
twitpic.com/bli9ak", "http:/ /www.bethkanter.org/listener/", "h"#biglistener", ttp:/ /
www.fastcoexist.com/node/1680610", "http:/ /www.youtube.com/watch?v=jmkevhbejla",
"https://bitly.com/wppomr", "international seafood sustainability foundation", "issf" AND
"fishing", "kanter" AND "big listener", "kanter" AND "big listening", "lack of predators lobster-on-
lobster violence", "leodicaprio" AND "upwell", "marine stewardship council", "maximum
sustainable yield", "menhaden" AND "sustainability", "menhaden" AND "sustainable", "no
overfishing guaranteed"~4, "ocean acidification" AND "google earth", "ocean wise", "ocean-
friendly aquaculture", "ocean-friendly seafood", "oysters" AND "sustainability", "oysters" AND
"sustainable", "political porpoise", "precautionary principle" AND "seafood", "responsibly caught
tuna"~3, "reuters" AND "lobster cannibalism", "safeway sustainable tuna"~15, "seafood choices
alliance", "seafood consumer guide", "seafood ecolabel", "seafood fraud is a serious issue",
"seafood pocket guide", "seafood ratings", "seafood summit", "seafood sustainability", "seafood
watch", "seafoodwatch", "sustainable fisheries", "sustainable fisheries act", "sustainable
fisherman", "sustainable fishermen", "sustainable fishery", "sustainable seafood", "sustainable
seafood"~9, "sustainable sushi", "sustainable" AND "tilapia", "the lobsters in maine are eating
46