Presentation by Ellen Bird & Christian Kugel at ESOMAR Global 2010

1. What online behavior reveals about digital culture
An assessment of techniques for identifying themes and interpreting them for insights
Christian Kugel & Ellen Bird December 2010
The contemporary online experience is a participatory one. This relatively recent development has had significant impact on the
nature of digital culture. Based on a belief that an understanding of digital culture is key in developing digital marketing ef-
forts, VivaKi created Reflecteur, an original cross-agency initiative. In this paper the authors describe a theoretical framework for
interpreting digital culture and compare and contrast two methods—user-generated content monitoring and crowdsourcing—for
uncovering digital cultural insights.
The Internet was invented to be a participative medium. increasingly be compelled to do so as well. The initiative
Without individuals participating, its value plummets. The described here was designed to help an agency network keep
Internet sans people influencing it simply makes no sense. its finger on the pulse of digital culture and arm every em-
Over the past five years, however, this attribute has been ployee with an understanding of the themes, characteristics
magnified by the birth and adoption of social media. Social and nuances of this ever evolving, dynamic subject.
platforms such as video sharing, blogging, networking and
photo sharing (just to name just a few) have truly revolution-
We christened this initiative with the project name “Reflec-
ized people’s digital experience. Social tools are built into
teur,” meant to convey our goal of being a voyeur of digital
nearly every site—people have come to expect the ubiquity
culture while reflecting the underlying cultural insights back
of the upload button, the comment field, and the “share this”
to our constituents. This initiative began in Q4 2007 and
icon.
continues to this day. Over
Figure 1: VivaKi Agency Network the course of three years, we
Digital marketing trends have have leveraged user-generated
clearly followed the migration content (UGC) monitoring,
to social experiences. It is the crowdsourcing, ethnographic
rare brand that does not have techniques and investigative
presence in multiple social en- methods to understand digital
vironments or infuse its digital culture, paint a picture of it for
ZenithOptimedia
assets with social functionality. our constituents and ultimately
As content diversified to include enable them to incorporate
a richer mix of user-generated these insights into digital
material, and as social platform- marketing initiatives. And in
enabled sharing became com- that time, Reflecteur has grown
mon, a distinct digital culture from a small project incubated
emerged—a culture rooted in inside Denuo, a boutique VivaKi
the act of participation. agency, to all of the agency
families within the entire VivaKi
network.
And increasingly, we believe
that understanding, internal-
izing and aligning with digital VivaKi comprises the digital
culture will be a price of entry and media agencies of Publi-
for crafting successful digital cis Groupe. Two media agency
marketing initiatives. Just as conventional advertising must families, Starcom MediaVest and ZenithOptimedia, as well as
recognize and address cultural cues, digital marketing will two digital agency networks, Razorfish and Digitas, are all
©2010 VivaKi. All Rights Reserved. Page 1

2. members of the VivaKi network. All VivaKi agencies believe theory of culture, but the Internet itself. Such characteristics
that the ability to win in the future requires comprehensive are precisely why we have a need for search engines, and
digital expertise and capabilities. Understanding, internalizing they describe the design philosophy of a site like Face-
and applying digital culture, for the reasons stated above, is book—which is never the same for even a second as people
a necessary element of that exhaustive expertise. continually add photos and post new status updates to it. In
1985, well before Twitter, Flickr or Blogger, Hassan described
postmodernism in opposition to modernism in a simple table.
He contrasted the postmodern attribute of “anarchy” to mod-
Theoretical Framework
ernism’s “hierarchy” and “idiolect” to “master code” and even
Precious little research has been published on the topic of
mentioned the postmodernist trait of “participation” com-
digital culture. The subject has not been extensively exam-
pared to the modernist trait of “distance” (Hassan, 1985).
ined in academia, due in large part to the fact that it does
not fit neatly into any one academic discipline. It certainly is
a relevant sub-topic in sociological cultural studies, but it also Table 1: Schematic differences between moderism
touches fields as diverse as cultural anthropology, communi- and postmodernism
cations and media theory, marketing, psychology and aspects modernism postmodernism
of natural sciences. Without an existing guiding framework
that we could readily use, we were left to build our own. purpose play
art object/finished work process/performance/happening
Fortunately, cultural theorists commonly incorporate aspects
distance participation
of media influence in their work, and many of the ideas pub-
creation/totalization decreation/deconstruction
lished in the last twenty years on the topic were relevant to genre/boundry text/intertext
our cause. hypotaxis parataxis
metaphor metonymy
selection combination
In our attempt to identify a working theoretical framework,
type mutant
we examined a number of established, competing theories.
paranoia schizophrenia
The framework that we ultimately embraced was postmod-
ernism. Most of the tenets of this theory of culture coincided Source: Hassan (1985)
with our early observations of the new participatory digital
Based on a dozen more supporting examples, we concluded
experience. Barker articulates postmodern culture quite
that postmodern cultural theory very naturally described how
succinctly by claiming that it is “marked by a sense of the
people interact with content, with brands and with each other
fragmentary, ambiguous and uncertain quality of the world
via the Internet.
along with high levels of personal and social reflexivity”
(2008). He further states that “postmodernism argues that
knowledge is: specific to language-games; [and] local, plural
and diverse” (2008). When each of these embedded concepts Initial approach
is examined independently, it describes aspects of the digital In Q4 2007, VivaKi contracted with a UGC monitoring vendor
experience almost perfectly. One has to look no further than to provide the source data for the Reflecteur initiative. We
status updates on Facebook or comments to videos posted are not naming the vendor here. The purpose of this paper
to YouTube to see the proliferation of language games, such is not to evaluate or assess any particular UGC monitor-
as jargon (e.g., noob, pwnage), acronyms (e.g., LOL, FTW) ing firm but rather to describe the characteristics of UCG
and emoticons. The notion of local/plural/diverse knowledge monitoring as they relate to understanding digital culture. We
is best exemplified by Wikipedia, which relies on millions of believe that the challenges detailed here are endemic to all
editors, each with their own type of expertise, to create, edit UGC monitoring vendors in this particular context. The plan
and police the site’s content. The idea of fragmentation is one called for an initial launch of Reflecteur in January, 2008.
that keeps brand managers and media planners up at night Two months prior, the vendor switched on the data feeds, at
and is the significant contributing factor for the formation of which point the team began exploring the data and engaging
online ad networks. And social reflexivity describes what we in mock analytical exercises to ensure that the launch date
see in action any time a new online meme catches on: online could me met. For six months, the team worked with the
users simultaneously are influenced by digital culture (by data every week, first in anticipation of the launch and then
consuming it) and directly influence it (by sharing or contrib- while producing weekly issues of the Reflecteur publication.
uting to new content).
Harvey echoes these points and perhaps goes even further Determining the method
by saying that “postmodernism swims, even wallows, in the The methods for capturing, aggregating and analyzing UGC
fragmentary and the chaotic currents of change as if that is data are varied. The specific method recommended by the
all there is” (1990). It is almost as if he is describing not a UGC vendor deviated meaningfully from a “typical” project.
©2010 VivaKi. All Rights Reserved. Page 2

3. Most UGC monitoring initiatives are focused on brands— projects with no built-in overlap. We deliberately varied the
counting the number of brand mentions, analyzing brand number of core blogs for each of the four topics. Knowing if
sentiment, identifying keywords that are frequently associat- there was a “right” number of core blogs would have been
ed with brands, and so on. For these applications, a com- an important finding in expanding the initiative to include
monly-accepted best practice is to cast as wide of a “data eli- other topics. All the core blogs were in English. The identified
gibility” net as possible. When trying to capture the entirety source blogs were distributed as follows:
of brand mentions in UGC, there is little sense in purposefully
• Parenting: 250 blogs
restricting or blocking entire streams of data. Any UGC data
should be eligible for inclusion, whether in the form of blog • Celebrity gossip: 89
posts, uploaded videos, message boards, comments, tweets,
• Consumer technology: 43
social network status updates and anything else.
• Timewasters: 25
For the Reflecteur initiative, the UCG vendor recommended
a blog-oriented approach, meaning that the eligible user- User-generated content eligible for inclusion in each of the
generated content was restricted to just blog posts. Because data streams originated with these source blogs. Every week,
we were interested in cultural concepts and not pre-defined the software pulled all the posts from each of the blogs in
keywords (as in the cases of brand monitoring projects), we these four categories. These blogs, therefore, were referred
were compelled to cast a narrower net. Including arbitrary to as “G0.” Any blogs that linked to G0 blogs in any given
message boards would prove problematic precisely because week were also eligible for inclusion. These were called the
of the lack of a keyword list. Without an a priori analyti- “G1” blogs. And finally, any blogs linking to the G1 blogs
cal framework, data from such broad and varied sources were also included (in any given week). These were called
becomes meaningless. Keyword lists are the backbone of the “G2” blogs (See figure 2).
the brand-oriented analytical framework. For the purpose of
uncovering insights in digital culture, there are no meaningful Figure 2: UGC blog distinctions
analogues.
G0 G1 G2
Blog Blogs Blogs
The blog-oriented approach to data eligibility effectively
meant that we deliberately chose to ignore large parts of Post Post Post
the UGC landscape. For the reasons described above, the
Post Post Post
vendor and the team did not believe that there was a feasible
alternative. Collectively, we all agreed that the major tradeoff Post Post Post
associated with the blog approach (altogether ignoring most
UGC data—everything that was not a blog) was necessary in
order to achieve clarity in the data.
Post Post
Data architecture Post Post
Having settled on a blog-oriented approach, the next step Post Post
was to apply sub-parameters to the blogosphere. For the
purposes of ensuring a successful launch, Reflecteur initially
focused on four aspects of digital culture: parenting, con-
The G0 blogs were chosen through a collaborative process
sumer technology, celebrity gossip and timewasters. Digital
between the team and the vendor. Having worked on projects
culture is, of course, much bigger than these four categories.
for brands that intersected the four categories (parenting for
But these four—which were chosen in consultation with our
brands targeting moms, for example), the vendor had al-
colleagues at VivaKi agencies—represented a nice cross-
ready established lists of relevant blogs. The team examined
section of both target audiences as well as content types.
each blog on the lists, removed the ones that were less rel-
Therefore, they represented a reasonable proof of concept.
evant or inactive and supplemented the lists with additional
If the process worked across these topics, the initiative could
blogs. This was a very manual process, but it needed to be.
be expanded to other relevant topics and subsequently grow
in scope.
Syndicated data does not exist for the parameters in which
we were working (blogs and specific areas of cultural focus).
After identifying these four categories, the vendor and the
And the team was creating a process—we did not have a di-
Reflecteur team assembled a list of core blogs which would
rect precedent. We certainly borrowed from the vendor’s past
power the data feeds. Each topic required its own sepa-
experience with similar subjects, but that experience was
rate data feed; the four were effectively treated as unique
©2010 VivaKi. All Rights Reserved. Page 3

4. directionally applicable, not directly applicable. The manual The reason for this analytical framework was rooted in
sorting only existed for the G0 blogs, however. The software pragmatism. We absolutely needed a measure of value;
automatically generated the rest of the eligible blogs. we needed to know where to look to identify digital culture
phenomena. We never envisioned the software uncover-
ing any actual insights—that would occur through human
The resulting data feeds, one for each of the four topics,
interpretation. But we needed a compass—a tool to help us
sourced the data from a pool of blogs; the pool being derived
focus on the most salient items each week. The vendor and
based on the presence of links. Any blog post originating
we assumed that links represented a reasonable proxy for
from any blog in the pool (G0, G1 or G2) was included in that
that compass. After all, if SiteA/Post1 garners ten times the
week’s analysis and data feed for that topic. Any blog post
number of links that SiteC/Post3 garners, it is reasonable to
originating outside of that pool was never sourced—it was
expect that the former is likely to be much more important.
never eligible for inclusion in the analysis.
It touches a nerve of some sort, or it inspires people’s desire
to share, or it generates a new online meme. It is more likely
It is important to note the nature of the actual data included to contain some insight into digital culture.
in the analyses. In this method, the “raw data” comprised
blog posts. Whether using Blogger, Wordpress or any other
blog publishing platform, posts share certain attributes,
which the vendor’s software was tuned to recognize. At the
Data reporting
Like most UGC monitoring projects (including brand-focused
core of the analytical process were these individual posts.
projects), data reporting for this initiative occurred through
To attribute a relative value to each post, the software then
an interactive dashboard. The primary mechanism for navi-
analyzed links—it looked for other blogs and blog posts that
gating the data was centered on the number of links any
linked back to any individual post in the core blogs.
given blog post amassed during a given week. The reporting
system ranked each post based on the total number of links
Imagine that SiteA has been identified as a core blog. SiteA/ and presented basic information about each post, including
Post1 is an original post appearing on that blog. SiteB/Post1 the title of the post and its URL. The “raw” data (the original
(a different blog) links to SiteA/Post1. That link is captured blog posts) was therefore available for examination via the
as a measure of relative value. SiteA/Post1 is credited with reporting interface. The interface was designed to add value
one link (See figure 3). to the raw data by focusing on “tabbed” data—the informa-
tion about link backs. An effective way to think about raw
and tabbed data for UGC monitoring is to compare it to
Figure 3: Blog post ranking conventional survey research. Raw data in survey research
Blog 1 Blog 2 are the respondent-level records, and tabbed data would
take the form of a set of cross tabs. In this example of UGC
monitoring, raw data are individual blog posts, and tabbed
Post Post data takes the form of the link information (number of link
1 1 backs) found in the reporting interface. In both cases, the
tabbed data serves only as a tool to analyze and interpret the
raw data.
Post Post Additional functionality was built into the reporting system.
Post
Post
Post
Post
Post Post
A “hot topic” section displayed a ranking of specific words
Post
Post Post Post
Post
Post that recurred in the pool of blogs for that week. Similar to
Post
the information presented in “hot topics” and “hot searches”
Post
Post found in Google Trends, or in the “trending topics” that Twit-
Post Post
Post
Post
Post
Post
Post
Post
ter compiles, this report simply counted the occurrences
Post Post
Post
Post
Post
Post
Post of words and short phrases found in the pool of blogs. And
Post
because the pool changed every week— G1 and G2 blogs
would migrate in and out of the pool from week to week—the
8 links 3 links
reporting interface also captured the sites comprising the
pool for each week.
This process continues until the system exhausts all the links
to the core blog post that it can find. In doing so, the soft-
ware builds the lists for the G1 and G2 blogs, for which the
Interpretation and insight dissemination
link-tracking process repeats. The result is a ranking of blog
Reflecteur was initially a weekly publication (it is now bi-
posts, one list for each of the four culture topics, based on
weekly), so the team constructed a weekly process for
the number of links each post receives.
©2010 VivaKi. All Rights Reserved. Page 4

5. accessing the reporting interface and interpreting the data resentative of some established or emerging trend, behavior
for insights. We relied heavily on the main reporting feature or aspect of the digital collective consciousness? We focus on
(the ranking of link backs) to determine which posts seemed qualitative insights, just as conventional ethnography does.
to generate broad interest. The team would examine the top We do not attempt to address the extent of cultural themes
10-15 posts directly, which began with visiting the blogs and that we identify because ethnography cannot adequately
reading the ranked posts. achieve a quantitative task such as this. But we do note re-
curring themes over time; and, in doing so, can directionally
gauge if a theme is particularly pervasive, if it evolves or if it
We relied on ethnographic techniques to arrive at the under-
fades away.
lying insights. Robert Kozinets (2002) described a framework
for online ethnography which he called “netnography.” Simply
put, netnography is the extension of ethnography to the digi- Once we were comfortable that we have a good grasp of the
tal space. Ethnography is particularly suited for adaptation digital cultural insights from any given week, the next chal-
to online research due to its inherent flexibility and open- lenge was disseminating those insights to our constituents.
endedness – traits well-suited to the dynamic nature of the At launch, the constituent-facing part of Reflecteur took the
Internet. Relying on online communities, Kozinets outlined form of a weekly publication comprising three elements:
five steps for conducting netnographic research: 1. Make cul-
• Four original articles describing pieces of user-generated
ture entrée, 2. Gather and analyze data, 3. Ensure trustwor-
content and the accompanying digital cultural insight.
thy interpretation, 4. Conduct ethical research and 5. Provide
These articles also linked to the content so that the
opportunities for culture member feedback (2002).
reader could experience it for themselves.
• Nine additional links to pieces of content that are note-
In addition to applying aspects of the netnography frame-
worthy but not necessarily worthy of a full article. These
work, we found that conventional ethnographic techniques
links either attain some level of virality or they support
translated quite nicely to UGC environments such as blogs.
a digital culture theme which we had previously uncov-
Rather than observing people in their homes interacting
ered.
with a website, however, we looked for evidence of interac-
tion that people left. These came in two primary forms: the • Top Reflecteur sites. Because the pool of blogs var-
comments that site visitors submit on the blog post itself and ied from week to week, it was important to highlight
the commentary that other bloggers include when they link a sample of the sites comprising the pool. Each week,
to the post. These user reactions have proven to be the most the publication linked to 25 sites that contributed to the
fertile ground for assessing the nature of people’s response. source data in the reporting system.
In UGC environments, people are increasingly comfortable
posting comments and engaging in mini debates with other
The publication was designed to calibrate the awareness of
commenters. Because the string of comments is cumula-
and sensitivity to digital culture among Denuo’s employee
tive and archived, it serves as a collective record of reader
base. It was not designed to address any specific marketing
response. For this reason, comments became the keystone
challenge or target audience research objective. Reflecteur
for understanding the underlying insight behind a piece of
was initiated to help the organization transform itself to align
user-generated content.
with trends in digital culture—a topic that until then had been
invisible at the institutional knowledge level. Individually,
Interpretation did not stop with examining comment threads, Denuo’s employees were tuned into digital culture at various
however. As it gained more ex- levels. Organizationally, it was
perience interpreting user-gen- Table 2: Checklist for Reflecteur content not; Reflecteur filled that gap.
erated content, the Reflecteur
team developed a list of ques-
Does it: As illustrated by:
Over time, the consciousness of
tions to ask when inspecting a Ellicit high levels of Dviersity and amount of the organization was re-orient-
piece of content. This checklist participation interactions
ed to be more aligned with dig-
served as a guide to ensure
Have cross community appeal ital culture. Individuals would
that the team approached each Diversity in linking sites
reference insights uncovered by
piece of content consistently
Have global appeal Links from sites around the Reflecteur through the course
(see table 2).
world/in various languages of their day-to-day work, com-
pare their interpretations of a
The question that we ultimately Show a larger understanding References to other digital new meme with one another
try to answer is this: what, if of digital culture culture memes/themes and connect the dots between
anything, does a given piece of their client deliverables and
Spawn a new digital meme or Creation of derivative works
user-generated content reveal meaningful recurring themes
activity by others
about digital culture? Is it rep- found within digital culture.
©2010 VivaKi. All Rights Reserved. Page 5

6. Problems with UGC monitoring the UGC monitoring system) was the link back. These sites
The first issue of the Reflecteur publication launched as all linked to each other, and they appeared to the monitoring
scheduled. As the team continued working with the UGC system as blogs. So their rankings relative to the legitimate
monitoring data over the course of the next several months, blogs increased. Over time, the team spent more of its time
however, we began to notice a series of problems. These sorting through the aggregator and spam blogs. It certainly
problems can be classified into two categories: substand- could be done, but the problem diminished the value of the
ard data quality and a disconnect between blog posts that data interface immensely because it reduced the power of
are merely rich in link backs with posts that illustrate some automation.
greater digital culture insight.
A second problem centered on attribution of value. On the
Poor data quality was not a problem that the team initially surface, attributing value based on the number of link backs
anticipated. This particular vendor was well-respected, and seems reasonable. If one post garners more link backs, it
various agencies in the network had commissioned reports is presumably more important and more likely to represent
from it in the past. A key problem with vendor-produced an insight into digital culture. True blog-to-blog link backs
reports, however, is that the underlying data (the “raw” and are not that common, however. Increasingly, people share
“tabbed data” as described earlier) are obfuscated. The raw content they like through Twitter (via any number of URL
and tabbed UGC data tend not to be displayed in a slide pres- shorteners), Facebook, instant messaging, Digg and dozens
entation report to clients. The vendor’s analysts obviously of other social media applications. To be sure, blogs are an
work with it, but the agency or client tends not to experience important part of the user-generated landscape, but to as-
the data firsthand—the reports focus on heavily summarized sign all the value of an individual post based on how many
data, implications of the findings and recommendations for other blogs link to it is problematic. It simply fails to capture
action. This was the first occasion where we experienced the the majority of the social links.
data itself without the perversion of summarization. Even
standard UGC monitoring dashboards tend to conceal the This omission results in problems in the data feed. The blog
underlying data—not as much as a report does, but certainly post ranked first might only have a handful more link backs
more so than we experienced with the data interface for this than the post ranked fifth, or tenth. Because blog link backs
project. are relatively rarer than sharing links in other social media
environments, the distinction between ranked posts is quite
The most disturbing example of this was the sudden visibility negligible. The other problem with attributing value to link
into the effect that aggregator and spam blogs had on the backs is that it ignores the importance of traffic and expo-
results. These computer-generated sites exist to tap into the sure. If a post has ten link backs but only 100 visits/views, it
paid search market. Software is written to automatically copy is probably less important than a post which has three links
entire blog posts and entire blogs, repost them to new sites, but 10,000 views. The team would have ideally incorporated
and repeat the process. Because search engine rankings are exposure metrics to help quantify the value of individual blog
based partly on cross-linking, these counterfeit sites link to posts, but that data simply did not exist—at least in a form
each other, thus creating the appearance of a healthy blog that could be integrated into the reporting system.
community. None of the content is original, the sites do not
have an established audience base, and there is no real rea- In addition to the attribution of value, the type of content
son why a person would want to visit them. But because of proved to be problematic as well. Automated monitoring
the mini networks that the cross-linking creates, these sites systems work great for analyzing text, but for anything
can garner quite attractive search rankings. Such placement multimedia the systems tend to be less-than-capable. So
naturally attracts free traffic, and the sites monetize the much of digital culture is defined by videos, photos, graphics,
unsuspecting visitors through paid search ads on the sites animations and other images (e.g., cartoon drawings). Yet,
as well as banner ads from lower-tier ad networks. Setting the UGC systems can really only handle text well. The rea-
aside the ethical or search marketing problems associated sons for this limitation are obvious—text comprises common
with these sites, this reality did introduce a very considerable symbols easily recognizable by software. But the negative
problem into our UGC data streams. impact to the Reflecteur initiative was unmistakable. In order
to surface multimedia content, it had to originate with some
The UGC monitoring software was frequently unable to text commentary on a blog post. This certainly happens, but
discern if a particular post was original or whether it was a it essentially places an artificial handicap on multimedia con-
result of the automated scripts described above. To a hu- tent. Because this content is so crucial to the fabric of digital
man, the difference is immediately very clear. But because culture, such a limitation became unacceptable.
the software was often confounded by these sites, the data
interface tended to be clogged up with counterfeit sites. After The final problem revolved around user reaction. In our at-
all, the primary measure of value for any given blog post (in tempts to uncover digital culture insights, we often turned to
©2010 VivaKi. All Rights Reserved. Page 6

7. user reactions to get a sense of how people responded to any tors captured tweets, websites, videos, photo galleries, new
particular piece of content. Comments posted to a YouTube platforms and news articles. While many of these items had
video, tweets, comments posted to a message board, etc. all been discussed on blogs, they were often not the items that
are tremendously valuable when trying to discern the insight. popped as the majority of user interaction had happened on
The reporting system accounted for none of these. the photo itself, for example – interactions the UGC monitor-
ing could not capture. The Internet is not as constrained by
traditional demographics – age, race, location, etc. Though
the crowd was almost completely composed of individuals in
Search for alternatives
their 20s and 30s living in the United States, we found that
Due to the problems with the UGC data detailed above, a
individual’s hobbies and interests had a much greater impact
new data collection process was called for. After examin-
on which sub-communities of digital culture were repre-
ing various possibilities we settled on using a crowdsourcing
sented.
approach. Crowdsourcing, which was first coined in Wired
magazine in 2006, delegates a specific task to a group, with
each individual often performing the same general role. It is The one area that remained a bias was around language, as
typically discussed as one route for cost effectively solving the vast majority of contributors spoke English as their first
marketing and business problems. However, in the case of language. However, this too ended up being a much smaller
Reflecteur, crowdsourcing could open up a wider swath of the issue than had been expected, as so much of what becomes
digital world for our examination. Much like the blind men popular within digital culture is visual and therefore able
and the elephant, each individual only sees a small portion of to cross-language boundaries. And we continued to refine
everything created and shared within the digital universe. We our source pool by expanding it even further. One benefit
each have our own concept of what digital culture is. of this extension has been an increase in the diversity of
backgrounds and demographics of our contributors, which is
already leading to higher quality data.
In order to make this process work, a tool was required to
collect the various crowdsourced items. We chose Diigo, a
social bookmarking site, for a number of reasons. First, it Though we were pleased with the quality of data we were
had the ability to create closed groups to share links. Though getting via the crowdsourcing process, it did require some
other sites also have this functionality, Diigo also had a clean changes in our interpretation process. We were no longer
and easy to use interface with an additional bonus of a web given a list of link backs to see (as a proxy) how popular an
browser toolbar plugin that allowed contributors to bookmark item was, as we had been with the UGC data. Instead, this
a site without leaving the page. This toolbar meant that the assessment of popularity and virality was left to the Reflec-
entire contribution process took a total of a few seconds, teur team to determine. In order to do so in a consistent
helping to alleviate any burdens on users and subsequently fashion, our criteria for inclusion in the Reflecteur publication,
increase participation. listed above, became more formalized. Once this process was
set into place we moved fully to the crowdsourced approach.
This shift was actually a six month migration as we tested
The team tested this process along side the UGC listening
and assessed which process worked best for our goals.
system for six months to assess whether the sites we col-
lected as a group were better, worse or the same as the sites
and content the UGC software surfaced. Of particular concern With this shift, a number of incremental benefits to the
was whether or not we’d see a dramatic shift in the types of crowdsourced approach emerged. The first was the unex-
sites and items tagged in the crowdsourcing process. Would pected usefulness of the database of links created within
we find that the crowd was too homogeneous and therefore Diigo. The manner of sharing links – via a simple web
missing important items across the rest of digital culture, browser toolbar – also allowed expanded participation to
invalidating our results? We actually found the opposite. non-researchers within the agency. Moreover, all links were
The crowdsourced data not only tended to overlap much of never deleted from the Diigo group, remaining accessible to
the usable UGC data, but it included items far beyond the all members. The simplicity of this new database of links,
original UGC areas of focus (parenting, consumer technology, compared to the reporting interface of the UGC software, al-
celebrity gossip and timewasters.) lowed this same group to peruse and access the data easily.
This database quickly became a core source for examples
and information for client presentations, pitches and various
The employees of Denuo are a digitally tuned-in group. But
agency projects.
each individual was tuned-in to a different area of digital
culture. Popular items in areas such as food, gaming, fashion
and design were captured within our Diigo group, but not by At the same time, everyone in the agency expanded their
the UGC software due to the required restrictions on subject understanding and awareness of digital culture beyond their
areas. The content formats were also more diverse. While the own experiences. RSS feeds of the link database and e-mail
UGC software could only capture blog posts, our contribu- alerts were used by a majority of contributors to consistently
©2010 VivaKi. All Rights Reserved. Page 7

8. remain aware of what was being talked about and shared Concurrently, Reflecteur’s new role within the wider VivaKi
online—allowing everyone to recalibrate digital efforts in organization required another design change. Though the
real-time. As a side benefit, the process of participating in core elements of the publication – four articles, nine addition-
Reflecteur promoted collaboration and new employee connec- al links and top sites – remained the same, each participating
tions across the entire agency. agency received its own white-labeled version of the publica-
tion. As participants in the data sourcing process, all could
legitimately claim collaboration in the publication’s creation.
Because we did not ask contributors to go outside their usual
They were also all given the opportunity to market their
routine to look for Reflecteur links, there is obvious bias in
agency on the third page of the publication, often showcasing
the group of sourced sites. They are reflective of the group
their top digital work. Finally, they were given free reign to
contributing. We began to combat this bias by inviting every-
share the publication with clients, partners, and anyone else
one within Denuo to participate in the crowdsourcing process.
outside the agency.
By adding these additional perspectives we hoped to piece
together a more complete image of what our elephant—digi-
tal culture—looked like as a whole. Many chose to also post the publication to their intranets and
company blogs. However, it is agency employees who are the
core distribution base for Reflecteur. This focus is due to the
Taken from the perspective of conventional research, whose
fact that it is the agencies that are first and foremost the cli-
foundation is built on the idea of representative sampling,
ents of Reflecteur. It is a tool for them to expand their knowl-
this bias would be glaring, and to some, too subjective. It
edge of and calibrate their digital efforts to what is happening
is important to note, however, that this bias is absolutely
within digital culture. Applied to the larger VivaKi network,
inherent in any crowdsourcing project. The two cannot be
Reflecteur has become a powerful tool for improving overall
separated. Crowdsourcing values expertise and perspective
organizational intelligence within the digital culture space.
over representativeness. Because Reflecteur was designed to
Recently, the initiative has expanded to include offices out-
be a qualitative initiative, the team was comfortable with the
side the United States. The global rollout will further diversify
migration to the new sourcing method and the accompanying
the content as well as the impact of Reflecteur in influencing
trade-offs.
digital marketing.
Migration to VivaKi
At the beginning of 2010 we made the decision to expand
Extending the value of the data
Along with the bi-weekly publications, the team also cre-
Reflecteur beyond Denuo, the original incubator agency. After
ates semi-annual presentations describing the recent trends
successfully using Reflecteur to calibrate this agency’s sen-
within digital culture. For these presentations we re-examine
sitivity to digital culture and refining our new crowdsourcing
everything published in the most recent six months, tak-
process for over 12 months, it was clear that other agencies
ing note of the core insights of each item. After selecting
within the VivaKi network could benefit from participating
the most prevalent, pertinent and recent themes, we craft
in the initiative. We had seen within Denuo that when more
a presentation that articulates each of these themes with
individuals participate, the result is broader visibility into the
a series of supporting examples. Examples of marketers
digital culture landscape and, therefore, better insights. By
aligning with these themes are also included to show the
expanding to all VivaKi agencies, each of which had a vested
“real world” application of digital culture insights. While the
interest in better understanding digital culture, we would be
Reflecteur publication focuses on the timeliest items, these
able to improve the effectiveness of Reflecteur.
presentations allow us to step back and examine a larger
period within digital culture – the point being to find recur-
Over a period of almost six months, we worked with the ring patterns.
leadership at each major agency to identify new contributors,
who were often junior members of the organization, to join
These semi-annual summary presentations do not cover new
the collaboration. These contributors were exposed to a short
content; the material is sourced strictly from past publica-
orientation and overview of Reflecteur. They were then given
tions. But the perspective is somewhat different. The publica-
access to the social bookmarking group and encouraged to
tions are topical; the presentations, on the other hand, are
post links. As expected, actual participation varies dramati-
designed to provide a broader view of the overall digital
cally, with some individuals posting regularly and some very
culture landscape. What we have seen over time is that there
infrequently. Every agency does have a number of active
are consistent, transitory and evolutionary themes within
contributors adding to the Reflecteur data pool every week.
digital culture. All of these are useful for the participating
With this additional participation, it took very little time to
agencies as inspiration for larger approaches and positioning
see an increase in the diversity and quality of links.
within the digital space as well as for short term campaigns
and initiatives.
©2010 VivaKi. All Rights Reserved. Page 8

9. Like the publication, the presentations are shared with the present, this gap is too large. As UGC monitoring technology
employee base at each agency and with those agencies’ progresses, perhaps this gap will narrow, in which case we
clients. But because the two are designed to share different will be willing to re-examine our conclusions.
perspectives on the same topic, they are used differently.
The publication is regularly shared, along with the agencies’
While UGC monitoring can not fulfill a role in the Reflecteur
other publications and thought leadership materials. The
initiative, technology certainly does. Crowdsourcing, the
presentation is shared as well, but usually in a different con-
technique with which we have seen much success, relies on
text. VivaKi agencies use the presentation to inspire brain-
technology platforms to be efficient. The expertise is decid-
storms, kick off working sessions to address key marketing
edly human, but the social bookmarking platform greatly
challenges and as special events to freshen the thinking of
facilitates the collaboration of people who have that exper-
both the agencies and the client teams.
tise. Without the enabling technology, crowdsourcing as an
effective data sourcing method is not possible. An additional
benefit of this method is its ability to quickly and easily
Concluding thoughts incorporate new aspects of digital culture. This adaptability
Reflecteur has now been in existence for three years. In that extends to future platform innovations and will allow us to
time we have changed data sources, enhanced the design of evolve Reflecteur in response to new innovations and facets
the user-facing product and expanded both our contributor of digital culture – we are relying on human collaboration and
pool as well as our distribution. The primary lesson we have expertise and not over-relying on technology to source data.
learned is that, like any culture, digital culture is continually
evolving. The digital experience is increasingly about par-
Through our three years, 70+ issues, 5000+ data items and
ticipation in the form of sharing and creating—hallmarks of
50+ contributors, we have witnessed firsthand that the digi-
social media. Because of this characteristic, people are more
tal space has a distinct culture (and subcultures) all its own.
empowered than ever to not just consume digital culture, but
As an arena of cultural studies, examining and understanding
to also actively contribute to and shape it. This is precisely
digital culture relies on established techniques. We rely on
why the ability to internalize digital culture will be a prereq-
classic ethnographic techniques and the lens of postmodern-
uisite to crafting effective digital marketing initiatives moving
ist cultural theory as our guide to recognizing salient insights.
forward. Any organization focused on digital marketing will
We also rely on the collective wisdom of VivaKi’s people, who
need to create institutional knowledge initiatives in order to
are wired into digital culture because of their personal pas-
build this capability.
sion for it. And finally, we rely on technology to facilitate the
collaboration between these people. This combination, es-
Barker states that “culture is not ‘out there’ waiting to be tablished observational techniques, well-documented theory,
correctly described…rather, the concept of culture is a tool crowdsourcing and a facilitating technology platform, is the
that is of more or less usefulness to us” (2008). A grasp of secret to how VivaKi has been able to create a network-wide
digital culture ultimately helps our constituents—our col- institutional knowledge base of the very important, yet still
leagues in the VivaKi agencies—better understand human emerging topic of digital culture.
motivations, behaviors, desires and attitudes. Reflecteur is
the mechanism that helps us properly use the tool that is
culture.
References
Harvey, David. (1990). The Condition of Postmodernity: An
At the commencement of the Reflecteur initiative, we un- Enquiry into the Origins of Cultural Change. Malden, MA:
derestimated the degree to which automation could play a Blackwell Publishing Ltd.
role in understanding digital culture. Our reliance on UGC
monitoring as the initial data collection mechanism demon-
Kozinets, Robert. (2002). “The Field Behind the Screen:
strated this. While many brands have success incorporating
Using Netography for Marketing Research in Online
UGC monitoring into their business and market intelligence
Communities”. Journal of Marketing Research. Vol 39,
work streams, it simply did not work for this initiative. Our
No.1, pp 61-72.
conclusion is that the subject of digital culture is just not
appropriate for software-generated, automated data col-
lection. Keyword lists, which tend to work well in brand- Barker, Chris. (2008). Cultural Studies. London: Sage
oriented UGC monitoring projects, do not translate to the
topic of culture. Additionally, culture, particularly postmodern Hassan, Ihab. (1985). “The Culture of Postmodernism.”
digital culture, is too nuanced and multifaceted. The level of Theory, Culture and Society. Vol 2, No. 3, pp 119-32.
complexity within the subject matter greatly outpaces the
level of sophistication of the UGC monitoring systems. At the
©2010 VivaKi. All Rights Reserved. Page 9

10. Christian Kugel
As a veteran of Denuo, Millward Brown and Starcom, Christian has developed a unique
and expansive perspective on how people interact with brands, each other and media.
In his current role, Christian is SVP at Publicis Groupe’s VivaKi unit, where he works with
senior leadership in the corporate strategy group. He created Reflecteur, a first-of-its kind
initiative that deconstructs, analyzes and explains digital culture. Prior to VivaKi, Kugel
was a member of the management team in Denuo. There, he led the HP account and
managed Denuo’s proprietary toolset. In 2007, Kugel created Socialight, Denuo’s propri-
etary consulting tool that measures the degree and influence of conversations and recom-
mendations among groups of consumers. Deployed internationally, Socialight has been
used to uncover insights on over 100 brands. Before Denuo, Kugel was an Account Group
Director at Millward Brown, where he managed multi-national research initiatives for
blue chip technology clients. Prior to that, Kugel was Director of Insights & Analytics at
Starcom IP. He was named an ‘Agency Innovator’ by Internationalist. His work with Socia-
light resulted in the ‘Most Innovative Research’ award by WOMMA, and he was nominated
for the prestigious Goodyear Award for best international research.
Ellen Bird
In the past two years, Ellen has established herself as a leading expert in digital culture
within Publicis Groupe’s VivaKi unit. She currently manages Reflecteur, which she trans-
formed from being an internal Denuo initiative into one that reaches all 15,000 VivaKi
employees. Prior to her current role in the VivaKi corporate strategy group, Ellen man-
aged emerging media initiatives at Denuo, such as leading the strategy and activation for
HP’s first formal presence on social networking platforms. She also managed a pioneering
marketing program for HP which brought the power of unbiased, third party product re-
views to shoppers’ hands—via their mobile devices in the store, at the point of purchase.
Prior to Denuo, Ellen worked at Starcom Worldwide, where she managed new media as
well as traditional marketing efforts for blue chip accounts including Bank of America and
Disney Parks. She has been a featured speaker at events for the Word of Mouth Market-
ing Association and Digital Hollywood. Although Ellen is a native of the East Coast, she
graduated with a degree in Sociology from Northwestern University and is currently based
in Chicago.
Authors’ note
Though our goal in creating Reflecteur is to uncover insights about digital culture, we did not focus on that here. The focus of this
paper was on the process of examining digital culture and, more specifically, the benefits and pitfalls of a UGC monitoring versus
a crowdsourcing method. We have collected a number of digital culture insights over the last three years. The bi-weekly Reflect-
eur publication is freely available for download at vivaki.com.
For more information
Contact Ellen Bird at ellen.bird@vivaki.com or Christian Kugel at christian.kugel@vivaki.com
©2010 VivaKi. All Rights Reserved. Page 10