Community Engagement. Innovation Management. Voice of the Customer. Crowdsourcing. Whatever you want to call it, communities are forming around companies, ideas, projects, start- ups, nonprofits, and people. These dialogues are happening organically and they’re happening online, which means that the transparency associated with these conversations presents a large variety of challenges and opportunities. Download this white paper to learn the importance of defining what success looks like when it comes to tracking: Impact Reach Engagement Satisfaction Costs of Implementation

1.
BEFORE YOU BEGIN
CROWDSOURCING BEST PRACTICES
JESSICA DAY
IDEASCALE

2.
There Is No ROI on Understanding:
Crowdsourcing Best Prac:ces
5 Metrics to Develop and Consider
Impact
Example Crowdsourcing Community Goals:
Reach
Engagement
Sa:sfac:on
Costs of Implementa:on
IdeaScale and Innova:on
3
3
4
4
5
6
7
8
Before You Begin

3.
5 Metrics to Develop and Consider
Community Engagement. Innovation Management. Voice of the Customer. Crowdsourcing.
Whatever you want to call it, communities are forming around companies, ideas, projects, start-
ups, nonprofits, and people. These dialogues are happening organically and they’re
happening online, which means that the transparency associated with these conversations
presents a large variety of challenges and opportunities.
The title of this piece is “There is No ROI on Understanding.” Which is not to say that
community managers and innovation experts shouldn’t try and understand their network, they
should. It’s just that for any of these investments to truly pay off, one must respond to those
insights. One must take action. Before they begin and in seeing innovation all the way through
the idea lifecycle.
Industry leaders are finding ways to involve themselves in these discussions and learn from
them. Coca-Cola used co-creation to generate a series of ads from their customers and
enthusiasts. Companies like Avid ProTools are improving their product based off of feature
requests in their customer lab. The new app called Waze uses information from the crowd to
generate real-time traffic reports and information. These are, of course, in addition to whole
host of other businesses large and small that are implementing programs that invite discussion,
ideas, and innovation.
Every conversation and community is different, however, and knowing where to begin is
oftentimes a daunting process. There are a few important metrics to define early on, however,
and they are:
• Impact
• Reach
• Engagement
• Satisfaction
• Costs of Implementation
There Is No ROI on Understanding:
Crowdsourcing Best Practices
3BEFORE YOU BEGIN

4.
“How do we find the one out of 1,000 ideas that is really
revolutionary unless we go through the torture of listening to the
4
Impact
In Romi Mahajan’s book Cool is For Fools, he introduced a character (and problem) that most
people are familiar with:
We all have that friend, peer, boss, employee, associate – you know, the guy who always knows
how everyone should do everything, how to run every company and every country, the space
program, and so on. He’s worse than an armchair quarterback, and even as a Monday morning
quarterback, he often loses the game […] Okay, fine, you get it. Arrogant pricks, right?
Yeah, well, sure. But here’s the deal: How do we find the one out of 1,000 ideas that is really
revolutionary unless we go through the torture of listening to the other 999?1
The most successful communities are places where great ideas and half-­‐baked ideas
germinate happily together and not every great idea arrives on schedule. So how do
businesses identify the right ideas? How do they find the great ideas and separate them from
the good ones?
The most successful companies that implement these programs are those that define their goals
early on. Some even organize challenges or campaigns based on those needs, but even those who
leave the conversation open need to know what they are looking for. What are the goals?
Example Crowdsourcing Community Goals:
• Cost Savings
• Revenue Increase
• Improving Company Offering
• Create Relevant Marketing Content
• Reduce Environmental Impact
• Improve Regulation Adherence
*Note: none of these goals is simply “customer/employee/stakeholder understanding.” All of these goals demonstrate
a response to improved audience understanding.
BEFORE YOU BEGIN

5.
“When members are rewarded for the impact or discussion or
collaboration that their work generates, idea quality improves by over
5
Every business is different. Every community presents a new field of opportunities. Defining
goals early on will help in the assessment process later.
One IdeaScale customer launched a feedback community to its restaurant employees in hopes
of identifying possible cost-savings solutions. Ideas were rewarded based off of their ability to
help meet this goal. One employee made a simple suggestion of replacing ketchup bottles with
2 oz. ramekins filled with ketchup instead. The idea was able to save the company $200,000/
year. A value that was easy to conclude, because the company knew what it was looking
for. The employee was rewarded and the company saved thousands. And that was just one idea.
IdeaScale also offers an assessment tool that makes it possible for clients to input their goal
bottom-lines and evaluate ideas against those goals based on projection.
Reach
When curating and considering a community, it is important to think of the invisible audience.
“a Facebook analysis of top 100 brand pages reveals that for every fan there are an additional 34
friends of fans that can be reached. This multiplier was found to grow even larger when one looked
beyond the top 100 brands.”2
This means that when asking a question, setting a challenge, seeking feedback by casting for it
out into the lake of ideas, that it’s not just about the voices that you’re hearing, you’re still in
conversation with and making an impression on those that are watching and not
participating. It’s like going to a party and telling a story in a large group of ten people. Maybe
only two other people dominate the discussion, but everyone goes home and talks about what
the party was like.
It also means that every person is potentially 34 more people that could have a solution. Bring
in a large community. It’s larger than you know.
The government issues 8,000 regulations every year and independent government agencies
like the FCC operate under the mandate to involve public opinion in any rule-making (so that
the government operates democratically and to improve law adoption when the time comes).
When the FCC used the IdeaScale tool to get feedback on the National Broadband plan, there
BEFORE YOU BEGIN

6.
Engagement
Innovation programs and engagement initiatives are communities. There should be an
opportunity for not just idea suggestions and feedback, but dialogue between members. This
means, that you’re not just looking for a community that speaks to you, but speaks to each
other.
Many community engagement programs offer incentives to contributors, but Professor Olivier
Toubia found that the best way to not only improve engagement, but also the quality of the idea,
was to offer rewards to members who generated the most participation with their ideas. He
called this model the impact model.
When members are rewarded for the impact or discussion or collaboration that their work
generates, idea quality improves by over 40%.3
IdeaScale is a system designed
around community and dialogue
(with the most relevant
conversation boiling to the top of
the most relevant ideas), but
members are also rewarded for
their participation in the IdeaScale
gamified badge system. There’s
even a badge that is automatically
awarded to the member whose
idea generates the most activity.
were over 60,000 responses from the public, (the highest number of Public Notices ever
published). As of the writing of this paper, the plan was 88% implemented, a nearly record-
breaking pace of adoption for any national plan, perhaps helped along by the voices of both
active and passive members that participated in the dialogue.
Finally, reach isn’t just about reaching everyone. It’s about reaching an audience that’s relevant
to you. If you reach out to your target audience (say: female entrepreneurs between the ages of
25 -50), it is more likely that their networks will include similar audience members rather just
putting out a blanket call for engagement to anyone.
6
Innovation programs and engagement
initiatives are communities.
BEFORE YOU BEGIN

7.
7
Satisfaction
One of the side benefits of engaging a group of stakeholders (beyond new ideas and fresh
insights) is that the more a group feels heard, the more likely they are to feel good about the
organization that’s doing the listening.
For example, only one of five employees feel engaged on the job. And, according to Gallup research,
engaged employees recommend their company’s products and services 78% of the time.4
The same goes for customers or other public communities, the more involved they feel in the
process, the more likely they are to feel loyal to the brand and glad to recommend it.
In an engagement scenario, people are more likely to feel heard and incorporated when there is
good moderation. As the moderator promotes ideas, rewards ideas, reviews ideas, and completes
ideas, all of these actions should be celebrated and shared not just for the sake of transparency,
but to show the efficacy of participation. When creating a community, one cannot
underestimate the value of a good moderator: reward them for actively curating a community.
One researcher concluded that using the IdeaScale tool to collect public feedback nearly
doubled customer service sa_sfac_on.5 Satisfied customers mean more recommendation and
the growth of a business, even as you
learn and grow from those customer
insights. Make sure that your
moderators are making your
community members feel heard.
“Engaged employees recommend their companies products and
services 78% of the time.”
BEFORE YOU BEGIN

8.
8
It’s easy to get excited about a great idea. But it’s not always easy to deliver on a great idea.
One of the most popular examples of crowdsourcing innovation is the Netflix prize in which
Netflix asked the community at large to generate an algorithm that would improve the Netflix
recommendation engine’s accuracy by 10% in exchange for a $1 Million purse. A few years later, a
team met that goal and happily walked away with their check for $1 Million and a sense of a job
well done.
But the story doesn’t end there, because it turns out that Netflix never implemented the final
solution for a number of reasons, stating: “we evaluated some of the new methods offline but the
additional accuracy gains that we measured did not seem to justify the engineering effort needed
to bring them into a production environment.”6
The market changed for Netflix and so did their priorities and goals and even though the
algorithm they purchased was great, they couldn’t make the commitment to implement it in the
long run. It is important when selecting and assessing ideas to consider your bottom lines, to
understand what it will take to meet the practical applications of innovation and see them though.
Conversely, the Veteran’s Association launched their IdeaScale community knowing exactly how
much money they had to spend on implementation. Ideas that didn’t fit the scope of the budget
had to be shelved for another time. The ideas were filtered from the top 125 to 32 finalists and
then the 32 finalists were evaluated against what was feasible.
You know your business best. You know what you can afford and what innovations are worth the
cost of investment.
Costs of Implementation
Businesses that use an online tool capable of not just prioritizing ideas and inspiration, but also
generating a community around those ideas are the ones that are leading their industry. They are
considering engagement, their brand reach, what the costs of innovation are and what the gains
of innovation are.
IdeaScale has been providing a flexible platform for numerous businesses since 2008. From the
beginning the guiding principles have been those of engagement and collabora_on. It’s not just a
list of ideas, it’s a community around those ideas. These are just a few of the metrics to consider
as you help build your community.
IdeaScale and Innovation
BEFORE YOU BEGIN

9. FOR MORE INFORMATION
sales@ideascale.com
Global / Americas
+1 800-­‐549-­‐9198
New Zealand
+64-­‐080-­‐099-­‐5088
Australia
+61-­‐02-­‐9037-­‐8414
United Kingdom
+44-­‐0-­‐808-­‐189-­‐1476
1. Mahajan, Romi. Cool Is For Fools: The Poetry of
Marketing. Ascentium Corporation, 2011, p. 86.
2. Swati. "To Expand Your Brand’s Reach on Facebook,
Focus on the Friends of Fans." Web log post. BuzzOm.
28 July 2011. Web. 01 Feb. 2012. http://
www.buzzom.com/2011/07/to-expand-your-brands-
reach-on-facebook-focus-on-the-friends-of-fans/
3. Toubia, Olivier. "Idea Generation, Creativity, and
Incentives." Marketing Science 25, no. 5 (2006):
411-25
4. Lauper, Elizabeth. “Social Knows: Employee
9
Engagement Statistics.” Social Workplace. 8 August
2011. http://www.thesocialworkplace.com/
2011/08/08/social-knows-employee-engagement-
statistics-august-2011-edition/
5. Newell, Angela. “Open Government is Dead, Long
Live Open Data.” GovLoop. 30 June 2011. http://
www.govloop.com/profiles/blogs/open-government-is-
dead-long
6. Amatriain, Xavier. “Netflix Recommendations:
Beyond the 5 stars.” The Netflix Tech Blog, 6 April
2012. http://techblog.netflix.com/2012/04/netflix-
recommendations-beyond-5-stars.html
BEFORE YOU BEGIN