Slides used in course given at CHI 2012. Based on material from our book, Building Successful Online Communities, available at http://www.amazon.com/Building-Successful-Online-Communities-Evidence-Based/dp/0262016575/
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Contributions andstartuptutorialkrautresnick
1. Evidenced-Based Social Design of
Online Communities
Robert E. Kraut
Carnegie Mellon University
Paul Resnick
University of Michigan
http://slidesha.re/KrautResnickCHI12
2. Agenda
• Our approach & nature of design claims (10 minutes)
• The challenge of contribution (90 min)
– Requests, goals & motivation (55 minutes)
– Design challenge (15 minutes)
– Break
– Debrief (20 minutes)
• Starting a community (80 minutes)
– Network externalities & getting to critical mass (45 minutes)
– Design challenge (15 minutes)
– Debrief (20 minutes)
3. Today’s goals
• Introduction to view of social design based on social
science theory and empirical results
• Application to
– Challenges of encouraging contribution to online groups
– Challenges of starting a community from scratch
• Format: Lecture combined with break-out groups for
design exercises
4. Instructors
• Robert Kraut
– Herbert A. Simon Professor of HCI
at Carnegie Mellon
– Social psychologist by training
– 12 years in industry at Bell Labs
and Bellcore
– Emphasis on social computing
• Paul Resnick
– Professor, School of
Information, University of Michigan
– Computer scientist by training
• Economics orientation
– 2 years in industry at AT&T
5.
6. Online communities face challenges
typical of off-line groups
• Community start-up
• Recruit, select and socialize members
• Encourage commitment
• Elicit contribution
• Regulate behavior
• Coordinate activity
But anonymity, weak ties, high turnover, & lack of
institutionalization make challenges more daunting online
7. Evidence-based Social Design
• Mine the rich empirical and theoretical literatures in
psychology and economics
• Develop design claims
– Hypotheses about the effects of social design decisions
• Sometimes directly tested in the online context and
sometimes only extensions of empirically tested
theories developed in offline settings
8. Inspiration
―There is nothing so
practical as a good theory‖
―If you want to understand Kurt Lewin
something, try to change it‖
9. The Roles of Theory and Evidence
• Identify Challenges
• Generate Solution Ideas
• Predict Consequences
10. Design Claims
• Our approach is to translate relevant social science
theory and empirical research to design claims
• Alternative X helps/hinders achievement of goal Y
under conditions Z
• E.g.,
– Coupling goals with specific deadlines leads to increases in
contributions as the deadlines approach
– Group goals elicit contribution most among people who
identify with the group
11. Design Claims Differ from Pattern
Languages
• Design pattern: a formal way of documenting a
solution to a design problem in a particular field of
expertise.
• May or may not document the reasons why a problem
exists and why the solution is a good one
• Captures the common solutions, but not necessarily
the effective ones
12. Design Levers
• Community structure
• Content, tasks & activities
• Selection, sorting & highlighting
• External communication
• Feedback & rewards
• Roles, rules, policies and procedures
• Access controls
• Presentation and framing
13. Morality of Social Design
• Isn't this "social engineering," manipulating people
and undermining their free choice?
14. Morality of Social Design
• Isn't this "social engineering"?
• Isn't it evil to manipulate people and undermine their
free choice?
15. We Don't Think So
• Moral imperative to make communities run well
• All action, even inaction, has an effect
– There is no neutral, non-manipulative configuration of online
communities
• Moral arguments should be about particular
goals, particular means
17. Online Communities Face Challenges
Typical of Off-line Groups
• Community start-up
• Recruit, select and socialize members
• Encourage commitment
• Elicit contribution
• Regulate behavior
• Coordinate activity
But anonymity, weak ties, high turnover, & lack of institutionalization make
challenges more daunting online
18. Reasons To Care
• Overall goal. Creating sufficient volume of contribution of the
resources the group values to provide benefits to group members
and others who rely upon the online community
• Different communities require different types of contribution
– Social support forums: Conversational acts, empathy, offers of help
– Recommender systems: Votes, opinions, comments
– Facebook: Invites, accepts, wall posts, pictures
– WoW guild: Time, particular skills
– Threadless: T-shirt designs
– OSS: Patches, code, translations, documentation
– Wikipedia: New articles, facts, copy-editing, administration
work, cash (& recently, letters to congressmen)
19. Under Contribution Is Rampant
• Across many Internet domains, a small fraction of
participants contribute the majority of material
– Code in open source projects
– Edits in Wikipedia
– Illegal music in Gnutella
– Answers in technical support groups
• Often leads to a power-law/Zipf curve distribution
• In many cases uneven contribution leads to an under
supply of needed content. E.g.,
– Assessments and content in Wikipedia
– Reviews of art movies in MovieLens
20. Wikipedia Stubs & Unassessed Articles
• Many Wikipedia articles haven‘t been assessed for
quality or importance
• 58% of important ones are of low quality
22. APS/WI Reviewing Goal
• Subgoal: Get psychologists & grad students to review
Wikipedia articles by adding comments to article talk
pages describing problems with an article
• ~300 have signed up for the APSWPI, improving >
700 articles
• But fewer than 15 have reviewed
• How you can apply any of the design claims
presented here to increase these reviews from APS
members?
24. Second attempt
• Simplifying the task
– Direct link to where the action is needed
• Highlighting ―social identity‖ in the invitation message
• Personalizing the message
– Specifying users‘ expertise
• Phrasing the task as ―rating‖ instead of ―reviewing‖
26. Naïve Task Analysis of Online Contribution
To get people to contributed needed content :
1. They need to understand what is wanted theories
of persuasion and goal setting
2. They have to be motivated to provide it theories
of motivation
1. Extrinsic motivations
2. Intrinsic motivations
3. How social situations influence motivation
3. They have to be competent to provide it
28. Requests Focus Attention on Needed
Contributions
• Make the list of needed contributions easily visible to
increase the likelihood that the community will provide
them
29. Identify Who Should Make The Contribution
• Request help in a chat room
• ―Can you tell me how to see someone‘s profile‖
– 400 Chat rooms
– DV=Time to response
• People are slower to respond when others are present
• Diffusion of responsibility is reduced when people are called by name
80 80
No name Name
70 70
Time to respond (seconds)
Time to respond (seconds)
60 No name
60
50 50
40 40
30 30
20 20
10
10
0
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Others present
Others present
Markey(2000)
30. Email Request to Contribute to Movielens
Quadruples Ratings
• In week after email reminder, contributes quadrupled, to ~ 20 ratings/person from
~5.4
• Is this sustainable?
31. Ask: Explicitly Asking for Needed Contributions
Increases Likelihood of Getting Them
• News site with a ―Leave a
comment‖ form at the end of
each article
• Fewer than 0.1% leave Comments by Type of Request
comments
No ask
• Experiment to estimate the Immediate
value of explicit requests Delayed
– No ask: ―Leave a comment‖ 0 20 40 60 80
Number of comments
100 120
form at end of article
(Walsh & Lampe, 2012)
– Immediate: Pop-up ―Leave
a comment‖ when user
opens article
– Delayed: Pop-up ―Leave a
comment‖ on closing article
32. Ask Someone Who Is Willing & Able to
Help: Intelligent Task Routing (Cosley, 2007)
35. Goal Setting Theory
• Goals motivate effort, perseverance & performance
– Trigger for both self-reward (e.g., self-efficacy) & external reward
(e.g., money, reputation, promotion)
• Goals are more effective if
– Specific & challenging rather than easy goals or vague ‗do your
best‘
– Immediate, with feedback
– People commit selves to the goals – because of
importance, incentives, self-esteem, …
– People envision the specific circumstance & method they will
use to achieve them
• Design claim: Providing members with specific and
highly challenging goals, whether self-set or system-
suggested, increases contribution.
36. Experiment Showing that Goals
Work:
• Send email to ~900 MovieLens subscribers
– Gave non-specific, do your best goal or specific, numerical
contribution goals
– Assigned goal to individual subscribers or a nominal group
of 10 subscribers (the ―Explorers‖)
38. In-game Goals in WoW
Weekly minutes playing World of Warcraft, by level
• In WoW players receive extra powers each 10-
levels implicit goals setting
• Ducheneaut, N., et al.(2007). The life and death of online gaming communities: A look
at guilds in world of warcraft. in SIGCHI conference on Human factors in computing
systems. San Jose, California, USA.
39. Featured Status in Wikipedia as a
Challenge
Wikipedia edits before and after reaching featured status
40. Design Claims Re: Goals
• Providing members with specific and highly
challenging goals will increase their contributions.
• Goals have larger effects when people receive
frequent feedback about their performance with
respect to the goals.
• Externally imposed goals can be as effective as self-
imposed ones, as long as the goals are important to
community members
• Time-delimited challenges enhance the effects of
goals
• Combining goals with appeals to social identity
enhances their effects
42. What Motivates Contributors?
• External personal value
– Reinforcement
– Pay
– Privilege
…
• Social utility
– Reputation
– Identification with the group
– Reciprocity
– Altruism
…
• Intrinsic value of task
(e.g., fun, curiosity, challenge)
These are leverage points for interventions to
increase motivation
43. Value-Expectancy Model Provides Leverage
Points for Reducing Social Loafing
3
individual individual
performance outcome
5
3, 4
individual 4 individual individual
effort utility motivation
6 6
group group
performance outcome
44. Value-Expectancy Model Provides Leverage
Points for Reducing Social Loafing
Frame request consistent with users’
values
Create incentives user values
3 Extrinsic
individual individual Intrinsic
performance outcome
5
3, 4
individual 4 individual individual
effort utility motivation
6 6
group group
performance outcome Liking for group members
Identification with group
History of interaction with group
Number of others
Own competence
Own unique skills
Group’s
incompetence Identifiably
Fairness of reward distribution
45. People Differ in Motivations to Volunteer
• Design claims: Matching experiences with motivations increases
– Recruiting
– Retention
Clary, E. and Snyder, M., (1999). The motivations to volunteer: Theoretical and
practical considerations. Current Directions in Psychological Science. p. 156-159.
46. Leverage the Variety of Motivations
• Don‘t assume everyone has the same motivation
Frame the request to match individual motivations
• E.g., Ads for recycling site for those with altruistic
versus financial motivations
Altruistic framing for Monetary framing for
less financially motivated more financially motivated
47. Assess Motivation from Sending Site
Items Donated by Framing and Source
3.5 Altrustic frame Financial frame
3.0
Items Donated
2.5
2.0
1.5
1.0
0.5
0.0
Craig's List Mturk
Source of participants
48. WikiProjects Use Collaborations of the Week
(COTW) as Time-Delimited Goals
Get designated to good status in a defined
period (e.g., a week or a month)
A COTW announcement in a project page
An example template identifying an article as a COTW
52
49. Goal doubles contribution
Edits per person on the
collaboration articles Self-identified group members
Non self-identified members
Pre-Collaboration Collaboration Post-Collaboration
53
50. Goal has much larger effect on group
members
Edits per person on the
collaboration articles Self-identified group members
Non self-identified members
Pre-Collaboration Collaboration Post-Collaboration
54
51. Goals and Identity
• Design claim: Goals have a more powerful effects
when achieving them benefits a group the target
identifies with
• Association for Psychological Science Wikipedia
Initiative appeals to PhD psychologists with this
technique
– Students motivated via extrinsic incentives (grades)
53. Extrinsic and Intrinsic Motivations
• Individual motivation influences behavior through external
motivators (e.g., rewards, incentives, reputation) and intrinsic
motivators (e.g., fun & curiosity)
Increase contributions by manipulating extrinsic incentives & intrinsic
motivations
– Extrinsic motivators: Offer rewards as incentive
(e.g., money, reputation, perks, grades)
• Larger rewards induce more contribution than smaller rewards.
• Luxury goods create better incentives than money as rewards for
more difficult tasks.
• Rewards of status, privileges, money, or prizes that are task-
contingent but not performance-contingent will lead to gaming by
performing the tasks with low effort.
• People won't game the system for private verbal reward
– Intrinsic motivators: Make the task fun or intrinsically interesting
57. Incentives vs. Reinforcements
• Incentives are promises given before the behavior
to cause people to produce it
• Reinforcements are rewards given after a
behavior that make it persist
60. Design Claims Re: Incentives &
Reinforcement
• Incentive Effects
– People do more of the behaviors that they anticipate will be
rewarded.
– Task non-contingent rewards will not create incentive to do more of
a task or exert more effort in doing it
– Larger rewards induce more contribution than smaller rewards
– Small gifts create more effective incentives than small payments
• Reinforcement effects
– Rewards delivered in response to behaviors cause people to do
more of those behaviors
– Rewards work better as reinforces if they are delivered right after
the desired behavior
– Rewards generate more consistent performance over time if they
are unpredictable
61. Intrinsic Motivators
• Intrinsic motivation is the process of working to
achieve the rewards that that come from carrying out
an activity rather from as a result of the activity.
• Comes from the pleasure one gets from the task itself
or from the sense of satisfaction in completing or
working on a task.
Redesign the task to make it more fun or interesting
62. ESP Game To Label Images
Truck
Red school bus
Red
Red school bus
• Example of playing the game
• Taboo words
64. What Makes a Contribution Fun?
Lessons from game design
Flow Criteria Principles of game design
Concentration Games should require concentration and the player should be able to concentrate
on the game
Challenge Be sufficiently challenging and match the player’s skill level
Skills Support player skill development and mastery
Control Support players sense of control over their actions
Clear Goals Provide the player with clear goals at appropriate time
Feedback Provide appropriate feedback at appropriate times
Immersion Players should experience deep but effortless involvement in the game
Social Interaction Games should support and create opportunities for social interaction
Mapping flow to principles of game design (from Sweetser & Wyeth, 2005)
66. Gamification
• Applying game-design thinking to non-game
applications
• Is the effect via fun (internal motivation) or
incentives (external motivations)?
67. Design Claims Re: Trade-offs Btw Intrinsic
& Extrinsic Motivation
• Adding a reward to an already interesting task will
cause people to be less interested in the task and to
perform it less often.
• While tangible rewards reduce intrinsic motivations for
interesting activities, verbal rewards enhance intrinsic
motivation.
• Verbal rewards will not enhance intrinsic motivation
and may undermine it while they are judged as
controlling.
• Verbal rewards enhance intrinsic motivations most
when they enhance the target‘s perceptions of
competence
68. Design Jam
• Groups of 4
• Task
– Redesign of one thing of
• request for review,
• reviewing page on APS/WI site
– Sample Interface for reviewing page:
http://hciresearch2.hcii.cs.cmu.edu/~rfarzan/wikipedia/tool/re
view/review.php?&cond=1 (also on next slide)
– Say which slides justify your proposal.
– Mockup your proposal.
– 15 minutes for Jam
– Readout after the break
• Show your mockup and narrate it: 1 minute!
72. Externalities
• Alice's adoption or production decisions have direct
or indirect effects on Bob's adoption or production
decisions
• Alice's decisions create costs and benefits external to her
• e.g., Size of telephone, email, and fax networks
• e.g., Complementary products—hardware & software
• e.g., Second hand smoke & other pollution
73. Positive Externalities in Napster
• Probability of a song appearing on Napster increased
with the number of users, at a declining rate
74. Negative Externalities in Napster
• Measures of congestion in Napster increased with the
number of users, at an increasing rate
75. Network Externalities in OLCs
• Negative externalities
– Server congestion; competition for attention
• Positive externalities
– People to interact with
– Content they produce
– Identity value
76. Implications of Positive Network
Externalities
• Winner-take-all competition between networks
• Need for critical mass
– minimum number of users that makes others want to join (or
not quit)
79. Getting to Critical Mass
• Leveraging Early Members
• Attracting Early Members
80. Getting to Critical Mass
• Leveraging Early Members
• Attracting Early Members
81. Join Now or Wait?
• Early members especially important
• Model provides insights into how to attract
utility (join now) =
participation_benefitstage1
- startup_cost
+ success_probability * (participation_benefitstage2 + early_adopter_benefit)
utility(wait) =
success_probability * (participation_benefitstage2 - startup_cost)
• Join now if utility(join_now) > utility(wait)
utility(join now) - utility(wait) =
participation_benefitstage1
– startup_cost * (1 – success_probability)
+ early_adopter_benefit * success_probability
82. Implications: Where to Look for Solutions
• Increase immediate benefits
util(join now) - util(wait) =
participation_benefitstage1
– startup_cost
* (1 – success_probability)
+ (early_adopter_benefit *
success_probability)
83. Implications: Where to Look for Solutions
• Increase immediate benefits
• Reduce effort to join
util(join now) - util(wait) =
participation_benefitstage1
– startup_cost
* (1 – success_probability)
+ (early_adopter_benefit *
success_probability)
84. Implications: Where to Look for Solutions
• Increase immediate benefits
• Reduce effort to join
• Promise future benefits to util(join now) - util(wait) =
early adopters participation_benefitstage1
– startup_cost
* (1 – success_probability)
+ (early_adopter_benefit *
success_probability)
85. Implications: Where to Look for Solutions
• Increase immediate benefits
• Reduce effort to join
• Promise future benefits to util(join now) - util(wait) =
early adopters participation_benefitstage1
– startup_cost
• Set expectations: probability * (1 – success_probability)
of success + (early_adopter_benefit *
success_probability)
86. Implications: Where to Look for Solutions
• Increase immediate benefits
• Reduce effort to join
• Promise future benefits to util(join now) - util(wait) =
early adopters participation_benefitstage1
– startup_cost
• Set expectations: probability * (1 – success_probability)
of success + (early_adopter_benefit *
success_probability)
• Conditional commitments
87. Implications: Where to Look for Solutions
• Increase immediate benefits
• Reduce effort to join
• Promise future benefits to util(join now) - util(wait) =
early adopters participation_benefitstage1
– startup_cost
• Set expectations: probability * (1 – success_probability)
of success + (early_adopter_benefit *
success_probability)
• Conditional commitments
• What's not worth focusing
on
– Expectation setting: stage 2
value if successful
88. Implications: Where to Look for Solutions
• Increase immediate
benefits
• Reduce effort to join util(join now) - util(wait) =
• Promise future benefits to participation_benefitstage1
– startup_cost
early adopters * (1 – success_probability)
• Set expectations: + (early_adopter_benefit *
success_probability)
probability of success
• Conditional commitments
• What's not worth focusing
on
– Expectation setting: stage 2
value if successful
89. Increase Immediate Benefits
• DC25: Productivity, Entertainment or Commerce
– E.g., Flickr offers picture storage and
management, services that are useful to the user even
if nobody else is using Flickr.
• DC26: Professionaly generated content
• DC27: Syndicated content
• DC28: Professional staff contributions
90. 45 300
Staff posts
40
Member posts
250
35 Active members
Current member count
30 Key Dates 200
Messages posted
25
150
20
15 100
10
50
5
0 0
Contest 1
Contest 2
Contest 5
Fishtanks
Contest 6
Feb 15
Thread 1
Thread 2
Contests 3, 4
Resnick, Paul, Janney, Adrienne, Buis, Lorriane R, and Caroline R Richardson, ―Adding
an online community to an Internet-mediated walking program. Part 2: Strategies for
encouraging community participation‖. Journal of Medical Internet Research. 2010.
12(4):e72.
91. Increase Immediate Benefits
• DC25: Productivity, Entertainment or Commerce
• DC26: Professionally generated content
• DC27: Syndicated content
• DC28: Professional Staff contributions
• DC30: (as a last resort)
• DC31: Bots
92. Promise future benefits to early adopters
• DC 32: Future discounts to the early adopters
– E.g., lower rates for life
• DC 35: Limited resources that tempt users to join early
– E.g., status & recognition with being an early adopter
– E.g., users sign up first to claim their username
• DC 36: Privileges
– E.g., administrator status
• Identity rewards
– "Won't you be proud that you helped this get off the ground?"
93. Expectation Setting:
Presenting Success at Different Stages
• DC43: Small and slow growing
– Display new members and content
• DC44: Small and fast growing
– Display percentage growth
• DC45: Big
– Display absolute numbers
96. Summary
• The Challenges
– Identifying a Niche
– Defending the Niche
– Getting to Critical Mass
• Leveraging Early Members
• Attracting Early Members
97. Attracting Early Members
• Increase immediate
benefits
• Reduce effort to join
• Promise future benefits util(join now) - util(wait) =
to early adopters participation_benefitstage1
– startup_cost *
• Set expectations: (1 – success_probability)
probability of success + (early_adopter_benefit *
• Conditional success_probability)
commitments
• What's not worth focusing
on
– Expectation setting: stage 2
value if successful
98. Challenge: Make Design Suggestions for
Getting a Specific Community to Critical
Mass
• SuccessfulOnlineCom • Increase immediate
munities.com benefits
• Or a community that • Promise future
someone in your benefits to early
group is trying to adopters
launch • Set expectations:
probability of
success
• Conditional
commitments
100. More information
Robert Kraut Paul Resnick
robert.kraut@cmu.edu presnick@umich.edu
www.cs.cmu.edu/~kraut presnick.people.si.umich.edu
Editor's Notes
In the first attempt, the link in the email was to the “Find Article” page and they had to first search for an article that would interest them with no pre-selection of relevant articles and then they had to click on “Review Now” button to view the review form and submit their reviews. Here is an example message we sent:Dear Hetz3486,The APS Wikipedia initiative has attracted a lot of enthusiasm so far, with more than 300 psychologists editing over 1000 Wikipedia articles. Psychology articles on Wikipedia are important, the typical article was visited more than 10,000 times over last six months. To ensure that the public will see high quality information, it is important for expert psychologists to evaluate the degree to which Wikipedia articles are comprehensive, accurate, well-written, and unbiased. You can help by assessing an article in your area of expertise and answering the following questions about it:Is the article accurate?Does this article provide enough detail about its topic?Does this article represent recent research in this area?Does this article provide adequate references to key research in the field?Is this article well-written and well structured?To submit your assessment of the article:Use the Find Articles section to browse articles in your area of interest or search for articles by keyword.Click on an article title to see its preview and information about its current status.Click on Review now button on the right side of the page to open the reviewing or form and enter the responses to the above questions in the comment box.
This is an example template on article’s talk page, identifying this particular article as the COTW target
During collaboration periods, non-self-identifiededitors increased their contributions.
The interaction effect is highly significant. p < 0.001Support our hypothesis that self-identified group members will voluntarily follow group directions and perform goal-related tasks.