Based on a review of the most successful international crowdsourcing projects, this talk will look at the attributes of successful crowdsourcing projects in cultural heritage, including interface and interaction design, participation in community discussion, and understanding participant motivations.
Public Lecture: "Designing Heritage Crowdsourcing Projects" at the Friedrich-Meinecke-Institute of the Free University of Berlin on 7 December 2015, 6 pm
1. Designing Successful Heritage
Crowdsourcing Projects
Mia Ridge, @mia_out
Digital Curator, British Library
Friedrich-Meinecke-Institute, Freie Universität Berlin, December 2015
8. What is crowdsourcing?
Crowdsourcing (Jeff Howe and Mark Robinson,
Wired, 2006): 'taking a function once
performed by employees and outsourcing it to
an undefined (and generally large) network of
people in the form of an open call'
9. What is crowdsourcing?
Crowdsourcing (Jeff Howe and Mark Robinson,
Wired, 2006): 'taking a function once
performed by employees and outsourcing it to
an undefined (and generally large) network of
people in the form of an open call'
Or, as Clay Shirky's cognitive surplus, 'the spare
processing power of millions of human brains'
11. Crowdsourcing in cultural heritage
Asking the public to help with tasks that
contribute to a shared, significant goal or
research interest related to cultural heritage
collections or knowledge.
12. Crowdsourcing in cultural heritage
Asking the public to help with tasks that
contribute to a shared, significant goal or
research interest related to cultural heritage
collections or knowledge.
The activities and/or goals should be inherently
rewarding.
41. Who participates in crowdsourcing?
• People who are passionate about your subject
/ people who like doing the task you're
offering
• People who can't volunteer in regular hours or
at your venues
• Super-volunteers and passers-by
• Amateurs, professionals, 'pro-ams'
42. Motivations for participation
• Altruistic
– helping to provide an accurate record of local
history
• Intrinsic
– reading 18thC handwriting is an enjoyable puzzle
or they're interested in the subject
• Extrinsic
– an academic collecting a quote from a primary
source
43. Intrinsic motivations for participation
• fun
• the pleasure in doing
hobbies
• the enjoyment in
learning
• mastering new skills,
practicing existing skills
• recognition
• community
• passion for the subject
State Library of Queensland, Australia
https://secure.flickr.com/photos/statelibraryqueensland/3198305152/
44. Motivations as design guide
People crave:
• satisfying work to do
• the experience of
being good at
something
• time spent with people
we like
• the chance to be a part
of something bigger
(Jane McGonigal, 2009)
48. Link text to motivations
'With a few keystrokes, you could bring a family
together'
'We know the names of these children; can you
help us tell their stories?'
'Kill Time. Make History.'
'Historians need your help!'
61. Project design
• Plan to store and process results from
crowdsourcing
• Plan to measure success
• Plan to contribute to your engagement
strategy and digitisation goals
• Understand the appetite for risk
• Reality check your plans
64. ‘no plan survives contact with the crowd’
With apologies to Helmuth von Moltke the Elder
65. Going off-piste
'...None of the above is ready to send to you as I
have more research to do'
https://www.flickr.com/photos/swedish_heritage_board/10207262464
67. Planning a graceful exit
https://www.flickr.com/photos/fylkesarkiv/4545543824
68. Thank you!
Questions?
Mia Ridge @mia_out
Digital Curator, British Library
Friedrich-Meinecke-Institute, Freie Universität Berlin, December 2015
Editor's Notes
Thank you for the invitation to speak.
Abstract: Based on a review of the most successful international crowdsourcing projects, this talk will look at the attributes of successful crowdsourcing projects in cultural heritage, including interface and interaction design, participation in community discussion, and understanding participant motivations.
Start with definitions; then show some key examples and discuss what can be learnt from them to apply when designing other projects. We’ll also look at the role of the participant community and the importance of using participant motivations as design guidance.
Before we get going...
Why am I here talking about this? Worked in museums for a long time, got interested in opportunity between public engagement and need to enhance collections. Made games c2010, then did PhD in digital history, including crowdsourcing as a stepping stone to engagement with the practices and skills of history. Edited a book.
Also some experience working with participatory WWI projects. In many ways did the opposite of the advice I will give you today, but in some ways that's how I know the advice is good. My advice - don't have a series of complicated, specialist tasks on a site with syntax that baffles even the reasonably tech-proficient.
Still, it matched what a tiny number of people wanted to do, so it did achieve some success in gathering structured data about allied WWI military units and collecting personal accounts of the war held in public collections.
Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
Where it came from... Open call to unknown people sometimes scares people in cultural heritage. Problematic coining - not usually a crowd or outsourcing - but a bit late now. I've given two other ways of looking at it, both closer to hobbies, leisure activities.
My definition – partly proscriptive as well as descriptive.
Tasks like tagging, collecting, transcribing, describing cultural heritage collections, undertaken by distributed, possibly anonymous participants. Participation possibly as altrustic or intrinsically-motivated 'volunteers', or as side-effect of playing games or own work on cultural / historical materials.
No financial rewards so has to be rewarding.
Those terms 'powerful purpose' and 'enjoyable tasks' are key and why I'll talk about design and participant motivations today.
GLAMs are galleries, libraries, archives, museums.
Manually enhancing collections records is expensive and time-consuming. Very few orgs have the resources for straight digitisation.
Even when metadata or information records are created by professional cataloguers, the content is often designed for internal or specialist users and doesn't use the everyday language our audiences might use to find material.
There's a lot of specialist expertise outside the museum. There's an online community for almost every topic or type of item under the sun.
Well-designed projects can help people discover new interests, or just encourage them to have a brief moment of deeper engagement with cultural heritage
If I'm talking about 'successful projects', what does that mean?
The number of tasks completed. To achieve this, must also have succeeded in letting people know about your project and providing an interface that lets enough of them get started on their first tasks.
You can also look at the number or type of people engaged in the tasks. This is often important for organisations whose mission is to reach the public, whether that's to give them an experience of contemporary science or access to their history through specific collections. The Zooniverse projects have reached well over a million people worldwide. Some of scientific projects might also look at the impact of publications on social media and in journals that result from their projects. Museums might look at the number of researchers who find their digitised collections.
Finally, you can look at the number of people who are deeply engaged - people whose feelings or knowledge about the material or the underlying disciplines change to the extent that they change some aspect of their behaviour.
http://herbariaunited.org/wiki/Harry_Corbyn_Levinge or http://herbariaunited.org/wiki/Augustin_Ley
Lots of types of tasks and output data, source materials, validation methods, organisational structures, goals...
You've all helped correct text (or transcribe audio)
Designed to let people get on with correcting errors they'd come across when doing their work but satisfying enough task that people do it for fun. Minimised barriers to participation.
Every time I talk about this site I have to go check their stats because they go up all the time. The task design is satisfying enough that some people go to the site just to transcribe. A few people spent a lot of time doing the task - a pattern we'll see in nearly every project.
Really focussed design. Altruistic and subject specialist motivation; clear sense of what to do next… Also topical content - if there are new menus, menus relevant to events (Superbowl, in this case); tantalising snippets of content...
Low friction design - friction is anything that makes it harder or delays participation. While minimising friction, also look for points that might cause anxiety. Empathy for your participants is a huge design asset.
Complex task - marking up transcriptions in XML - on difficult source material. Has a small number of very productive super-taggers… manual validation creates backlog and delay in approving content reduces feelings of reward. Post to the blog about progress help make up for it. Media coverage helped - each round drew in a few contributors.
Old Weather is a 'Zooniverse' project that aims to extract weather information from historic ships logs for use by climate scientists. The 'Zooniverse' is the organisation behind many citizen science projects, some of which have developed into partial citizen history projects. Powerful purpose!
Zooniverse originally created forums to help answer questions when busy after launch; but turned out to provide place for wonderful discussion. Here volunteers have compiled detailed guides to understanding handwriting on particular ships
The power of finding the right audience, particularly through something as fast as social media. There are a lot of map fans in the world and they were eager to interact with the BL's collection.
Able to learn from lots of previous projects; put own twist on
Matched with computational techniques. Fed data to software, asked it to learn the visual characteristics of things depicted. Able to collect images with 'sideburns'. In this interface you can correct or mark for review, helping the software learn. (Makes you question the definition of a sideburn!)
There is a long tradition of avocational historians, sometimes working on collaborative projects - local historians in the UK sometimes undertake long-term, complex projects. FreeBMD began in 1998 to 'provide free Internet access to the Civil Registration index information from England and Wales'. It has over 250 million records. Polish vs passion projects
Two key stages in UX - convincing people to start and convincing people to stay
http://discovery.ucl.ac.uk/171071/ 'many 'amateur' content creators are 'extremely self motivated, enthusiastic, and dedicated' and test the boundaries between 'between definitions of amateur and professional, work and hobby, independent and institutional' and quotes Leadbeater and Miller's 'The Pro-Am Revolution' on people who pursue an activity 'as an amateur, mainly for the love of it, but sets a professional standard'.'
Why will people join in? Lots of reasons - same task (such as transcribing sections of a historic document) could be undertaken for altruistic, intrinsic or extrinsic reasons… Usually more than one motivation per person. Motivations change over time – different when deciding to start participating to when continuing. Often similar to motivations for volunteering in a GLAM. Much research on motivations for participation in non-commercial crowdsourcing projects comes from citizen science, or other 'community-based peer-production projects' like open source software.
(Intrinsic motivation - behavior, such as a hobby, that is initiated without obvious external incentives. External motivation is activated by external incentives, such as direct or indirect monetary compensation, or recognition by others.)
Some things are just intrinsically fun, whether it's playing with a hose or collecting football cards.
Some of the sense of progression or mastery comes from the community, not only from the interface. Given the availability of digital platforms, the largest investment should be community interaction. Think about the hobbies of people you know (or your hobbies). There's a passionate community for every topic, if you look hard enough on the internet.
Another way of looking at it.. Source: http://www.aam-us.org/resources/publications/museum-magazine/museums-as-happiness-engineers and http://www.youtube.com/watch?v=zJ9j7kIZuoQ&feature=plcp
'once you pop you can't stop' is your design goal: make a task small and satisfying enough that people want more.
Those terms 'powerful purpose' and 'enjoyable tasks' are key and why I'll talk about design and participant motivations today.
Design is important because lots of projects around; people will compare yours to others
Match 'microcopy' messages to motivations. Demonstrate a close match between the crowdsourcing project and the mission of the organisation running it
Your contribution makes a difference immediately… Effective design that makes correcting text a satisfying interaction. The user experience is further enhanced by the immediate appearance of the corrected text on the page (alongside the editing history). This shows participants the value of their contribution by making their corrections immediately available for the benefit of other users.
'16,400 little boxes – one for each person who's contributed to oldWeather. The area of each box is proportional to the number of pages transcribed, between us all we've done 1,090,745 pages.'
Some people do a lot of the work, and a lot of people do some of the work. This represents all 16,400 people who have transcribed at least one page for Old Weather (back in 2012)
Source: http://blog.oldweather.org/2012/09/05/theres-a-green-one-and-a-pink-one-and-a-blue-one-and-a-yellow-one/
Gives people a chance to get familiar with the material before trying other tasks
Minimised everything not essential to the task on this interface. This tightly scaffolded user experience means low cognitive overhead, faster and more satisfying tasks. Excellent example of a 'microtask'.
Easy-to-learn game-play; Simple controls; 'Forgiving' game-play with low risk of failure; Carefully managed complexity levels with a shallow learning curve, guidance through early levels, and inclusive, accessible themes; Sense of rapid progress and achievement
Build any tests for skill or experience requirements into the interface
Build tutorials for new skills into application at the point where its needed; provide good feedback on actions
Easy-to-learn game-play; Simple controls; 'Forgiving' game-play with low risk of failure; Carefully managed complexity levels with a shallow learning curve, guidance through early levels, and inclusive, accessible themes; Sense of rapid progress and achievement = flow!
Build any tests for skill or experience requirements into the interface; Build tutorials for new skills into application at the point where its needed; provide good feedback on actions
Further down the page, requests for help with specialist tasks - 'need a second pair of eyes. Help fix misspellings, fill in missing data... ' If you can't bear to see a misspelled word go by, you might not be able to resist that. Ecosystems of tasks are a good way to attract a wider range of contributors to help with different processes within a project. They also include proof of the value of your contributions - 'x menus digitized and counting...' and a way to check out the content before committing to action.
Motivations for participation change over time... One way to keep people motivated is to provide more responsibilities, or more complex tasks as their skills and knowledge grow. In some cases, new research questions or projects emerge from the community – this helps keep people engaged and is a great way to demonstrate impact.
Allow participants to return to records, add info not strictly required for your task.
Tiny tasks that collectively contribute to getting all the needed data
Tiny screens and lack of physical keyboards a challenge but big opportunity re leisure time
Lesson – you can’t just issue invitations, you have to stick around. Allow time for community interaction and marketing
Release early and often (if you can). Projects change once a community finds them. Allow time to update after launch as things will need to be tweaked and participants often have good ideas
Sometimes participants get distracted by the task
Find balance between participants' interests and organisational goals, even in projects that seem purely contributory
With any luck, one day you will run out of content to crowdsource - what then? Where does your community go?