Playing with metadata / Gavin Willshaw, Scott Renton (University of Edinburgh)
•Télécharger en tant que PPTX, PDF•
1 j'aime•204 vues
Learn about Edinburgh University Library's metadata games platform, a crowdsourcing initiative which has improved descriptive metadata and become a vital engagement tool both within and beyond the library. Hear how they have developed their games in collaboration with Tiltfactor, a Dartmouth College-based research group which explores games design for social change, and learn what they're doing with crowd-sourced data. Presented at the CIG Scotland seminar 'Somewhere over the Rainbow: our metadata online, past, present & future' (Metadata & Web 2.0 Series) at the National Library of Scotland, Edinburgh, 5th April 2017
Recommandé
Recommandé
Contenu connexe
Similaire à Playing with metadata / Gavin Willshaw, Scott Renton (University of Edinburgh)
Similaire à Playing with metadata / Gavin Willshaw, Scott Renton (University of Edinburgh) (20)
Plus de CIGScotland
Plus de CIGScotland (20)
Dernier
Dernier (20)
Playing with metadata / Gavin Willshaw, Scott Renton (University of Edinburgh)
- 1. Playing with metadata Gavin Willshaw, Digital Curator Scott Renton, Digital Developer
- 2. Digitisation • Increasing digitisation = more digital content • collections.ed.ac.uk – collections portal • How can this be made more discoverable and usable?
- 3. University image collection • Large collection of digital images • http://images.is.ed.ac.uk • Images mostly available CC BY at high resolution
- 4. Image metadata • Good “volume level” metadata • Data often says very little about what is in the image • Makes it very difficult for user to locate!
- 5. How did we get here? • No dedicated image cataloguer • Size of collection • Legacy content • Initial intention to improve description of collections items
- 6. The Game • Retro theme • Simple interface • “Say what you see” • Open to all
- 7. Stage 1: tag • Harvest initial tags • Open text box • No quality review • Points for tags entered
- 8. Stage 2: vote • Vote on quality of others’ tags • Closed system – good / bad / don’t know • Gain points for good tags; lose points for bad tags • Important quality review stage
- 9. • Tags QA-ed and imported into image management system • Distinction made between formal metadata and crowdsourced tags • Link included to add more tags
- 10. What we mean by crowdsourcing “--crowdsourcing not as extracting labor from a crowd, but as a way for us to invite the participation of amateurs (in the non-derogatory sense of the word)—those with every bit of the potential to be a Darwin or Mendel—in the creation, development, and further refinement of public good.” Owens, T. (2013) Digital Cultural Heritage and the Crowd. Curator: The Museum Journal, Vol. 56. Iss. 1.
- 11. What we mean by “the crowd” • Crowd: large, anonymous masses of people • Sourcing: outsourcing work and labour • “Knowledge Communities” - volunteers, engaged citizens, peers • Participation: production, development, refinement
- 12. What communities exist? • Academic researchers • Hobbyists • Students • Public • Educationalists • Self forming groups • many more . . .
- 13. Why engage communities? • Increased users • Widening participation • Feedback for improvements • Frees staff to focus on other tasks • Access to expertise
- 14. Metadata games for engagement • Metadata Games “on tour” • Integrated into other events on campus • Ran events which coincided with exhibitions • Events necessary for success
- 15. Motivation • Altruism • Display your skills/knowledge • Entertainment • Compound motivations
- 16. Information Services culture of play
- 17. Tiltfactor: metadatagames www.metadatagames.org 18,000+ tags for UoE images
- 20. Pros and Cons • Pros: • Hosted service • Easy to create an account • Easy to set up and play • Range of options – not just tagging • Easy to load images in from dropbox / Flickr • Cons • Limited functionality for more advanced options e.g. translation / transcription • Crowdcrafting requires technical expertise for modifications
- 21. What we have learned • Difficult to create engaging platform • Combine with events / activities – target theme and collections • Incentives / prizes help • Considerable staff time needed • Crowdsourced tags a compliment rather than substitute for formal metadata http://libraryblogs.is.ed.ac.uk/librarylabs/
- 22. PART 2 Scott Renton, Digital Development
- 23. Metadata Games Workflow • Why we needed them • How we built them • How we got on • Where do we go from here?
- 24. We get ‘skeleton’ records
- 25. We need ‘meat on the bones’
- 26. Options for enriching data http://images.is.ed.ac.uk/luna/servlet/UoEgal~2~2
- 28. How we built it • Emphasis on lo-fi • LAMP (Linux, Apache, MySQL, PHP)- not cool! • Front-end design decisions- retro feel • EASE authentication • Part of the Library Labs site
- 29. Design decisions • Moderation module- causing a backlog! • Voting module- how many is enough? • Scoreboard • Stars for high contributors • More complex games: • set no of items, clock, featured items • “Easter Eggs” within the game
- 30. Authenticity • Moderation • Voting module • Thresholds • Demarcation • Could we do this better?
- 31. Proliferation
- 34. How we’ve got on (so far) • 283 users • 34070 tags in system • 15616 tags from our game • 18454 tags from Tiltfactor metadata games pushed in • 6212 tags pushed back into native systems • 2875 rejections • 1993 tags still in ‘voting’ pool • 22332 tags in moderation purgatory • Not bad for only 20 hours of play • With more engagement we maybe couldn’t cope… (!)
- 35. Where do we go next? • MSc projects • Revamp- intern signed up for the summer • Responsiveness • Links to social media • “Gamification”- make the games more playable • Make them more incentive-driven • Make them more secure (or less difficult to get into) • Computer-generated metadata?
- 36. IIIF • Beautification • Streamlining • Thumbnails through URL • Photoshopping through URL • Ditch the JPEGs! • Interoperability • Annotations • Translation and transcription
- 38. The Polyglot Project (coming soon!)
- 39. Questions and more info • Email • gavin.willshaw@ed.ac.uk • scott.renton@ed.ac.uk • lddt@mlist.is.ed.ac.uk • Sites • http://librarylabs.ed.ac.uk (metadata games) • http://images.is.ed.ac.uk (image repository) • http://collections.ed.ac.uk (collections repository) • http://iiif.io (IIIF website)
Notes de l'éditeur
- Thank you very much Natasha. I’m Gavin and I’m the Digital Curator at the University Library and this is Scott Renton, digital developer in the library digital development team I’m currently responsible for managing the library’s PhD digitisation project – a three year project to digitise 17,000 PhDs – but I am also involved in a number of other projects to raise awareness of and use of the library’s collections. Scott is the technical genius behind our systems We’re going to talk to you today about our metadata games project – I’ll give you an overview of the project – how it started, why we created it and what we have learned from the process. Scott will then give you an overview of the more technical aspects of the project. We also have a couple of laptops with the games installed, so if you’d like a shot of the game then feel free to do so after the talk. We’d be very interested to hear your feedback.
- As I said, I manage the PhD digitisation project – and digitisation is steadily becoming a larger and larger element of the library’s work. Whereas a few years ago we only had two photographers, we now have three photographers, a scanner, a team of five mass digitisation assistants and digitisation internships. We also have conservators specifically employed on digitisation activity. So we’re now creating a lot more digital content, and we need to continue to work on new ways to make this content more discoverable and usable. We started on the journey by launching the collections.ed website, our one stop shop for digital images. This provides collections level descriptions for our collections as well as links to them. Brings all the collections together. This website include links to all of our collections, including our images collection. Hello everyone. We’re really pleased to be here at DCDC15 and sharing thoughts and ideas with you all. I’m Rachel Hosker and I’m Archives Manager for the University of Edinburgh, my colleague Claire Knowles is Library Digital Development Manager. We hope that by sharing what we’re doing at the University of Edinburgh we can have some productive conversations about having fun with our collections and opening them up for all. It’s my job to start us off by setting the scene for you. There are a number of reasons we’ve got ourselves into developing crowdsourcing resources for our collections. Claire would be able to tell you about my frustration with the difficulties we can have getting people to find our stuff and the times we’ve heard the ‘I didn’t know the University of Edinburgh had that item or collection’! Part of this is the inherent issue of being an ‘ancient’ University with a long, highly academic (unsurprisingly) reputation. people just don’t think they can come and use our stuff unless they’re a student or member of staff. This is despite many attempts to communicate this in a variety of ways. We thought we try a different approach. One where we could allow people to enjoy the collection and have fun in a different way that researchers usually do. We needed to find a way for people to get over the perceived view of ‘priviledge’ and understand we wanted everyone to come and use our collections, that they were there for all to see and use and in a variety of ways. We also had discovered that our vast collection of digital images of our collections has metadata issues. I’m expecting groans around the room at this point. I’ve chosen this image on the screen to illustrate this point - its from an incunabula we hold. Nowhere in the metadata for the image does it say its Noah’s Ark. Nor does it tell us about all the lovely animals including the mermaids and the lovely dog, cat and rat on the deck of the boat! We had high quality image but had inherited some from a time when metadata either wasn’t captured or was taken from other places than the authoritative catalogues. How were we going to sort this out. We equally were and are still thinking about how our catalogues and discoverability met user needs. Did an ISAD(G) professionally created description cover all bases in terms of archives? Did a Rare Book Catalogue entry have enough information to describe the image taken from an individual page. The answer was no. This meant that we had to think how we could capture information about the image and what was being seen as well as the integral what it was or part of. -- copied from RH Hello everyone. We’re really pleased to be here at DCDC15 and sharing thoughts and ideas with you all. I’m Rachel Hosker and I’m Archives Manager for the University of Edinburgh, my colleague Claire Knowles is Library Digital Development Manager. We hope that by sharing what we’re doing at the University of Edinburgh we can have some productive conversations about having fun with our collections and opening them up for all. It’s my job to start us off by setting the scene for you. There are a number of reasons we’ve got ourselves into developing crowdsourcing resources for our collections. Claire would be able to tell you about my frustration with the difficulties we can have getting people to find our stuff and the times we’ve heard the ‘I didn’t know the University of Edinburgh had that item or collection’! Part of this is the inherent issue of being an ‘ancient’ University with a long, highly academic (unsurprisingly) reputation. people just don’t think they can come and use our stuff unless they’re a student or member of staff. This is despite many attempts to communicate this in a variety of ways. We thought we try a different approach. One where we could allow people to enjoy the collection and have fun in a different way that researchers usually do. We needed to find a way for people to get over the perceived view of ‘privilege’ and understand we wanted everyone to come and use our collections, that they were there for all to see and use and in a variety of ways. We also had discovered that our vast collection of digital images of our collections has metadata issues. I’m expecting groans around the room at this point. I’ve chosen this image on the screen to illustrate this point - it's from an incunabula we hold. Nowhere in the metadata for the image does it say it's Noah’s Ark. Nor does it tell us about all the lovely animals including the mermaids and the lovely dog, cat and rat on the deck of the boat! We had high quality image but had inherited some from a time when metadata either wasn’t captured or was taken from other places than the authoritative catalogues. How were we going to sort this out. We equally were and are still thinking about how our catalogues and discoverability met user needs. Did an ISAD(G) professionally created description cover all bases in terms of archives? Did a Rare Book Catalogue entry have enough information to describe the image taken from an individual page. The answer was no. This meant that we had to think how we could capture information about the image and what was being seen as well as the integral what it was or part of.
- We have a large database of approximately 50,000 high quality digital images. Images can be viewed online at high resolution and downloaded for free. The majority of the images are available on a CCBY licence, where there are no copyright / IP restrictions – this is our default position for digitised images.
- Images have good metadata describing the volume the image came from – author, title, year of publication etc But the metadata tended to say very little about the image itself. E.g. turtle In the images above, the word “turtle” “X” and “Y” don’t / didn’t appear anywhere in the descriptive metadata. As you’d expect, this makes it very difficult for a user to locate an image – if they typed “turtle” into the search box, the image on the screen would never have appeared!
- Reasons for this are varied, but include: No dedicated image cataloguer Size of collection Much of the content legacy – decision only taken to put online in 2013 (?) but digital images been created for 10 – 15 years. These images often from reader requests or for projects and no intention then to put them online. Have explored different ways of enhancing this descriptive metadata – volunteers do their bit. LUNA software had a crowdsourcing module built in but the interface not ideal (image) and required a user account (check) Given the difficulties associated with our descriptive metadata, we decided to try creating a new interface and game system to encourage users to add tags – our metadata game is the result! So, to be clear, our initial intention was to create metadata games first and foremost as a tool to improve the descriptions of our digital collections.
- To do so, we created a fun game with elements which, we hoped, would encourage people to take part The game itself is fairly simple – players are given a selection of images from the collection and they need to look at them and “say what they see” then enter tags into the box at the bottom of the screen.
- So, in this example, the user would type “pillar”, “ruins”, clouds etc into the box The user is then awarded one point for each tag they enter, with their total points displayed at the top of the screen
- Following this, in the second stage, the player is provided with tags from another user and they need to decide whether these tags are good or bad: good tags earn extra points and bad tags lose points! This second stage, as well as adding variety to the game and a new source of points, also acts as an initial quality assessment stage.
- Crowdsourcing been around for some time now, in particular some very high profile projects including Zooniverse, Transcribe Bentham, and of course the projects described earlier by our NLS colleagues… This quote sums our approach up – crowdsourcing not as extracting labour from a crowd but collaboration of communities in the creation, development and refinement of public good.
- The word ‘crowd’ implies an anonymous mass of people We prefer to see the crowd as ‘knowledge communities’ that is, volunteers, engaged citizens, citizen scientists who are engaged in the production, development and refinement of knowledge.
- Communities can come in many forms and can include academic researchers, students, members of the public and many others At our sessions, we have engaged with academics and students, but also members of the public ranging from 8 years old through to people in their 70s!
- So the value of engaging communities: It increases the numbers of people using your systems and collections It widens participation to include non-typical groups Provides a forum for feedback for improvements to the system Frees up staff to focus on other tasks Access to expertise – our collections in different language etc
- We quickly realised that, as well as being a tool for sourcing tags, the games also had considerable potential for increasing awareness of and engagement with our collections. We initially used the game only with colleagues in Information Services and the Library but saw great potential for taking it to other locations. To engage people we: Took the game on tour around the campus (including KB) and Edinburgh libraries Integrated the game into other events happening throughout the University year Including Innovative Learning Week, Pop Up Library and Ada Lovelace Day Ran events which coincided with exhibitions (Where’s Dolly) Worth noting that we have limited success in attracting people to take part of their own accord - far more successful when we make the session part of an event.
- Incentivised people to take part by offering rewards (calendars, coffees, dolly teddy) Make it competitive Also: Altruism Display your skills/knowledge Entertainment Compound motivations
- Our metadata games also tie in more broadly to a culture of fun and play that is being encouraged across Information Services. The culture promotes the value of fun and play as important to learning and encourages an environment and atmosphere of experimentation and acceptance of trying things out and it being “ok to fail”. This culture is exemplified at events such as: Ada Lovelace Day, celebrating the achievements of women in tech through fun activities such as LEGO raspberry pi, Sonic Pi etc Wikipedia editathons: a fun and engaging atmosphere for updating Wikipedia Board Game Jams: staff and students develop board games using the library’s digital collections – learn about copyright, licensing etc in fun context.
- As well as developing our games, we have also been exploring crowdsourcing in several other areas. We have worked with Tiltfactor in the US to put our images into their crowdsourcing platform. Tiltfactor is a design organisation that works out of Dartmouth College which focuses on serious fun / play. Through their metadatagames project, they have created a range of games which have harvested over 300k tags to date Games include: One player and multiplayer Audio and video tagging Coooperative and competitive games We entered around 2,500 of our images into the Tiltfactor system and have gained around 18,000 tags from these.
- In addition, we have experimented with other platforms including Zooniverse and Crowdcrafting In the Zooniverse project, users are given a page from the letters of Lady Grange, who was exiled to St Kilda by her husband for allegedly revealing her husband’s Jacobite sympathies Users need to read the text, identify names, dates and places, tag them and say whether they can understand the story
- In Crowdcrafting, the user is given an image from the collection and needs to say whether it is an animal, vegetable or mineral. Running score along the bottom of the page
- The pros of using these services are that they are hosted, it’s easy to get an account and they’re quick to set up and play with. However, still early days. Crowdcrafting: requires technical and HTML expertise to make modifications, issues around permissions for images to be hosted remotely Zooniverse: limited functionality for translation / transcription
- Creating engaging platform is very difficult Need to combine with other events / activities to make it work Activities like this free up staff time but also requires significant staff time for quality assessment Needs significant input of staff resources (intern hired to do so) Improve description of collections Increase use of and engagement with collections Frees up staff time
- So, Gavin and myself are both very involved in tackling this issue, but from slightly different angles; even after an attempt at choreographing our halves of this particular tango, we're inevitably going to walk over the same boards a few times (hopefully without treading on any toes). It'll drum a few of the points home, I'm sure, though! I'd like to talk about this from the perspective of digital development, so effectively why, as a techie, I had an interest in tackling this issue, so I'll mention the problem, why we did what we did, the design decisions involved, how it fit into our workflow, and where, with the benefit of two years' additional experience and interest in the community, we can see this going. Apologies, I tend to explain this through the medium of skeleton.
- So, from my perspective, I'm very interested in workflows, and despite 7 years of hacking away at this, there is still something of an atavistic perspective to how we get image data in the department. A lot of digitisation is based on readers' orders being passed to the photographers, and this actually still involves paper forms. Therefore, you're only going to get a small amount of identification metadata for a digitised page, because it needs to be copied out. Most of this stuff isn't really catalogued, and those that hold the purse strings will be far likelier to pay for a new camera than a cataloguer.
- This generally means there's a lot more that could be captured for an item than- we've got our skeleton records, but we need meat on the bones. This particularly macabre scene is from one of our most popular collections- the Incunabula- our earliest printed books, effectively, and very popular (mainly because there was no political correctness around the gruesome images therein). That's generally what, if we're lucky, will go straight onto a record, along with the EXIF data that comes directly from the camera but where's the rich, contextual information, and where are the tags that tell us "skeleton", "horse", etc.
- The systems that we had prior to this didn't have really any capability for enrichment, beyond hiring interns to come in and properly describe our images through the LUNA backend. LUNA does have an annotation module- annotations are stored in an database, so we can look at and garner this information but nobody knows about it, as it's pretty well hidden, so we've never been able to get very much. We needed something extra.
- So, we decided to build the games. Gavin's shown you what they look like- very deliberately lo-fi, with the space invaders feel, almost to emphasise their simplicity. They're really tenuously described as games- there's nothing particularly clever going on, and they've only worked because they're surprisingly addictive! We could do so much more if we had the time.
- We built them using straightforward lamp: it's really just form data writing directly to a database, although that can be surprisingly powerful. There is authentication, using the University's single sign-on module, and we put them directly onto our Library Labs site, which is where all of our investigative and innovative work goes.
- And again, as Gavin said, we have had to take a number of decisions as to how to control the flow of what we get, and to try to incentivise the users. Random images seemed to work best, based on collections that were safe to surface and needed tags most. We went initially for a moderation module to try and exert a human eye over some of the data before anyone else got to see it- we are using stopwords downstream for this kind of thing- but the problem has been a massive backlog, because the crowd aren't involved at this point and approved moderators don't have time. We probably need to bite the bullet and ditch this. The voting module is a crucial part of the game, an easy way to score a lot of points and the thing that determines whether data gets into the core system. The scoreboard is hugely effective in incentivising users- I've watched people get very worked up trying to win during a session. We've introduced a "star" branding for high scoring players, and we've also tried to make games more complex, introducing clocks and set numbers of images (see the Art game), as well as easter eggs (the hidden Dolly the Sheep).
- Authenticity is an important point- we have a threshhold of net 2 votes allowing something to get into the core system, because we don't have enough activity to make it higher. That said, the fact that we clearly demarcate the tags to keep the public/scholarly separate probably means we have the leeway to do very little. It is difficult though, and is a regularly bone of contention. I have spoken to people whose metadata librarians would be horrified at the very thought of what we're doing.
- The proliferation of the games has been an interesting point. We started off with one, endless game, and let that fly, and then we added the Art game, which had a bit more structure and sense of competition to it (we fudged it so that the items physically on show always came up first for that particular event, so they're very well-described). The Where's Dolly was made to look like a Wellcome/Roslin glass plate slide and, as I say, would cause the game to haemorrhage excitement if Dolly The Sheep came up. It was pretty straightforward to skin a new game and give it distinct data, so we could easily do more of the same (but at roughly the same level of complexity).
- Then, just a little bit about how we get the data back into core systems- mysql data exported into csv, and through the LUNA API, it gets onto the main record.
- For collections sites (not all our collections see LUNA as their golden copy), it goes into the DSpace database via a Java curation task, and is pretty heavily grilled for stopwords at this point.
- So, here's how we did. It's not bad off approximately 20 hours of play- we get, as Gavin's said, no action when we don't have an event on, and it is something we need to address.
- So, that's what we've done, but there's a lot that can still happen. We had some informatics students looking at metadata games using this as basis last year, and it gave us some ideas. We have an intern signed up for the summer to give them an overhaul- that's to make them responsive initially, but possibly to do the things they currently do more elegantly, and certainly engagingly. Gamification (if that's a word) should be addressd: I initially got involved in this from the selfish perspective of trying to meet our users' needs of making the collections searchable, and not to any great end making it an altruistic "bring the community together". It was all about free labour doing the things we couldn't do. We need to find ways to bring the users back, beyond competitivity and incentives, and it is one of the challenges of this internship. Can we do things with computer-generated metadata, recognition of shapes etc, or OCRing? Should we be using oAuth as a means of getting in, and hooking the sites up to Twitter and Facebook to allow greater security, engagement and ease of access?
- And then, there's this new thing that we've been getting into in the Library Communtiy called IIIF. This is a framework for interoperability across images, with the intention of getting away from silos. The idea is the image is only ever hosted once, and can then be reused by bespoke viewers and in customised presentations by "photoshopping" on the fly using a URL. That's one side of what it can do, so it allows us to beautify our websites by bringing in Deep Zoom functionality embedded into the page, means we can get rid of jpegs, and it means we can often hide collections that are revealed in two places, so we don't need to keep metadata in sync! The other side is that it presents a standard for interoperable PRESENTATION, where you may have multiple images relating to one item, to determine the sequence, direction etc of how these should be shown, and crucially ANNOTATION can come in at this point.
- Just to give you a couple of examples of this: the existing game has a link to the LUNA system for high-res zooming, that opens a separate window- this will get away from that. Using the OpenSeadragon viewer, we will be able to zoom directly on the page, which is very useful for checking what is there to be tagged.
- And then , for crowdsourced material, IIIF is a much more standardised way of bringing in annotations for translations and transcriptions. I think it is something that we will set our intern off to look at, getting away from the mysql inserts, html form data approach. Last year, we got funding for an intern to look at a transcription and translation tool for researchers- as opposed to "the layman", and she came up with something way beyond our expectations. I'm not sure these screenshots quite do it justice, but she used IIIF (image API) to bring in content, and IIIF (presentation API) to manage and administer translations. She uses Leaflet IIIF and Draw to allow vectors to be drawn round the area we're interested in, and stores the annotations that people submit, and, yes annotate again, as well as votes and discussion on the annotations. She's also introduced an array of keyboards so you can transcribe in more or less any language, although she struggles a bit with languages which are no longer with us in that form (and of course a lot of this manuscript data is in such a form!). It's not live yet, so I can only show screenshots, but I will let you all know when it is available!