Project output versus influence in practice: impact as a dimension of research quality

1. Project output versus influence inpracticeImpact as a dimension of researchquality6th International Conference on Evidence based Librarianship and InformationPracticeKeynote presentation by Professor Hazel Hall, Director, Centre for SocialInformatics, 30th June 2011

2. http://lisresearch.orghttp://lisresearch.org

3. If you are tweeting thisIf you are tweeting thissession, please refer to mesession, please refer to meas @hazelh. Thank you!as @hazelh. Thank you!

4. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach

5. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach

6. Impact themes in LIS literatureImpact themes in LIS literature1. Evaluation of services – general use (debated)1. Evaluation of services – general use (debated)

7. Impact themes in LIS literatureImpact themes in LIS literature2. Evaluation of service impact on end user behaviours2. Evaluation of service impact on end user behaviours

8. Impact themes in LISImpact themes in LISliteratureliterature3. Evaluation of services as3. Evaluation of services asrelated to specificrelated to specificoutcomesoutcomes

9. Impact themes in LIS literatureImpact themes in LIS literature4. How to evaluate services4. How to evaluate services

10. Impact themes in LIS literatureImpact themes in LIS literature5. Bibliometrics5. Bibliometrics

11. Impact themes in LIS literatureImpact themes in LIS literature5. Bibliometrics – publication impacts5. Bibliometrics – publication impacts

12. Impact themes in LIS literatureImpact themes in LIS literature6. Bibliometrics research evaluation6. Bibliometrics research evaluationWe can stand on the shoulders of giantsWe can stand on the shoulders of giantsbecause librarians and informationbecause librarians and informationscientists have maintained a curated recordscientists have maintained a curated record

13. Impact themes in LIS literatureImpact themes in LIS literature7. LIS research output  LIS practice7. LIS research output  LIS practice

14. Impact themes in LIS literatureImpact themes in LIS literature7. LIS research output  LIS practice7. LIS research output  LIS practice

15. Impact themes in LIS literatureImpact themes in LIS literature8. LIS practice  LIS research8. LIS practice  LIS research(Brookes, 1976)(Brookes, 1976)

16. Impact routes varyImpact routes varyCumulative and indirect effect ofCumulative and indirect effect ofexposure to research outputexposure to research outputDirect policy mandate thatDirect policy mandate thatderives from a particular studyderives from a particular study

17. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach

18. Measurement challenge 1Measurement challenge 1Attribution, and cause and effect, areAttribution, and cause and effect, arenot always accurate nor obviousnot always accurate nor obvious

19. Greer Garson Greer Garson Best actress 1942 for role in Mrs Miniver Best actress 1942 for role in Mrs Miniver Acceptance speech 5 minutes 30 seconds Acceptance speech 5 minutes 30 secondsMeasurement challenge 1Measurement challenge 1Attribution, and cause and effect, areAttribution, and cause and effect, arenot always accurate nor obviousnot always accurate nor obvious

20. Measurement challenge 2Measurement challenge 2Sources of mediated training material areSources of mediated training material arenot always appreciated by practitionersnot always appreciated by practitioners

21. Measurement challenge 3Measurement challenge 3End-users are not normally aware of researchEnd-users are not normally aware of researcheffort to support services design and deliveryeffort to support services design and delivery

22. Measurement challenge 4Measurement challenge 4Certain types of research output attract moreCertain types of research output attract moreattention than othersattention than others

23. Measurement challenge 5Measurement challenge 5The impact of some is overwhelmingThe impact of some is overwhelming(and disproportionate)(and disproportionate)

25. Practitioner interest in research outputPractitioner interest in research outputIncreases value of research undertakenIncreases value of research undertakenNarrows research-practice gap Narrows research-practice gap benefits health of the professionbenefits health of the profession

26. Local contexts influence attitudesLocal contexts influence attitudestowards impact measurementtowards impact measurementREF2014 in the UKREF2014 in the UK

28. NB presented here are strongNB presented here are strongmessages that have emerged frommessages that have emerged fromRiLIES project empirical dataRiLIES project empirical dataanalysed to dateanalysed to datehttp://lisresearch.org/rilies-project/http://lisresearch.org/rilies-project/

29. Literature review; poll; focusLiterature review; poll; focusgroups; case studies; surveygroups; case studies; survey

30. Research ideas for impact in practiceResearch ideas for impact in practiceFocus on research areas of direct relevanceFocus on research areas of direct relevanceto practitioners, both in terms of subjectto practitioners, both in terms of subjectand context -- time, place, sectorand context time, place, sector(Perth and London focus groups)(Perth and London focus groups)

31. Research approach for impact in practiceResearch approach for impact in practiceDeploy action research approachDeploy action research approachMix in development and engagement, e.g. capacityMix in development and engagement, e.g. capacitybuilding as hook, workshops throughout projectbuilding as hook, workshops throughout projectEncourage community creation around project forEncourage community creation around project forsustainabilitysustainabilityInvolve high profile, prestigious partnersInvolve high profile, prestigious partners(Evalued, Evaluating the impact of clinical librarian(Evalued, Evaluating the impact of clinical librarianservices, Open to all case studies)services, Open to all case studies)

32. Project scale and diversity for impact in practiceProject scale and diversity for impact in practiceData collection needs to be on a scale for projectData collection needs to be on a scale for projectrecommendations to be applicable elsewhererecommendations to be applicable elsewhereEnsure that projects are outward-looking toEnsure that projects are outward-looking tosafeguard against accusations of biassafeguard against accusations of bias(Perth focus group)(Perth focus group)

33. Face-to-face dissemination for impact in practiceFace-to-face dissemination for impact in practiceExploit face-to-face dissemination as the practitioners’Exploit face-to-face dissemination as the practitioners’favoured routefavoured routeTarget dissemination according to career stage preferencesTarget dissemination according to career stage preferencesInvest project budget in training activities for project legacyInvest project budget in training activities for project legacyLaunch events effective for policy maker target audiencesLaunch events effective for policy maker target audiences(Poll, focus groups, case studies)(Poll, focus groups, case studies)

34. Accessible textual sources for impact in practiceAccessible textual sources for impact in practiceTailor a variety of reports according to intended audiences:Tailor a variety of reports according to intended audiences:tools they can access, career stage, sector…tools they can access, career stage, sector…Make reports accessible: easy to locate e.g. open access;Make reports accessible: easy to locate e.g. open access;understandable, e.g. possible to act on project outcomesunderstandable, e.g. possible to act on project outcomesWork with profile dissemination partners (e.g. HEFCE, JISC, Work with profile dissemination partners (e.g. HEFCE, JISC,OCLC, Pew, RIN, SCONUL) to enhance authority of work and OCLC, Pew, RIN, SCONUL) to enhance authority of work andincrease its reach increase its reachEnsure work is picked up by digest/alert services and bloggersEnsure work is picked up by digest/alert services and bloggers

35. http://www.twitter.com/LISResearchhttp://www.twitter.com/LISResearchprovides an alert servicesprovides an alert services Conference registration open Conference registration open Training day announcement Training day announcement Publication of research report Publication of research report Study day announcement Study day announcement Conference registration to close Conference registration to close Conference registration open Conference registration open Study day announcement Study day announcement Publication of blog post Publication of blog post

36. Exploit network hubsExploit network hubs5 key individuals in UK5 key individuals in UKemerge from RiLIES project…emerge from RiLIES project…

37. 5 UK research connectors5 UK research connectors1. Rhona Arthur @RhonaEA1. Rhona Arthur @RhonaEA

38. 5 UK research connectors5 UK research connectors2. Andrew Booth @scharrlock2. Andrew Booth @scharrlock

39. 5 UK research connectors5 UK research connectors3. Phil Bradley @Philbradley3. Phil Bradley @Philbradley

40. 5 UK research connectors5 UK research connectors4. Alison Brettle @BrettleAli4. Alison Brettle @BrettleAli

41. 5 UK research5 UK researchconnectorsconnectors5. Hazel Hall @hazelh5. Hazel Hall @hazelh

42. Face-to-face dissemination for impact in practiceFace-to-face dissemination for impact in practiceWhat happens when formal opportunities for face-to-What happens when formal opportunities for face-to-face dissemination are restricted:face dissemination are restricted: training/travel budgets cut? training/travel budgets cut? reduced professional memberships? reduced professional memberships?… and when research infrastructures are fractured? This… and when research infrastructures are fractured? Thishas been addressed in UK in the formation of thehas been addressed in UK in the formation of theLibrary and Information Science Research CoalitionLibrary and Information Science Research Coalition

43. Researcher priorityResearcher priorityPresent research output in an accessiblePresent research output in an accessiblewaywayService manager priorityService manager priorityFacilitate research output consumption,Facilitate research output consumption,e.g. reverse mentoring, job descriptions,e.g. reverse mentoring, job descriptions,reporting back from conferencesreporting back from conferencesCourse/accreditation priorityCourse/accreditation priorityApplication of research in practiceApplication of research in practice

44. A greater role for social media? A greater role for social media? Currently divides the practitioner Currently divides the practitioner community, e.g. blogs very popular with community, e.g. blogs very popular with academic librarians but inaccessible to academic librarians but inaccessible to some others; Twitter essential to some, some others; Twitter essential to some, irrelevant to others irrelevant to othershttp://www.bite.ca/bitedaily/2010/07/social-media-monopoly-board-game/

45. A greater role for self-organisingA greater role for self-organisingnetworks as communities of practice?networks as communities of practice?(with technical infrastructure provided by(with technical infrastructure provided bysocial media)social media)

46. Research funding to support researcher-Research funding to support researcher-practitioner linkspractitioner linkshttp://www.lisresearch.org/dream-projecthttp://www.lisresearch.org/dream-project

47. DREaM project launch conferenceDREaM project launch conferenceTuesday 19th July, LondonTuesday 19th July, LondonPlaces still availablePlaces still available

48. Key points LIS impact takes a number of forms Research impact measurement is tricky Research impact measurement matters Where impact is measured as a dimension of research quality, a number of strategies can be deployed to extend project outcome reach  Preliminary findings of RiLIES project dominated by influence of dissemination strategy

49. Bringing it back to Salford:anyone recognise these lads?

50. Bringing it back to Salford:anyone recognise these lads?

51. What difference does it make? (1984)

52. Playlist 20 March 1984Playlist 20 March 1984Hand in glove; Still ill; Heaven knows I’mHand in glove; Still ill; Heaven knows I’mmiserable now; This charming man; Girl afraid;miserable now; This charming man; Girl afraid;Pretty girls make graves; This night has openedPretty girls make graves; This night has openedmy eyes; Barbarism begins at home; Whatmy eyes; Barbarism begins at home; Whatdifference does it make? II don’t owe youdifference does it make? don’t owe youanything; Handsome devil; You’ve gotanything; Handsome devil; You’ve goteverything now; These things take timeeverything now; These things take time

55. Photo creditsImage: I Can Gather All the News I+Need+on+the+Weather+Report Image: Greer Garson & Robert Younghttp://www.flickr.com/photos/51035555243@N01/2671536366 http://www.flickr.com/photos/44186100@N00/2490143154Image: These Books are Sweet Image: ‘Waiting for the library to open’http://www.flickr.com/photos/47823583@N03/4474869956 http://www.flickr.com/photos/davepattern/1424246518/in/set- 72157602122007409Image: ‘Radar Graph – Library Usage v Final Gradeshttp://www.flickr.com/photos/davepattern/4172110833/in/set-72157622847182901/ Image: BC007 http://www.flickr.com/photos/51548378@N05/4745027706Image: 50lb weightshttp://www.flickr.com/photos/62327186@N00/2185667691 Image: Money to burn http://www.flickr.com/photos/8363329@N05/811315055Image: Article-level metrics at PLoShttp://www.flickr.com/photos/14829735@N00/3929622080 Image: Warhols Light Bulbs http://www.flickr.com/photos/66814335@N00/3036254720Image: 365: day 119http://www.flickr.com/photos/33563858@N00/2451800342 Image: Cheers http://www.flickr.com/photos/71477195@N00/204879104Image: The locker miseryhttp://www.flickr.com/photos/33685308@N00/513100970 Image: P1010121 http://www.flickr.com/photos/67614763@N00/4304374413Image: 90" engine valveshttp://www.flickr.com/photos/61132483@N00/2975215 Image: My social Network on Flickr, Facebook, Twitter+and+MyblogLog http://www.flickr.com/photos/49503019876@N01/1824234195Image: Oscarhttp://www.flickr.com/photos/45756179@N03/5068766020 Smiths at Salford Lads Club (inside cover of The Queen is dead) by Stephen WrightImage: ed vaizeyhttp://www.flickr.com/photos/25414047@N00/4065621328

56. Reference & acknowledgements Brookes, B.C. (1976). A new paradigm for information science. The Information Scientist, 10(3), 103-12. Peter Cruickshank, Ella Taylor-Smith, Jenny Gebel (RiLIES project team) Stephanie Kenna (LIS Research Coalition) All those who have contributed data to date

57. Professor Hazel Hallh.hall@napier.ac.uk @hazelh

Project output versus influence in practice : impact as a dimension of research quality

Hazel Hall