Library Metrics and Measurement: Counting What Counts & Making it Matter
•Télécharger en tant que PPTX, PDF•
6 j'aime•2,428 vues
Wild Wisconsin Winter Web Conference 2014 Wednesday, January 15 2:30 - 3:30 pm Register: https://www1.gotomeeting.com/register/278806136 #wwwc14
Recommandé
Contenu connexe
En vedette
En vedette (9)
Similaire à Library Metrics and Measurement: Counting What Counts & Making it Matter
Similaire à Library Metrics and Measurement: Counting What Counts & Making it Matter (20)
Plus de Emily Clasper
Plus de Emily Clasper (20)
Dernier
Dernier (20)
Library Metrics and Measurement: Counting What Counts & Making it Matter
- 1. LIBRARY METRICS AND MEASUREMENT: COUNTING WHAT COUNTS & MAKING IT MATTER Emily Clasper Wild Wisconsin Winter Web Conference 2014
- 4. Library Statistics Circulation Transactions Door Count Reference Questions Web site visits Program Attendance Collection Inventory Staff Actions http://farm7.staticflickr.com/6074/6033655868_6b53ef4ebb_b.jpg
- 5. The Panicked Call… We’ve been doing so well! How can our stats all be down? http://farm9.staticflickr.com/8006/7706143020_e533940f18_b.jpg
- 6. Depending on Our Numbers Securing Funds Justifying expenses Complying with mandates Gaining support from stakeholders Demonstrating the value of services, ROI Data-driven decision making Allocation of our internal resources Service development http://farm3.staticflickr.com/2602/3678747186_b63d82fe2c_o.jpg decisions
- 7. Moving Beyond Circ Stats http://connecttocrlibrary.files.wordpress.com/2010/12/graph.jpg
- 8. Asking The Right Questions http://farm3.staticflickr.com/2318/2201907500_6f0741be40_o.jpg
- 9. The Wrong Question We bought a 3-D printer and it seems to get a lot of use, but circulation down! http://static.flickr.com/7062/6845000770_ebd5fb87d5.jpg
- 10. The Wrong Question That long jumper seems like a really strong athlete, but she doesn’t run very fast! http://farm5.staticflickr.com/4141/4817786416_282faf5324_o.jpg
- 11. Asking New Questions Is this a question that we can Will the answer lead to new Does this knowledge help us assess our Can this assessment be used to
- 12. Go Back to Your Goals http://farm3.staticflickr.com/2773/4318924726_85eae8b991_o.jpg
- 14. Outcomes Measurement Process Inputs Activities Library Perspective: What Library Does/Provides Results of Process Output Interim Outcomes Outcomes Goals User Perspective: Results of the User’s Experience Adapted from: Rubin, Rhea Joyce. 2006. Demonstrating Results: Using Outcome Measurement in Your Library. PLA Results Series. Chicago: American Library Association.
- 15. What Defines Success? What is it that we’re actually trying to DO? How will we know if we’ve accomplished that goal?
- 17. What Data Do We Need? http://farm9.staticflickr.com/8514/8450658347_06c56211f9_b.jpg
- 18. Key Performance Indicators Measureable Indicates progress towards goals Actionable Measured frequently and consistently May be objective or subjective
- 19. 12am-1am 3-4 am 4-5 am 5-6 am 6-7 am 7-8 am 8-9 am 9-10 am 10-11 am 11am-12pm 12-1 pm 1-2 pm 2-3 pm 3-4 pm 4-5 pm 5-6 pm 6-7 pm 7-8 pm 8-9 pm 9-10 pm 10-11 pm 11pm-12am Library KPIs Can Take Many Forms 250,000 200,000 150,000 100,000 50,000 0 http://www.bu.edu/library/files/2013/03/Faculty-Satisfaction-Services.bmp
- 20. KPIs as Strategic Goals By 2015, 80% of users surveyed will indicate that they waited no longer than three minutes for staff assistance at a public service desk. By 2015, at least 85% of all purchased items will be invoiced, cataloged, processed, ad sent for shelving within two weeks of receipt. By 2016, the number of children aged 6-11 with library cards will increase from X to Y. Each year, a minimum of 80% of adults and teens who attend a training session on how to use technology to create content will indicate that the training session was very good or excellent.
- 26. Questions for Data Collection Are we counting effectively? What structures do libraries have in place for gathering performance data? What kind of models can we emulate for improving our data collection strategies? How do we get closer to gathering qualitative measures as well? http://farm3.staticflickr.com/2197/2200500024_33038620af_o.jpg
- 28. Accessing and Storing Data http://farm4.staticflickr.com/3245/2294144289_3102e0de26_o.jpg
- 30. Now That We Have The Data… http://static.flickr.com/6017/5961260280_f38d3cbee9.jpg
- 32. What Does This Really Mean? http://farm3.staticflickr.com/2891/9354301543_30e405da0a_b.jpg
- 33. Using Data for Good, Not Evil http://farm4.staticflickr.com/3674/10868394285_6a10dc9e99_b.jpg
- 35. Using and Presenting Data http://farm9.staticflickr.com/8330/8436405977_d0613c3cac_b.jpg
- 36. Making Data Easy to Understand http://farm6.staticflickr.com/5144/5659510216_657431c231_b.jpg
- 43. Promoting a Brand http://obieosobalu.files.wordpress.com/2011/01/history-of-microsoft-final.jpg http://www.coca-colacompany.com/our-company/infographic-coca-cola-at-a-glance
- 45. Turning Outcomes Into Impact http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
- 47. Questions for Libraries What areas of measurement might we get into in response to changing organizational goals? How can we present the data we gather and the conclusions we draw from it to more effectively demonstrate success, promote the library and its goals, boost productivity, make more informed decisions, tell the library’s story to stakeholders? What skills do we, as Information Pros, Managers, and Advocates need to develop in order to effectively make analytics the powerful tool we need it to be?
- 48. Do We Have the Skills? http://farm8.staticflickr.com/7171/6632168217_bdc32df214_b.jpg
- 49. Do We Have The Tools? http://farm1.staticflickr.com/113/286709039_105881e4b9_o.jpg
- 50. Can We Make it Matter? http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
- 51. Recommended Reading • Brophy, Peter. 2006. Measuring Library Performance: Principles and Techniques. London: Facet. • Hernon, Peter, Robert E. Dugan, and Joseph R. Matthews. 2014. Getting Started with Evaluation. Chicago: American Library Association. • Ishak, Das. 2014. “Discovering the Right Key Performance Indicators in Libraries: A Review of Literatures.” Accessed January 14. http://www.academia.edu/1805631/Discovering_the_right_key_performance_indicators_in_libraries_a_ review_of_literatures. • Larkin, Richard. 2013. “Using Outcomes to Measure Nonprofit Success.” NPQ - Nonprofit Quarterly. July 2. http://www.nonprofitquarterly.org/management/22549-using-outcomes-to-measure-nonprofitsuccess.html. • Matthews, Joseph R. 2007. The Evaluation and Measurement of Library Services. Westport, Conn: Libraries Unlimited. • Rubin, Rhea Joyce. 2006. Demonstrating Results: Using Outcome Measurement in Your Library. PLA Results Series. Chicago: American Library Association. • Webbmedia Group. 20:12:02 UTC. “Key Performance Indicators For Libraries”. Education. http://www.slideshare.net/webbmedia/key-performance-indicators-for-libraries. • Wolf, Lesile. 2010. “What Gets Measured Gets Done: Key Performance Indicators.” California Digital Library. http://www.cdlib.org/cdlinfo/2010/09/15/what-gets-measured-gets-done-key-performanceindicators/. • Zappone, Marisa. 2013. “Why It Makes Sense for Nonprofits To Focus on Contribution, Not Causation.” Greenlights for Nonprofit Success. http://www.greenlights.org/blog/2013/09/16/why-it-makes-sense-for-
Notes de l'éditeur
- This is a presentation riddled with questions, very few of which have concrete answers.
- The public libraries I work with usually use combinations of these
- But I’ve been getting more and more of this, lately
- Many public Libraries depend on circ stats almost exclusively. In a world where these same liraries see over 50% of their circs come from DVDs and CDs, these libraries are in trouble.
- We are doing ourselves a disservice by relying so heavily on Circ stats. We’re not telling our whole story. We’re not making our case effectively. We’re not arming ourselves with enough data to make truly informed decisions. Where do we go from here?
- Start with the questions we using metrics to answer.
- Actual question I got.
- Not important to the competition at hand.When we rely on the same old data to measure new initiatives, it can lead us astray. We need to formulate questions that reflect on the goals we want to accomplish in a meaningful way.
- When looking at what we want metrics to tell us, ask yourself: Is this a question that we can ANSWER? – scope, framingWill the answer lead to new KNOWLEDGE? - verifiableDoes this knowledge help us assess our PROGRESS? - Can this assessment be used to suggest ACTION? - Will the actions help lead us closer to our GOALS?
- Strategic planning. Mission, Goals should drive everything. If goals are clearly identified, we can start asking questions that matter and determining new ways of answering those questions.
- When considering the goals, we need to think about BOTH outputs and Outcomes.Outputs What you’ve done (Make 100 Cupcakes)Outcomes Impact of the Outputs (100 Happy, sugared up kids) – libraries usually focus on Outputs. It’s easier, funders ask for it, etc.Outcomes are harder to get at, but more in line with progressing towards our goals.
- Outcomes measurement – see this excellent book for practical examplesEvaluation of both process and results. Results are harder to measure, but more important. ROI often focuses on Input/Output equation, which is valuable. However, it often fails to get at the true impact of the initiative. Resume writing assistance: Inputs: 100 hours staff time, specialized staff training Outputs: 35 resumes for jobless library membersOutcomes – interviews, jobs attained; customer satisfaction; community engagement and awareness
- Come back to these questions. This leads to…
- Deciding what to measure in order to do our ouput and Outcomes analyses?What can we get?What would indicate a successful outcome?
- One way of organizing your data collection is to define KPIs in line with the strategic goals.Much documentation on this process. My own requirements…
- Specific Circulation StatisticsComputer terminal, facilities, equipment usage Ratio of professional staff (FTE) per capitaWorkflow efficiency ratesCustomer satisfaction survey feedback
- KPIs should be framed as specific goals with a defined timeframe. Assessment of progress can be measured against the degree of success in reaching these KPIs
- Factors to consider in determining Data collection methodology:Qualitative measuresDon’t forget qualitative measures!How do we quantify the intangibles of our successes? Constant vigilance – don’t sacrifice quality. It is at the core of your mission.
- Are there standards within the industry or community that can be applied?
- What kind of data is available?Open data resources are multiplying!
- Impacts what you decide to measure>ILS Tools – use and collection statisticsIndependent analysis products – GIS, demographic comparisonsObservational methods – “field work”
- Hi Tech makes new measures possible: Movement Sensors (Measuring Behaviors: Attention, Movement, Location, Response) GeotrackingHow long did someone browse that section before making a selection? Issues of compiling and storing data – track length of time between visits, borrowing habits, behaviors – privacy issues
- Effective measurement should not impact the data itself – hard with qualitative or subjective dataWhat are we doing already, or what can be adapted from current procedures>Look at models from inside and outside the industry!The ever-present issue of qualitative measurement
- Love this data collection tool. Simple, easy, at point of service. Fallable, yes, but reasonably straightforward. We need to make sure that the tools we use are appropriate to the questio, data, and situation.
- Data access and security, protecting patron privacy. Anonymization of data, security measures, privacy policies, staff training
- Once the data has been collected, we need to do something with it. Often I see library directors and staff take a pile f statistics, survey results, etc. and then stop the process. They stare at the numbers, usually looking for a straightforward gain or decline. But what can this data really tell us?
- Determination of value, implications for each measure (How much is a “Circ” worth in 2014?)Identification of trendsWhat skills are needed for this? What do we, as a profession, need to work on?
- Popularity vs. Importance – More people attended the concert on the lawn than the college search seminar. Causation vs. Correlation
- Gotta read the numbers right. Someone sent me a chart much like this one once, telling me that they had an exponential growth in new members. Nope. Reasonably steady new membership, growing overall membership. Time to cull out the inactive cards.
- Use data responsibly. Guard privacy. Use to further favorable goals, not personal agendas.
- Photo: Phil Bradley
- Presentation of data connecting with stakeholdersMaking strategic decisionsMotivating staffHow can we better use statistics and measures to tell our stories and make our case? What can we do to make the data more meaningful to our stakeholders?
- Bigger = MoreNot just a list of numbers
- I know how big a soda can is. This is a measurement I can relate to. I can connect with this data because it means something real to me.
- Compare with Library card data. Where should we focus more early literacy campaigns?Compare with results of recent Affordable Care Act info initiative. How successful were we in reaching target demographics?
- Providing context.Social media = connections (I’ve talked little about social and Web analytics)
- New York City's daily carbon dioxide emissions as one-ton spheresIllustrating your point to stakeholders. Envision something like this for circ stats instead of a number.
- Make a bold statement to get results. We recently needed to make a point to our executive board about the complexity f our ILS setup. We made a chart of system elements that covered an entire wall… point taken. Action approved. How can you make an impression on funders, legislators, community members by translating metrics into something that they can’t ignore?
- Telling that story. How can we show (not tell!) the impact of our services to those we need to sell it to. What is it that makes our library important to the community we serve?
- More storytelling – What can we tell about the mission of an organization by looking at how they represent themselves via data?
- Bottom line: Turning the outcome measurements into something that can make an impact. Using them to further our goals.
- We’ve asked a lot of questions about library metrics, and brought up some concerns. So where do we go from here?
- MORE QUESTIONS!To consider:
- If not, were do we get them?
- If not, what tools need to be developed? Can we engage in partnerships?
- How can we use it to Change the World?