Recommandé
Contenu connexe
Similaire à Oh benchmarking talk
Similaire à Oh benchmarking talk (20)
Plus de Schlow Centre Region Library
Dernier
Dernier (20)
Oh benchmarking talk
- 1. By the Numbers: Benchmarking Your Community and Library
- 6. State College, Pennsylvania Ranked #1 America's Best and Worst Cities for Crime (Smallest Metro Areas) Ranked #2 Most Secure Places to Live 2011 (Small Towns) Ranked #31 2005 Best Places to Live Ranked #45 Best Green Cities
- 9. YOUR PEER LIBRARIES • Service area population • Total operating expenditures • Number of outlets • Ethnicity • Poverty level • Education level • Average income
- 12. Library Benchmarks • LJ Index of Public Library Service • Hennen’s American Public Library Ratings (HAPLR) • Public Library Data Service (PLDS) • National Center for Education Statistics (NCES) • Institute of Museum and Library Services (IMLS) • Urban Libraries Council Technology Benchmarks • US IMPACT Survey (IMLS, Gates, U-WA) • State and consortium reports
- 13. PER CAPITA Visits Circulation Program Attendance Public Internet Use
- 15. HENNEN WEIGHTS Expenditure Per Capita 3 Cost Per Circ 3 Visits Per Capita 3 Materials % of Budget 2 Collection Turnover 2 Circ Per FTE Staff Hour 2 Circ Per Capita 2 Circ Per Hour 2 Reference Per Capita 2 Materials $ Per Capita 2 FTE staff Per 1000 Pop 2 Periodicals Per 1000 Pop 1 Volumes Per Capita 1 Visits Per Hour 1 Circ Per Visit 1
- 17. Print - $135 Online - $200
- 19. Develop a library strategic plan that achieves a dynamic school library program and includes a mission and vision, measurable goals, operational management procedures, instructional strategies and an evaluation component.
- 22. LIBRARY POP SCORE %ILE Hotwings Mem PL 25K 559 58% Beefstock Mem PL 25K 464 43% Bacon Mem PL 25K 444 40%
- 23. LIBRARY POP SCORE %ILE Hotwings Mem PL 25K 559 58% Beefstock Mem PL 25K 464 43% Bacon Mem PL 25K 444 40% 2 week loan period, 2 renewals = 3 circs for 6 weeks 3 week loan period, 1 renewal = 2 circs for 6 weeks
- 24. PUBLIC LIBRARY SUCCESS STORIES Lower poverty Higher education $$$$ Large collection More outlets
- 26. Large Suburban County Library 30% fewer staff < 2 books per capita Less expenditures per capita Highest program attendance
- 27. Urban Library Peers better funded Lowest education and income demographic Highest diversity demographic Peer had more visible marketing
- 28. School Library Test scores Economic impact Advocates
- 30. DIY •Focus on important #s •Schedule annually •Get help and resources •Find data partners
- 31. By the Numbers: Benchmarking Your Community and Library
Notes de l'éditeur
- How do you know if your community and its library is good, or great? How can government and libraries measure effectiveness? What does success look like? One tool for measuring performing is the process of benchmarking. Today, you’ll get an introductory look to comparison tools available for local governments and libraries. Comparisons are just one way to assess community and library performance, in addition to gathering customer feedback, analyzing long-term data, and other measurement activities. How many of you look at Hennen or LJ ratings? Do ICMA or IMLS benchmarks? Other comparisons? Benchmarking tends to focus on what you have done in the past, and compares it to what others have done in the past. It doesn’t indicate quality, or a lot about your users. It measures outputs, such as children’s program attendance, but doesn’t fully measure outcomess, such as the impact of Summer Reading Program attendance on reading scores. So why do it?
- This is why. When I became the director of my library two years ago, my initial staff interviews showed a lot of exhaustion. State aid cuts in PA had required my predecessor to cut 2 FTEs the year before at a time when library use was at an all time high. Benchmarking with peers allowed me to show elected officials our staffing compared to other libraries – very low – and resulted in some small staffing gains. That is an example of what benchmarking can do for you. If you, too, are understaffed, I understand the resistance to doing comparisons. But it isn’t a major project. Keep it simple, stick to comparing key indicators that your funders and stakeholders and decision-makes are interested in. For example: we care more about the quality and nature of reference questions, not the numbers. Gadget training is ok. BENEFITS: Find peer libraries for deeper benchmarking, best practices, and information exchanges Neutral evidence for advocacy, grants – elected officials respond to comparisons Accountability for public and private dollars given to the library Set targets for improvement Marketing ---Particularly large impact when library board or elected officials don’t know excellence; are complacent or self-satisfied; or library has poor self-image
- What can benchmarking do for you? Here’s another example. One library we worked with was doing a strategic plan and had a vague impression that its bookmobile was underperforming. They had entrenched outreach staff, no changes or excitement, little data on performance. Found peers with bkms and benchmarked and collected data for the first time ever. The comparison found their bookmobile was underperforming; low circ, high cost per circ compared to peers. Their new plan addressed the issues. Then they looked at community data and reports to see if they could put send the bookmobile to areas of growth, or special needs. This is how numbers can contribute to improvement in your operations.
- Community benchmarks First step is to find comparables. Find “sister cities”: commonalities of population, setting (tourist towns, college towns), demographics. Ask government officials Explore Census data Get involved with peer community activities “ Sister Cities International” advice “…carefully explore your purpose for building a sister city relationship. Is it to investigate economic development opportunities? Is it for youth exchange? Questions such as these are extremely important and will assist you in identifying the goals of your partnership.”
- Community ratings – see Sperling website and other sources, i.e. magazines. Cons Methodology unknown or unscientific Community pride or dismay Pros Potential for peers Perceptions, how your community seen by outside world
- Community Benchmarks and Comparisons ICMA – International City/County Management Association is big on benchmarking government services – “Center for Performance Measurement.” They benchmark libraries, too. Digitally Inclusive Communities ICMA partnering with Gates Foundation on library public computing data. To help community leaders make strategic decisions about technology investments, they are working with IMLS and the University of Washington Information School to develop tools to help communities assess their current capabilities, work across institutional boundaries and create plans, using a common set of principles, that would help business, anchor institutions (schools, hospitals, libraries), public safety and cultural institutions work together to fully and effectively realize the value of a digitally inclusive community. Studies show early tech adopter libraries – more use, $. Your local government staff may be active and participate in these benchmarks. Good source of performance data for local government. Measure financials, public works expenditures, capital improvements, etc. They collect various kinds of data including results of citizen feedback surveys that can be used for comparison. Can be pricey to belong – partner with your municipality.
- What’s the relevance of community benchmarking to the library? In State College: Knight: Social offerings and aesthetics are important. Endorsed a free hot dog giveaway outside on party weekends. Aesthetics important – prioritize upkeep of 6 year old destination building. ICMA showed we not used a great deal by borough residents – mostly students -- but are well-appreciated and supported. How could there be this disconnect? Qualitative data (observations) of their use of DVDs, Wi-Fi, and international families use Family Place. “ Best place to live for retirees” – started delivery and pickup service to retirement communities. Considering responses to these findings in our next strategic plan.
- Finding peer libraries is a critical step in benchmarking and evaluation. They provide opportunities for sharing, learning, exchanges of best practices. Schools and academics might want to look at enrollment instead of population, some kind of budget figure, and number of outlets is important if you are looking at the entire campus or school system, otherwise you should look to other institutional markers of your context. The libraries should ideally be within about 20% of your stats in these areas to be similar. Library of Norman, Arkansas, not operating but on National Register of Historic Places, originally part of the old waterworks building. Shed is in Williamsburg, Brooklyn and phone booth in Belbroughton, UK
- There is a compare academic libraries on the NCES website. The academics can find peers using this feature. Medical Library Association as 2007/2008 Health Science Library Stats and Benchmarking but must be member to access. School libraries – cannot find a peer identification database, must be tools at the school level??? Again, academics, schools and specials may want to look at peers their institution already identifies as peers.
- Benchmarking: Lots of data out there.
- The LJ Index, identifies “America’s Star Libraries.” Authored by librarian/statistician Ray Lyons and Keith Curry Lance, groups libraries by total operating expenditures instead of population. It uses these 4 per capita outputs from IMLS data weighed equally : circulation, program attendance, visits, and public computer use. Critiques went back and forth between authors of the 2 systems. What do you think of your library being ranked by only four variables? Which system is best? The LJ authors state emphatically: “this should be one among several sources of information…decide how to incorporate it into a more comprehensive assessment process.” Or, as I said before, use them to start you on the benchmarking path. The ratings will prompt questions and concerns and drive research that may help improve library performance.
- Hennen and LJ trends show NY and OH having many highly rated libraries. Here is the LJ map from the Nov. 1, 2011 article by Keieth Curry Lance and Ray Lyons. Sparks the question- what are we doing wrong in PA? We have relatively high state funding for public libraries which is eroding annually, and poor local funding, you can see the effect that has on us.
- Hennen’s American Public Library Ratings are administered by Thomas Hennen, a Wisconsin library director who first published his formulas and ratings in 1999 in American Libraries . He was an independent researcher looking for some way to compare and benchmark libraries in a meaningful manner. Hennen uses the data you submit in your state annual report, that is forwarded to the Federal Government. It is now received and processed by the federal Institute for Museum and Library Services (IMLS) and the data is published, unfortunately, 1-2 years after you submit it, much like census data takes time to be processed. Hennen groups libraries by population served and then looks at 15 inputs, such as funding and staffing, and outputs such as circulation and visits per capita that have varying weights in the formula, producing a final numerical score. You can find the scores of libraries by state on Hennen’s website. He also, for a fee, provides a customized report comparing your libraries to others with some additional data not found on the website.
- Hennen and LJ scores group libraries into large cadres which limit their value. The rating systems both have generated much debate. High-ranked libraries celebrate their success and low-ranking ones agonize over the reasons behind their numbers. American Libraries ’ observed a few years ago that they “have never fielded as many media inquiries over an AL article as the Hennen ratings.” So what should we make of these systems? We’ll take a closer look at them, but our advice is: despite their shortcomings, use them to start you on the benchmarking path. The results will prompt questions and concerns and drive research that may help improve library performance.
- Public Library Association Statistical Report The Public Library Association conducts an annual survey and publishes results roughly a year earlier than IMLS for more timely comparisons. Published annually, the PLDS Statistical Report collects information on finances, library resources, annual use figures and technology from more than 800 US/Canada public libraries who voluntarily participate. BONUS: Each year there is a special, different add-on survey highlighting one service area like children’s programs, salaries, etc. It costs $135.00 for the print version, but for $200/year you can have online access which is much, much, more useful. Online database users can view all tables, export them into Excel, compose graphs, etc. “ Counting Opinions” just took over from U-IL, new look and functions.
- Academics and Schools can use NCES website. The Association of Research Libraries uses a service called LibQUAL+ for collecting and interpreting user feedback and benchmarking with peers. Fee of $3,200.
- OH Dept. of Education had nice document for school libraries about benchmarks but no data connected to it. Your benchmarks are complicated because you must work with educators to first define college or school learning outcomes, and then benchmark the library activities that affect those outcomes.
- If Hennen and LJ have whetted your appetite for more comparison data, turn to IMLS, source of their data. Libraries annually report their output measures to their state libraries, and they pass the information on to the federal Institute of Museum and Library Services (IMLS). After processing the data, it is posted online for anyone to use. The most recent data available is from fiscal year 2009. Some states, like NJ, have more recent info online for their own states. Better yet, IMLS has its own comparison tool online which provides a more detailed and useful benchmark than the 2 previous ratings.
- Here’s an article from a 2007 posting on a blog, “The Well Dressed Librarian.” This is the part where I tell you DON”T PANIC about your numbers. Comparing stats with peer libraries allows you to flag any extreme performance, and then you have to dig deeper to get the reason behind the numbers.
- Case study of how Hennen sparks question: Here are 3 public libraries in a midwestern state that serve populations of 25,000 to 49,999. Bacon Memorial is really puzzled as to their low scores. They dug into the statistics. Hotwings had much higher circulation but a similar materials budget and collection size. What was causing the high circ? Better books? Large building and shelf capacity? What can it be? The answer was: different loan periods. Bacon could adjust circ period or – lower the status of circ as an important indicator.
- These qualities of LJ and Hennen high ranked libraries are related – not necessarily a cause and effect, but many “winners” have 3 or more qualities. If you put a well-managed library in a new setting what different outcomes would you see? If you lack these qualities – is it possible to have high scores? And what score or ranking indicates success? We have no accepted benchmarks except the 12% materials and 70% personnel guidelines. No answers yet.
- Hennen, LJ, and other benchmarks don’t, and can’t, measure customer service, leadership, management, collection quality and other factors that play a role in a library’s community success. The proof of that is that winners of the Bill & Melinda Gates Foundation and Library Journal “ The Best Small Library in America ” often have ratings in the bottom half. MILANOF-SCHOCK LIBRARY MOUNT JOY 10 K 454 41% So why look at them? It’s a rough “place-marker” as where a library stands among others, Lyons - “I think of them as a kind of filter…we do it as an advocacy tool and predictor of potential or additional success.” Leads to exploration of values, additional data.
- Large suburban county library with 18 outlets. Culturally diverse, mixed income and education and good Hennen and LJ scores. Bond approved for capital projects – branch construction and renovation. But the economic downturn threatened operating income to staff the new or enlarged branches. How could they build their case for more funding in a very competitive county government budget process? IMLS data showed: 30% less staff than peers. Fewer than 2 books per capita, lower than peers, which affect circ. Expenditures per capita were less than peer libraries. Didn’t realize their program attendance was extraordinary and something to brag about. They had many famous author visits which put them over the top. Numbers allowed them to: cheerleaders for staff productivity; have neutral objectives; presented funding comparison to county commissioners; trustees had some 1-on-1 meetings with elected officials and released data to the press. Commissioners funded additional staff positions. Library commentary became more objective because it was backed by data and true descriptions of other libraries.
- Urban library case study: LJ and Hennen scores in top quarter, but not “stars.” In addition to these attributes: Surprises: Program attendance for library outstanding and was not promoted enough; one of the top in the country and they didn’t realize or celebrate it. Library had fewer service hours than peers, which reduces circulation and other outputs. Helped to light a fire under a complacent board.
- There are other things to measure such as economic impact, ROI, customer satisfaction, that can help explain/expand benchmarks and define goals. Combine with other processes: High staff turnover – benchmarking showed low personnel expenditures, but exit interviews needed to determine extent of bad work environment or management. Balanced scorecard – how can we improve? Decide what the goal is – to get in top half, above mean/median of peers?
- When you produce a report, don’t snow them with tons of data.