Presented at the Library Assessment Conference, Charlottesville, VA, 31st of October 2012.

1. Library Assessment
Conference 2012
Chalottesville, VA.
Applying Performance
Measurement to Safeguard Budgets:
Qualitative and Quantitative Measurement of
Electronic Journal Packages.
Selena Killick
Library Quality Officer,
Cranfield University
@SelenaKillick

2. Cranfield University
• The UK's only wholly postgraduate university
focused on science, technology, engineering and
management
• One of the UK's top five research intensive
universities
• Annual turnover 2010/11 £169m
• Over 40% of our students study whilst in
employment

3. Expenditure on Journals
£
2006-07 2007-08 2008-09 2009-10 2010-11 2011-12

4. Information Expenditure
by Format 2010-11 Print Books
4% eBooks
4%
Print Journals
Databases 5%
24%
eJournals
63%

5. Information Expenditure
by Format 2010-11 Print Books
4% eBooks
4%
Print Journals
Databases 5%
24%
eJournals
25%
Big Deals
38%

6. Evaluating the Big Deals
Requirements:
• Systematic
• Sustainable
• Internal benchmarking
• Elevator pitch
• So what?
• Enable informed decision making
• Demonstrate smart procurement

7. New Approach
Quantitative: Qualitative:
• Size • Academic Liaison
• Usage • Reading Lists Review
• Coverage
• Value for Money
CONYERS, A., 2007. What do publisher usage statistics tell us? The Analysing Publisher Deal project from Evidence Base.
SCONUL Focus, no.40, pp.72-76, Available at: http://www.sconul.ac.uk/publications/newsletter/40/25.pdf.

8. Quantitative Metrics
Including:
• Average number of downloads per title
• % of titles with zero downloads
• Average cost per title
• Usage of core titles
• Cost per full-text download
• Overall
• For each core title
• Three-year trends for most popular titles
• Number of core titles in Top 30 most popular titles

9. Considerations
• When to measure from/to?
• calendar, financial/academic, or contract year?
• Which titles make up our core collection?
• Do we have access to all of the ‘zero use’ titles?
• What constitutes Low/Medium/High?
• What about the aggregator usage statistics?
• Do we trust the usage statistics?
• What is the size of the target population?

10. Subscribed Titles
For each core title the cost, downloads, and cost-per-
download categorised:
 Zero
 Low
 Medium
 High
 Cancel?

11. Systematic Analysis
• Excel Template using three main data sources:
• COUNTER JR1
• Subscription agent financial report
• Student and Staff population data
• Automated as much as possible
• Match formulas working with ISSNs to link title price
to usage/holdings
• All calculations are completed automatically when the
data sources are added
• Results fit onto a one-page printout

12. Quantitative Reporting
• Systematic 
• Sustainable 
• Internal benchmarking 
• Elevator pitch 
• So what? 
• Enable informed decision making 
• Demonstrate smart procurement 

13. Qualitative Measures:
Academic Liaison
• Who is using it?
• Why?
• How?
• Who is recommending it?
• How valuable is it?
• What will be the impact if we cancel?
• Teaching?
• Research?

14. Effective Knowledge
Management
• The Barrington Liaison Tool (BLT)
• Customer Relationship Management tool developed
to capture academic liaison
• Integrated with Enquiry Tracking system
• Web based & device generic
• Reporting on resource usage

15. Reading List Review
Analysis of course reading lists:
• What are our academic recommending?
• Where is it published?
• How often is it recommended?

16. Quantitative & Qualitative
Reporting
• Systematic 
• Sustainable 
• Internal benchmarking 
• Elevator pitch 
• So what? 
• Enable informed decision making 
• Demonstrate smart procurement 

17. Using the Results
What they can do:
• Both qualitative and quantitative measures tell the
story of the resource
• Aid decision making
• Justify procurement
• Safeguard budgets…?

18. What they can’t do:

19. Closing thoughts
• Is it worth investing in this?
• Qualitative AND Quantitative
• Danger of relying on cost-per-download

20. References
• CONYERS, A., 2010. Usage Statistics and Online Behaviour
(2). In: G. STONE, R. ANDERSON and J. FEINSTEIN, eds,
The E-Resources Management Handbook – UKSG. Burford:
UKSG. Available at:
http://uksg.metapress.com/link.asp?id=084t98646x2rn62k
• CONYERS, A., 2007. What do publisher usage statistics tell
us? The Analysing Publisher Deal project from Evidence Base.
SCONUL Focus, no.40, pp.72-76, Available at:
http://www.sconul.ac.uk/publications/newsletter/40/25.pdf.
• Lib-Stats discussion list and archive:
http://www.jiscmail.ac.uk/LIB-STATS
• TAYLOR-ROE, J. and SPENCER, C., 2005. A librarian's view
of usage metrics: through a glass darkly? Serials: The Journal
for the Serials Community, 18(2), pp. 124-131.
• Joint Usage Statistics Portal (JUSP): http://jusp.mimas.ac.uk/

21. Thank You!
Selena Killick
s.a.killick@cranfield.ac.uk
@SelenaKillick
Tel: +44(0)1793 785561