Charleston Conference
Saturday, November 6, 2010
11:15 AM - 12:00 PM
The data in our library catalogs can be recombined in many ways to tell us about our users. In this presentation, I will describe an evaluation project I did that used easy-to-access data about library holdings, circulation, and InterLibrary Loan requests to reveal user behavior from several different angles. As my data all came directly from the software we already use, my methods should be replicable without using a dedicated collection analysis product.
The collection evaluation includes two angles: course-based circulation statistics, and a comparison of ILL and circulation. In order to determine which courses most heavily use library resources, each 200-level course was assigned a call number range, so that I could look at holdings and circulation history for each course. I will discuss what these numbers told me, what they didn’t tell me, and further steps that might help me make meaning from the data. As another measure of how well we are meeting user needs, I also compared InterLibrary Loan data to circulation data to see how often users’ needs were met by books we own versus books they need to request from other libraries. I will discuss how my library plans to use this information to inform collection development in the future and will note other data that attendees could consider collecting.
Audience members will benefit from seeing concrete steps that can be replicated in almost any library. I hope that the audience will also share their own systems for evaluation.
Collection Development with Numbers by Karen Kohn, Arcadia University
1. Collection Development with Numbers:
Mining our Circulation and ILL Data to Understand
Patrons’ Needs
Karen Kohn
Collection Development Manager
Arcadia University
Glenside, PA
www.arcadia.edu
Charleston Conference
Issues in Book and Serial Acquisition
Saturday, November 6, 2010
2. Master’s Level
3156 FTE
~130,000 items, including online
~112,000 books in circulating collection
No approval plans:
Librarians and faculty responsible for all selection
3. 1. To analyze our collection as it relates to our
curriculum
2. To identify both what is being used and what is
wanted that we don’t have.
3. To do so with quantitative data
4. Usage-based
• How much of what we have gets used?
• How much of what students use do we have?
• Knievel, J. E., Wicht, H., and Connaway, L.S. (2006). Use of
circulation statistics and Interlibrary Loan data in collection
management. College & Research Libraries, 67(1), 35-49.
• Ochola, J.N. (2002). Use of circulation statistics and
Interlibrary Loan data in collection management. Collection
Management, 27(1), 1-13.
Comparison to Other Institutions
• White, H. D. (2008). Better than brief tests: coverage power
tests of collection strength. College & Research Libraries,
69(2), 155-174.
5. • How much of what we have gets used?
• Circulation Data
• Course-level analysis
• How much of what students use do we have?
• Checkouts v InterLibrary Loan
• Department-level analysis
6. How much of what we have gets used?
• For course-specific call number ranges
• How many items held
• How many of these circulated
• How many total checkouts
• Ratios
o Checkouts to books owned
o Number of books used to books owned
o Uses per book
How much of what students use do we have?
• For department-specific call number ranges
• Number of ILL requests
• Number of checkouts from our library
• Ratio of checkouts to ILL requests
7. I felt I knew what they meant, could translate the numbers into words.
I had access to the data with existing reports
8. 200 level courses
Presumed to involve more research than 100-level
Manageable number (240)
Of these 240 courses
-21 needed same resources as other courses
(ex: Photography I and Photography II)
-19 presumed not to need books
(ex: fieldwork, conversational language, physical activity)
-2 were special topics, no real course description
198 courses were assigned call numbers or ranges
9. Great Trials in History (CJ208)
KF8910-9066, F74.S1, E451, B310-318, QB36.G2, KZ1175-1179,
KF224.S3, KFM2478.8.W5, TR140.M347, KD372.W53
Modern Programming Languages (MA228)
QA76.7-QA76.73.Z2
Human Anatomy (BI205)
QM
10. 0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Does not include:
19 courses that had no books
54 courses that had books but no circulation
Health and Human Rights
Introduction to Art
Therapy
Intermediate Italian I
Feminist Theory
Script
Analysis
Digital Imaging II
History of Modern Architecture
Acting: Scene
StudySurvey of Latin-American
Literature
13. If ratio is high:
Keep buying books!
If ratio is low:
Less clear
Possibilities
Course has no research assignment
Subject lends itself to journal use
Our books are too old
Our books are off-base in some way
If no books:
Have to ask faculty their needs
15. Buy more books, or more copies of the same books?
Looked at circulation rates only for books that circulated
Divided total number of checkouts by number of books checked out
How heavily are these books being used?
17. Painting I
Printmaking II
Plant Biology
Financial Accounting
Conceptual Physics I
Video Production I
Writing Poetry and Fiction
Native American Fiction
Jacksonian America
Linear Algebra
Introduction to Italian Literature
Hispanic Experience in
Philadelphia
Spanish for Medical Careers
Symphonic Literature and
Repertoire
Developmental Psychology of the
Black Child
Psychology of Work Behavior
Script Analysis
Stage Makeup
54 courses total
18. •Seek more books for courses with high checkout rate
•Buy similar books to those with high use
•Weed ranges with very low % of books used
•Where no checkouts or no books, speak with faculty
19. Only collected 1 year. Not all courses were offered!
Would need more years of non-usage to justify weeding.
Had only rudimentary guesses of which courses required library
research.
Some courses might be using journals rather than books.
Do not have a record of in-house use.
20. How much of student needs are being met by our collection?
What do students need that we don’t have?
Could not break this down by courses because not enough requests
2,802 ILL requests for books v 10,198 checkouts from our stacks
21. Anthropology GN, GT
Math/Computer Science QA
Art and Design N, TR, TT Modern Language PA-PD, PF-PM, PQ,
PT
Biology
QH-QM, QP501-end,
RC212-RC320, RC571-
RF, TD
Music M
Business HB-HJ
Peace and Conflict
Resolution
JZ4835-end of JZ,
KZ
Chemistry/Physics QB-QD, TA, TK Physical Therapy RM695-893
Criminal Justice
HV8079-end of HV,
HV6001-8069 Political Science J-JZ3875, K-KW
Education L Psychology
BF, QP351-495,
RC321-571, RJ499-
507
English P, PE, PN, PR, PS Public Health
QP-QP345, RA-
RA1000
Forensic Science
HV8073, RA1001-1171,
RS189-190, RS441,
TH9025-9745 Religion/Philosophy B-BD, BH-BX
History C-F Sociology HM-HV5840
22. Department Call
Numbers
ILL
Requests
Checkou
ts
Percentage of
Book Needs
Met By Us
Anthropology GN, GT 27 59 68.60%
Art and Design N, TR, TT 99 1341 93.13%
Biology
QH-QM, QP501-end, RC212-
RC320, RC571- RF, TD 28 124 81.58%
Business HB-HJ 131 308 70.16%
Chemistry/Physics QB-QD, TA, TK 12 82 87.23%
Criminal Justice
HV8079-end of HV, HV6001-
8069 15 90 85.71%
Education L 112 455 80.25%
English P, PE, PN, PR, PS 51 158 75.60%
Forensic Science
HV8073, RA1001-1171,
RS189-190, RS441, TH9025-
9745 8 11 57.89%
History C-F 397 1805 81.97%
Math/Computer Science QA 31 168 84.42%
Modern Languages PA-PD 34 133 79.64%
Music M 26 144 84.71%
Peace and Conflict
Resolution JZ4835-end of JZ 11 65 85.53%
Physical Therapy RM695-893 2 13 86.67%
Political Science J-JZ3875 73 300 80.43%
Psychology BF 77 364 82.54%
Public Health QP-QP345 6 35 85.37%
Religion/Philosphy B-BD 45 123 73.21%
Sociology/Anthropology HM-HV5840 191 736 79.40%
24. Patrons may not be requesting everything they need
• Would skew satisfaction rate up
Some books requested may not fall within our Collection Plan
(e.g., textbooks, leisure reading)
• Would skew satisfaction rate down, mostly in the Ps.
25. Look at what books and topics were requested
Make recommendations for purchasing
26. What’s wrong with the books that aren’t circulating?
• Could not operationalize this numerically
Are books selected by librarians circulating more than
books selected by faculty?
• We do keep track of who selected, but harder than expected to
decide what counts as librarian-selected.
• Ex: I make a list and faculty member says please order # 1,5,7, and 9.
• Ex: Faculty member orders literary criticism, librarian realizes we
don’t have the literary work and orders it.
• Ex: Librarian submits the order but does not specify where it
originated.
27. Actions
Librarians have been looking at ILL data, approaching faculty
Will bring course-based data to faculty soon, possibly in the Spring
Future Analysis
Repeat analysis for 300-level courses
Look at 5 years of usage data
Hi.. I’m going to talk about a project I did to evaluate my library’s collection using usage data. Although the title I proposed emphasizes how numerical the project is, I think what’s distinctive about it is how closely the evaluation was tied to our curriculum. Also, really focused on usage rather than comparing us to other libraries or selection lists. Usage or patron needs and curriculum are two most important things driving collection development.
Why no approval plan – culture of faculty expecting to know about purchases ahead of time. For a long time, faculty were doing all the ordering. They like that librarians are involved, but would not want the task removed from the hands of people they know. What this means for us is we have room to do very targeted collection development.
These are my broad goals. (read them)
This evaluation project was distinctive in that it looked at the collection at the course level. Concerns about using analysis tools, they don’t know what you need, only what you have.
This goal looks at both sides of the coin – what are we doing well, so we can keep doing it, and how can we improve. This is based on ILL as indicator of what would they like us to have that we don’t.
There are a few studies that use quantitative evaluations, but most of what I’d been taught was vague, like “look at what people are using.”
Was not a lot of literature on this topic. I divided the studies into two main types, and within the first type there are two ways of looking at it. I phrased them similarly to show that they are complementary. These two are the questions I used for my study. Did not do Comparison, as I’ll explain later.
Knievel et al: Transactions per item, % of a given collection that circulated, Ratio of ILL requests to holdings, 25 subject categories, mapped to LCC numbers
John N. Ochola, % of collection compared to % of circulation, % of ILL requests to % of holdings, LC subclass (i.e., two letters)
(yes, these articles really have the same title)
White uses WorldCat to search by class number, to see what percentage of the titles his library owned. Compared by how many other libraries had it; e.g. should have a higher percentage of the titles that most libraries have than the titles that few libraries have. The scale would not have worked well for us, as our percentages would be very low and we’d be comparing ourselves to much larger institutions.
Same questions, broadly, that the others were answering, but I measured slightly differently and looked at it at course level. Keep relaying back to these 2 questions.
First list is for the first question, about what’s being used.
Explain ratios: If 8 books, and two were used, but total checkouts was 8. Each of these told me different things, which I will go over.
Usage data only include what circulates; we don’t have a way of keeping in-house use statistics.
Second list is about overall needs and how much of the need we’re fulfilling.
Did not do ILLs by course because there weren’t enough of them to map something to every course. (2802 book requests in 2009-10). For each department, they used, say 6 books in our library for every 1 they requested on ILL.
This was a pilot.
Some have a big range, others small, others a full class number or several.
(Socrates, Scopes Monkey Trials, Nuremberg Trials)
Variation is okay because I wasn’t looking to see if we had “enough” books, based on a set definition of “enough.” Enough is all defined by usage, which is a ratio, so that allowed me to compare apples to apples.
Didn’t label all the dots, hard to read. I took out the zeroes here. Those are important to look at, just removed them for visual purposes.
Give several examples, keep repeating “this means lots of usage, this means low usage”
Renewals are included.
Intro to Art Therapy: 14 books, 14 checkouts
History of Modern Architecture: 27 books, 1 checkout
Digital Imaging II: 18 books, 13 checkouts
Intermediate Italian I: 13 books, 13 checkouts
Feminist Theory: 14 books, 12 checkouts
Health and Human Rights: 4 books, 8 checkouts so doesn’t necessarily mean our collection is great, but means do more of the same
Script Analysis: 14 books, 11 checkouts
Acting: Scene Study: 70 books, 1 checkout!
Keep repeating “Crime Scene Investigation had a high number of checkouts compared to the size of the collection” etc.
(don’t say years unless asked)
Psychology of Work Behavior has zero uses, 6 books, average year 1984. Adolescence has 24 books, 1 use, average year 1977. Forensic Psychology has 4 books, 2 uses, average year 2002.
Some courses with high ratios had very few books and could use more. Others may have a lot of books, but whatever is being used, we could stand to buy more of.
If low ratio – I do have an average year, but it’s an average, doesn’t tell me a cutoff for when a book gets too old to be used. We could weed and/or talk to faculty about their needs.
I felt like these data were useful, but was unsure about possible distortion from lots of uses of one book. Instead of using total number of checkouts, uses number of books checked out. (give examples) This was for comparison, to see if specific books were being used a lot, and also because it tells us where to weed.
*Intro to Art Therapy: 14 books, 10 were borrowed (14x)
Renaissance Art: 57 books, 2 were borrowed (2x)
Modern Africa: 18 books, 5 were used (5x)
Health Psychology: 8 books, 5 used (5x)
Forensic Psychology: 4 books, 2 borrowed (2x)
Feminist Theory: 14 books, 8 were borrowed (12x)
**Health and Human Rights: 4 books, 2 were used (8x)
Shows where we need to do weeding. Using the 80/20 rule, we’d weed whenever less than 20% of collection is used, but this is conservative.
Courses with zero usage are not in the chart. Don’t weed these, but could try to look for newer books to buy.
I wondered about this…
This is in order of highest ratio. Highest ratio I found was 3, meaning maybe 2 books were used 6 times, or one book was used 3 times… You can see the vast majority are less than 1.5. This means no particular book is getting really heavy usage, should be sufficient to look at previous numbers (circ ratio) to see where we need to keep collecting diligently. If ratio was 1.5 or more, I looked to see if particular books were getting very heavy use. No book (in the ranges I looked at) had >4 checkouts in the last year, which means we probably don’t need more copies of the same book, but more similar books.
54 courses. Not terribly concerned that Stage Makeup is not using books, but Hispanic Experience in Philadelphia or Native American Fiction we should contact faculty. Take a look at whether our books are old, how many we have, ask if they have assignments that involve using the library, would they like us to look for books for them.
Again, this was a pilot. Some of these limitations can be corrected and some can’t.
So, that’s how are we doing so far with what we have. Remember, my two guiding questions were “what gets used” and “how much of what they use do we have”. I answered first question with circulation data, ILL will get at the second. At this stage I expanded the question to 2 parts.
Another way to do this would be with student bibliographies, which I did not do.
Still varies in how broad the range is. Some departments fit neatly into disciplines, like Education or Music, and get a single letter. Others are more profession-based, like Forensic Science, and may cross disciplines. In a few cases we have multiple disciplines in the same department.
dates used in ILL were June 09-May 10. locations for checkouts: stx, curric, media/closed stk, reserves, oversize, McN (no curric-fic)
Run through an example: for English, students used 209 books total, 75% of these came from our library.
PoliSci, used 373 books total, 80% came from our library.
Forensic Science, used only 19 books, 11 from us.
Same thing graphically. Mostly pretty high.
We did this for all departments, regardless of satisfaction rate. Started here because this data made the most sense to librarians, this is what jumped out at them when I presented the data. Also it felt more manageable.
(I wanted to compare average age with how many books, but can’t show correlation between hard number and average.)
Wanted some way to show what was wrong with books that weren’t circulating, but I couldn’t operationalize it.
Also wanted to compare whether librarian requests circulated more than faculty requests, but harder to distinguish than I expected.
Did not do any comparisons to other libraries because White’s approach didn’t work technologically, and with a selection-list approach I kept wanting to buy the books.
Mention White’s use of WorldCat to do a comparison with other libraries.
A lot of action steps, so not ready to repeat the analysis unless we’ve made more progress on the actions.
Felt like a pilot, would find it more useful to have 5 years of data, then do analysis infrequently