ICT role in 21st century education and it's challenges.
Discovery impact erl2014
1. Not always discovered: Phase two
of a study of the effect of discovery
systems on online (journal) usage
ER&L
March 18, 2014
Michael Levine-Clark, University of Denver
John McDonald, University of Southern California
Jason Price, SCELC Consortium
http://bit.ly/discovery-impact-erl2014
2. “…a steep increase in full text downloads
and link resolver click‐throughs suggests
Summon had a dramatic impact on user
behavior and the use of library collections
during this time period.”
The Impact ofWeb-scale Discovery on the Use of a Library
Collection
DougWay (2010)
http://scholarworks.gvsu.edu/library_sp/9/
6. What did we measure?
• Whether there is an
effect
• NOT why that effect
exists (that’s a future
study!)
7. • “Society will need to shed some of its obsession
for causality in exchange for simple correlations:
not knowing why, but only what”
• (Cukier & Mayer-Schonberger. 2013. Big data: A revolution that will
transform how we live, work, and think.)
8.
9.
10. Data collection
• List of libraries with discovery services
> Searched on lib-web-cats
• Surveyed Libraries
> Discovery service Implemented
> Implementation Date (month/year)
> Search box location
> Marketing effort
• 149 Libraries Gave Approval
> 33 libraries selected for this phase
> 6 for each of the 4 major discovery services and a
group of 9 libraries with no service
11. Dataset• 33 Libraries
– 28 US, 2 CA, 1 each from UK, AUS, NZ
– WorldCat book holdings
> Average: 1,114,193 ; Range: ~300k to ~2.6mil
• Implementation dates (Discovery Libraries):
> 2010 (3), 2011 (19), 2012 (2)
• 6 Publishers
• 9,206 Journals
• 163,545 Usable Observations
12. Methodology
Compared COUNTER JR1 total full text article views for the
12 months before vs 12 months after implementation date
June2010Start
Implementation
May2011
May2012
End
Year 1 Year 2
Included implementation month in Year 1 to ensure that
both periods included an entire academic year
18. Analyzing Usage Change: % vs Total
Use 12
months
before
Use 12
months
after
% Change
Total
Change
Journal A 500 600 20% 100
Journal B 5 15 200% 10
Which is the better measure?
Is it the same for publisher- & journal-level data?
19. Reducing variation due to institution size
Currently converting to change per FTE
Values are shown as x 1,000 to bring the change
metric back per journal-library combination to a
minimum of 0.1
2013 JISC Discovery study took a similar approach
29. Results
Can we detect differences between Discovery
Services, Publishers, and/or Libraries and/or
their interactions?
• Library – Yes
• Publisher – No
• Discovery Service – Yes
• Differential discovery service effect by
publisher – Yes
30. Next Steps
• Design & test for effects of:
– Aggregator full text availability
– Publisher Size
– Journal Subject
– Overall usage trends (Requires Disc Srvc ‘control’)
– Configuration options in Discovery services
• Expand pool of libraries
• Perhaps explore WHY
31. Sharing Data
• With participating libraries
– Customized reports for each library
• With participating publishers
– Customized reports for each publisher
– Presentations as requested
• With discovery vendors
– Presentations as requested
• In publications and presentations
– Maintaining anonymity of data