McDonald, John, Jason Price, and Michael Levine-Clark, “Discovery or Displacement: A Large Scale Longitudinal Study of the Effects of Discovery Platforms on Online Journal Usage,” Plenary. UKSG Annual Conference, Harrogate, U.K., April 16, 2014.

1. Discovery or Displacement?
A Large-Scale Longitudinal Study of the
Effect of Discovery Systems on Online
Journal Usage
UKSG
Harrogate
April 16, 2014
Michael Levine-Clark, University of Denver
John McDonald, University of Southern California
Jason Price, SCELC Consortium
http://tinyurl.com/pg686r4

2. Does implementation of a discovery
service impact usage of publisher-
hosted journal content?

3. Publisher-hosted journals are
only part of the picture
eBooks, pBooks, aggregator journal content,
etc.
publisher journal content
The six publishers in this study

4. What did we measure?
• Whether there is an
effect
• NOT why that effect
exists (that’s a future
study!)

7. Data collection
• List of libraries with discovery services
> Searched on lib-web-cats
• Surveyed Libraries
> Discovery service Implemented
> Implementation Date (month/year)
> Search box location
> Marketing effort
• 149 Libraries Gave Approval
> 33 libraries selected for this phase
> 6 for each of the 4 major discovery services and a
group of 9 libraries with no service

8. Dataset• 33 Libraries
– 28 US, 2 CA, 1 each from UK, AUS, NZ
– WorldCat book holdings
> Average: 1,114,193 ; Range: ~300k to ~2.6mil
• Implementation dates (Discovery Libraries):
> 2010 (3), 2011 (19), 2012 (2)
• 6 Publishers
• 9,206 Journals
• 163,545 Usable Observations

9. Methodology
Compared COUNTER JR1 total full text article views for the
12 months before vs 12 months after implementation date
June2010Start
Implementation
May2011
May2012
End
Year 1 Year 2
Included implementation month in Year 1 to ensure that
both periods included an entire academic year

10. Observations by Publisher

11. Journals by Library & Service
5327
3262
5328
6294
4384

13. Average Usage Change By Discovery & Publisher
Per Journal &
Per 10,000 FTE

15. Examine Data for Outliers

16. Full Model
Including Discovery Service, Publisher, and Library
Including Discovery Service, Publisher, and Library

17. Testable Effects
• Discovery Tool
– Implemented by multiple libraries
– Used to find content from all publishers
• Publisher
– Accessible in all discovery tools
– Accessible across multiple libraries
• Library
– Uses content from multiple publishers
– Uses only one discovery tool (so only within DT)

18. Nested ANOVA Model
[all three factors – preliminary results]

19. Does usage change vary across libraries?
Institution (sorted by Mean Change)

20. Does usage change vary across libraries using the same service?
Library 10-15 Library 16-21 Library 22-27 Library 28-33Library 1-9

21. Does usage change vary across publishers?
Publisher (sorted by Mean Change)
B
AAA A A

22. Does usage change vary across discovery services?
A
BB
C
D

23. Publisher
Does the effect of discovery service differ across publishers?

24. Results
Can we detect differences between Discovery
Services, Publishers, and/or Libraries and/or
their interactions?
• Library – Yes
• Publisher – No
• Discovery Service – Yes
• Differential discovery service effect by
publisher – Yes

25. Next Steps
• Design & test for effects of:
– Aggregator full text availability
– Publisher Size
– Journal Subject
– Overall usage trends
– Configuration options in Discovery services
• Expand pool of libraries
• Perhaps explore WHY

26. Past Presentations
• Ithaka Sustainable Scholarship Conference (October 2013)
• Charleston Conference (November 2013)
• ER&L/Library Journal Webinar (December 2013)
• Shangai Jiao Tong Univ / Beijing Univ (Jan 2014)
• SCELC Colloquium (March 2014)
• ER&L (March 2014)
• Presentations posted on slideshare :
– http://visualcv.com/lpq4t1s
– http://tinyurl.com/pg686r4
michael.levine-clark@du.edu | johndmcd@usc.edu | jason@scelc.org