authors: Randy Oldham, Alison Hitchens, Dillon Moore Presentation at ELUNA, Salt Lake City, 2012

1. PRIMO CENTRAL TRIAL,
USABILITY TESTING, AND
IMPLEMENTATION
OPTIONS
Alison Hitchens, University of Waterloo
Dillon Moore, Wilfrid Laurier University
Randy Oldham, University of Guelph

2. Agenda
 Setting up the trial (Alison)
 User testing during the trial (Randy)
 Decisions after the trial (Dillon)
 Questions?

3. TriUniversity Group (TUG)
University of Waterloo
University of Guelph
Wilfrid Laurier University
We use: Primo, SFX,
Primo Central,
Voyager, Verde, bX
Recommender

4. About our institutions
U Waterloo U of Guelph Laurier
# of Undergrad
Students
28,200 18,296 14,536
Library employee
FTE
118.5 95.4 61
ACQ Budget 6.9 Million 6.8 Million 3.1 Million

5. Expanding discovery
 There are several options for expanding the
scope of our Primo search
 Add Primo Central index
 Create a deep search API to search:
 Summon
 Ebsco
 Scholars Portal e-journals, e-books, e-data etc.
 Others…
 Harvest records directly from various places, e.g.
Hathi Trust

6. Why trial Primo Central Index?
 Large size- “encompasses tens of millions of records of
global or regional significance harvested from primary
and secondary publishers and aggregators”*
 Scholarly material- “records include a mix of scholarly
material, primarily articles and e-books but also
conference proceedings, newspaper articles and more”*
 Provider neutral- Ex Libris doesn’t own or sell access
to the content
 Interoperability- indexed using Primo software
 Reduced set-up time- don’t need to develop an API
 Free- after initial release Primo Central was free to
current Primo customers for a year
*Primo Central Index Configuration Guide

7. Why trial Primo Central Index?
 We wanted to answer some questions:
 How are local resources discovered when blended
into a much larger index?
 What is the subject coverage like? How does it match
up with our collections?
 What search options do our users prefer?
 Does adding this index enhance the discovery
experience of our users?
 Does it impact the workload of library staff?
 What does web-scale discovery mean for information
literacy?

8. Setting up the trial
 Winter 2011 - staff testing and feedback
 Summer 2011 - Launched as production trial
 Created Primo Central focused FAQ for users
 Provided staff training
 Waterloo included a search box preferences question as part
of a larger library web site survey
 The three institutions took different approaches to
activating collections, for example:
 Waterloo activated anything that was free for search
 Guelph activated collections that corresponded to their
acquisitions

9. Initial Decisions
 Primo Implementation team initially decided on
three tabs
 Web and Information Architecture cross-
functional team at Guelph
 Recommended having no tabs
 Single search box for Guelph
 3 tabs for Waterloo and Laurier
 Compare user feedback with our consortial partners

10. The Design – Waterloo / Laurier
 Waterloo and Laurier
 3 tabs: Blended, Primo only, Primo Central only
 Tabs have quick limits that can be applied

11. The Design – U of G
 U of Guelph
 Single search box
 No tabs, no quick limits

12. The Trial Launch
 Waterloo & Laurier launched as separate trial,
parallel to existing non-Primo Central Primo
view
 Guelph replaced access to its Primo-only view,
essentially forcing patrons into the PC view.
 Feedback links on the Guelph PC view sent
patrons to a quick three-question survey
 Random PRIZE: one of five $10 gift cards for
campus food services/textbooks/gifts

13. The Usability Testing &
Surveys

14. The Survey Responses
 88 responses
 Generally +ve
 staff vs users
 Number 1 +ve:
 Easy to use
 Number 1 –ve:
 How do I find a known item?

15. Immediate tweak
 Changed facet order
Before: After:

16. Usability Studies – UW
 @ U of Waterloo
 Purpose: to discover how uWaterloo students,
faculty and staff use PC, what they like, and what
they dislike.
 Came up with specific scenarios:
 “Please search for the journal article title Strategic
decision-making. Could you tell me if you find the
search results easy to understand? Why, or why not?”
 “I’d like to draw your attention to the facet ‘Expand
beyond library collections’. Before I ask you to click on
it, could you tell me what it means to you?”

17. Usability Studies - UW
 UW Participants:
 Undergraduates: 38
 Graduates: 32
 Faculty: 2
 Staff: 4
 Alumni: 1
 UW Incentive PRIZE: iPad 2
 Lots of feedback to help with labels and to get
direction. Lots of non-PC related comments.

18. Usability Studies - UG
 @ U of Guelph usability testing
 Participants completed 4 real-life tasks. Before
the tasks were assigned, users were asked
pre-test questions and after the test a number
of follow-up questions were asked.
 Use PC to find information on something that
interests you, or that you expect to be
researching in the near future.
 Take some time to look at the results of your
search. What have you found?

19. Usability Testing - UG
 UG Participants:
 3 Undergraduates
 4 Graduates
 Incentive: $10 Dollar gift card for campus
food/gifts/textbooks
 Learned a lot about GENERAL principles of
discovery vs search and user
expectations…less about the actual PC labels
etc.

20. Survey Library Staff - UW
 Sent survey to Library staff
 Asked label questions…
 Asked some task-based questions…
 Offered chance for comments/feedback

21. Survey Library Staff - UG
 Did group interview with Library staff from
Learning &Curriculum Support team and
Research Help Team
 How do you use facets/with students?
 When do you use Adv search/with students?
 How effective is PC to find what you/students
need?
 What has PC been good/lousy for
 What do you wish PC could/not do?
 Has the availability of PC changed your search
habits?

22. Label testing
 UG/WLU - Final stab at label testing for our
tabs
 Gorilla testing…

23. Gorilla Testing

24. Guerilla Testing

25. Guerilla Usability Testing

26. Incentives:
 UG – Coffee & muffin coupons
 WLU – Bowl of candy

27. Testing Complete
 All Usability testing, survey responses and
comments complete:
 UG, UW and WLU Staff survey results
 UG, UW and WLU Patron feedback
 UG and UW Usability feedback
 UG, UW and WLU Staff group interview feedback
 UG WIA feedback
 UG and WLU Guerilla label testing
 Now… it was decision time…

28. Decision Time

29. Implementation Planning
 Common across all views:
 3 search tabs
 Facet order
 Changes to views
 Addressing comments from staff survey and
usability testing
 Usability testing at Laurier to get quick
assessment
 Individual view changes
 Static HTML files

30. Implementation Plan
 Creation of duplicate views with Primo Central
and all changes implemented
 Testing by staff with testing scripts and
usability testing at Laurier
 Communication to staff
 Updated FAQs, staff guides, tutorials
 Moving all new features of duplicate views to
existing production views

31. Search Box
 Embedded search box changes
 Example Boolean search under the search
box

32. Tooltips
 Tooltips for search tabs
 Hover text gives detailed explanation of what
each tab is searching

33. Above the Results
 Top-level facets now only on left side, removed
from top of results list
 “blended” search scope renamed to match tab
name

34. Expand Results
 Entirely removed this option
 Removed using jQuery
 Potential for confusion greater than benefit
(demonstrated during staff survey)
 Not promoting Primo with Primo Central as a
complete literature review tool
 Might choose to turn it back on in the future

35. Top-level Facets
 Label changes
 Removed “Cited Articles” facet entirely
(jQuery)
 New normalization rule for “Available” to
contain available physical items in the library

36. Brief Results
 “Full text available” now “Available online”
 Changed CSS to bring attention to the
versions tabs

37. Usability Testing - Laurier
 April 11-18
 Laurier Participants
 3 graduate students
 9 undergraduate students
 Quick feedback on some of the decisions that
were made in the implementation process
 Generally positive feedback and results

38. Usability Testing - Laurier
 Opening questions on experience with Primo /
Primo Central
 Simple tasks (search, find an available book in
library, find and retrieve an article)
 Follow up questions
 What the tab names mean to you
 What Primo is searching
 What changes would improve searching in Primo

39. Still Ahead
 Laurier launch for the fall
 Accessibility testing
 Creation of ongoing usability testing plan
 Potential of Hathi Trust through Primo Central
 Assessment of usage statistics

40. Questions?
 Alison Hitchens
 ahitchen@uwaterloo.ca
 Randy Oldham
 roldham@uoguelph.ca
 Dillon Moore
 dimoore@wlu.ca