7. Powered by
Q7: In your estimation, on how many separate occasions have you used
the Summon search since September 2014?
Answered: 99 Skipped: 6
8. Powered by
Q8: When searching for information on a research topic, which
online research tools do you commonly use? (check all that apply).
Answered: 99 Skipped: 6
Other: U of T library (3) York
(1) Ryerson (1) TPL Databases
(1) LAC (1) WorldCat (2) ILL (1)
arhive.org (1) ebrary (1)
Amazon (1)
10. Powered by
Q10: How relevant do you find Summon search results to your research
topic(s)?
Answered: 97 Skipped: 8
11. Powered by
Q11: Would you like to have access to any of the following instructional
resources on using Summon? (mark all that apply)
Answered: 86 Skipped: 19
12. Powered by
Q12: Have you used the search filters, located on the left-hand side menu, to
narrow your search results? Examples of filters include Library Location, Content
Type, and Discipline.
Answered: 97 Skipped: 8
13. Powered by
Q13: How useful were the search filters in improving your search results?
Answered: 64 Skipped: 41
14. Powered by
Q14: Do you have further comments on the utility of the search filters? For
example, would it be beneficial to arrange them differently, eliminate some or add
others?
Answered: 15 Skipped: 80
Filter: Arrangement Filter: Suggestions Filter: Comments
Move, or remove library location
Library location and language facets not useful,
move down the page.
Move pub. date up
Library location not useful. Move or delete.
Library location redundant with catalogue filter
Narrowing by author if you have a last name but
not the first
Filters can be even more specific e.g.
commercial photography and fine art
photography and design photography
Spilt books/ebooks
Disciple and subject confusing, what’s the
difference? Can one be removed?
Library location expanded by default for
searching physical collection only
FT, peer review, content filters are the most
useful.
Functional as is / fine as they are
Don’t return relevant results
Great!
Huge improvement, easier to understand and
navigate
Filters are great, very user-friendly. Easy to
search by publication date (unlike U of T’s)
OCAD’s library system ‘incoherent’ hard to find
what I’m looking for.
15. Powered by
Q15: What aspect or feature of Summon do you like best?
Answered: 61 Skipped: 44
“It actually provides real-time access to research documents - which is precisely
what graduate students need. The previous system was not actually able to
provide access to digital documents for some reason - at least in my experience
- in a large number of cases. This one works very well - and actually provides
access to documents that even Google Scholar doesn't seem to be able to find.”
16. Powered by
Q15: What aspect or feature of Summon do you like best?
Relevance Ease of Use & Efficiency Facets / Refinements Access to Content Other
Very easy to use
and relevant to
my topic
How easy it is to
find one or more
relevant articles
That it can access
so much of the
catalogue without
having to narrow
the search
criteria.
Great relevancy
ranking and nice
clean, user-
friendly design.
Easy to figure out / use
Searchable information
all in one tool. Automatic
access to resources.
Quite easy to use...I get
the needed results
quickly!
Easy access to a wide
variety of research
material
It’s quick and easy to
find links to the info I
need when researching
Cuts down steps
involved in our other
database searches
After searching, the subheadings
and categories have options to
search deeper into my topic
Having the ability to sort by results
that have their full-text online is a
great advantage.
Clear tags describing type of
resource (i.e. eBook, journal article,
newspaper article, etc.)
Able to search for certain categories
that are not available through
regular library catalogue (e.g.
publisher name, language, etc.)
The faceting (and cluster-analysis) is
well implemented.
Able to search most resources at
once.
Its easy accessibility to scholarly
articles
It holds a lot of information.
It's nice to have all the search
results come up in one big
search and from there being able
to narrow it down easily.
I can search either E-Books,
Media, Catalogues, Databases
and Journals immediately
Its broad based search results
and the various sources that are
available.
Provides real-time access to
research documents
I don’t know I’ve never used it!
That it aggregates online and print sources in one place
(i.e., not having to log into e-journals separately)
Able to search most resources at once. Able to search
for certain categories that are not available through
regular library catalogue (e.g.: publisher name,
language, etc.)
Very helpful to my research
It works most of the time
The mouse-over feature that shows detailed info on
the right
Ease of searching across multiple resources at once.
I like how it is the default search engine on the OCADU
Library homepage
17. Powered by
Q16. If you could change one aspect or feature of Summon, what would it
be?
Relevance Ease of Use & Efficiency Facets /
Refinements
Access to Content Other
Searched for very
specific terms as I
knew there was a
specific journal
article I was looking
for It did not come
up on the search
Maybe important
information
highlighted:
hierarchy of
information
Needs visual
hierarchy; focal
point and clear
layout to make the
viewer easily
understand the
results.
Not clear enough what the actual
article is, whether an e-book,
journal or book review
Boolean search seems dated
Navigation and user interface
could be improved
A clearer indication of the media
of the return results, such as books
vs. online articles vs. abstracts
Lack of pagination of results, also
can’t tell where in the result set
you are with scroll
Add regional
sorting
Side buttons
on the left
are difficult
to use
Narrow by
books and
ebooks not
books/ebook
s
It doesn't actually link you to anything
Have specific types of searches within
summon
More recent sources and more downloadable
[non-drm] pdf files. I downloaded a file that
needed adobe digital editions and it only
allowed me to view it for a few days and I
have to re-download it.
Link failure or not linking to full-text article
but rather DB or journal level
Lots of duplicate results for periodical
literature, even when the source was
identical (i.e., it wasn't a case of the content
being available in various aggregations).
Lack of access to multimedia content or poor
ranking of content
That when you switched between journal, catalogue,
etc. it didn't wipe out the text in the search bar
If I click on catalogue and enter a search term -- and
then change my mind and select Summon -- the text
that I entered in the box disappears and I have to
enter it again. Is there any way to keep the text in the
box?
Make it more colorful
Summon My Account being connected with other
OCADU network (say, Canvas, My Record) and login
with our student number instead of that long serial
number on our card. And it should be easy to login in
from the beginning of the search instead of going
through multiple steps
I'd get rid of the links on the redirect page (when you
log in after clicking a search item when you're off-
campus) since the links don't work!
18. Powered by
Q17: Overall, how satisfied are you with your experience of the Summon
search?
Answered: 95 Skipped: 10
19. Powered by
Q18: How likely are you to continue to use Summon in your research?
Answered: 95 Skipped: 10
20. Powered by
Discussion: Enhancements to Summon Service?
Issue: Possible Enhancements:
Links not returning full-text; or linking
to DB or journal level records:
Summon support:
Authentication and access:
Known item searching:
Filters/Facets:
Results / access for non-textual
content:
PQ has expanded their Indexed Enhanced Direct Linking program which does not rely upon OpenURL. Majority of
content is now IEDL. In some cases, where packages are problematic like ACUP package we can load MARC
records directly into Summon and use 856 links. Intelex has been loaded and will be followed by ACUP. OpenURL
does not work well for aggregated news content (due to metadata / entitlement discrepancies and lack of full-text
subs). When checked 1 year ago news resolution link was 52%. A solution would be to exclude news content by
default and provide a toggle on the tabbed search box.
Create a guide (LibGuide/LibAnswers) explaining Summon, content is searched, search strategies for different
content types, strengths & weaknesses of the service that links to the Summon search box on the homepage, how
relevancy ranking and clustering works vs. using Boolean and wildcards. When and why Journal A– Z or DB A – Z
pages should be used. Known issues and solutions can also be included such as adding exceptions to pop-up
blockers for ArtStor content. Expand efforts to promote Summon through information literacy program.
Move EZProxy to hosted solution and implement SSO (CAS). Explore CAS option for Horizon (My Account). Add
My Account login to OCAD website “Login” page. Remove broken links on EZProxy login page.
Improvements to known item searching made by PQ in April/May release.
Remove “Library Location” now redundant due to “Library Catalogue” filter. Move publication date slider higher
in ranking. Remove either "Discipline” (HILCC) or “Subject” facet. Remove or lower “language” facet.
Build a multi-media search (filter by content type) to add to tabbed search box. Load Criterion on Demand and
NFM MARC records directly to Summon.
Notes de l'éditeur
Following upon the Library’s official launch of Summon in September 2014 a user survey of the OCAD University community was undertaken in January – February 2015. The aim of the survey was to gauge user satisfaction with the new system over the course of the fall academic term and to obtain feedback to inform improvements to Summon as well as any initiatives to strengthen user support and promote the service. Survey participants were asked between 12 – 15 open and closed-ended questions ranging from user demographic questions to satisfaction with specific features of the Summon service. Research ethics approval was sought for the survey by OCAD’s Research Ethics Board; however, the board determined that approval was not necessary and this type of survey, “quality assurance”, was out of scope for the board.
The survey was promoted in a variety of ways: Library news & social media, LibGuides, Facebook advertisements, several signs were posted throughout the Library, banner ads and survey invitation pop-ups, in information literacy sessions, and survey computer “kiosks” both in the Learning Zone and at the Library’s winter book sale. As incentives to participate candies were made available to survey participants that used the survey computer kiosks in the LZ and over the course of the book sale and survey respondents were invited to enter a draw to win one of three $50 gift cards for Above Ground Art Supplies. Of the promotional methods used, promoting the survey at the book sale was the most effective method followed by promoting the survey in information literacy sessions and kiosk set-up in the Learning Zone. $10 was spent promoting the survey on Facebook; however, it’s difficult to determine how effective this method was in generating responses or in promoting the survey. Facebook ads were targeted to an audience living in Toronto, with post-secondary education level, who listed OCAD University amongst their interests. While the FB advertisement campaign had extensive reach, displaying in the newsfeeds of 4,815 users, the click-through rate was relatively low (39) and of those click-throughs very few resulted in completed survey responses.
The survey collected a total of 105 responses (est. 2% of total population) with 96% (101) consenting and 4% (4) declining to participate in the survey. The largest number of survey responses came from undergraduate students, 65% (65), followed by library staff at 13% (13) then faculty at 9% (9). Additionally, graduate students accounted for 4% (4) of responses and staff (6) & other (3) for 9%.
4th year students made up the largest proportion of undergraduate respondents at 38%, followed by 3rd year at 23 %, 2nd at 21%, and first year at 12%. Given that the 1st years represent the largest group of undergraduates yet the lowest number of responses to the survey; other methods of promoting future surveys to this group should be considered to increase the number of responses.
Of the undergraduate students surveyed, the largest numbers of respondents are enrolled in Illustration at 22% followed by Drawing & Painting at 17%, Environmental Design 11%, and Advertising 9%.
Art 2, Design 2, Liberal arts 5
First time 33, 2 – 10 29, 11 – 25 16, 26 – 50 6, 50+ 15
Library databases were selected as the commonly most used online tool, followed by the catalogue and Google.
79% of users felt that Summon was either somewhat or very easy to use. Obviously, library staff found it the easiest of all to use with 75% saying it’s very easy – undergraduates and graduates reported more difficulty using Summon than others.
88% found the results somewhat to very relevant to their research topics. Not at all 4, not very relevant 8, somewhat relevant 63, very relevant 22
Surprisingly, a very high number of users claimed to apply the filters/facets in their searches.
Most users agreed that search filters improved their search results. Most commonly mentioned filter was the date slider to narrow down results for more current articles. Not at all useful 0, not very useful 7, somewhat useful 31, very useful 26
Comments on the positive attributes ranged from the speed, and relative ease of discovering content useful to research projects to specific features of the UI including faceted display and sorting of results.
Comments on negative features of Summon included issues with icons used to differentiate content types or print / online material. Issues with Linking were prominent in the responses – in some cases an article level link cannot be built via OpenURL, in others linking may fail entirely. Duplication of results is a problem (When FoD was first indexed I noticed that all the records appeared twice). Complaints about known-item searching.
Overall, 85% of users reported being somewhat or very satisfied with Summon. The breakdown is: Very dissatisfied 5, somewhat dissatisfied 9, somewhat satisfied 50, very satisfied 31.
87% indicted that they were somewhat or very likely to continue using Summon in their research. Very unlikely 6 (4 undergrads 1 faculty and 1 library staff member), somewhat unlikely 6, somewhat likely 30, very likely 53