Canadian Library Association Annual Conference, June 2, 2012, Ottawa

2. Lorie Kloda, MLIS, PhD(c), McGill University
Denise Koufogiannakis, MA, MLIS, PhD(c), University of Alberta
Alison Brettle, MLIS, PhD, University of Salford
Canadian Library Association Annual Conference, Ottawa, 2012

4. Structured abstract
objective – design – setting – subjects –
method – main results – conclusion
Commentary
 300-400 words
 appraisal of validity, reliability, applicability
 significance, implications for practice
4

5. 5

6. To investigate the impact of evidence
summaries on library and information
professionals and their practice
Knowledge
Practice
Users

7. Phase 1
Development and face-validation of tool
Phase 2
Survey questionnaire to readers (QUANT)
Phase 3
Interviews (QUAL)

8. Development of a tool to assess impact

9. Findings (Phase 1)
Development of
Impact
Assessment
Method
Grad, R., Pluye, P., &
Beauchamp, M.-E.
(2007). Validation of
a method to assess
the clinical impact of
electronic
knowledge
resources, e-Service
Journal, 5(2), 113-
135.
http://iam2009.pbworks.com/

10. Survey questionnaire to readers of evidence summaries

11. Findings (Phase 2)
Survey
175 153 emailed 101
Respondents recruited IAM survey respondents
1 unusable 15
3 bounced
email incomplete
86
21 duplicates 49 no reply completed
IAM survey
153
remaining
101 total
respondents n=62

12. Findings (Phase 2)
USA
Country
Canada
UK
Australia
USA Ireland
Australia Finland
Malaysia
UK Saudi Arabia
Iran
Puerto Rico
Spain
Canada
Brazil
Hong Kong

13. Findings (Phase 2)
50+
25+
Number of 15
Evidence 14
13
Summaries 12
11
Read in Past 10
9
Year 8
7
6
5
4
3
2
1
0
0 2 4 6 8 10
No. of respondents

14.  Decline in Reference Transactions with Few Questions Referred to
Librarian when the Reference Desk is Staffed by a Paraprofessional (8)
 The Presence of Web 2.0 Applications Is Associated with the Overall
Service Quality of Library Websites (6)
 Google Scholar Out-Performs Many Subscription Databases when
Keyword Searching (4)
 Statistical Measures Alone Cannot Determine Which Database
(BNI, CINAHL, MEDLINE, or EMBASE) Is the Most Useful for Searching
Undergraduate Nursing Topic (4)
 A Graduate Degree in Library or Information Science Is Required, but not
Sufficient, to Enter the Profession (3)

15. Findings (Phase 2)
Reason for Freq. %
Reading For general interest or curiosity 15 24%
Evidence
Summary For personal continuing 18 29%
professional education
n=62 To answer a specific question 21 34%
or address a specific issue in
my practice
Other 8 13%

16. Findings (Phase 2)
When the > 1 year
Evidence 6-12 months
Summary was
3-6 months
Read
1-3 months
1 week-1 month
1-7 days
Today
0 5 10 15 20 25

17. Findings (Phase 2)
Freq. %
Cognitive
My practice was (will be) improved 11 13%
Impact I learned something new 36 42%
I recalled something I already knew 14 16%
It prompted me to investigate more 23 27%
It confirmed I did (I am doing) the right thing 17 20%
I was reassured 13 15%
I was dissatisfied: There is a problem with the 1 1%
presentation of this evidence summary
I was dissatisfied: I disagree with the content 0 0%
of this evidence summary
It is potentially harmful 0 0%
Other 9 10%

18. Findings (Phase 2)
Practice Freq. %
Impact Change my service approach 5 6%
“You reported: Change my approach to collections 1 1%
My practice was
(will be) Change my management approach 4 5%
improved. What
Change my approach to teaching 4 5%
did you (will you)
do differently Change my professional approach 4 5%
after reading the
Evidence Other 3 3%
Summary?”

19. Findings (Phase 2)
Community  None
Impact
 Hypothesized future/potential
“If reading this
Evidence Summary impacts on users
resulted in some  Reinforced cognitive or practice
change to your
individual practice, do
you think it led to an
impacts, not user outcomes
impact on anyone  5 reported actual impact at this
within the community
you serve or level:
environment in
which you work?  Change in teaching LIS students
Please explain in the
comment box.”  Observed (anecdotal) changes

20. Interviews with subsample of survey respondents

21. Findings (Phase 3)
13 participants Subsample from survey
respondent
Various settings, education
levels, countries, and professional
positions

22. Proposed Impacts Suggested Confirmed
Phase 1 Phase 2 Phase 3
C1 Practice was improved ? ✗
C2 Learned something new ✔ ✔
C3 Recalled something ✔ ✓
C4 Prompted to investigate ✔ ✔
C5 Confirmed ✔ ✓
C6 Reassured ✔ ?
C7 Dissatisfied – presentation ? 0
C8 Dissatisfied - content 0 0
C9 Potentially harmful 0 0
P1 Service ✔ ✔
P2 Collections ✔ ✓
P3 Management ✔ ✓
P4 Teaching ✔ ✓
P5 Professional practice ✔ ✓
U User Community ? ?

23. Findings (Phase 3)
Potential Discovery
 New research
Impacts  Interesting topics
 Methods
Uncovered  Keeping current
Sharing
 With colleagues, managers
 Report writing
 Recommended reading
Assistance
 Research
 Writing, presentations
 Teaching (for professors)

24. Findings (Phase 3)
“Likes” Format of evidence summaries
 Descriptive
 Accuracy of abstract
 Concise
Time saving
Database of Evidence Summaries

25.  One evidence summary assessed per
respondent
 Cognitive impact comparable to findings in
Grad, Pluye, et al. (2006)
 Practice impact – two-tiered
 Low community impact

26. How to best assess impact of reading about
research?
Do the items in the tool resonate with you?
How important is documenting the impact of
research on practice?

27. Revise Impact Assessment Tool
 Include new or revised impact items
 Improve wording of items
Conduct survey of a larger sample of evidence
summary readers
Adapt tool for other forms of research
dissemination

28. Canadian Association of Research Libraries
Research in Librarianship Grant
Roland Grad and Pierre Pluye, McGill University,
for their feedback
All of our survey respondents and interview
participants

29. denise.koufogiannakis@ualberta.ca
lorie.kloda@mcgill.ca
a.brettle@salford.ac.uk
@eblip