Presented at the 2015 Northumbria Conference in Edinburgh, Scotland.

1. Measure for Measure
“Using collaborative assessment to build stronger information literacy skills”
PRESENTED BY: Susan Gardner Archambault

2. • Private Jesuit and Marymount University in Los Angeles, CA
• 6,064 Undergraduates
• 2,189 Graduates
• Library Open 24/5
• New Core Curriculum in Fall of 2013
LMU

3. Info. Literacy Learning Outcomes

4. Info lit flag
Core Structure

6. Rhetorical Arts Course Criteria
Assign at least 10% of the final course grade on the basis of information
literacy, with a librarian-led workshop and one or more course-integrated
assignments
INFORMATION LITERACY OUTCOMES:
1. Conceptualize an effective research strategy and then collect, interpret,
evaluate and cite evidence in written and oral communication
2. Distinguish between types of information resources and how these
resources meet the needs of different levels of scholarship and different
academic disciplines

7. Spring 2014
• 1272 first-year students
had a required library visit
• Across 72 sections of
Rhetorical Arts

8. 3 Info Lit Course Assignments
in Common Syllabus

9. Library Visit Agenda
1. Intro to LibGuide
2. Why use the library?
3. Evaluating sources
(RADAR game)
4. Finding sources
(demo)

10. Library Support Material

11. Rhetorical Arts 1000 LibGuide

12. RADAR Game
• Students were paired
up and given a source
to evaluate
• Using the resources in
the course LibGuide
the student pairs
answered questions
about the source they
were given
• Discussion with the
librarian after
completing the RADAR
Game

13. Gamification
• Incorporating Friendly
Competition (points)
• Motivational Feedback
(stars)
• Measure Progress
(showing star count)
• Reward effort (getting
the answer right on the
first try earns more
points)

14. Research Strategies Tutorial
(Recommended Homework)

15. Research
Questions
• How effective was the
face-to-face instruction?
• How effective were the
course-integrated
assignments?
• How effective was the
library support material?

16. Methodology

17. Direct Measures
Overall Averages:
• Student scores across 100
sampled annotated
bibliographies. Scored with a
calibrated rubric by a group of
volunteer R.A. instructors

18. Indirect Measures
Survey:
• Stratified random sample of 300
students (48% response rate)
• All instructors teaching a section
(57% response rate)

19. Comments
Survey comments
were analyzed using
NVivo and coded

20. Major Findings
& Applications

21. High-Scoring Areas:
o citing sources correctly
o identifying the main
purpose of a source
Low-Scoring Areas:
o identifying intended
audience
o authority of the author
o accuracy of the evidence
o bias
Annotated Bibliography

22. 3.54
4
0
1
2
3
4
5
Helpfulness/Evaluating Evidence
Students
Instructors
Annotated Bibliography

23. Comment No. of Times
Not helpful for writing/research process 7
Too many steps/make simpler 3
Annotated Bibliography/Comments

24. Comment No. of Times
Fun/Engaging 21
Helpful 15
Competition 8
Interactive 7
Liked partner work 6
Not fun/engaging 6
Too easy 6
Not useful 5
RADAR Game/Student Comments

25. Students who listed a
major in the College of
Science & Engineering
liked the RADAR game
more than the average
student
Statistically Significant

26. Annotated Bibliography Revised

27. Added Hints Added More Discussion
RADAR Game Revised

28. Pre-Search Worksheet

29. 56%22%
22%
Instructors (Mean = 3.44 / 5)
Agree
Disagree
Neutral
51%31%
18%
Students (Mean = 3.17 / 5)
Agree
Disagree
Neutral
Helpfulness in Developing a Topic

30. Comment No. of Times
Too easy/not complex enough 17
Did not integrate with or apply to other projects/assignments 5
Add “working bibliography” of sources 4
Layout needs more space 4
Add reflective questions re: terminology & inclusion/exclusion
choices
3
Add citation tracking element 1
Have instructor approve the topic 1
Allow space to change positions 1
ID stakeholder and their position 1
Pre-Search Worksheet Student/Faculty Comments

31. Research Diary

32. 3.29
3.71
3.33
3.5
2.77
2.58
0 1 2 3 4 5
Reflecting on search process
Helped collect info
Grading rubric/significant
aspects of learning
Grading rubric/clear criteria
Student
Instructor
Research Diary

33. Comment No. of Times
Busy work/not helpful 24
Add section on relevancy of sources/source
reflection/how to use/more source interaction
6
Timing/better integration with other
assignments (earlier in semester)
2
Research Diary Student/Faculty Comments

34. Pre-Search Worksheet Revised

35. Continued…

36. Library Visits/Library Instruction

37. 75%
17%
8%
Instructors (Mean = 3.92 / 5)
Valuable
Somewhat
Not Very
Valuable
46%
30%
24%
Students (Mean = 3.22 / 5)
Valuable
Somewhat
Not Very
Valuable
How Valuable Are Library Visits?

38. 4.3
4.32
4
3.88
3.57
3.57
3.55
3.67
3.8
3.8
3.41
3.58
3.5
3.53
0 1 2 3 4 5
Scholarly vs. Popular
Getting Help from Librarian
Finding Info
Finding a Topic
Evaluating Appropriateness/ Info
Citing Sources
Using Evidence/Make Arguments
Student
Instructor
How Helpful Are Library Visits?

39. Comment No. of Times
LibGuide 30
Services Offered by Librarians 29
Research Databases 24
Research Strategies 16
Evaluating Sources 13
What Did Students Find Most Helpful?

40. Comment No. of Times
Fine – no suggestions for improvement 50
Research databases/keyword searching 16
Finding scholarly sources 13
Repetition of FYS or other course 10
Too easy 10
Better relevance or integration with real assignment 8
Citing 8
Navigating library web site or LibGuide 8
What Needs Improvement? Student/Faculty Comments

41. Revised: Finding Sources Activity
(Team-based Active Learning)

42. LibGuide Student Ranking

43. Comment No. of Times
Couldn’t find/unaware of 14
Too complicated/confusing/too many links 9
LibGuide: Student/Instructor Comments

44. Research Strategies Tutorial

45. 29%
50%
21%
Instructors Who
Required Tutorial
Yes
No
Not Sure
55%
45%
Students Who
Completed Tutorial
Yes
No
Research Strategies Tutorial: Usage

46. Student Ranking of Tutorial
3.71 out of 5

47. Revised: Tutorial as Mandatory
Homework Assignment

48. Encourage better integration
of library visit and info lit
assignments by assigning
the same research topic
throughout multiple
assignments
Additional Improvement

49. Thank You:
William H. Hannon Library
Research Incentive Travel Grant
Susan Gardner Archambault
susan.archambault@lmu.edu
@susanarcham