Presented at LILAC 2009

1. Forget about the 3Rs, our
students need the 3Cs:
citation, connectors and
critical thinking!
Jacqui Weetman DaCosta

2. Two more Cs – Curriculum Vitae
From August 2006 (as DaCosta)
• Information Literacy
Librarian at The College of
New Jersey
From 1999-2006 (as Weetman)
• Academic Team Manager at
De Montfort University,
Leicester (UK)
From birth-1999
• Variety of academic libraries
(and one public library)

3. The College of New Jersey
• Small 4 year college (6,000 FTE)
• Mainly undergraduate
• State-funded (public)
• Highly selective
• National acclaim:
– Top public college in northern region
– Ranked 10th
nationally in best value public colleges
– 4th
best college library in country

4. What’s this all about?
Testing of learning to
inform teaching
• How and why testing began
• Results of the testing
– TCNJ and others
• How the results inform our
teaching

5. The 3Cs that students want

6. IDS 102 – Information Literacy
Proficiency
• Mandatory online course for all new students
since 2004
– Six modules, assessed using internal CMS/VLE
– Need to pass course in order to graduate
– Offered in Fall and Spring semesters
– Replaced first year induction
• LINKS tutorial available for anyone to access
from web pages http://www.tcnj.edu/~liblinks/

7. Course Evaluation
• Do the students like the
course?
– Review of the tutorial
– Redesign of the tutorial
(launched July 2008)
• Do the students learn
anything?
– Use of the tutorial
– Limited assessment attempts
to 3 per module

8. Pre- and Post-Course Testing
Started with voluntary participation using an
online survey
– Educational Opportunity Fund (EOF) – 100
students
Surveys Response Rates %
July 2007 100
August 2007 59
January 2008 19

9. Testing within the assessment
Simple Online Courseware System
• SOCS enabled pre- and post-course testing
– Score given just for completion
– Students see it as a mandatory part of the
assessment
– In reality, do not fail them if they don’t complete it
• Questions on:
– Knowledge
– Where learned skills
– Feelings about learning information skills

10. The C pattern emerges
Citation (EOF – Summer 2008)
In the following citation, what is the title of the journal?
Macklin, A.S. (2001) Integrating information literacy using problem-
based learning. Reference Services Review, 29(1), 306-313.
Pre-
Course
%
Post-
Course
%
Macklin 4 1
Integrating information literacy using problem-based
learning
73 57
Reference Services Review 23 42
306-313 0 0
Don’t know 0 0

11. The C pattern emerges
Citation (Fall 2008)
In the following citation, what is the title of the journal?
Macklin, A.S. (2001) Integrating information literacy using problem-
based learning. Reference Services Review, 29(1), 306-313.
Pre-
Course
%
Post-
Course
%
Macklin 1 1
Integrating information literacy using problem-based
learning
62 44
Reference Services Review 35 53
306-313 1 1
Don’t know 1 1

12. The C pattern emerges
Connectors (EOF – Summer 2008)
When performing a search in a database, which of
the following word combinations is likely to give you
more results?
Pre-
Course
%
Post-
Course
%
election OR presidential 14 42
election AND presidential 64 48
election NOT presidential 2 0
national AND security 15 8
Don’t know 5 2

13. The C pattern emerges
Connectors (Fall 2008)
When performing a search in a database, which of
the following word combinations is likely to give you
more results?
Pre-
Course
%
Post-
Course
%
election OR presidential 39 68
election AND presidential 43 24
election NOT presidential 4 2
national AND security 7 4
Don’t know 6 2

14. The C pattern continues
Citation – IDS 102 Assessment – Module 4
If you had the following citation for an article, what would
you search to see if your library had it in print or in full text
in a database?
Thomas, Susan Gregory. ‘Capitalists on Campus: students with
computer skills cash in.’ US News and World Report, 7 Sept. 1998, p.
82.
Summer
2008
%
Fall
2008
%
Search the title - ‘Capitalists on Campus: students with
computer skills cash in.’
39 63
Search the title - US News and World Report 55 31
Search the author - Thomas, Susan Gregory 5 5
Search the subject – college students work 1 1

15. The C pattern continues
Critical Thinking – IDS 102 Assessment – Module 4
Which of the following is an appropriate
search strategy for selecting search terms in
a TCNJ Library periodical database, like
EBSCOhost Academic Search Premier?
Summer
2008
%
Fall
2008
%
Brainstorm synonyms 56 79
Brainstorm many different topics 2 3
Use only the words in your topic 42 18
Type the search words twice 0 0

16. The C pattern continues
Critical Thinking – IDS 102 Assessment – Module 4
For your biology class, you need to track down the following
citation:
Hikmat, A. Al-Ani, Strain, B. R., and Mooney, H. A. (1972). The physiological ecology
of diverse populations of the desert shrub Simmondsia chinesis. Journal of Ecology, 60,
41-57.
A search of the TCNJ list of electronic periodicals gives the
results below. Which database will provide full text of this
article?
Journal of ecology (0022-0477)
from 02/01/1998 to 06/01/2004 in EBSCOhost EJS
from 03/01/1913 to 12/01/2000 in JSTOR
from 03/01/1997 to 1 year ago in Academic Search Premier
Summer
2008
%
Fall
2008
%
EBSCOhost EJS 7 5
JSTOR 60 70
Academic Search Premier 28 12
TCNJ Library Catalog 5 13

17. Other 3Cs evidence
TCNJ (Spring 2008) – Bennett & Lasher
undertook a pre- and post-course assessment,
in class, with Senior Capstone Business
students.
• Identification of journal title in a citation
– 61% correct pre-course
– 68% correct post-course
MMU (2006-) – Glass & Griffiths undertook IL
audits where found that students struggled with:
• constructing and implementing effectively
designed search strategies
• refining the search strategy
• understanding the source of an item from its
citation description

18. Other 3Cs evidence
UCLA (Spring 1999) – undertook an Information
Competence Survey and found:
• 60% did NOT identify a journal citation from a
varied list of citations
• 45.5% did not know that OR retrieves more
records than AND or NOT
California State University San Marcos (2006-)
“One finding that reinforces anecdotal data is that
students do not make connections between one course
and another. If they learn about it in one area they do
not necessarily apply it in another area.”
(Sonntag, 2008)

19. How do you solve a problem like
Citation?
• Give students references to
hunt down in library sessions
• Ask academic staff to
incorporate tasks within
assessed work
• Include in any online tutorial or
CMS
• Be more explicit on web pages

20. How do you solve a problem like
Connectors?
• Explain the importance of
keywords and how they link
together
• Get students to practise
searches in class, using
AND/OR/NOT
• As previously, utilise:
– Academic staff (assessment)
– Online tutorials, CMS, etc.
– Web pages

21. How do you solve a problem like
Critical Thinking?
• First – find, or agree upon, a
definition!
“Critical thinking involves the
conceptualization, analysis, synthesis,
evaluation, and ultimate application of
information so that the learner may
reach conclusions or form independent
judgments based upon what the learner
has experienced combined with previous
knowledge.” (Allen, 2008)

22. How do you solve a problem like
Critical Thinking?
• Contextualise, Contextualise,
Contextualise!
• Must work with academic
staff
• SCONUL’s Seventh Pillar sees
this as a higher level skill:
– “The ability to synthesise and
build upon existing information,
contributing to the creation of
new knowledge.”

23. Test to inform your teaching
• Informal
– Show of hands
– Student/Classroom Response Systems, e.g.
Clickers, NetSupport
– Short tests/surveys in class, or pre- and post-
class with support of lecturer
• Formal
– Include as part of assessment
– Online or on paper
– Working with academic staff
– Commercial, e.g. iSkills, SAILS, TRAILS
• Defining appropriate questions

24. Some useful sources
for test questions
Avery, E.F. (ed.) (2003) Assessing Student Learning Outcomes for
Information Literacy Instruction in Academic Institutions. Chicago
(IL): American Library Association.
Emde, J. and Emmett, A. (2007) Assessing information literacy skills using
the ACRL standards as a guide. Reference Services Review, Vol. 35 (2),
pp 210-229.
Lawson, M. (1999) Assessment of a college freshman course in information
resources. Library Review, Vol. 48 (2), pp 73-78.
Mittermeyer, D. and Quirion, D. ( 2005) Information literacy: study of
incoming first-year undergraduates in Quebec. Education for
Information, Vol. 23 (4), pp 203-232.
Radcliff, C.J. et al. (2007) A Practical Guide to Information Literacy
Assessment for Academic Librarians. Westport (CT): Libraries
Unlimited.

25. Do you want to test me?
Questions?
Email: dacosta@tcnj.edu

26. References
Allen, M. (2008) Promoting critical thinking skills in online information literacy instruction using a
constructivist approach. College and Undergraduate Libraries, Vol. 15 (Issue 1/2), pp 21-38.
Caravello, P.S. et al. (2001) Information Competence at UCLA: Report of a Survey Project. Los
Angeles (CA): UCLA. Available from: http://repositories.cdlib.org/uclalib/il/01
Ellis, E. L. and Whatley, K. M. (2008) The evolution of critical thinking skills in library instruction,
1986-2006: a selected and annotated bibliography and review of selected programs, College &
Undergraduate Libraries, Vol. 15 (Issue 1/2), pp 5-20.
Glass, B. and Griffiths, J. (2008) Online information literacy audits: a longitudinal study. LILAC
(Librarians’ Information Literacy Annual Conference) 2008, Liverpool (UK), March 17-19, 2008.
Available from:
http://www.lilacconference.com/dw/resources/2008/2.07%20Presentations/LILAC08Glass.ppt
SCONUL (1999) Information Skills in Higher Education: a SCONUL position paper. Society of
College, National and University Libraries [WWW] Available from:
http://www.sconul.ac.uk/groups/information_literacy/publications/papers/Seven_pillars2.pdf
Sonntag, G. (2008) We have evidence, they are learning: using multiple assessments to measure
student information literacy outcomes. World Library and Information Congress: 74th
IFLA
General Conference and Council, Quebec (Canada), 10-14 August 2008. Available from:
http://www.ifla.org/IV/ifla74/papers/134-Sonntag-en.pdf