Research study - EDF5481

1. Effects of Face-to-Face
and Online
Information Literacy
Instruction on
Community College
Students’ Web
Evaluation Skills and
Confidence
Sila Lott

2. Introduction
– Internet = access to more information
– Students go through information overload (Head, 2013)
– Students don’t know how to discern credible info (Lim & Simon, 2011)
Percentage Skill/Problem Source
87% of college students Information illiterate (Head & Eisenberg, 2011)
26% of college students Can determine website
credibility
(Head & Eisenberg, 2011)
89% of college students* Have difficulty starting a
research assignment
(Head, 2013)
50% of college students* Have issues assessing quality
of research efforts
(Head, 2013)
80% of college students* Report rarely (if ever) utilizing
a Librarian for help
(Head, 2013)
*Sample of 11,000 students
Background
Literature:

3. Justification
– Information literacy skills are necessary in both professional and personal
settings
– Given our digital age rich with mass media, but also fake news and clickbait,
students need to learn skills to effectively determine reliable and credible
sources
Gap Purpose
– A review of literature reveals few studies
that have evaluated information literacy
instruction to improve web evaluation
skills of community college students
(Dewald, 1999; Reinhardt & Isbell, 2002; Samson, 2000;
Walton & Archer 2004 )
– The most comprehensive study of information literacy
skills to date only had a 1% sample of community college
students (87/ 8353 institutions) (Head, 2011)
– Evaluate effectiveness of face-
to-face and online information
literacy instruction for
improving community college
students’ web evaluation skills.

4. Research Questions and Hypotheses
Q1: Does face-to-face and online
information literacy instruction of
community college students improve their
web evaluation skills in comparison to
receiving no information literacy
instruction?
Q2: Does face-to-face information literacy
instruction versus online information
literacy instruction in comparison, improve
web evaluation skills of community college
students ?
Q3: Does face-to-face and online
information literacy instruction improve
confidence in student web evaluation
skills?
H1: Face-to-face and online information
literacy instruction will improve community
college students’ web evaluation skills.
H2: There is no significant difference in the
outcome of community college students’
web evaluation skills after face-to-face
versus online information literacy
instruction.
H3: Face-to-face and online information
literacy instruction will improve community
college students’ confidence in their web
evaluation skills.

5. Methods
Participants /
SamplingDesign
Measures
Operationalizations:
Information Literacy: Ability to
locate, find and evaluate information
Instruction: Active dissemination of
knowledge & information offered in
a face-to-face or virtual classroom.
Web Evaluation: Ability to effectively
evaluate a website for reliability
(trustworthiness) and credibility
(authority/ credentials)
ABCD Framework – modified by
Kathleen Schrock (2002)
A=Authority, B=Bias, C=Currency,
D=Documentation
Confidence: Positive report/ high
rating by a student that they can
effectively evaluate a website using a
Likert scale on a scale of 1 to 10
reporting how confident they are to
evaluate the website.
Between groups
pre-test and post-
test experimental
design
-Community
College Students
-Selected from
Tallahassee
Community
College
-Non-probability
Convenience
Sample
-Voluntary/random
sample
-Sample of 80
students (4 classes
- 20 per class)

6. Methods
Treatment
Class Pre/Post-Test Treatment/Control F2F /
Online
Sample
Class One Y Treatment
(Info Lit Instruction)
F2F 20 students
Class Two Y Control
(No Info Lit Instruction)
F2F 20 students
Class Three Y Treatment
(Info Lit Instruction)
Online 20 students
Class Four Y Control
(No Info Lit Instruction)
Online 20 students
Total: 80 students

7. Methods
Procedure
Step 1: Pre-Test
Step 2: Skill Test
5-Question Survey
1) On a scale of 1 to 10 (1 being not at all, and 10 being very), how familiar are you with evaluating websites?
2) How skilled would you say you are in evaluating websites on a scale of 1 to 10?
3) List attributes of a credible website (to your knowledge).
4) List attributes of a not credible website (to your knowledge).
5) How confident are you on a scale of 1 to 10 regarding your website evaluation skills?
Step 3: Treatment
(or Control)
Step 4: Skill Test
Step 5: Post-Test
The treatment will have the students look at three websites on varying levels of credibility. Website one = very
credible, two = fair, three = not credible. The students are to take 5-10 minutes reviewing the website, before
answering seven questions measuring their assessment of website credibility.
Inclusion / Exclusion of Information Literacy Instruction. Lesson comprises of: 1) Why is Research / Information
Literacy Important, 2) Explaining the Deep Web, 3) Google Web Searching Pros and Cons 4) Authority is Constructed
and Contextual 5) Web Evaluation - ABCD Framework 6) ABCD Checklist 7) Practice . OR dummy lesson.
Same as step 2, but different websites.
Same as step 1.

8. Discussion
• Validity Strengths and Weaknesses
– Shortcomings: Non-probability convenience sample – not truly random –
classes utilized after granted permission from instructor – voluntary –
Selection threat (internal validity – observed phenomenon may be due to
instructor characteristics) AND to (external validity – hard to generalize to
non-voluntary instructors).
– Only one community college measured
– Small sample size (threat to external validity)
– Strengths: Groups will be experimentally controlled – increases internal validity
and reduces internal threat of interaction of selection and treatment. Also helps
reduce internal threat of history. Strengthened by controlling for classes (same
academic level / discipline) and instructor (same pedagogical style)

9. References
Dewald, N. H. (1999). Transporting good library instruction practices into the Web environment: an analysis of online tutorials. Journal of Academic Librarianship,
25(1), 26–31. https://doi-org.proxy.lib.fsu.edu/10.1016/S0099-1333(99)80172-4
Head, A. (2013). Project Information Literacy: What can be learned about the information-seeking behavior of today's college students? Retrieved
from https://www.projectinfolit.org/uploads/2/7/5/4/27541717/head_project.pdf.
Head, A., & Eisenberg, M. (2011). How college students use the Web to conduct everyday life research. Retrieved from
https://firstmonday.org/article/view/3484/2857Introduction.
Lim, S., & Simon, C. (2011). Credibility judgment and verification behavior of college students concerning Wikipedia. First Monday, 16(4). Retrieved
from https://www.ojphi.org/ojs/index.php/fm/article/view/3263/2860
Reinhardt, J., & Isbell, K. (2002). Building Web literacy skills. The Reading Matrix, 2(2). Retrieved from
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.120.2808&rep=rep1&type=pdf
Samson, S. (2000). What and when do they know? Web-based assessment. Reference Services Review, 28(4), 335-342. Retrieved from
https://doi:http://dx.doi.org.proxy.lib.fsu.edu/10.1108/00907320010359650
Schrock, K. (2002). Teaching media literacy in the age of the internet. Classroom Connect, 4-6. Retrieved from https://www.slideshare.net/KarenHahn1/the-abc-of-website
-evaluation?from_action=save
Walton, M., & Archer, A. (2004). The Web and information literacy: scaffolding the use of web sources in a project-based curriculum. British Journal of Educational Technology,
35(2), 173–186. https://doi-org.proxy.lib.fsu.edu/10.1111/j.0007-1013.2004.00379.x