Presented at LILAC 2016

1. Using screen-capture technology to
understand health information
seeking behaviors and assess
e-health literacy
Danielle Carlock
Faculty Librarian
Scottsdale Community College, AZ, USA
d.carlock@scottsdalecc.edu

2. OVERVIEW
• Institutional Profile
• Methodology
• Results
• Next Steps

3. SCOTTSDALE COMMUNITY COLLEGE
INSTITUTIONAL PROFILE
• Public, two year college in
Scottsdale, AZ, USA
• Approximately 10,000 students
• Part of the Maricopa Community
College District, serving 250,000
students
• General education requirements for
Associate degree include a Natural
Sciences course

4. FON 241LL COURSE
• FON 241LL “Principles of Human Nutrition
Laboratory” is a 1 credit course that fulfills the
Natural Science requirement
• Taken mainly by non-health science majors
• Students evaluate their risk for cardiovascular
disease, hypertension, type II diabetes, and
obesity

5. eHEALTH LITERACY
“The ability to seek, find, understand,
and appraise health information from
electronic sources and apply the
knowledge gained to addressing or
solving a health problem.”
(Norman and Skinner, 2006)

6. INSTRUCTIONAL METHODOLOGY
PRETEST
(AUGUST)
3 LABS THAT
REINFORCED THE
SKILLS
(SEPT-NOV)
POSTTEST
(DECEMBER)

7. PRETEST/POSTTEST METHODOLOGY
• Before and after instruction students were
given two searching prompts:
1. A personal health information need
2. An academic health information need
• Searches screen-captured using Panopto
• Searches scored against rubrics

8. SCHOLARLY INFORMATION PROMPT
& RUBRIC
Prompt: Your nutrition instructor has assigned a
research paper on the role of nutrition in type II
diabetes. Locate one scholarly source on the role of
nutrition in type II diabetes.
Rubric (5 points):
CRITERIA NO YES
ARTICLE IS SCHOLARLY 0 2
ARTICLE IS RESEARCH ARTICLE 0 2
ARTICLE IS RELEVANT 0 0.5
ARTICLE IS CURRENT 0 0.5

9. OVERALL RESULTS-SCHOLARLY
PRETEST
MEASURE RESULTS (n=36)
Mean score 35%
Mean search
duration
2.5 minutes
Mean # of sites
viewed
0.6 sites
Areas of weakness Not knowing what a scholarly
source is or how to find one

10. SEARCH SOURCES (Prior to
instruction)
SEARCH SOURCE # OF
STUDENTS
% OF
STUDENTS
GOOGLE 19 53%
LIBRARY WEB SCALE 7 19%
GOOGLE SCHOLAR 3 8%
LIBRARY DATABASE 2 5.5%
LIBRARY RESEARCH GUIDE 2 5.5%
MULTIPLE SEARCH SOURCES 2 5.5%
CONSUMER HEALTH SITE 1 2.5%

11. TYPES OF SOURCES SELECTED
(Prior to instruction)
TYPE OF SOURCE # OF STUDENTS % OF
STUDENTS
CONSUMER SITE 16 44%
REVIEW 7 19.5%
POSITION PAPER 6 17%
RESEARCH 4 11%
ONLINE REFERENCE
WORK
2 5.5%
NONE 1 3%

12. SEARCHING TRENDS (Prior to
instruction)
• For about half, no distinction in search
methods/sources for consumer/personal vs.
scholarly searches
• Use of filters/limits, Boolean & synonyms virtually
non-existent
• Search revisions did not necessarily lead to better
outcomes

13. PRETEST/POSTTEST
COMPARISON-SCHOLARLY
MEASURE PRETEST POSTTEST
Mean score 35% 72% **
Mean search duration 2 minutes, 30
seconds
2 minutes,
38 seconds
Mean # of sites viewed 0.6 1.5
Areas of weakness Knowing what
scholarly sources
are and where to
find them
Relevancy & source
distinction
Use of filters
**Pretest and posttest scores significantly different using a Wilcoxon Signed Rank Test,
(V=35, p<0.001)

14. TYPES OF SEARCH SOURCES
(Before and after instruction)
SOURCE % PRIOR TO
INSTRUCTION
% AFTER
INSTRUCTION
Google/internet search engine 53% 11%
Google Scholar 8% 28%
PubMed or PubMed Central 0% 20%
Science Direct 0% 17%
BioMed Central 0% 6%
Library Web Scale 19% 3%
Academic Search Premier 5.5% 0%
Library Research Guide 5.5% 0%
Consumer health website 2% 0%
Multiple sources 5.5% 17%

15. SOURCES SELECTED
(Before and after instruction)
TYPE OF SOURCE % OF STUDENTS
BEFORE INSTRUCTION
% OF STUDENTS
AFTER
INSTRUCTION
Research article 11% 67%
Review article 19.5% 11%
Consumer health site 44% 8%
Position paper 17% 5.5%
Letter to the editor 0% 3%
Book chapter 0% 3%
Online reference work 5.5% 0%
None 3% 3%

16. SEARCHING TRENDS
(After instruction)
•Increase in the use of Boolean (from 8% to 53%)
•Increase in the use of filters (from 11% to 30%)
•Large number of multiple search sources &
revisions suggests difficulties with recognizing
topically relevant research articles

17. IMPLICATIONS FOR
INSTRUCTION
• Introduce fewer databases and
emphasize filters/limits
• More practice exercises on topical
relevancy and source type

18. TAKE AWAYS
• Students must be explicitly taught what
scholarly sources are and how to find them
• If there is not enough time for practice and
checking for understanding instruction may
lead to confusion, over-searching and
reversion to Google
• One shots: less is more

19. NEXT STEPS
• Repeating the study with
approximately 100 students this
semester
• Interested in finding others that want
to replicate the study

20. WORK CITED
Norman CD, Skinner HA. 2006. eHealth
Literacy: Essential Skills for Consumer
Health in a Networked World. Journal of
Medical Internet Research:8(2):e9.

21. ACKNOWLEDGEMENTS
Thank you to:
• Maricopa Center for Learning and Instruction which
funded the fellowship
• Robert H. Martin, SCC Nutrition faculty, for his
collaboration on the project
• SCC Library Division faculty for feedback on the
presentation