In the age of internet and social media, Dr. Carl Abelardo Antonio teaches us how to evaluate online health resources so we can tell which of them is gold and which of them is junk.

1. Not all that gets viral is gold:
Evaluating online health resources
Carl Abelardo T. Antonio, MD, MPH
Assistant Professor of Health Policy & Administration
College of Public Health | University of the Philippines Manila
ctantonio@up.edu.ph
21st PHICS Annual Convention
Crowne Plaza Manila Galleria, Quezon City, Philippines | 29 May 2015

2. FIVE PILLARS
Fundamental principles of
Wikipedia1
Wikipedia is an encyclopedia
Wikipedia is written from a
neutral point of view
Wikipedia is free content that
anyone can use, edit, and
distribute
Editors should treat each
other with respect and civility
Wikipedia has no firm rules
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 2

3. 21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 3
RESULTS: 1433 full text articles from 1008 journals indexed in
Medline, PubMed, or Embase with 2049 Wikipedia citations
were accessed. The frequency of Wikipedia citations has
increased over time; most citations occurred after December
2010. More than half of the citations were coded as definitions (n
= 648; 31.6%) or descriptions (n=482; 23.5%). Citations were
not limited to journals with a low or no impact factor; the search
found Wikipedia citations in many journals with high impact
factors.2

4. Presentation Outline
Information
exchange
in the
Internet
age
Challenges
of
information
on the
internet
Making
sense of
online
health
information
421st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015

5. Presentation Outline
Information
exchange
in the
Internet
age
Challenges
of
information
on the
internet
Making
sense of
online
health
information
521st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015

6. INFORMATION EXCHANGE IN THE INTERNET AGE
Growing internet usage @ PHL
33-M estimated number of
Filipino internet users3
531%
growth in internet use
among Filipinos in the past
five years4
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 6

7. INFORMATION EXCHANGE IN THE INTERNET AGE
The internet as a marketplace of ideas5
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 7
• Art. 19. Everyone has the right to freedom of opinion
and expression; this right includes freedom to hold
opinions without interference and to seek, receive
and impart information and ideas through any media
and regardless of frontiers.6

8. INFORMATION EXCHANGE IN THE INTERNET AGE
Data never sleeps7
English Wikipedia8
Articles 4,878,955
Pages 36,359,218
Files 855,776
Edits 773,080,212
Users 25,251,523
Admins 1,350
Active users 126,044
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 8
▲800 articles/day

9. Presentation Outline
Information
exchange in
the Internet
age
Challenges
of
information
on the
internet
Making
sense of
online
health
information
921st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015

10. CHALLENGES OF INFORMATION ON THE INTERNET
Anyone can create content9
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 10
sites.google.com wordpress.org
tumblr.com www.blogger.com

11. CHALLENGES OF INFORMATION ON THE INTERNET
Content quality not always assured9
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 11
Wikipedia comparison Results of study
vs peer-reviewed literature
for 10 most costly medical
conditions in the US
Statistically significant discordance between 9 of the 10
selected Wikipedia articles and their corresponding peer-
reviewed sources (P<.05) .10
vs dietary supplement
references for herbal
supplement information
Articles in Wikipedia in mid-2012 for the 19 top-selling
herbal supplements were frequently incomplete, of variable
quality, and sometimes inconsistent with reputable sources
of information on these products. Safety information was
particularly inconsistent among the articles.11
vs standard medical
textbook for respiratory
topics
Despite the effort placed in creating Wikipedia respiratory
articles by anonymous volunteers (wikipedians), most
articles had knowledge deficiencies, were not accurate.12
vs standard medical
textbook for
gastroenterology and
hepatology topics
Wikipedia is not a reliable source of information for
gastroenterology and hepatology articles. Several
limitations, deficiencies, and scientific errors have been
identified in the articles examined.13

12. CHALLENGES OF INFORMATION ON THE INTERNET
Information may not always be recent9
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 12
• It is always easy to assume that websites are kept
up to date, but this is not necessarily true.9

13. Presentation Outline
Information
exchange in
the Internet
age
Challenges
of
information
on the
internet
Making
sense of
online
health
information
1321st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015

14. MAKING SENSE OF ONLINE HEALTH INFORMATION
Tools and techniques
• Quick review method:
▫ Rapid appraisal questions
• Quality checklist:
▫ Health on the Net (HON)
• Critical appraisal tool:
▫ DISCERN Instrument
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 14

15. MAKING SENSE OF ONLINE HEALTH INFORMATION
Rapid Appraisal Questions9
• Who is the author of the website or who owns the
site? Can you find anything about their credibility?
• Is the information provided accurate?
• How up-to-date is the material?
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 15

16. HONcode Site Evaluation
Form (http://www.hon.ch/)
HONcode Principles14
 Authority (Give qualifications of
authors)
 Complementarity (Information to
support, not replace)
 Confidentiality (Respect the
privacy of site users)
 Attribution (Cite the sources and
dates of medical information)
 Justifiability (Justification of claims
/ balanced and objective claims)
 Transparency (Accessibility,
provide valid contact details)
 Financial disclosure (Provide
details of funding)
 Advertising (Clearly distinguish
advertising from editorial content)
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 16

17. DISCERN Instrument
(http://www.discern.org.uk/)
DISCERN Criteria15
 Have explicit aims
 Achieve its aims
 Be relevant to consumers
 Make sources of information explicit
 Make date of information explicit
 Be balanced and unbiased
 List additional sources of information
 Refer to areas of uncertainty
 Describe how treatment works
 Describe the benefits of treatment
 Describe the risks of treatment
 Describe what would happen without
treatment
 Describe the effects of treatment
choices on overall quality of life
 Make it clear there may be more than
one possible treatment choice
 Provide support for shared decision-
making
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 17

18. Summary
• With the internet, information production and
exchange occurs at an exponential rate.
• The quality of information online is not always
guaranteed.
• Tools have been developed to help consumers of
information appraise information on the Internet.
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 18

19. References
1Five Pillars [Internet]. 2015 May 2 [cited 2015 May 28]. Available from link.
2Bould MD, Hladkowicz ES, Pigford AA, Ufholz LA, Postonogova T, Shin E, Boet S. References that anyone can edit: review of Wikipedia
citations in peer reviewed health science literature. BMJ. 2014 Mar 6;348:g1585.
3Montenegro B. PHL Internet users now 38 million, two-thirds under 30 –IMMAP [Internet]. 2014 Sep 25 [cited 2015 May 28]. Available from
link.
4Magdirila P. Philippines records the biggest internet population growth globally [Internet]. 2014 Mar 13 [cited 2015 May 28]. Available from
link.
5McLaughlin S. Tech Beat: Universal access, open Internet and the marketplace of ideas [Internet]. 2011 Dec 02 [cited 2015 May 28].
Available from link.
6Universal Declaration of Human Rights [Internet]. Adopted by the United Nations General Assembly on 10 December 1948 [cited 2015 May
28]. Available from link.
7James J. Data Never Sleeps 2.0 [Internet]. 2014 Apr 23 [cited 2015 May 28]. Available from link.
8Wikipedia:Statistics [Internet]. 2015 May 24 [cited 2015 May 28]. Available from link.
9Keen C, Otter ME. Critical appraisal. In: Walker DM (ed.). An Introduction to Health Services Research. London: Sage; 2014.
10Hasty RT, Garbalosa RC, Barbato VA, Valdes PJ Jr, Powers DW, Hernandez E, John JS, Suciu G, Qureshi F, Popa-Radu M, San Jose S,
Drexler N, Patankar R, Paz JR, King CW, Gerber HN, Valladares MG, Somji AA. Wikipedia vs peer-reviewed medical literature for
information about the 10 most costly medical conditions. J Am Osteopath Assoc. 2014 May;114(5):368-73.
11Phillips J, Lam C, Palmisano L. Analysis of the accuracy and readability of herbal supplement information on Wikipedia. J Am Pharm Assoc
(2003). 2014 Jul-Aug;54(4):406-14.
12Azer SA. Is Wikipedia a reliable learning resource for medical students? Evaluating respiratory topics. Adv Physiol Educ. 2015 Mar;39(1):5-
14.
13Azer SA. Evaluation of gastroenterology and hepatology articles on Wikipedia: are they suitable as learning resources for medical students?
Eur J Gastroenterol Hepatol. 2014 Feb;26(2):155-63.
14The HON Code of Conduct for medical and health Web sites (HONcode) [Internet]. 2013 Sep 19 [cited 2015 May 28]. Available from link.
15Quick reference guide to the DISCERN criteria [Internet]. n.d. [cited 2015 May 28]. Available from link.
21st PHICS Annual Convention || Evaluating Online Health Resources || CT Antonio || 29 May 2015 19