Presented at the Department of Community Medicine, Faculty of Medicine Ramathibodi Hospital, Mahidol University on October 15, 2019

1. Evidence-Based
Medicine III:
Systematic Reviews
RAID 515
Nawanan Theera-Ampornpunt
Department of Community Medicine
Faculty of Medicine
Ramathibodi Hospital
สไลด์บางส่วนดัดแปลงจากเอกสารประกอบการสอนของคณาจารย์ภาควิชาเวชศาสตร์ชุมชน คณะแพทยศาสตร์โรงพยาบาลรามาธิบดี มหาวิทยาลัยมหิดล

2. 2
 Systematic Reviews
▪ Lecture
▪ Case & Paper Discussion
▪ Critical Appraisal
 Group Presentation
Outline

3. Common Pitfalls
for EBM

4. 4
 PICO: Question structure
Example:
P: Are patients with pain symptoms
I: who take conventional NSAIDs,
C: compared to those who take selective
NSAIDs,
O: at an increased risk of peptic ulcer or
GI bleeding?
Common Pitfalls for EBM

5. 5
 PICO Question: Not too specific, but not
too broad
 Focus on patient population similar to
this patient, but not exactly this patient
 Pertinent features in PICO Question
should be part of keywords
 Selecting paper that best fits the
question & your critical appraisal reveals
good (valid, important & applicable)
evidence
Common Pitfalls for EBM

6. 6
 Avoid “Free full text” filter or limit
▪ PubMed only shows those publicly
available for free
▪ PubMed doesn’t know which journals the
university subscribes to (the ones that you
can access full texts even though they’re
not free)
▪ You could miss many relevant papers
Common Pitfalls for EBM

7. 7
 When using OVID MEDLINE
▪ Basic Search vs. Advanced Search
▪ Advanced Search: Always put “AND”
between words, otherwise will search for
exact match
▪ risk AND breast cancer AND contraceptive
Common Pitfalls for EBM

8. Systematic Reviews
(Lecture)

9. 9
1.2.5.1 Identifying and selecting studies
1.2.5.2 Quality of evidence assessments
1.2.5.3 Combining the findings of
independent studies
1.2.5.4 Variation between study findings
1.2.5.5 Summarizing and interpreting
results
เกณฑ์แพทยสภาเรื่อง Systematic Reviews
http://www.tmc.or.th/file_08062012.pdf

10. 10
Types of Review Studies
http://www.docstoc.com/docs/126908275/Meta-analysis

11. 11
A clearly stated set of objectives with
pre-defined eligibility criteria for studies
An explicit, reproducible methodology
A systematic search that attempts to identify all
studies that would meet the eligibility criteria
An assessment of the validity of the findings of
the included studies, for example through the
assessment of risk of bias
A systematic presentation, and synthesis, of the
characteristics and findings of the included
studies
Key Characteristics of
Systematic Reviews
http://handbook.cochrane.org/

12. 12
 To collate all empirical evidence that
fits pre-specified eligibility criteria in
order to answer a specific research
question
 Uses explicit, systematic methods to
minimize bias and provide reliable
findings
Why Do Systematic Reviews?
http://handbook.cochrane.org/

13. 13
 To provide more precise estimates of
effects than estimates from individual
studies, and increase power of
statistical tests over individual studies,
by combining information from all relevant
studies
 To facilitate investigations of consistency
of evidence, and exploration of
differences, across studies
Why Do Meta-Analyses?
http://handbook.cochrane.org/ http://en.wikipedia.org/wiki/Meta-analysis

14. 14
The power of a statistical test is
 The probability that it correctly rejects the
null hypothesis when the null hypothesis
is false.
 In other words, the ability of a test to
detect an effect, if the effect actually
exists.
 Avoiding Type II errors (False-negatives)
Review: Statistical Power
http://en.wikipedia.org/wiki/Statistical_power

15. 15
เกณฑ์ 5 ข้อของแพทยสภาเป็นวิธีทา
Systematic Reviews
1.2.5.1 Identifying and selecting studies
ค้นหาบทความและคัดเลือก
1.2.5.2 Quality of evidence assessments
ประเมินคุณภาพของแต่ละบทความ
1.2.5.3 Combining the findings of independent
studies
รวมข้อค้นพบจากแต่ละบทความ
1.2.5.4 Variation between study findings
ความแตกต่างของข้อค้นพบระหว่างบทความ
1.2.5.5 Summarizing and interpreting results
สรุปและแปลผลภาพรวม
http://www.tmc.or.th/file_08062012.pdf

16. 16
 Systematic review vs. Meta-analysis
 Cochrane database of
systematic reviews
 Risk of bias in primary studies
 Random effect vs. Fixed effect models
 Heterogeneity
(variation among studies)
 Forest plot vs. Funnel plot
Keywords of Systematic Reviews

17. 17
1.2.5.1 Identifying and Selecting Studies
ค้นหาบทความและคัดเลือก
ค้นหาบทความในฐานข้อมูลต่างๆ เช่น
 MEDLINE: PUBMED
 COCHRANE: reviews and database
 EMBASE: Excerpta Medica Database
 Grey Literature: thesis, dissertation, unpublished
studies
ค้นหาอย่างน้อย 2 คนโดยเป็นอิสระต่อกัน นามาคัดเลือก
ร่วมกันหรืออาจต้องมีบุคคลที่สามช่วยตัดสินในกรณีที่
ความเห็นไม่ตรงกัน

18. 18
1.2.5.2 Quality of Evidence Assessments
ประเมินคุณภาพของแต่ละบทความ
ตัวอย่างแนวทางการประเมินคุณภาพบทความ RCT
The Cochrane Risk of Bias Tool
 Selection bias: allocation concealment
 Performance bias: blinding participants &
personnel (not always possible)
 Detection bias: blinding assessors
 Attrition bias: incomplete data due to withdrawals
 Reporting bias: selective outcome reporting
 Other biases: bias in other domains (e.g. carry-
over effect in cross-over trials, contamination of
intervention)
http://handbook.cochrane.org/

19. 19

20. 20
1.2.5.3 Combining the Findings of
Independent Studies
รวมข้อค้นพบจากแต่ละบทความ
Meta-analysis: การประมวลผลรวมของการศึกษาต่างๆ
เข้าด้วยกันด้วยวิธีการทางสถิติ มี 2 models
 Fixed Effects Model
o Assumes that the true effect of treatment is
the same for every study
 Random Effects Model
o Assumes that the true effect estimate for each
study varies
http://handbook.cochrane.org/

21. 21
1.2.5.4 Variation between Study Findings
ศึกษาความแตกต่างของข้อค้นพบระหว่างบทความ
Concepts and sources of heterogeneity
 Clinical Heterogeneity: Differences in patients
characteristics or treatment regimen (interventions)
 Methodological Heterogeneity: Variation in
study design, defining & measuring outcomes,
duration of follow-up & other methodological factors
(e.g. use of blinding & allocation concealment)
 Statistical Heterogeneity (or Heterogeneity):
Variability in intervention effects observed in
different studies due to Clinical Heterogeneity
and/or Methodological Heterogeneity
 The Play of Chance: Random errors
http://handbook.cochrane.org/

22. 22
1.2.5.5 Summarizing and Interpreting Results
สรุปและแปลผลภาพรวม

24. 24
1.2.5.5 Summarizing and Interpreting Results
สรุปและแปลผลภาพรวม (Forest Plot)
http://handbook.cochrane.org/

25. 25
Measuring Heterogeneity
Standard Chi-square Test (Cochran Q Test)
▪If significance p-value < 0.10, there is significant
heterogeneity.
Tau-squared (in random-effects model)
I2 (0-100%, Larger percentages indicate larger
heterogeneity) (See interpretation below)

26. 26http://handbook.cochrane.org/
Single True Effect among
Studies (Fixed Effects)
Multiple True Effect among
Studies (Random Effects)
Single vs. Multiple True Effects Among Studies

27. 27
Funnel Plot to Detect
Publication Bias (“File Drawer Problem”)
http://handbook.cochrane.org/

28. Fig 1 Example of symmetrical funnel plot.
Jonathan A C Sterne et al. BMJ 2011;343:bmj.d4002
©2011 by British Medical Journal Publishing Group

29. 29
Funnel Plot:
No Publication Bias Suspected
http://en.wikipedia.org/wiki/Meta-analysis/

30. 30
Funnel Plot:
Publication Bias Suspected
http://en.wikipedia.org/wiki/Meta-analysis/

31. Systematic Reviews
(Example)

32. 32
Case Scenario & Search Strategy
แพทย์ปฏิบัติงานที่โรงพยาบาลชุมชนแห่งหนึ่งพบผู้ป่วยชายอายุ 50 ปี เป็น
เบาหวาน แต่ไม่เคยเป็นโรคหัวใจมาก่อน ถามว่า จะกินยาลดไขมันเพื่อป้องกัน
การเกิดโรคหัวใจได้หรือไม่
P Are patients with diabetes
I who take statin for primary prevention
C when compared to placebo
O at a decreased risk of cardiovascular diseases?
พิมพ์ search terms ใน PUBMED Clinical Queries ดังนี้
diabetes statins "primary prevention"
"cardiovascular diseases" meta-analysis
พบ 7 บทความ ในคอลัมน์ Systematic Reviews

33. 33
Paper Selection
เลือกบทความชื่อ
Statins for primary prevention of cardiovascular and
cerebrovascular events in diabetic patients without
established cardiovascular diseases: a meta-analysis.
Chen YH, Feng B, Chen ZW. Exp Clin Endocrinol
Diabetes. 2012 Feb; 120(2):116-20

34. Systematic Reviews
(Critical Appraisal)

35. 35
Critical Appraisal Checklist for
Systematic Reviews
 What question (PICO) did the systematic review
address?
 Is it unlikely that important, relevant studies were
missed?
 Were the criteria used to select articles for inclusion
appropriate?
 Were the included studies sufficiently valid for the
type of question asked?
 Were the results similar from study to study?
 What were the results?
 Will the results help locally?
http://www.cebm.net/