This document outlines a lecture on systematic reviews and meta-analyses. It discusses the rationale for systematic reviews in healthcare, the steps to conduct one, and how meta-analyses aggregate and statistically analyze results. Advantages include providing the best evidence and reducing bias compared to traditional reviews. Disadvantages include more effort required and insufficient high-quality studies. Heterogeneity between studies must be assessed and addressed. Publication bias can skew results if smaller negative studies are not published.

1. Research Methodology and
Evidence Based Healthcare (EBHC481)
Systematic review and
meta-analysis
Dr. Samer Rastam MD, PhD
1
Lecture 16
Level 8
Year 4

2. Research Methodology and
Evidence Based Healthcare (EBHC481)
Objectives
1. Be familiar with the rationale for systematic reviews.
2. Understand the steps involved in conducting a systematic review.
3. Be able to explain what meta-analyses are and why they are
conducted.
4. Be able to interpret the results of a meta-analysis.
5. Know the difference between subgroup and sensitivity analyses and
be able to interpret their results.
6. Be able to explain the main characteristics of a forest plot.
7. Understand the common sources of bias implicated in systematic
reviews.
8. Be able to list the advantages and disadvantages of systematic
reviews.2

3. Research Methodology and
Evidence Based Healthcare (EBHC481)
3

4. Research Methodology and
Evidence Based Healthcare (EBHC481)
Rationale for systematic reviews
• With the introduction of online access, there is a lot of
information out there. we only really want to use high-quality
evidence when making decisions.
• Decision-makers therefore require reviews of the evidence
available.
• Systematic reviews use a peer-reviewed protocol in an attempt to
provide an unbiased summary of the best available evidence.
4

5. Research Methodology and
Evidence Based Healthcare (EBHC481)
The role of systematic reviews in healthcare
• Required to establish the clinical benefit and cost effectiveness of
an intervention
• Crucial when there is an important, potentially lifesaving, question
which has been addressed by a number of primary randomized
controlled trials but there is still uncertainty about the answer.
5

6. Research Methodology and
Evidence Based Healthcare (EBHC481)
Traditional reviews
• Also called: narrative reviews, commentaries or literature reviews.
• Involve reviewing and summarizing the existing knowledge on a particular
topic.
• Influential but have the following disadvantages:
• Considering they are not based on a peer-reviewed protocol, the findings
are often non-reproducible.
• Bias is likely to be an issue if the collection, appraisal or summarizing of
information stages of the review are influenced by personal beliefs on the
particular topic.
• Different reviewers often reached different conclusions about the same
topic.
• The need for a systematic approach to reviewing the research evidence was
emphasized in 1992 with the publication of two landmark papers (Antman,
Lau).
6

7. Research Methodology and
Evidence Based Healthcare (EBHC481)
Developing a systematic review
• Involves a number of steps, which are based on the key principles of evidence-
based medicine:
1. Formulate and define an appropriate healthcare question.
2. Identify studies that address this question (inclusion criteria)
3. Studies then assessed for methodological quality using a critical appraisal
framework.
• A scale should be used for assessing the quality of the individual studies.
• Studies appraised as having a low-quality score are excluded.
• The impact of excluding these studies can be assessed by carrying out a
sensitivity analysis.
4. Combine the results (and conduct a meta-analysis, if appropriate).
5. Interpret the findings, taking bias into account.
7

8. Research Methodology and
Evidence Based Healthcare (EBHC481)
Evidence synthesis
• Aggregating the findings from the individual studies identified is
known as evidence synthesis.
• The type of evidence synthesis depends on the type of data being
reviewed:
• For qualitative data: Meta-synthesis
• For quantitative data: Meta-analysis
• Meta-analysis: a systematic review with a statistical technique
which combines results of several independent studies in order to
obtain a more precise estimate of a treatment effects.
8

9. Research Methodology and
Evidence Based Healthcare (EBHC481)
Why do a meta-analysis?
1. Pool all the results on a topic, resolving controversies if
individual study findings disagree.
2. Improve the estimates of the ‘effect size’ of the intervention.
3. Increase the overall sample size and therefore the statistical
power of the pooled results.
9

10. Research Methodology and
Evidence Based Healthcare (EBHC481)
Advantages of systematic review and meta-analysis
• Appear at the top of the ‘hierarchy of evidence’, thus giving us the best possible
estimate of any true effect.
• Can shorten the time lag between research practice and the implementation of new
findings into clinical practice.
• Relatively quicker and less costly to perform than a new study.
• Large amounts of information are critically appraised and synthesized in order to
reduce errors (including bias) and improve the accuracy and reliability of the findings.
• Compared to a single study, the results can often be generalized to a broader
population across a wide range of settings.
• If the studies included give consistent results, it provides evidence that the
phenomenon is robust.
• If the studies included in the review give inconsistent results, any sources of variation
can be studied.
• A meta-analysis has high power to detect exposure effects and estimate these effects
with greater precision than single studies.10

11. Research Methodology and
Evidence Based Healthcare (EBHC481)
Disadvantages of systematic reviews and meta-analysis
• Require considerably more effort than traditional reviews.
• The clinical questions posed are often too narrow, thus reducing
the applicability of the findings to your patient.
• There may be an insufficient number of high-quality studies
available for review.
• Sometimes the interventions reviewed do not reflect current
practice.
• The underlying physiological effects of an intervention are not
considered.
11

12. Research Methodology and
Evidence Based Healthcare (EBHC481)
Combining estimates in a meta-analysis
• Use a clear strategy to include studies (inclusion criteria).
• Exclude studies appraised as having a low-quality score.
• Combine the results from valid studies into a single summary estimate,
together with a measure of reliability of that estimate, the confidence interval.
• When combining the results, the following points should be considered:
1. Study participants treated should only be compared with control
participants from the same study (other studies may have different
features).
2. Combining the results may not be appropriate if there are considerable
differences in the study participants, interventions or outcomes.
3. It is important to take the relative size of each study into account when
combining the results.
4. Depending on the presence of statistical heterogeneity, the fixed-effects
or random-effects model may be used
12

13. Research Methodology and
Evidence Based Healthcare (EBHC481)
Heterogeneity
• The presence of observed variability between study estimates,
beyond that expected by chance.
• Estimating the degree of heterogeneity: by using I2 statistic (range:
0% to 100%). The more heterogeneity, the larger the I2 statistic.
13

14. Research Methodology and
Evidence Based Healthcare (EBHC481)
Sources of this heterogeneity
• From clinical or methodological diversity, or both.
• Clinical sources:
1. Age and sex of study participants.
2. Disease severity of study participants.
3. Treatment differences in randomized controlled trials, e.g. dose or intensity of
the intervention.
4. Outcome definition.
5. Location and setting of the study.
• Methodological sources:
1. Crossover versus parallel group design for RCT.
2. Randomized by individuals or clusters (e.g. by school or by family).
3. Case-control versus cohort for observational studies.
4. Different approaches to analyzing the results.
5. Differences in the extent to which bias was controlled, e.g. allocation
concealment, measurement bias, etc.14

15. Research Methodology and
Evidence Based Healthcare (EBHC481)
Dealing with heterogeneity
1. Not performing a meta-analysis
2. Random-effects meta-analysis
3. Subgroup analysis.
Presenting Meta-analyses
• The results of meta-analyses are often presented in a standard
way known as a ‘forest plot’
15

16. Research Methodology and
Evidence Based Healthcare (EBHC481)
16

17. Research Methodology and
Evidence Based Healthcare (EBHC481)
• The individual study results are represented by a circle or a square
(study estimate).
• The size of the circle or square is proportional to the weight for
that individual study.
• The horizontal line running through the circle or square
corresponds to the 95% confidence interval.
• The center of the diamond represents the summary effect
estimate of the meta-analysis (vertical broken line).
• The 95% confidence interval for the summary effect estimate
corresponds to the width of the diamond
• The studies are often displayed in chronological order
17

18. Research Methodology and
Evidence Based Healthcare (EBHC481)
Bias in meta-analyses
• Production of evidence:
• It is crucial that threats to the internal validity of a study
(confounding, and bias) are reviewed for all studies .
• Use methodological checklists for critical appraisal.
• Dissemination of evidence:
• Failure to include all relevant studies in a meta-analysis may lead to
the exposure effect being under or overestimated.
• The analyses reported in a published article are more likely to show
a statistically significant finding , while all outcomes should be
included in the final research report.
18

19. Research Methodology and
Evidence Based Healthcare (EBHC481)
• In general, those studies with significant, positive results are more
likely to be:
1. Considered worthy of publication (publication bias).
2. Published in English (language bias).
3. Published quickly (time lag bias).
4. Published in more than one journal (multiple publication
bias).
5. Cited in subsequent journals (citation bias).
19

20. Research Methodology and
Evidence Based Healthcare (EBHC481)
Publication bias
• The measure of exposure effect (risk ratio or odds
ratio) is on the horizontal axis. The exposure
effects are usually plotted on a logarithmic scale.
• As the sample size of a study increases, there is
an increase in the precision of the study
(reduction in the standard error)
• In the absence of publication bias, the plot will
resemble a symmetrical inverted funnel.
• If there is publication bias, where smaller studies
showing no statistically significant effect remain
unpublished, the funnel plot will have an
asymmetrical appearance with the lower right-
corner of the plot missing.
20

21. Research Methodology and
Evidence Based Healthcare (EBHC481)
Preventing publication bias
• To put all ongoing established trials on a register.
• Some journals will no longer consider trials for publication unless
they are registered from the start.
• Journals should consider studies for publication based only on the
literature review and study methodology.
21

22. Research Methodology and
Evidence Based Healthcare (EBHC481)
22

23. Research Methodology and
Evidence Based Healthcare (EBHC481)
References
1. Introduction to Clinical Research for Residents, Saudi
Commission for Health Specialties, 2014
2. Oxford Handbook of Clinical and Healthcare Research,
Sumantra Ray, Oxford, 2016
3. Crash Course Evidence-Based Medicine- Reading and Writing
Medical Papers, Amit Kaura (201)
23