This slide show explores how to review literature and develop an understanding of the quality of the clinical evidence for a treatment modality. Reviews the development of a guideline based on evidence based GRADE process.

1. C R E A T I N G T H E F O U N D A T I O N F O R E V I D E N C E
B A S E D P R A C T I C E
Evidence Based Research

2. What is an Evidence Based Guideline?
"Guidelines are recommendations intended to assist
providers and recipients of health care and other
stakeholders to make informed decisions.
Recommendations may relate to clinical interventions,
public health activities, or government policies."
- WHO 2003, 2007

4. Grade Approach to Clinical Recommendations
 The Grade Approach is one tool that organizations
can use to distill a body of research to make a clinical
claim.
 An “evidenced based treatment” is a treatment that
has research evidence supporting it.
 Research evidence can be of high, moderate, low or
very low quality.
 Based on the quality of the research a clinician can
make a claim about an intervention.

5. Further Assessing the Quality of Evidence
 Randomized trials = high quality evidence.
 Observational studies (i.e., case-control studies and
controlled prospective or retrospective cohort studies) = low
quality evidence.
 Unsystematic observations (i.e., case series, case reports,
clinical experience) = very low quality evidence.

6. Quality of Research = Quality of Evidence
The “quality of evidence” is the confidence that you
have that the direction and the magnitude of the
estimated effect are correct.
Quality of evidence Suggested implications
High Further research is unlikely to change the confidence in an
estimated effect; we are confident that we can expect very
similar effect in a population for which the recommendation is
intended
Moderate Further research is likely to have an important impact on the
confidence in an estimated effect and may change that
estimate
Low Further research is very likely to have an important impact on
the confidence in an estimated effect and is likely to change
that estimate
Very low Any estimate of an effect is very uncertain

7. Quality of Evidence = Strength of Claim
 The higher the quality of the evidence, the stronger
the claim one can make about a treatment.
 As a high quality study is published other
researchers can begin to gain interest in the SE.
 With increased publications it becomes easier to get
funding for research.
 With increased publications it becomes easier to get
organizations to provide SE.

8. Levels of Evidence – Levels of Recommendations
The Image on the Right
Explores how a review of
sleep treatments classified
the level of evidence to
support a given treatment.
They might for instance
find Level I evidence for
CBT but a much lower
level of evidence for other
interventions.
Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters for the
psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN
WESTCHESTER-, 29(11), 1415.

9. Levels of Evidence = Levels of Recommendations
This image explores
recommendations a
clinician could make to
his clients or patients
based on evidence.
For instance if the
evidence is high we
would say it is an
intervention is a
“standard of
practice”
If the evidence is low
but positive we might
call the treatment an
“option for care”
Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters for the
psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN
WESTCHESTER-, 29(11), 1415.

10. Grade Approach to Clinical Recommendations
Below is an example of a clinical claim about sleep treatment that
was made using a similar approach to the Grade approach.
Recommendation: Psychological and behavioral interventions are effective and
recommended in the treatment of primary chronic insomnia .
Recommendation Level: This is a clinical standard
Review of Evidence: This is a new recommendation implied but not specifically
stated in the prior practice parameters. There were 17 studies identified in the
current review that evaluated the effects of treatment for primary insomnia,
including 5 randomized control trials with level I evidence that demonstrated the
effectiveness of psychological and behavioral interventions.
Plain Language Translation: Psychological and Behavioral treatments
are effective in treating chronic insomnia and should be offered to all patients with
insomnia as a standard of care.
Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters for the
psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN
WESTCHESTER-, 29(11), 1415.

11. Risk of Bias – Blinding Risk of Bias – General
 Blinding: Keeps the people
administering the study “blind” to the
which group they are in or working
with. It reduces the risk of unconscious
and conscious bias.
 Patient Blinded: If patients are aware
of receiving the treatment it can impact
outcomes. This is often inevitable in
psychological research.
 Caregiver blinded: The provider
offering treatment can impact the
outcomes with unconscious behaviors.
 Assessor blinded: The person
administering the tests or assessments
can unconsciously or consciously
impact the outcomes of a study with
their knowledge of what group a
participant is in.
 Base Line Differences: Happen when
there is a difference between the
control group and treatment group at
start of study.
 Selection Bias: When the subjects
chosen could impact the outcomes of a
study.
 Attrition: People leaving the study
during the study.
 Timing: Stopping study prior to timing
expected to find benefit.
Important Factors in Study Design

12. It is important to assess the quality of a study…
 Reasons to downgrade the quality of evidence (i.e. factors
that lower your confidence in estimated effect):
 Risk of bias – This is the risk that the researcher could consciously
or indirectly influence the outcome of a study.
 Inconsistency – This occurs when there is conflicting data in
different studies e.g. one study implies a treatment is effective
another implies less efficacy.
 Indirectness – Does the study compare groups or interventions
that are similar.
 Imprecision – This happens when there is a small sample size
(number of participants in a study) or the statistics have a wide
variation in them. Particularly when the confidence interval is wide.
 Reporting bias – The tendency of journals to publish positive
results, profit motive, or

14. Evidence GRADE: A,B,C,D

15. References
Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T.,
& Swick, T. (2006). Practice parameters for the psychological and behavioral
treatment of insomnia: an update. An American Academy of Sleep Medicine
report. SLEEP-NEW YORK THEN WESTCHESTER-, 29(11), 1415.
Schunemann, H., Ahmed, F., & Morgan., R. (2009) Guideline Developmetn Using
GRADE. McMaster University and Center for Disease Control. Pulled from web:
http://www.cdc.gov/vaccines/acip/recs/GRADE/downloads/guide-dev-grade.pdf