Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Presentation s rs
Similaire à Presentation s rs (20)
Dernier
Dernier (20)
Presentation s rs
- 1. Systematic Reviews What are they, and how do you conduct one? Julia Mueller School of Nursing, Midwifery and Social Work School of Computer Science
- 2. Structure 1) 2) 3) 4) 5) Definition The Cochrane Collaboration Review protocol Conducting the systematic review Conclusions 2
- 3. Definition 3
- 4. What are systematic reviews? Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions Systematic Review • A literature reviewing technique typically used in evidence-based medicine, but applicable to other fields of study • Gathering, critically appraising and summarising all relevant studies that address a certain research question Basis for decision making Cook et al., 1997 4
- 5. Why do we need systematic reviews? Definition The Cochrane Collaboration • The problem: Too many papers, too little time The Review protocol Conflicting information and Conducting the recommendations from different papers systematic Conclusions com Interpreting results www.bvallc. review Summaries of all literature on a given topic are needed so that the evidence is easily and quickly accessible 5
- 6. Why is it important to be systematic? Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions – – – – Concise and rigorous literature search Explicit search strategy Explicit inclusion and exclusion criteria Explicit methods for extracting and summarising data from primary studies Ensures reproducibility, reduces bias – Overall: COMPREHENSIVE, EXPLICIT, REPRODUCIBLE, RATIONAL 6
- 7. Systematic vs. traditional reviews Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions Traditional Systematic Research question Broad Specific Search strategy, data collection and interpretation Informal, subjective, not explicit Explicit, clear, with rationale Critical appraisal May or may not take Systematic using of included place, not using specific standardised tools studies tools Cipriani and Geddes, 2003 7
- 8. Different forms of literature reviews Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Literature reviews Systematic reviews Metaanalyses Traditional reviews Interpreting results Conclusions 8
- 9. What is a meta-analysis? Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results = “the use of statistical techniques to integrate and summarise the results of included studies” • Combining information from several studies larger sample size higher precision higher statistical power (= the probability that the statistical test will detect an effect that is really there) Conclusions Liberati et al., 2009 9
- 10. What is a meta-analysis? Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results • Whether you can conduct a meta-analysis depends on: – No. of available studies – Variability of methods and outcome measures • If a meta-analysis is not possible, a narrative/qualitative systematic review can be conducted Conclusions 10
- 11. Who can conduct a systematic review? Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • The review team should have skills in – – – – – Systematic review methods Information retrieval The relevant topic area Statistics Qualitative research methods • It is good practice to have at least 2 researchers involved minimise bias and error 11
- 12. 12
- 13. Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions = an international network • named after Archie Cochrane (1909-1988), a British epidemiologist, who advocated evidence-based medicine • Aim: Provide resources to enable well-informed decisions about health care • http://www.cochrane.org/ 13
- 14. Cochrane reviews Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions = Systematic reviews and meta-analyses published by the Cochrane Collaboration in the Cochrane Library • Specific methods, structure and format: Cochrane Handbook for Systematic Reviews of Interventions • “internationally recognised as the highest standard in evidence-based health care” • Existing Cochrane Reviews are updated regularly as new information becomes available Can result in changed conclusions 14
- 15. Cochrane Reviews over 5,000 Cochrane Reviews currently available in The Cochrane Library (November 2013) 15
- 16. Cochrane reviews in Manchester Definition • Cochrane groups with editorial base in Manchester The Cochrane Collaboration The Review protocol Conducting the systematic review • Cochrane Oral Health Group, School of Dentistry, The University of Manchester • The Cochrane Bone, Joint and Muscle Trauma Group (BJMTG) Interpreting results Conclusions 16
- 18. Review Protocol Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • Describes the methods to be used in the review – – – – – – – – – Background Review question Inclusion/Exclusion criteria Search strategy Study selection Quality assessment Data extraction Data synthesis Plans for dissemination • Specifying the methods in advance reduces the risk of bias • If modifications are required, they should be documented and justified 18
- 19. Background Definition The Cochrane Collaboration • Key contextual factors relevant to the review question • Explain why the review is required The Review protocol Conducting the systematic review Interpreting results Conclusions 19
- 20. Background: CS-related example Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions Example Kitchenham et al. (2007): “A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies.” • Two different types of cost estimation models • Within-company models developed using one company’s data • Cross-company models developed using datasets from several companies • Several studies have compared prediction accuracies with contradictory results • Important for small companies who do not have their own project data Systematic review to determine factors that influence the outcome of studies comparing within and cross-company models 20
- 21. Inclusion criteria Definition The Cochrane Collaboration The Review protocol • Language: The ideal is to include all available relevant evidence, but this often not feasible Language bias • Publication type/status – Full papers in peer-reviewed journals Conducting the systematic review Interpreting results Conclusions “Publication bias”: When studies with significant results are more likely to get published – – – – – – Contact authors Ongoing studies partially published as conference abstracts Reports and discussion papers Book chapters Theses … 21
- 22. Inclusion criteria Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • Use the “PICOS” elements: Population: Patients or Users (disease, age, gender, disability), software, type of technology, websites … Interventions: Type, main features …, e.g. assistive technology for disabled users… Comparison: No treatment, treatment as usual; between or within organisations … Use this to define your question Outcomes: e.g. survival, disease remission, interactions with the keyboard, commands executed, … Study design: Randomised controlled trials, cohort studies, case-control studies… 22
- 23. Review question Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results • Clear, as specific as possible • Example 1: Are psychological interventions effective in reducing chronic headache in children and adolescents? • Example 2: Are cross-company models significantly worse than within-company estimation models for predicting effort for software/web projects? Conclusions 23
- 24. Building the search strategy Definition The Cochrane Collaboration The Review protocol Conducting the systematic review 1. Describe all relevant PICOS elements (you might not want to include all!) 2. Organise them into concepts 3. Find all related words and synonyms 4. Search all synonyms for one concept with “OR” 5. Combine all concepts with “AND” Interpreting results Conclusions 24
- 25. Search: Health-related example Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions Example: Are psychological interventions effective in reducing chronic headache in children and adolescents? Population 1. Concept: “Children and adolescents” Child* OR adolescen* OR juvenile OR paediatric 2. Concept: “Headache” Headache OR migraine OR tension headache Interventions 3. Concept: “Psychological interventions” psycholog* OR psychotherapy OR biofeedback OR relaxation OR cognitive OR behavio#ral 1. AND 2. AND 3. 25
- 26. Search: CS-related example Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • Kitchenham et al. (2007). A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies. 1. Population: software OR application OR product OR Web OR WWW OR Internet OR World-Wide Web OR project OR development 2. Intervention: cross company OR cross organisation OR cross organization OR multiple-organizational OR multipleorganisational model OR modeling OR modelling effort OR cost OR resource estimation OR prediction OR assessment 3. Comparison: within-organisation OR within-organization OR within-organizational OR within-organisational OR single company OR single organisation 4. Outcome: Accuracy OR Mean Magnitude Relative Error • 1. AND 2. AND 3. AND 4. 26
- 28. Conducting a literature search Definition The Cochrane Collaboration Search electronic databases, e.g. ACM Digital Library, CiteSeer, INSPEC, … The Review protocol Conducting the systematic review Visually scan reference lists from identified studies Handsearch key journals and conference proceedings Interpreting results Conclusions Contact study authors, experts, manufacturers, other organisations … … 28
- 29. The Review protocol Conducting the systematic review Interpreting results # of records after duplicates removed # of records screened (title, abstracts) # of records excluded # of full-text articles assessed for eligibility # of full-text articles excluded, with reasons included Conclusions # of records identified through other sources screening The Cochrane Collaboration # records identified through database screening eligibility Definition identification Conducting a literature search # of studies included in data synthesis Liberati et al., 2009: The PRISMA statement for reporting systematic reviews 29
- 30. Quality assessment Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • A review should be based on the best quality evidence available • The quality of the included studies will impact on the reliability of the results of the review • Useful tools for quality assessment – CASP tools (Critical Appraisal Skills Programme), e.g. CASP checklist for randomised controlled trials – JADAD scale – … 30
- 31. Quality assessment: CS Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions Kitchenham et al. (2007) • 6 questions to assess the quality of included studies, developed by the researchers themselves • Examples: – Is the analysis process description complete? – Is it clear how accuracy was measured? – Were all model construction methods fully defined (tools and methods used)? 31
- 32. Data extraction Definition The Cochrane Collaboration • What information will be extracted from studies? • Data extraction sheet The Review protocol Conducting the systematic review Interpreting results Conclusions 32
- 33. Data synthesis Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results • If a meta-analysis is possible, data can be summed up statistically using quantitative methods • If not, data can be synthesised narratively • Narrative data synthesis usually ocurrs in form of table(s) and textual description Conclusions 33
- 34. Narrative data synthesis: Health-related example Author, dat e and location Participant demographics, attrition Intervention length, content and groups Measures and followup Reported results Author’s conclucions Segatto et al., 2010. Brazil N= 175 90.3% male Age: 21.8 (2.6) 16 – 25 years Mostly Caucasian (71%) Patients in emergency room Attrition: 14,9% , no differences btw. completers and dropouts found Motivational Interview (MI): person-centred techniques, also included EB (45 min.) Educational brochure (EB): 3 page informative general guidance on risks of alcohol (5 min.) Alcohol Consumption Questionnaire (ACQ), Rutgers Alcohol Problems Index (RAPI), Alcohol Consumption Risk Questionnaire (ACRQ), Measured at baseline and 3 months Time effect for: days of alcohol use, days with moderate use, days with heavy use, negative consequences, being in action stage No sig. differences in long-term change in riskperception No group x time effects MI alone not able to promote sig. and lasting changes Limitations: brief interventions usually target risk populations at an early stage, too short observation time … … … … … … 34
- 35. Narrative data synthesis: CS-related example 35
- 36. Interpreting results Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • Was there consensus across all studies? • Where did studies differ? What might be the reason? Did these studies differ in participant populations, methods/measures used …? • Did it make sense to pool the results? (heterogeneity/homogeneity) • Discuss the quality of the included studies: Can the results of the review be valid? • Discuss possible sources of bias: Publication bias, language bias… 36
- 37. Conclusions 37
- 38. Conclusions Definition The Cochrane Collaboration The Review protocol Conducting the systematic review • Systematic Reviewing is designed to ensure a comprehensive, reproducible and critical review of all the available literature on a given topic • Not all techniques shown here will be equally applicable to CS Interpreting results Conclusions • Other methods may be legitimate as long as they are stated explicitly and a rationale is provided 38
- 39. Thanks for listening. Any questions? Julia Mueller julia.mueller@postgrad.manchester.ac.uk http://jnmueller.wordpress.com/ School of Computer Science, LF1 Kilburn Building, Oxford Road, MANCHESTER, M13 9PL, UK 39
- 40. References Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions 2009. Systematic Reviews: CRD's guidance for undertaking reviews in health care. CRD, University of York. CIPRIANI, A. & GEDDES, J. 2003. Comparison of systematic and narrative reviews: the example of the atypical antipsychotics. Epidemiol Psichiatr Soc, 12, 146-53. COOK, D. J., MULROW, C. D. & HAYNES, R. B. 1997. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med, 126, 376-80. LIBERATI, A., ALTMAN, D. G., TETZLAFF, J., MULROW, C., GOTZSCHE, P. C., IOANNIDIS, J. P., CLARKE, M., DEVEREAUX, P. J., KLEIJNEN, J. & MOHER, D. 2009. The PRISMA statement for reporting systematic reviews and metaanalyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ, 339, b2700. 40
- 41. References Definition The Cochrane Collaboration The Review protocol Conducting the systematic review Interpreting results Conclusions • Here’s an example of a well-conducted systematic review and meta-analysis: COVENTRY, P. A. & HIND, D. 2007. Comprehensive pulmonary rehabilitation for anxiety and depression in adults with chronic obstructive pulmonary disease: Systematic review and meta-analysis. J Psychosom Res, 63, 551-65. Cochrane Handbook for systematic reviews for interventions: http://handbook.cochrane.org/ 41
- 42. References Definition The Cochrane Collaboration The Review protocol Conducting the systematic review • This is a systematic review in the area of CS: Kitchenham, B., Mendes, E., Travassos, G.H. (2007) A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies, IEEE Trans on SE, 33 (5), pp 316329. • Guidelines for performing Systematic Literature Reviews in Software Engineering Interpreting results Conclusions 42