Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
[Poster title]
[Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Ph...
Prochain SlideShare
Chargement dans…5
×

Internal 2012 - Software and model selection challenges in meta-analysis

Software and model selection challenges in meta-analysis: MetaEasy, model assumptions, homogeneity and IPD

Livres associés

Gratuit avec un essai de 30 jours de Scribd

Tout voir

Livres audio associés

Gratuit avec un essai de 30 jours de Scribd

Tout voir
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Internal 2012 - Software and model selection challenges in meta-analysis

  1. 1. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary Software and model selection challenges in meta-analysis MetaEasy, model assumptions, homogeneity and IPD Evan Kontopantelis David Reeves Centre for Primary Care Institute of Population Health University of Manchester Health Methodology Research Seminar Manchester, 6 Nov 2012 Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  2. 2. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  3. 3. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  4. 4. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Timeline ‘Meta’ is a Greek preposition meaning ‘after’, so meta-analysis =⇒ post-analysis Efforts to pool results from individual studies back as far as 1904 The first attempt that assessed a therapeutic intervention was published in 1955 In 1976 Glass first used the term to describe a "statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings" Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  5. 5. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Meta-analysing reported study results A two-stage process the relevant summary effect statistics are extracted from published papers on the included studies these are then combined into an overall effect estimate using a suitable meta-analysis model However, problems often arise papers do not report all the statistical information required as input papers report a statistic other than the effect size which needs to be transformed with a loss of precision a study might be too different to be included (population clinically heterogeneous) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  6. 6. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Individual Patient Data IPD These problems can be avoided when IPD from each study are available outcomes can be easily standardised clinical heterogeneity can be addressed with subgroup analyses and patient-level covariate controlling Can be analysed in a single- or two-stage process mixed-effects regression models can be used to combine information across studies in a single stage this is currently the best approach, with the two-stage method being at best equivalent in certain scenarios Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  7. 7. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Outline 1 Meta-analysis overview The heterogeneity issue 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  8. 8. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue When Robin ruins the party... Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  9. 9. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Heterogeneity tread lightly! When the effect of the intervention varies significantly from one study to another It can be attributed to clinical and/or methodological diversity Clinical: variability that arises from different populations, interventions, outcomes and follow-up times Methodological: relates to differences in trial design and quality Detecting quantifying and dealing with heterogeneity can be very hard Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  10. 10. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Absence of heterogeneity Assumes that the true effects of the studies are all equal and deviations occur because of imprecision of results Analysed with the fixed-effects method Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  11. 11. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary The heterogeneity issue Presence of heterogeneity Assumes that there is variation in the size of the true effect among studies (in addition to the imprecision of results) Analysed with random-effects methods Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  12. 12. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  13. 13. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Challenges with two-stage meta-analysis Heterogeneity is common and the fixed-effect model is under fire Methods are asymptotic: accuracy improves as studies increase. But what if we only have a handful, as is usually the case? Almost all random-effects models (except Profile Likelihood) do not take into account the uncertainty in ˆτ2. Is this, practically, a problem? DerSimonian-Laird is the most common method of analysis, since it is easy to implement and widely available, but is it the best? Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  14. 14. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Challenges with two-stage meta-analysis ...continued Can be difficult to organise since... outcomes likely to have been disseminated using a variety of statistical parameters appropriate transformations to a common format required tedious task, requiring at least some statistical adeptness Parametric random-effects models assume that both the effects and errors are normally distributed. Are methods robust? Sometimes heterogeneity is estimated to be zero, especially when the number of studies is small. Good news? Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  15. 15. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Outline 1 Meta-analysis overview 2 Two-stage meta-analysis the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  16. 16. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Organising Data initially collected using data extraction forms A spreadsheet is the next logical step to summarise the reported study outcomes and identify missing data Since in most cases MS Excel will be used we developed an add-in that can help with most processes involved in meta-analysis More useful when the need to combine differently reported outcomes arises Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  17. 17. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What it can do Help with the data collection using pre-formatted worksheets. Its unique feature, which can be supplementary to other meta-analysis software, is implementation of methods for calculating effect sizes (& SEs) from different input types For each outcome of each study... it identifies which methods can be used calculates an effect size and its standard error selects the most precise method for each outcome Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  18. 18. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What it can do ...continued Creates a forest plot that summarises all the outcomes, organised by study Uses a variety of standard and advanced meta-analysis methods to calculate an overall effect a variety of options is available for selecting which outcome(s) are to be meta-analysed from each study Plots the results in a second forest plot Reports a variety of heterogeneity measures, including Cochran’s Q, I2, HM 2 and ˆτ2 (and its estimated confidence interval under the Profile Likelihood method) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  19. 19. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Advantages Free (provided Microsoft Excel is available) Easy to use and time saving Extracted data from each study are easily accessible, can be quickly edited or corrected and analysis repeated Choice of many meta-analysis models, including some advanced methods not currently available in other software packages (e.g. Permutations, Profile Likelihood, REML) Unique forest plot that allows multiple outcomes per study Effect sizes and standard errors can be exported for use in other meta-analysis software packages Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  20. 20. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Installing Latest version available from www.statanalysis.co.uk Compatible with Excel 2003, 2007 and 2010 Manual provided but also described in: Kontopantelis E and Reeves D MetaEasy: A Meta-Analysis Add-In for Microsoft Excel Journal of Statistical Software, 30(7):1-25, 2009 Play video clip Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  21. 21. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Outline 1 Meta-analysis overview 2 Two-stage meta-analysis the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  22. 22. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Stata implementation MetaEasy methods implemented in Stata under: metaeff, which uses the different study input to provide effect sizes and SEs metaan, which meta-analyses the study effects with a fixed-effect or one of five available random-effects models To install, type in Stata: ssc install <command name> help <command name> Described in: Kontopantelis E and Reeves D metaan: Random-effects meta-analysis The Stata Journal, 10(3):395-407, 2010 Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  23. 23. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Outline 1 Meta-analysis overview 2 Two-stage meta-analysis the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  24. 24. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Many random-effects methods which to use? DerSimonian-Laird (DL): Moment-based estimator of both within and between-study variance Maximum Likelihood (ML): Improves the variance estimate using iteration Restricted Maximum Likelihood (REML): an ML variation that uses a likelihood function calculated from a transformed set of data Profile Likelihood (PL): A more advanced version of ML that uses nested iterations for converging Permutations method (PE): Simulates the distribution of the overall effect using the observed data Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  25. 25. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Performance evaluation our approach Simulated various distributions for the true effects: Normal Skew-Normal Uniform Bimodal Created datasets of 10,000 meta-analyses for various numbers of studies and different degrees of heterogeneity, for each distribution Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  26. 26. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Performance evaluation our approach Compared all methods in terms of: Coverage, the rate of true negatives when the overall true effect is zero Power, the rate of true positives when the true overall effect is non-zero Confidence Interval performance, a measure of how wide the (estimated around the effect) CI is, compared to its true width. Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  27. 27. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Homogeneity Zero between study variance 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Coverage 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Power (25th centile) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE zero between study variance Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  28. 28. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Coverage performance Small and large heterogeneity under various distributional assumptions 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Coverage 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Coverage Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  29. 29. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Power performance Small and large heterogeneity under various distributional assumptions 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Power (25th centile) 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Power (25th centile) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  30. 30. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 CI performance Small and large heterogeneity under various distributional assumptions 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=1.18 − I²=15% (normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=1.18 − I²=15% (skew−normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=1.18 − I=15% (bimodal distribution) Overall estimation performance (medians) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=1.18 − I=15% (uniform distribution) Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=0, ku=3 H²=2.78 − I²=64% (normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk=2, ku=9 H²=2.78 − I²=64% (skew−normal distribution) Confidence Interval performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE sk1=0, ku1=3, var1=.1, sk2=0, ku2=3, var2=.1 H=2.78 − I=64% (bimodal distribution) Overall estimation performance (medians) 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies FE DL ML REML PL PE H=2.78 − I=64% (uniform distribution) Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  31. 31. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Coverage by method Large heterogeneity across various between-study variance distributions 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Coverage 0.70 0.75 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Coverage 0.80 0.85 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Coverage 0.90 0.95 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Coverage Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  32. 32. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Power by method Large heterogeneity across various between-study variance distributions 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Power (25th centile) 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Power (25th centile) 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Power (25th centile) 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 % 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Power (25th centile) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  33. 33. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 CI performance by method Large heterogeneity across various between-study variance distributions 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% FE − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% DL − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% ML − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% REML − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PL − Overall estimation performance 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2 5 8 11 14 17 20 23 26 29 32 35 number of studies Zero variance Normal Skew−normal Bimodal Uniform H=2.78 − I=64% PE − Overall estimation performance Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  34. 34. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Which method then? Within any given method, the results were consistent across all types of distribution shape Therefore methods are highly robust against even severe violations of the assumption of normality Choose PE if the priority is an accurate Type I error rate (false positive) But low power makes it a poor choice when control of the Type II error rate (false negative) is also important and it cannot be used with less than 6 studies Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  35. 35. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Which method then? For very small study numbers (≤5) only PL gives coverage >90% and an acccurate CI PL has a ‘reasonable’ coverage in most situations, especially for moderate and large heterogeneiry, giving it an edge over other methods REML and DL perform similarly and better than PL only when heterogeneity is low (I2 < 15%) The computational complexity of REML is not justified Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  36. 36. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Outline 1 Meta-analysis overview 2 Two-stage meta-analysis the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  37. 37. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 Bring on the champagne? Does not necessarily mean homogeneity Most methods use biased estimators and not uncommon to get a negative ˆτ2 which is set to 0 by the model We identified a large percentage of cases where the estimators failed to identify existing heterogeneity In our simulations, for 5 studies and I2 ≈ 29%: 30% of the meta-analyses were erroneously estimated to be homogeneous under the DL method 32% for REML and 48% for ML-PL Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  38. 38. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary the MetaEasy add-in metaeff & metaan Methods and performance ˆτ2 = 0 What does it mean? In these cases coverage was substandard and was over 10% lower than in cases where ˆτ2 > 0, on average The problem becomes less profound as the number of studies and the level of heterogeneity increase Better estimators are needed There might be a large number of meta-analyses of ‘homogeneous’ studies which have reached a wrong conclusion Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  39. 39. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  40. 40. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Forest plot One advantage of two-stage meta-analysis is the ability to convey information graphically through a forest plot study effects available after the first stage of the process, and can be used to demonstrate the relative strength of the intervention in each study and across all informative, easy to follow and particularly useful for readers with little or no methodological experience key feature of meta-analysis and always presented when two-stage meta-analyses are performed In one-stage meta-analysis, only the overall effect is calculated and creating a forest-plot is not straightforward Enter ipdforest Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  41. 41. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis A practical guide ipdforest methods ipdforest examples Power 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  42. 42. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power The hypothetical study Individual patient data from randomised controlled trials For each trial we have a binary control/intervention membership variable baseline and follow-up data for the continuous outcome covariates Assume measurements consistent across trials and standardisation is not required We will explore linear random-effects models with the xtmixed command; application to the logistic case using xtmelogit should be straightforward In the models that follow, in general, we denote fixed effects with ‘γ’s and random effects with ‘β’s Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  43. 43. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 1 fixed common intercept; random treatment effect; fixed effect for baseline Yij = γ0 + β1jgroupij + γ2Ybij + ij ij ∼ N(0, σ2 j ) β1j = γ1 + u1j u1j ∼ N(0, τ1 2) i: the patient j: the trial Yij: the outcome γ0: fixed common intercept β1j: random treatment effect for trial j γ1: mean treatment effect groupij: group membership γ2: fixed baseline effect Ybij: baseline score u1j: random treatment effect for trial j τ1 2: between trial variance ij: error term σ2 j : within trial variance for trial j Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  44. 44. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 1 fixed common intercept; random treatment effect; fixed effect for baseline Possibly the simplest approach In Stata it can be expressed as xtmixed Y i.group Yb || studyid:group, nocons where studyid, the trial identifier group, control/intervention membership Y and Yb, endpoint and baseline scores note that the nocons option suppresses estimation of the intercept as a random effect Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  45. 45. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 2 fixed trial specific intercepts; random treatment effect; fixed trial-specific effects for baseline Common intercept & fixed baseline difficult to justify A more accepted model allows for different fixed intercepts and fixed baseline effects for each trial: Yi j = γ0j + β1j groupi j + γ2j Ybi j + i j β1j = γ1 + u1j where γ0j the fixed intercept for trial j γ2j the fixed baseline effect for trial j In Stata expressed as: xtmixed Y i.group i.studyid Yb1 Yb2 Yb3 Yb4 || studyid:group, nocons where Yb‘i’=Yb if studyid=‘i’ and zero otherwise Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  46. 46. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 3 random trial intercept; random treatment effect; fixed trial-specific effects for baseline Another possibility, althought contentious, is to assume trial intercepts are random (e.g. multi-centre trial): Yi j = β0j + β1j groupi j + γ2j Ybi j + i j β0j = γ0 + u0j β1j = γ1 + u1j wiser to assume random effects correlation ρ = 0: i j ∼ N(0, σ2 j ) u0j ∼ N(0, τ2 0 ) u1j ∼ N(0, τ2 1 ) cov(u0j , u1j ) = ρτ0τ1 In Stata expressed as: xtmixed Y i.group Yb1 Yb2 Yb3 Yb4 || studyid:group, cov(uns) cov(uns): allows for distinct estimation of all RE variance-covariance components Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  47. 47. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 4 random trial intercept; random treatment effect; random effects for baseline The baseline could also have been modelled as a random-effect: Yi j = β0j + β1j groupi j + β2j Ybi j + i j β0j = γ0 + u0j β1j = γ1 + u1j β2j = γ2 + u2j as before, non-zero random effects correlations: u0j ∼ N(0, τ2 0 ) u1j ∼ N(0, τ2 1 ) u2j ∼ N(0, τ2 2 ) cov(u0j , u1j ) = ρ1τ0τ1 cov(u0j , u2j ) = ρ2τ0τ2 cov(u1j , u2j ) = ρ3τ1τ2 In Stata expressed as: xtmixed Y i.group Yb || studyid:group Yb, cov(uns) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  48. 48. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Model 5 Interactions and covariates A covariate or an interaction term can be modelled as a fixed or random effect Assuming continuous and standardised variable age we can expand Model 2 to include fixed effects for both age and its interaction with the treatment: Yi j = γ0j +β1j groupi j +γ2j Ybi j +γ3agei j +γ4groupi j agei j + i j β1j = γ1 + u1j In Stata expressed as: xtmixed Y i.group i.studyid Yb1 Yb2 Yb3 Yb4 age i.group#c.age || studyid:group, nocons If modelled as a random effect, non-convergence issues more likely to be encountered Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  49. 49. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis A practical guide ipdforest methods ipdforest examples Power 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  50. 50. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power General ipdforest is issued following an IPD meta-analysis that uses mixed effects two-level regression, with patients nested within trials and a linear model (xtmixed) or logistic model (xtmelogit) Provides a meta-analysis summary table and a forest plot Trial effects are calculated within ipdforest Can calculate and report both main and interaction effects Overall effect(s) and variance estimates are extracted from the preceding regression Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  51. 51. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Process ipdforest estimates individual trial effects and their standard errors using one-level linear or logistic regressions Following xtmixed, regress is used and following xtmelogit, logit is used, for each trial ipdforest controls these regressions for fixed- or random-effects covariates that were specified in the preceding two-level regression User has full control over included covariates in the command (e.g. specification as fixed- or random-effects) But we strongly recommend using the same specifications Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  52. 52. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Estimation details In the estimation of individual trial effects, ipdforest controls for a random-effects covariate (i.e. allowing the regression coefficient to vary by trial) by including the covariate as an independent variable in each regression Control for a fixed-effect covariate (regression coefficient assumed constant across trials and given by the coefficient estimated under two-level model) is a little more complex. Not possible to specify a fixed value for a regression coefficient under regress and the continuous outcome variable is adjusted by subtracting the contribution of the fixed covariates to its values in a first step prior to analysis For a binary outcome the equivalent is achieved through use of the offset option in logit Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  53. 53. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Reported heterogeneity For continuous outcomes, ipdforest reports, I2 and H2 M, based on the xtmixed output For binary outcomes, an estimate of the within-trial variance is not reported under xtmelogit and hence heterogeneity measures cannot be computed Between-trial variability estimate ˆτ2 and its confidence interval is reported under both models Random-effects model approach is more conservative, accounting for even low levels of τ2 Described in: Kontopantelis E and Reeves D A short guide and a forest plot command (ipdforest) for one-stage meta-analysis The Stata Journal, in print Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  54. 54. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis A practical guide ipdforest methods ipdforest examples Power 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  55. 55. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Depression intervention We apply the ipdforest command to a dataset of 4 depression intervention trials Complete information in terms of age, gender, control/intervention group membership, continuous outcome baseline and endpoint values for 518 patients Results not published yet; we use fake author names and generated random continuous & binary outcome variables, while keeping the covariates at their actual values Introduced correlation between baseline and endpoint scores and between-trial variability Logistic IPD meta-analysis, followed by ipdforest Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  56. 56. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Dataset . use ipdforest_example.dta, . describe Contains data from ipdforest_example.dta obs: 518 vars: 17 6 Feb 2012 11:14 size: 20,202 storage display value variable name type format label variable label studyid byte %22.0g stid Study identifier patid int %8.0g Patient identifier group byte %20.0g grplbl Intervention/control group sex byte %10.0g sexlbl Gender age float %10.0g Age in years depB byte %9.0g Binary outcome, endpoint depBbas byte %9.0g Binary outcome, baseline depBbas1 byte %9.0g Bin outcome baseline, trial 1 depBbas2 byte %9.0g Bin outcome baseline, trial 2 depBbas5 byte %9.0g Bin outcome baseline, trial 5 depBbas9 byte %9.0g Bin outcome baseline, trial 9 depC float %9.0g Continuous outcome, endpoint depCbas float %9.0g Continuous outcome, baseline depCbas1 float %9.0g Cont outcome baseline, trial 1 depCbas2 float %9.0g Cont outcome baseline, trial 2 depCbas5 float %9.0g Cont outcome baseline, trial 5 depCbas9 float %9.0g Cont outcome baseline, trial 9 Sorted by: studyid patid Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  57. 57. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power ME logistic regression model - continuous interaction fixed trial intercepts; fixed trial effects for baseline; random treatment and age effects . xtmelogit depB group agec sex i.studyid depBbas1 depBbas2 depBbas5 depBbas9 i > .group#c.agec || studyid:group agec, var nocons or Mixed-effects logistic regression Number of obs = 518 Group variable: studyid Number of groups = 4 Obs per group: min = 42 avg = 129.5 max = 214 Integration points = 7 Wald chi2(11) = 42.06 Log likelihood = -326.55747 Prob > chi2 = 0.0000 depB Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] group 1.840804 .3666167 3.06 0.002 1.245894 2.71978 agec .9867902 .0119059 -1.10 0.270 .9637288 1.010403 sex .7117592 .1540753 -1.57 0.116 .4656639 1.087912 studyid 2 1.050007 .5725516 0.09 0.929 .3606166 3.057303 5 .8014551 .5894511 -0.30 0.763 .189601 3.387799 9 1.281413 .6886057 0.46 0.644 .4469619 3.673735 depBbas1 3.152908 1.495281 2.42 0.015 1.244587 7.987253 depBbas2 4.480302 1.863908 3.60 0.000 1.982385 10.12574 depBbas5 2.387336 1.722993 1.21 0.228 .5802064 9.823007 depBbas9 1.881203 .7086507 1.68 0.093 .8990569 3.936262 group#c.agec 1 1.011776 .0163748 0.72 0.469 .9801858 1.044385 _cons .5533714 .2398342 -1.37 0.172 .2366472 1.293993 Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] studyid: Independent var(group) 8.86e-21 2.43e-11 0 . var(agec) 5.99e-18 4.40e-11 0 . Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  58. 58. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power ipdforest modelling main effect and interaction . ipdforest group, fe(sex) re(agec) ia(agec) or One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group) Study Effect [95% Conf. Interval] % Weight Hart 2005 2.118 0.942 4.765 19.88 Richards 2004 2.722 1.336 5.545 30.69 Silva 2008 2.690 0.748 9.676 8.11 Kompany 2009 1.895 0.969 3.707 41.31 Overall effect 1.841 1.246 2.720 100.00 One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Interaction effect (group x agec) Study Effect [95% Conf. Interval] % Weight Hart 2005 0.972 0.901 1.049 19.88 Richards 2004 0.995 0.937 1.055 30.69 Silva 2008 0.987 0.888 1.098 8.11 Kompany 2009 1.077 1.015 1.144 41.31 Overall effect 1.012 0.980 1.044 100.00 Heterogeneity Measures value [95% Conf. Interval] I^2 (%) . H^2 . tau^2 est 0.000 0.000 . Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  59. 59. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Forest plots main effect and interaction Overall effect Kompany 2009 Silva 2008 Richards 2004 Hart 2005 Studies 0 2 3 4 5 6 7 8 9 101 Effect sizes and CIs (ORs) Main effect (group) Overall effect Kompany 2009 Silva 2008 Richards 2004 Hart 2005 Studies 0 .2 .4 .6 .8 1.2 1.41 Effect sizes and CIs (ORs) Interaction effect (group x agec) Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  60. 60. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power ME logistic regression model - binary interaction fixed trial intercepts; fixed trial effects for baseline & age cat; random treatment effect . xtmelogit depB group i.agecat sex i.studyid depBbas1 depBbas2 depBbas5 depBba > s9 i.group#i.agecat || studyid:group, var nocons or Mixed-effects logistic regression Number of obs = 518 Group variable: studyid Number of groups = 4 Obs per group: min = 42 avg = 129.5 max = 214 Integration points = 7 Wald chi2(11) = 42.67 Log likelihood = -326.24961 Prob > chi2 = 0.0000 depB Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] group 1.931763 .5702338 2.23 0.026 1.083151 3.445233 1.agecat .7389353 .213031 -1.05 0.294 .4199616 1.300179 sex .7173044 .154959 -1.54 0.124 .469698 1.095439 studyid 2 1.029638 .5608887 0.05 0.957 .3539959 2.994823 5 .828577 .6082301 -0.26 0.798 .1965599 3.492777 9 1.266728 .6812765 0.44 0.660 .4414556 3.634794 depBbas1 3.196139 1.515334 2.45 0.014 1.261999 8.094542 depBbas2 4.625802 1.923028 3.68 0.000 2.047989 10.44832 depBbas5 2.354493 1.692149 1.19 0.233 .5756364 9.630454 depBbas9 1.902359 .7183054 1.70 0.089 .9075907 3.987446 group#agecat 1 0 .9277371 .3558053 -0.20 0.845 .4374944 1.967331 1 1 1 (omitted) _cons .6175744 .2800575 -1.06 0.288 .253914 1.502076 Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] studyid: Identity var(group) 2.24e-19 1.24e-10 0 . Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  61. 61. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power ipdforest modelling main effect and interaction . ipdforest group, fe(sex i.agecat) ia(i.agecat) or One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group), agecat=0 Study Effect [95% Conf. Interval] % Weight Hart 2005 3.594 1.133 11.402 19.88 Richards 2004 2.814 1.034 7.657 30.69 Silva 2008 3.750 0.585 24.038 8.11 Kompany 2009 1.010 0.475 2.147 41.31 Overall effect 1.792 1.079 2.976 100.00 One-stage meta-analysis results using xtmelogit (ML method) and ipdforest Main effect (group), agecat=1 Study Effect [95% Conf. Interval] % Weight Hart 2005 1.166 0.376 3.617 19.88 Richards 2004 2.566 1.007 6.543 30.69 Silva 2008 1.935 0.340 11.003 8.11 Kompany 2009 3.551 1.134 11.126 41.31 Overall effect 1.932 1.083 3.445 100.00 Heterogeneity Measures value [95% Conf. Interval] I^2 (%) . H^2 . tau^2 est 0.000 0.000 . Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  62. 62. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Forest plots main effect for each age group Overall effect Meyer 2009 Lovell 2008 Willemse 2004 Mead 2005 Studies 0 2 4 6 8 10 12 14 16 18 20 22 24 261 Effect sizes and CIs (ORs) Main effect (group), agecat=0 Overall effect Meyer 2009 Lovell 2008 Willemse 2004 Mead 2005 Studies 0 2 4 6 8 10 121 Effect sizes and CIs (ORs) Main effect (group), agecat=1 Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  63. 63. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis A practical guide ipdforest methods ipdforest examples Power 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  64. 64. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Power calculations to detect a moderator effect The best approach is through simulations As always, numerous assumptions need to be made effect sizes (main and interaction) exposure and covariate distributions correlation between variables within-study (error) variance between-study (error) variance - ICC Generate 100s of data sets using the assumed model(s) Estimate what % of these give a significant p-value for the interaction Can be trial and error till desired power level achieved Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  65. 65. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary A practical guide ipdforest methods ipdforest examples Power Research on power calculations with simulations The Stata Journal (2002) 2, Number 2, pp. 107–124 Power by simulation A. H. Feiveson NASA Johnson Space Center alan.h.feiveson1@jsc.nasa.gov Abstract. This paper describes how to write Stata programs to estimate the power of virtually any statistical test that Stata can perform. Examples given include the t test, Poisson regression, Cox regression, and the nonparametric rank-sum test. Keywords: st0010, power, simulation, random number generation, postfile, copula, sample size 1 Introduction Statisticians know that in order to properly design a study producing experimental data, one needs to have some idea of whether the scope of the study is sufficient to give a reasonable expectation that hypothesized effects will be detectable over experimental error. Most of us have at some time used published procedures or canned software to obtain sample sizes for studies intended to be analyzed by t tests, or even analysis of variance. However, with more complex methods for describing and analyzing both continuous and discrete data (for example, generalized linear models, survival models, selection models), possibly with provisions for random effects and robust variance es- timation, the closed-form expressions for power or for sample sizes needed to achieve a certain power do not exist. Nevertheless, Stata users with moderate programming ability can write their own routines to estimate the power of virtually any statistical test that Stata can perform. 2 General approach to estimating power 2.1 Statistical inference Before describing methodology for estimating power, we first define the term “power” and illustrate the quantities that can affect it. In this article, we restrict ourselves to classical (as opposed to Bayesian) methodology for performing statistical inference on the effect of experimental variable(s) X on a response Y . This is accomplished by calculating a p-value that attempts to probabilistically describe the extent to which the data are consistent with a null hypothesis, H0, that X has no effect on Y . We would then reject H0 if the p-value is less than a predesignated threshold α. It should be pointed out that considerable criticism of the use of p-values for statistical inference has appeared in the literature; for example, Berger and Sellke (1987), Loftus (1993), and COMMENTARY Open Access Simulation methods to estimate design power: an overview for applied research Benjamin F Arnold1*† , Daniel R Hogan2† , John M Colford Jr1 and Alan E Hubbard3 Abstract Background: Estimating the required sample size and statistical power for a study is an integral part of study design. For standard designs, power equations provide an efficient solution to the problem, but they are unavailable for many complex study designs that arise in practice. For such complex study designs, computer simulation is a useful alternative for estimating study power. Although this approach is well known among statisticians, in our experience many epidemiologists and social scientists are unfamiliar with the technique. This article aims to address this knowledge gap. Methods: We review an approach to estimate study power for individual- or cluster-randomized designs using computer simulation. This flexible approach arises naturally from the model used to derive conventional power equations, but extends those methods to accommodate arbitrarily complex designs. The method is universally applicable to a broad range of designs and outcomes, and we present the material in a way that is approachable for quantitative, applied researchers. We illustrate the method using two examples (one simple, one complex) based on sanitation and nutritional interventions to improve child growth. Results: We first show how simulation reproduces conventional power estimates for simple randomized designs over a broad range of sample scenarios to familiarize the reader with the approach. We then demonstrate how to extend the simulation approach to more complex designs. Finally, we discuss extensions to the examples in the article, and provide computer code to efficiently run the example simulations in both R and Stata. Conclusions: Simulation methods offer a flexible option to estimate statistical power for standard and non- traditional study designs and parameters of interest. The approach we have described is universally applicable for evaluating study designs used in epidemiologic and social science research. Keywords: Computer Simulation, Power, Research Design, Sample Size Background Estimating the sample size and statistical power for a study is an integral part of study design and has profound consequences for the cost and statistical precision of a study. There exist analytic (closed-form) power equations for simple designs such as parallel randomized trials with treatment assigned at the individual level or cluster (group) level [1]. Statisticians have also derived equations to estimate power for more complex designs, such as designs with two levels of correlation [2] or designs with two levels of correlation, multiple treatments and attrition [3]. The advantage of using an equation to esti- mate power for study designs is that the approach is fast and easy to implement using existing software. For this reason, power equations are used to inform most study designs. However, in our applied research we have routi- nely encountered study designs that do not conform to conventional power equations (e.g. multiple treatment interventions, where one treatment is deployed at the group level and a second at the individual level). In these situations, simulation techniques offer a flexible alterna- tive that is easy to implement in modern statistical software. Here, we provide an overview of a general method to estimate study power for randomized trials based on a * Correspondence: benarnold@berkeley.edu † Contributed equally 1 Arnold et al. BMC Medical Research Methodology 2011, 11:94 http://www.biomedcentral.com/1471-2288/11/94 Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  66. 66. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary Outline 1 Meta-analysis overview 2 Two-stage meta-analysis 3 One-stage meta-analysis 4 Summary Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  67. 67. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary Two-stage meta-analysis MetaEasy can help you organise your meta-analysis and can be especially useful if you need to combine continuous and binary outcome. Methods implemented in Stata under metaeff and metaan A zero ˆτ2 is a reason to worry; heterogeneity might be there but we cannot measure or account for in the model If ˆτ2 > 0, even if very small, use a random-effects model The DL method works reasonably well, under all distributions, especially for low levels of heterogeneity Profile likelihood, which takes into account the uncertaintly in ˆτ2, works better when I2 ≥ 15% Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  68. 68. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Meta-analysis overview Two-stage meta-analysis One-stage meta-analysis Summary One-stage meta-analysis A few different approaches exist for conducting one-stage IPD meta-analysis Stata can cope through the xtmixed and the xtmelogit commands The ipdforest command aims to help meta-analysts calculate trial effects display results in standard meta-analysis tables produce familiar and ‘expected’ forest-plots The easiest way to calculate power to detect complex effects is through simulations Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  69. 69. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Appendix Thank you! Key references Comments, suggestions: e.kontopantelis@manchester.ac.uk Kontopantelis, Reeves Software and model selection challenges in meta-analysis
  70. 70. [Poster title] [Replace the following names and titles with those of the actual contributors: Helge Hoeing, PhD1; Carol Philips, PhD2; Jonathan Haas, RN, BSN, MHA3, and Kimberly B. Zimmerman, MD4 1[Add affiliation for first contributor], 2[Add affiliation for second contributor], 3[Add affiliation for third contributor], 4[Add affiliation for fourth contributor] Appendix Thank you! Key references References Kontopantelis E, Reeves D. A short guide and a forest plot command (ipdforest) for one-stage meta-analysis. The Stata Journal, in print. Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A simulation study. Stat Methods Med Res, published online Dec 9 2010. Kontopantelis E, Reeves D. metaan: Random-effects meta-analysis. The Stata Journal, 10(3):395-407, 2010. Kontopantelis E, Reeves D. MetaEasy: A Meta-Analysis Add-In for Microsoft Excel. Journal of Statistical Software, 30(7):1-25, 2009. Kontopantelis, Reeves Software and model selection challenges in meta-analysis

    Soyez le premier à commenter

    Identifiez-vous pour voir les commentaires

Software and model selection challenges in meta-analysis: MetaEasy, model assumptions, homogeneity and IPD

Vues

Nombre de vues

512

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

4

Actions

Téléchargements

7

Partages

0

Commentaires

0

Mentions J'aime

0

×