Solas Webinar Slides

Statistical Solutions | Registered in Ireland, Reg. No 233638 | www.statsols.com
Visual Sensitivity Analysis
for Missing Data
Aodhán O’Leary
Andrew Grannell
Webinar
4th June 2015

Overview
• Missing Data Analysis & Imputation
• Sensitivity Analysis & Tipping Points
• SOLAS Tipping Points Demo
• Round-up and Q&A

Aodhán O’Leary
Statistician
MSc. (Stat.) , University College Cork
Andrew Grannell
Head of Research and Development
MSc. (Stat.), University College Cork
Who we are

Missing Data Analysis
• Missing Data Mechanisms (MCAR, MAR, MNAR)
• Complete Case analysis…
• LOCF, LVCF, BOCF, BVCF…
• Inverse probability weighting
• Maximum Likelihood estimation
• Multiple Imputation
• And others…

Imputation Overview
• “We use MI to replace missing values with plausible
values in order to exploit the information that was
recorded”
- Andridge & Little, 2010
• MI methods can be classed into two main types
– Model based methods
– Hot deck methods
• Techniques incorporate variability to give a number of
imputed values for each missing data point

Sensitivity Analysis
• Consists of THREE main steps:
1. Drawing conclusions under working
assumptions about missing data,
2. Identifying a set of plausible alternative
assumptions,
3. Studying the variation in the statistical
output & conclusions under the alternative
settings

Regulatory Advice on Sensitivity Analysis
• “Examining sensitivity to the assumptions about the
missing data mechanism should be a mandatory
component of reporting.” [1]
• “A targeted range of…‘sensitivity analyses’ can
help to investigate and understand the robustness
of estimates…” [2]
• “‘Tipping point’ analysis recommended by NRC
appealing for regulators” [3]

Regulatory Advice on Sensitivity Analysis
• “This…survey of Japan- and foreign-based
pharmaceutical manufacturers revealed that
missing data are frequent in confirmatory
trials…the most popular method for handling
missing data was LOCF imputation…LOCF is not
justified in disease areas in which missing data are
frequent” [4]
• “One type of supportive analysis might investigate
treatment effects according to a range of different
estimands.” [1]

Treatment Control
Treatment Effect Estimation

Treatment Control
= 𝑆𝑢𝑐𝑐𝑒𝑠𝑠
= 𝐹𝑎𝑖𝑙𝑢𝑟𝑒

Treatment Control
Treatment: 𝑡𝑖 = 1
Control: 𝑡𝑖 = 0

Treatment Control
Treatment: 𝑡𝑖 = 1 Success: 𝑦𝑖 𝑡 = 1
Control: 𝑡𝑖 = 0 Failure: 𝑦𝑖 𝑡 = 0

Treatment Control
𝜏 =
𝑖=1
𝑁
𝑦𝑖(1) −
𝑖=0
𝑁
𝑦𝑖(0) 𝑁

Treatment Control
𝜏 =
𝑖=1
𝑁
𝑦𝑖(1) −
𝑖=0
𝑁
𝑦𝑖(0) 𝑁 𝜏 = 𝑦 𝑇 − 𝑦 𝐶

T C T C
Effect Estimation with Missing Data

T C T C
𝜏 = 𝑦 𝑇 − 𝑦 𝐶

T C T C
𝜏 = 𝑦 𝑇 − 𝑦 𝐶
𝜏 =
𝑦 𝑇 𝑁 𝑇 + 𝑦 𝑇 𝑁 𝑇
𝑁 𝑇
−
𝑦 𝐶 𝑁 𝐶 + 𝑦 𝐶 𝑁 𝐶
𝑁 𝐶
obs obs obs obsmis mis mis mis

Tipping Point Plotting

=
=
Control Group
Treatment Group
Worst Case

=
=
Control Group
Treatment Group
=
=
Worst Case Best Case
Control Group
Treatment Group

Enhanced TP Plots Background
• Uses the above plots as the basis for
analysing missing data.
• Includes the data from imputations.
• Makes it possible to compare the results
of analysis under multiple assumptions.
• Gives a more detailed visual analysis, and
allows you to compare effect estimates.

Constructing Enhanced TP Plots
1. Get # of missing outcomes for
Treatment & Control group
(outline x and y axis range),
Set Up the Axes

2. Potential # of success in
Treatment group on x-axis and
potential # of success in Control
on y-axis,
Set Up the Axes

on y-axis,
3. Calculate estimated treatment
effect ( 𝜏) and p-values for a
given hypothesis for each cell,
Set Up the Axes

on y-axis,
4. Set up grid to highlight all
combinations that result in
rejecting the null hypothesis,
Draw the Tipping Point ‘Staircase’

on y-axis,
4. Set up grid to highlight all
combinations that result in
rejecting the null hypothesis,
5. Overlay imputation information
(squares, contours)
Overlay Imputation Info

Enhanced Tipping Point Plotting

Overview of SOLAS
• Stand alone Missing data package
• FIVE Multiple Imputation techniques
• Drag & Drop interface
• SOLAS syntax language feature
• Plotting & Visualisation techniques

ETP Plots - 1000 Imputations

SOLAS Tipping Points Demo

Round Up
• Background & History
• Overview of Missing Data analysis,
Imputation and Tipping Points
• SOLAS Demo

ICH Expert Group Survey
• The purpose of this survey is to gather data on related current
practices, primarily focused on clinical trials involving drugs,
vaccines and biologics. The survey is being distributed to various
regions within ICH, including industry and regulatory communities.
If a question is not applicable to your job role, please select “I am
unable to answer this question/not applicable to my job role”.
• The ICH E9 Revision 1 working group thanks you in advance for your
participation. Please complete the survey by Friday 5th June 2015.
• Link to survey: https://www.surveymonkey.com/r/VPMV57Z

Questions?
Thank you
andrew.grannell@statsols.com
ICH Survey:
https://www.surveymonkey.com/r/VPMV57Z

References
1. NRC-Panel (2010) The Prevention and Treatment
of Missing Data in Clinical Trials. National
Academies Press.
2. ICH E9(R1) (2014) Addendum to Statistical
Principles of Clinical Trials. CPMP/ICH/363/96.
3. LaVange, L.M. (February 2015) Missing Data
Issues in Regulatory Clinical Trials. Symposium
conducted at the meeting of the Japanese
Pharmaceutical Manufacturers Association.

References
4. Tanaka, S., Fukinbara, S., Tsuchiya, S., Suganami, H., and
Ito, Y.M. (2014). Current Practices in Japan for the
Prevention and Treatment of Missing Data in Confirmatory
Clinical Trials: A Survey of Japanese and Foreign
Pharmaceutical Manufacturers.Therapeutic Innovation &
Regulatory Science, 48(6) 717-723.
5. Campbell, G., Pennello, G., and Yue, L. (2011). Missing
Data in the Regulation of Medical Devices. Journal of
Biopharmaceutical Statistics, 21(2):180–195.
6. Victoria Liublinska and Donald B. Rubin (2014). Sensitivity
analysis for a partially missing binary outcome in a two-
arm randomized clinical trial. Statistics in Medicine,
33(24):4170-4185.

Solas Webinar Slides

Recommandé

Contenu connexe

Plus de nQuery

Plus de nQuery (19)

Dernier

Dernier (20)

Solas Webinar Slides