4. EUROPEAN GUIDELINES
“Clinical data must be provided for ALL medical devices. This may be
literature review or clinical investigation depending on device class and
use.”
When must/should a clinical investigation be undertaken?
To ensure a high level of safety and performance, demonstration of
compliance with the general safety and performance requirements
should be based on clinical data that, for class III medical devices and
implantable medical devices should, as a general rule, be sourced
from clinical investigations to be carried out under the responsibility of
a sponsor who can be the manufacturer or another legal or natural
person taking responsibility for the clinical investigation.
Depending on clinical claims, risk management outcome and on the
results of the clinical evaluation, clinical investigations may also have
to be performed for non-implantable medical devices of classes I, IIa
and IIb.
4
Clinquest Services Qserve Conference
5. FDA GUIDELINES
The collection and evaluation of sound clinical data are the basis of
the approval process for many medical devices.
Goals for device studies (including IVD studies) are:
•
Producing valid scientific evidence, demonstrating
reasonable assurance of the safety and effectiveness of the
product
•
Protecting the rights and welfare of study subjects
Clinquest Services Qserve Conference
5
6. FDA GUIDELINES
Valid scientific evidence is defined as:
“Evidence from well-controlled investigations, partially controlled
studies, studies and objective trials without matched controls,
well-documented case histories conducted by qualified experts,
and reports of significant human experience with a marketed
device, from which it can fairly and responsibly be concluded by
qualified experts that there is a reasonable assurance of the
safety and effectiveness of a device under its conditions of use.”
Clinquest Services Qserve Conference
6
7. GUIDELINES
Clinical Investigation Plan
The clinical investigation plan (CIP) shall define:
•
•
•
•
•
•
•
•
Rationale of the clinical investigation
Study objectives
Study design
In/exclusion criteria
Endpoint choice
Proposed analyses: statistical considerations
Monitoring
Conduct and record-keeping: data management
7
Clinquest Services Qserve Conference
8. GUIDELINES
Statistical Section Clinical Investigation Plan
The statistical section should at least include:
•
•
•
•
•
•
•
•
Analysis populations used
Missing data strategy
Definition of endpoints
Hypotheses to be tested (efficacy/safety endpoints)
Planned statistical methodology
Sample size justification
Sensitivity analyses planned
Planned interim analyses
•
Detailed Statistical Analysis Plan
8
Clinquest Services Qserve Conference
10. PRECISION
X
/ BIAS
x
X
x
x
x
x x
x
xxx
Precision small
Bias large
X
Precision large
Bias large
x
X
x
x
x x
x
x x
x
x
Precision small
Bias small
Precision large
Bias small
10
Clinquest Services Qserve Conference
11. PRECISION
/ BIAS
•
Precision: random fluctuations
Maximizing precision by:
s
s.e.
- work with large samples
N
- work with homogeneous samples
- use paired designs (patient/sample is its own control)
- use extra information, e.g. baseline measurements
•
Bias: systematic deviate from true result
Minimizing bias by:
- randomization (decreases selection bias and impact of
confounding)
- blinding (decreases observation bias)
- use extra information, e.g. baseline measurements (decreases
impact of confounding)
- avoid missing data
Clinquest Services Qserve Conference
11
12. EXAMPLE
STENT STUDY
Comparison of Stenting versus Balloon Angioplasty (PTA) for the
treatment of below the knee artery disease (Class III).
Efficacy Study Objective:
- Does our Stent perform better than balloon angioplasty?
Ethical, control standard care
Design:
-
Parallel Arm, Multiple Centers
Superiority Trial
Randomized?
Yes
Blinded?
No
Duration FU?
Long enough for acceptable evaluation of
performance and safety (1 year)
- Interim Analyses? No
Primary efficacy endpoint:
- Binary in-segment stenosis at 12 months by angiography
12
Clinquest Services Qserve Conference
13. STENT STUDY
Possible bias:
- Objective primary endpoint
- Observation bias, blinding not possible
- Use ITT analysis population in case of
randomized study
- Confounding possible?
- Missing data
13
Clinquest Services Qserve Conference
14. ANALYSIS DATASETS
• ITT Analysis set:
All randomized patients; analyzed as
randomized.
• Full Analysis set: The analysis set that is as complete as
possible and as close as possible to the
intention to treat ideal of including all
randomized subjects (ICH).
• Per Protocol set:
Set of patients with minimal violations against
the protocol as: errors in treatment assignment,
use of excluded medication, poor compliance,
loss to follow-up and missing data; analyzed as
treated.
To be set before database lock.
14
Clinquest Services Qserve Conference
15. ANALYSIS DATA SETS
Angioplasty N1=50
5 patients too severe
5 patients go over on
stenting
R
Stent N2=50
•
•
ITT
PP
50 patients on stent 50 patients on angioplasty
55 patients on stent 45 patients on angioplasty
PP analysis generally used as sensitivity analysis
15
Clinquest Services Qserve Conference
16. BIAS BY CONFOUNDING VARIABLES
Mortality rate comparison
Hospital
A
Died
Survived
Total
Mortality rate
63
2037
2100
3.0%
Died
Survived
Total
Mortality rate
B
16
784
800
2.0%
Relative risk hosp. B vs. hosp. A = 0.66,
34% lower risk of dying in hosp. B than in
hosp. A
Total
79
2821
2900
Good Health
A
B
6
8
594
592
600
600
1.0%
1.3%
Relative risk hosp. B vs. hosp. A = 1.33,
33% higher risk of dying in hosp. B than
in hosp. A
Poor Health
A
B
57
8
1443
192
1500
200
3.8%
4.0%
Total
14
1186
1200
Total
65
1635
1700
Relative risk hosp. B vs. hosp. A = 1.05,
5% higher risk of dying in hosp. B than
in hosp. A
16
Clinquest Services Qserve Conference
17. BIAS BY CONFOUNDING VARIABLES
Mortality rate confounded by health status because :
- Health status has impact on outcome
- Health status not equally distributed over the two hospitals
Died
Survived
Total
Mortality rate
Good Health
A
B
6
8
594
592
600
600
1.0%
1.3%
Total
14
1186
1200
Poor Health
A
B
57
8
1443
192
1500
200
3.8%
4.0%
Total
65
1635
1700
Statistical analysis: Adjusted relative risk hospital B vs. hospital A = 1.14,
14% higher risk of dying in hospital B than in hospital A, taking health status
into account.
17
Clinquest Services Qserve Conference
18. BIAS BY CONFOUNDING VARIABLES
HOW TO COPE WITH PROGNOSTIC/CONFOUNDING FACTORS ?
FACTORS?
•
Identical distribution of prognostic factors over treatment groups
(stratified randomization)
•
Subgroup analyses (problem of multiple testing and low power)
•
Retrospective control for prognostic/confounding factors during
statistical analysis
Confounding variable is a background factor which:
–
is not equally distributed over the risk groups
–
influences the outcome variable(s)
18
Clinquest Services Qserve Conference
19. STENT STUDY, HYPOTHESES TO BE TESTED
Primary efficacy endpoint:
H0 : π A = π B
no stent therapy effect
π A π B 0
HA : π A π B
stent therapy effect
π A π B 0
Try to reject H0
-If p-value < 0.05
-If p-value ≥ 0.05
reject H0: there is a statistically significant therapy effect
accept H0: there is no statistically significant therapy effect.
19
Clinquest Services Qserve Conference
20. STENT STUDY, STATISTICAL INFERENCE
ITT Population
50.0%
40.0%
Percentage with Stenosis
30.0%
20.0%
Stent Treatment
10.0%
PTA Treatment
0.0%
Difference
-10.0%
-20.0%
-30.0%
-40.0%
20
Clinquest Services Qserve Conference
22. STENT STUDY, MISSING DATA
Restenosis
No Restenosis
Total
Missing
Therapy
Stent (N=113)
PTA (N=115)
15 (22.4%)
31 (41.9%)
52
43
67
74
46 (40.7%)
41 (35.7%)
Total
46
95
141
87 (38.2%)
best case
Restenosis:
23 Stent,
21 PTA
22
Clinquest Services Qserve Conference
worst case
23. STENT STUDY, SAMPLE SIZE CALCULATION
Stent study:
With 61 lesions in each therapy arm, a clinically meaningful difference of
25% in 12 months in-segment restenosis rate can be detected with a
power of 80% at a two sided significance level of 5%, assuming a 12
months restenosis rate of 45% in the group of subjects who only receive
balloon angioplasty. The total sample size was increased to almost 110
lesions per therapy arm to account for a 45% drop out rate.
Restenosis
No Restenosis
Total
Missing
Therapy
Stent (N=113) PTA (N=115)
15 (22.4%)
31 (41.9%)
52
43
67
74
46
41
Total
46
95
141
87
Drop out rate: 38.2%
Power to detect 25% difference = 85.8%
23
Clinquest Services Qserve Conference
24. STENT STUDY, SAMPLE SIZE CALCULATION
•
Performed on the basis of assumptions concerning:
- How accurate can the primary endpoint be measured?
- What is the clinically significant difference you like to see
(HA)?
- What do you expect as outcome for the control group?
•
Performed on the basis of agreements about:
- significance level: (e.g. 5%)
- power: 1-
(e.g. 90%)
24
Clinquest Services Qserve Conference
25. STENT STUDY, SAMPLE SIZE CALCULATION
Hypothesis Testing
H0
Type I error / Type II error
HA
(Type I error)= Significance level
P(reject H0 | H0 is true) = P(false positive)
of the test
(1 - ) = Power of the test:
P(reject H0 | HA is true)
(Type II error):
P(accept H0 | HA is true) = P(false negative)
Power: Probability to accept correctly, by a given , the alternative
hypothesis.
25
Clinquest Services Qserve Conference
26. ADAPTIVE DESIGNS
Adaptive Design Clinical Study: a study that includes a
prospectively planned opportunity for modification of one or more
specified aspects of the study design and hypotheses based on
analysis of data (usually interim data) from subjects in the study
(FDA).
Goal: to make the study more efficient: shorter duration, fewer
patients, more information
Possible problems:
Operational bias: revisions not previously planned and made or
proposed after an un-blinded interim analysis raise major
concerns about study integrity.
Multiple testing: control of type I error rate
Regulatory concerns: FDA communication/review needed
26
Clinquest Services Qserve Conference
27. ADAPTIVE DESIGNS
Planned Adaptive Design
• Allocation Rule
- Can be fixed but can change based
on accruing data
• Sampling Rule
- How many subjects sampled at next
stage (cohort)
- Sample size recalculation based on
interim results
• Stopping Rule
- When to stop a trial: efficacy, futility
•
Decision Rule
- E.g. dropping study arm
Fixed Sample Design
- Randomization remains
fixed throughout the study
- Only one stage
- Fixed sample size
- No early stopping
- No changes
27
Clinquest Services Qserve Conference
28. EXAMPLE CIN-EVENT STUDY
Comparison experimental device system with standard hydration
protocol for preventing the incidence of CIN (Contrast Induced
Nephropathy), after the administration of contrast media. (Class
IIb).
Efficacy Study Objective:
- Evaluation of the comparability between experimental device and
standard care in CIN events within 3 days post contrast administration.
Design:
-
Parallel Arm, multiple centers
Superiority Trial
Randomized? Yes, stratified by Y/N NSTEMI
Blinded? No
Duration FU? 3 days for primary efficacy endpoint, 90 days for safety
Interim Analysis? Yes
Primary efficacy endpoint:
- CIN event within 3 days; missing information for CIN: failure
28
Clinquest Services Qserve Conference
29. CIN-EVENT STUDY, SAMPLE SIZE CALCULATION
Assumptions:
3-day CIN event rate control arm: 15%
Assuming 80% power and with a 1:1 randomization allocation ratio,
155 subjects in each group are required to demonstrate
the expected difference of 10% in the incidence of CIN between
groups at a significance level of 0.05 based on a 2-sided test.
The study will randomize a total of 326 patients to account for a 5%
lost to follow-up but based on the interim analysis, could
randomize up to a maximum of twice the initial sample size, or 652
patients.
Clinquest Services Qserve Conference
29
30. CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION
ˆ
1 Measured difference at interim analysis
1 Expected difference at final analysis
Clinquest Services Qserve Conference
30
31. CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION
Sample size recalculation will be performed based on the
method of modification of sample size in group sequential
clinical trials employing conditional power and maintaining type I
error rate. The sample size will be adjusted such that the
calculated conditional power obtained is 80%.
The trial can be stopped for reasons of overwhelming efficacy (pvalue<0.003) or futility (conditional power under the current trend
<20%).
31
Clinquest Services Qserve Conference
