Comparison of effect sizes associated with surrogate and final primary endpoints in randomised clinical trials.

1. Comparison of effect sizes associated with
surrogate and final primary endpoints in
randomised clinical trials
Ciani O., Garside R., Pavey T., Stein K., Taylor R.S.
1

2. Background
Classic Definition for surrogates
Disease-centered characteristics Patient-centered characteristics
Biomarkers Surrogate outcomes Final outcome
A characteristic that is A characteristic
objectively measured A biomarker that is that reflects how
and evaluated as an intended to substitute patient feels,
indicator of normal, and predict for a final functions or
pathogenic or outcome. survives.
pharmacologic
responses to a
therapeutic intervention.
Cardiovascular
e.g. LDL-cholesterol
Mortality
e.g. Intraocular pressure Loss of vision

3. Background
HTA-based Definition of surrogates
Disease-centered characteristics Patient-centered characteristics
Biomarkers Surrogate outcomes Final outcome
A characteristic that is A characteristic
objectively measured A biomarker - or clinical that reflects how
and evaluated as an or patient-relevant patient feels,
indicator of normal, outcome - that is functions or
pathogenic or intended to substitute survives.
pharmacologic and predict for a final
responses to a outcome, namely
therapeutic intervention. survival or HRQoL.
e.g. Rate of hip fracture Mortality/HRQoL
e.g. Event-free Survival Overall Survival

4. Objectives of the study
I. To study the association between primary endpoint
(surrogate vs final) and treatment effect estimates in
RCTs
II.To compare the risk of bias in trials reporting a
surrogate endpoint vs trials reporting a final primary
endpoint
4

5. Methods
Study selection
Initial sample of abstracts (N = 639)
Initial sample of abstracts (N = 639)
Excluded (N = 55)
Excluded (N = 55)
NotRCTs (N = 17)
Not RCTs (N = 17)
Economicevaluation studies (N = 11)
Economic evaluation studies (N = 11)
Noninterventional treatment (N = 25)
Non interventional treatment (N = 25)
Secondaryanalysis (N = 2)
Secondary analysis (N = 2)
For outcomes classification (N = 584)
For outcomes classification (N = 584)
Composite mixed outcomes (N = 73)
Composite mixed outcomes (N = 73)
Eligible for the study (N = 511)
Eligible for the study (N = 511)
Matching procedure
Matching procedure
Surrogate outcomes based (N = 137)
Surrogate outcomes based (N = 137) Final outcomes based (N = 137)
Final outcomes based (N = 137)
Excluded (N = 36)
Excluded (N = 36)
Excluded (N = 53)
Excluded (N = 53) Compositemixed outcomes (N = 9)
Composite mixed outcomes (N = 9)
Equivalence/Non-inferioritystudy (N = 15)
Equivalence/Non-inferiority study (N = 15) Earlytermination (N = 1)
Early termination (N = 1)
UnpooledMuliti-arm (N = 33)
Unpooled Muliti-arm (N = 33) Equivalence/Non-inferioritystudy (N = 11)
Equivalence/Non-inferiority study (N = 11)
Noanalysable data (N = 5)
No analysable data (N = 5) UnpooledMuliti-arm (N = 11)
Unpooled Muliti-arm (N = 11)
Noanalysable data (N = 4)
No analysable data (N = 4)
Surrogate outcome trials (N = 84)
Surrogate outcome trials (N = 84) Final outcome trials (N = 101)
Final outcome trials (N = 101)
Binaryendpoint (N = 51) Binaryendpoint (N = 83)
5
Binary endpoint (N = 51) Binary endpoint (N = 83)

6. Methods
Data extraction
 Effect Size
 Binary endpoints: n/N data for each arm
 Continuous endpoints: SS, Mean, SD for each arm
 TEs(95%CI) as reported by authors
Study characteristics: sample size, follow-up, type of intervention,
patient population, sponsor (i.e. FP, NFP and mixed), positive outcome in
favour of the new treatment
Risk of bias: adoption of the intention to treat (ITT) principle, adequate
randomized sequence generation and allocation concealment, double-
blind/placebo-control
Surrogate outcomes: type of surrogate (i.e. imaging, histo/biochemical,
instrumental, other), authors’ statement about validation and use of a
substitute outcome
6

7. Methods
Data analyses
 Primary Analysis
 Random-effects meta-analysis
 Binary endpoints: TEs expressed as ORs
 Meta-regression models
 Binary endpoints: Ratio of ORs (95%CI)
 ROR > 1 → greater TEs of the surrogate endpoints
 Adjustment for key trial characteristics
 Sensitivity Analyses
 Pooled Relative Risk Ratios estimate (RRR)
 Combined continuous and binary endpoints ROR estimation
 Within-pair comparison of differences in ln(OR)
 Secondary Analysis
 Logistic regression model
 OR of reporting result in favour of the new treatment
 Risk of bias assessment
 χ2 - test of methodological quality dimensions across groups
7

8. Results
Study characteristics
Surrogate Final P-
Characteristics
outcomes (N = 84) outcomes (N = 101) value
Intervention, N(%) 0.33
Pharmaceuticals 49 (58) 61 (60)
Medical Devices 7 (8) 7 (7)
Surgical procedures 4 (5) 8 (8)
Health promotion activities 7 (8) 2 (2)
Other therapeutic technologies 17 (20) 23 (23)
Sponsor, N(%) 0.86
Profit 24 (29) 28 (28)
Not-for-Profit 49 (58) 57 (56)
Mixed 11 (12) 16 (16)
Sample size, Median (IQR) 371 (162-787) 741 (300-4731)
<0.001
Follow up, [days] Median (IQR) 255 (137-540) 180 (40-730) 0.73 8
*Chi-square test, Fisher exact test, Mann-Whitney U test

9. Results
Comparison of TEs – primary analysis
Method of Analysis Surrogate Final outcome
Adjusted^
(Nr of Surrogate trials vs. Nr of outcome Trials Trials ROR (95% CI)
ROR (95% CI)
Final Outcome trials) OR (95% CI) OR (95% CI)
Primary analysis
Binary outcomes 0.51 0.76 1.47 1.46
(51 vs. 83) (0.42 to 0.60) (0.70 to 0.82) (1.07 to 2.01) (1.05 to 2.04)
ORs = Odds ratios pooled using DerSimonian & Laird random effects meta-analyses. ROR: Relative Odds Ratio; ^Adjusted for trial-level
characteristics of clinical area of intervention, patient population, type of intervention, sponsor, journal, mean sample size and mean follow
up time
9

10. Results
Comparison of TEs – sensitivity analyses
Method of Analysis Adjusted^
Surrogate Trials Final Trials ROR or RRR
(Nr Surrogate trials vs. ROR or RRR
RR (95% CI) RR (95% CI) (95% CI)
Nr Final Outcome trials) (95% CI)
Inclusion of risk ratios as
0.56 0.80 1.38 1.36
reported by authors
(0.48 to 0.65) (0.75 to 0.86) (1.12 to 1.71) (1.08 to 1.70)
(57 vs. 86)
Inclusion of continuous
0.46 0.68 1.44 1.48
outcomes
(0.39 to 0.54) (0.62 to 0.74) (0.83 to 2.49) (0.83 to 2.62)
(84 vs. 101)
Binary outcomes
0.48 0.68 1.38
matched-pairs -
(0.39 to 0.59) (0.61 to 0.77) (1.01 to 1.88)
(43 vs. 43)
RRR: Relative Risk Ratio; ^Adjusted for trial-level characteristics of clinical area of intervention, patient population, type of intervention,
sponsor, journal, mean sample size and mean follow up time 10

11. Results
Risk of bias
Surrogate Final outcomes
Risk of Bias Assessment, N(%) P-value
outcomes (N=84) (N=101)
ITT adoption 62 (74) 83 (82) 0.17
Adequate Randomization 54 (64) 65 (64) 0.99
sequence generation
Adequate Randomization 61 (73) 74 (73) 0.92
allocation concealment
Double Blinding/Placebo control 42 (50) 43 (43) 0.31
*Chi-square test
11

12. Discussion and limitations
 Between-trial comparison of treatment effects
 Possible role of smaller trial sample size in surrogate outcome
trials
 ~40% ‘overestimation’ of TEs in surrogate outcomes trials
 Consistent result across sensitivity analyses, confirmed by
secondary analyses
 Findings not explained by methodological quality or other key
trial characteristics
12

13. Thanks for your attention
Q&A
oriana.ciani@pcmd.ac.uk
13

14. Main References
1. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints:
preferred definitions and conceptual framework. Clinical Pharmacology and
Therapeutics 2001; 69: 89–95.
2. Bucher, H. C. et al. Users' guides to the medical literature: XIX. Applying clinical trial
results. A. How to use an article measuring the effect of an intervention on surrogate
end points. Evidence-Based Medicine Working Group. JAMA, 1999: 282, 771-8.
3. Fleming TR, DeMets DL. Surrogate endpoints in clinical trials: Are we being misled?
Annals of Internal Medicine 1996; 125: 605–13.
4. Lassere M. The Biomarker-Surrogacy Evaluation Schema: a review of the
biomarker-surrogate literature and a proposal for a criterion-based, quantitative,
multidimensional hierarchical levels of evidence schema for evaluating the status of
biomarkers as surrogate endpoints .Statistical Methods in Medical Research 2007;
17: 303–340.
5. Taylor RS, Elston J. The use of surrogate outcomes in model-based cost-
effectiveness analyses: a survey of UK Health Technology Assessment reports.
Health Technol Assess 2009; 13(8).
6. Weir CJ, Walley RJ. Statistical evaluation of biomarkers as surrogate endpoints: a
literature review. Stat Med 2006; 25: 183-203.
14