Meta-analysis of time-to-event outcomes from randomized trials using restricted mean survival time: application to individual participant data

Bias and precision of methods for estimating the difference in restricted mean survival time from an individual patient data meta-analysis

Background

The difference in restricted mean survival time (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗), the area between two survival curves up to time horizon \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗, is often used in cost-effectiveness analyses to estimate the treatment effect in randomized controlled trials. A challenge in individual patient data (IPD) meta-analyses is to account for the trial effect. We aimed at comparing different methods to estimate the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗ from an IPD meta-analysis.

Methods

We compared four methods: the area between Kaplan-Meier curves (experimental vs. control arm) ignoring the trial effect (Naïve Kaplan-Meier); the area between Peto curves computed at quintiles of event times (Peto-quintile); the weighted average of the areas between either trial-specific Kaplan-Meier curves (Pooled Kaplan-Meier) or trial-specific exponential curves (Pooled Exponential). In a simulation study, we varied the between-trial heterogeneity for the baseline hazard and for the treatment effect (possibly correlated), the overall treatment effect, the time horizon \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗, the number of trials and of patients, the use of fixed or DerSimonian-Laird random effects model, and the proportionality of hazards. We compared the methods in terms of bias, empirical and average standard errors. We used IPD from the Meta-Analysis of Chemotherapy in Nasopharynx Carcinoma (MAC-NPC) and its updated version MAC-NPC2 for illustration that included respectively 1,975 and 5,028 patients in 11 and 23 comparisons.

Results

The Naïve Kaplan-Meier method was unbiased, whereas the Pooled Exponential and, to a much lesser extent, the Pooled Kaplan-Meier methods showed a bias with non-proportional hazards. The Peto-quintile method underestimated the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗, except with non-proportional hazards at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗= 5 years. In the presence of treatment effect heterogeneity, all methods except the Pooled Kaplan-Meier and the Pooled Exponential with DerSimonian-Laird random effects underestimated the standard error of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗. Overall, the Pooled Kaplan-Meier method with DerSimonian-Laird random effects formed the best compromise in terms of bias and variance. The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast },=,10,\kern0.5em ,\mathrm{years}\right) $$\end{document}rmstDt∗=10years estimated with the Pooled Kaplan-Meier method was 0.49 years (95 % CI: [−0.06;1.03], p = 0.08) when comparing radiotherapy plus chemotherapy vs. radiotherapy alone in the MAC-NPC and 0.59 years (95 % CI: [0.34;0.84], p < 0.0001) in the MAC-NPC2.

Conclusions

We recommend the Pooled Kaplan-Meier method with DerSimonian-Laird random effects to estimate the difference in restricted mean survival time from an individual-patient data meta-analysis.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0137-z) contains supplementary material, which is available to authorized users.

Difference in Restricted Mean Survival Time for Cost-Effectiveness Analysis Using Individual Patient Data Meta-Analysis: Evidence from a Case Study

Objective

In economic evaluation, a commonly used outcome measure for the treatment effect is the between-arm difference in restricted mean survival time (rmstD). This study illustrates how different survival analysis methods can be used to estimate the rmstD for economic evaluation using individual patient data (IPD) meta-analysis. Our aim was to study if/how the choice of a method impacts on cost-effectiveness results.

Methods

We used IPD from the Meta-Analysis of Radiotherapy in Lung Cancer concerning 2,000 patients with locally advanced non-small cell lung cancer, included in ten trials. We considered methods either used in the field of meta-analysis or in economic evaluation but never applied to assess the rmstD for economic evaluation using IPD meta-analysis. Methods were classified into two approaches. With the first approach, the rmstD is estimated directly as the area between the two pooled survival curves. With the second approach, the rmstD is based on the aggregation of the rmstDs estimated in each trial.

Results

The average incremental cost-effectiveness ratio (ICER) and acceptability curves were sensitive to the method used to estimate the rmstD. The estimated rmstDs ranged from 1.7 month to 2.5 months, and mean ICERs ranged from € 24,299 to € 34,934 per life-year gained depending on the chosen method. At a ceiling ratio of € 25,000 per life year-gained, the probability of the experimental treatment being cost-effective ranged from 31% to 68%.

Conclusions

This case study suggests that the method chosen to estimate the rmstD from IPD meta-analysis is likely to influence the results of cost-effectiveness analyses.

Comparison of Treatment Effects Measured by the Hazard Ratio and by the Ratio of Restricted Mean Survival Times in Oncology Randomized Controlled Trials

PURPOSE

We aimed to compare empirically the treatment effects measured by the hazard ratio (HR) and by the difference (and ratio) of restricted mean survival times (RMST) in oncology randomized trials.

METHODS

We selected oncology randomized controlled trials from five leading journals during the last 6 months of 2014. We reconstructed individual patient data for one time-to-event outcome from each trial, preferably the primary outcome. We reanalyzed each trial and compared the treatment effect estimated by the HR with that by the difference (and ratio) of RMST. We estimated an average ratio of the HR to the ratio of RMST; an average ratio less than one indicates more optimistic assessments with HRs.

RESULTS

We analyzed 54 randomized controlled trials totaling 33,212 patients. The selected outcome was overall survival in 21 (39%) trials. There was evidence of nonproportionality of hazards in 13 (24%) trials. The HR and RMST-based measures were in agreement regarding the statistical significance of the effect, except in one case. The median HR was 0.84 (Q1 to Q3 range, 0.67 to 0.97) and the median difference in RMST was 1.12 months (range, 0.22 to 2.75 months). The average ratio of the HR to the ratio of RMST was 1.11 (95% CI, 1.07 to 1.15), with substantial between-trial variability (I(2) = 86%). Results were consistent by outcome type (overall survival v other outcomes) and whether the proportional hazard assumption held or not.

CONCLUSION

On average, the HR provided significantly larger treatment effect estimates than the ratio of RMST. The HR may seem large when the absolute effect is small. RMST-based measures should be routinely reported in randomized trials with time-to-event outcomes.