Bias and precision of measures of survival gain from right-censored data

A Comparison of Alternative Network Meta-Analysis Methods in the Presence of Nonproportional Hazards: A Case Study in First-Line Advanced or Metastatic Renal Cell Carcinoma

OBJECTIVES

Network meta-analysis (NMA) of time-to-event outcomes based on constant hazard ratios can result in biased findings when the proportional hazards (PHs) assumption does not hold in a subset of trials. We aimed to summarize the published non-PH NMA methods for time-to-event outcomes, demonstrate their application, and compare their results.

METHODS

The following non-PH NMA methods were compared through an illustrative case study in oncology of 4 randomized controlled trials in terms of progression-free survival and overall survival: (1) 1-step or (2) 2-step multivariate NMAs based on traditional survival distributions or fractional polynomials, (3) NMAs with restricted cubic splines for baseline hazard, and (4) restricted mean survival NMA.

RESULTS

For progression-free survival, the PH assumption did not hold across trials and non-PH NMA methods better reflected the relative treatment effects over time. The most flexible models (fractional polynomials and restricted cubic splines) fit better to the data than the other approaches. Estimated hazard ratios obtained with different non-PH NMA methods were similar at 5 years of follow-up but differed thereafter in the extrapolations. Although there was no strong evidence of PH violation for overall survival, non-PH NMA methods captured this uncertainty in the relative treatment effects over time.

CONCLUSIONS

When the PH assumption is questionable in a subset of the randomized controlled trials, we recommend assessing alternative non-PH NMA methods to estimate relative treatment effects for time-to-event outcomes. We propose a transparent and explicit stepwise model selection process considering model fit, external constraints, and clinical validity. Given inherent uncertainty, sensitivity analyses are suggested.

Individual patient data network meta-analysis using either restricted mean survival time difference or hazard ratios: is there a difference? A case study on locoregionally advanced nasopharyngeal carcinomas

BACKGROUND

This study aimed at applying the restricted mean survival time difference (rmstD) as an absolute outcome measure in a network meta-analysis and comparing the results with those obtained using hazard ratios (HR) from the individual patient data (IPD) network meta-analysis (NMA) on the role of chemotherapy for nasopharyngeal carcinoma (NPC) recently published by the MAC-NPC collaborative group (Meta-Analysis of Chemotherapy [CT] in NPC).

PATIENTS AND METHODS

Twenty trials (5144 patients) comparing radiotherapy (RT) with or without CT in non-metastatic NPC were included. Treatments were grouped in seven categories: RT alone (RT), induction CT followed by RT (IC-RT), RT followed by adjuvant CT (RT-AC), IC followed by RT followed by AC (IC-RT-AC), concomitant chemoradiotherapy (CRT), IC followed by CRT (IC-CRT), and CRT followed by AC (CRT-AC). The primary endpoint was overall survival (OS); secondary endpoints were progression-free survival and locoregional control. The rmstD was estimated at t* = 10 years in each trial. Random-effect frequentist NMA models were applied. P score was used to rank treatments. Heterogeneity and inconsistency were evaluated.

RESULTS

The three treatments that had the highest effect on OS with rmstD were CRT-AC, IC-CRT, and CRT (respective P scores of 92%, 72%, and 64%) compared to CRT-AC, CRT, and IC-CRT when using HR (respective P scores of 96%, 71%, and 63%). Of the 32 HR and rmstD analyzed, 5 had a different interpretation, 3 with a direction change (different direction of treatment effect) and 2 with a change in significance (same direction but a change in statistical significance). Results for secondary endpoints were overall in agreement.

CONCLUSION

The use of either HR or rmstD impacts the results of NMA. Given the sensitivity of HR to non-proportional hazards, this finding could have implications in terms of meta-analysis methodology.

Bias and precision of methods for estimating the difference in restricted mean survival time from an individual patient data meta-analysis

Background

The difference in restricted mean survival time (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗), the area between two survival curves up to time horizon \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗, is often used in cost-effectiveness analyses to estimate the treatment effect in randomized controlled trials. A challenge in individual patient data (IPD) meta-analyses is to account for the trial effect. We aimed at comparing different methods to estimate the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗ from an IPD meta-analysis.

Methods

We compared four methods: the area between Kaplan-Meier curves (experimental vs. control arm) ignoring the trial effect (Naïve Kaplan-Meier); the area between Peto curves computed at quintiles of event times (Peto-quintile); the weighted average of the areas between either trial-specific Kaplan-Meier curves (Pooled Kaplan-Meier) or trial-specific exponential curves (Pooled Exponential). In a simulation study, we varied the between-trial heterogeneity for the baseline hazard and for the treatment effect (possibly correlated), the overall treatment effect, the time horizon \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗, the number of trials and of patients, the use of fixed or DerSimonian-Laird random effects model, and the proportionality of hazards. We compared the methods in terms of bias, empirical and average standard errors. We used IPD from the Meta-Analysis of Chemotherapy in Nasopharynx Carcinoma (MAC-NPC) and its updated version MAC-NPC2 for illustration that included respectively 1,975 and 5,028 patients in 11 and 23 comparisons.

Results

The Naïve Kaplan-Meier method was unbiased, whereas the Pooled Exponential and, to a much lesser extent, the Pooled Kaplan-Meier methods showed a bias with non-proportional hazards. The Peto-quintile method underestimated the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗, except with non-proportional hazards at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ {t}^{\ast } $$\end{document}t∗= 5 years. In the presence of treatment effect heterogeneity, all methods except the Pooled Kaplan-Meier and the Pooled Exponential with DerSimonian-Laird random effects underestimated the standard error of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast}\right) $$\end{document}rmstDt∗. Overall, the Pooled Kaplan-Meier method with DerSimonian-Laird random effects formed the best compromise in terms of bias and variance. The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ rmstD\left({t}^{\ast },=,10,\kern0.5em ,\mathrm{years}\right) $$\end{document}rmstDt∗=10years estimated with the Pooled Kaplan-Meier method was 0.49 years (95 % CI: [−0.06;1.03], p = 0.08) when comparing radiotherapy plus chemotherapy vs. radiotherapy alone in the MAC-NPC and 0.59 years (95 % CI: [0.34;0.84], p < 0.0001) in the MAC-NPC2.

Conclusions

We recommend the Pooled Kaplan-Meier method with DerSimonian-Laird random effects to estimate the difference in restricted mean survival time from an individual-patient data meta-analysis.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0137-z) contains supplementary material, which is available to authorized users.

Difference in Restricted Mean Survival Time for Cost-Effectiveness Analysis Using Individual Patient Data Meta-Analysis: Evidence from a Case Study

Objective

In economic evaluation, a commonly used outcome measure for the treatment effect is the between-arm difference in restricted mean survival time (rmstD). This study illustrates how different survival analysis methods can be used to estimate the rmstD for economic evaluation using individual patient data (IPD) meta-analysis. Our aim was to study if/how the choice of a method impacts on cost-effectiveness results.

Methods

We used IPD from the Meta-Analysis of Radiotherapy in Lung Cancer concerning 2,000 patients with locally advanced non-small cell lung cancer, included in ten trials. We considered methods either used in the field of meta-analysis or in economic evaluation but never applied to assess the rmstD for economic evaluation using IPD meta-analysis. Methods were classified into two approaches. With the first approach, the rmstD is estimated directly as the area between the two pooled survival curves. With the second approach, the rmstD is based on the aggregation of the rmstDs estimated in each trial.

Results

The average incremental cost-effectiveness ratio (ICER) and acceptability curves were sensitive to the method used to estimate the rmstD. The estimated rmstDs ranged from 1.7 month to 2.5 months, and mean ICERs ranged from € 24,299 to € 34,934 per life-year gained depending on the chosen method. At a ceiling ratio of € 25,000 per life year-gained, the probability of the experimental treatment being cost-effective ranged from 31% to 68%.

Conclusions

This case study suggests that the method chosen to estimate the rmstD from IPD meta-analysis is likely to influence the results of cost-effectiveness analyses.