Quantifying and estimating the predictive accuracy for censored time-to-event data with competing risks

Explained variation and degrees of necessity and of sufficiency for competing risks survival data

In this contribution, the Schemper-Henderson measure of explained variation for survival outcomes is extended to accommodate competing events (CEs) in addition to events of interest. The extension is achieved by moving from the unconditional and conditional survival functions of the original measure to unconditional and conditional cumulative incidence functions, the latter obtained, for example, from Fine and Gray models. In the absence of CEs, the original measure is obtained as a special case. We define explained variation on the population level and provide two different types of estimates. Recently, the authors have achieved a multiplicative decomposition of explained variation into degrees of necessity and degrees of sufficiency. These measures are also extended to the case of competing risks survival data. A SAS macro and an R function are provided to facilitate application. Interesting empirical properties of the measures are explored on the population level and by an extensive simulation study. Advantages of the approach are exemplified by an Austrian study of breast cancer with a high proportion of CEs.

Comparison of models for stroke-free survival prediction in patients with CADASIL

Cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, which is caused by mutations of the NOTCH3 gene, has a large heterogeneous progression, presenting with declines of various clinical scores and occurrences of various clinical event. To help assess disease progression, this work focused on predicting the composite endpoint of stroke-free survival time by comparing the performance of Cox proportional hazards regression to that of machine learning models using one of four feature selection approaches applied to demographic, clinical and magnetic resonance imaging observational data collected from a study cohort of 482 patients. The quality of the modeling process and the predictive performance were evaluated in a nested cross-validation procedure using the time-dependent Brier Score and AUC at 5 years from baseline, the former measuring the overall performance including calibration and the latter highlighting the discrimination ability, with both metrics taking into account the presence of right-censoring. The best model for each metric was the componentwise gradient boosting model with a mean Brier score of 0.165 and the random survival forest model with a mean AUC of 0.773, both combined with the LASSO feature selection method.

Accurate prediction of HCC risk after SVR in patients with hepatitis C cirrhosis based on longitudinal data

BACKGROUND

Most existing predictive models of hepatocellular carcinoma (HCC) risk after sustained virologic response (SVR) are built on data collected at baseline and therefore have limited accuracy. The current study aimed to construct an accurate predictive model incorporating longitudinal data using a novel modeling strategy. The predictive performance of the longitudinal model was also compared with a baseline model.

METHODS

A total of 400 patients with HCV-related cirrhosis who achieved SVR with direct-acting antivirals (DAA) were enrolled in the study. Patients were randomly divided into a training set (70%) and a validation set (30%). Informative features were extracted from the longitudinal variables and then put into the random survival forest (RSF) to develop the longitudinal model. A baseline model including the same variables was built for comparison.

RESULTS

During a median follow-up time of approximately 5 years, 25 patients (8.9%) in the training set and 11 patients (9.2%) in the validation set developed HCC. The areas under the receiver-operating characteristics curves (AUROC) for the longitudinal model were 0.9507 (0.8838-0.9997), 0.8767 (0.6972,0.9918), and 0.8307 (0.6941,0.9993) for 1-, 2- and 3-year risk prediction, respectively. The brier scores of the longitudinal model were also relatively low for the 1-, 2- and 3-year risk prediction (0.0283, 0.0561, and 0.0501, respectively). In contrast, the baseline model only achieved mediocre AUROCs of around 0.6 (0.6113, 0.6213, and 0.6480, respectively).

CONCLUSIONS

Our longitudinal model yielded accurate predictions of HCC risk in patients with HCV-relate cirrhosis, outperforming the baseline model. Our model can provide patients with valuable prognosis information and guide the intensity of surveillance in clinical practice.

Evaluation of competing risks prediction models using polytomous discrimination index

For competing risks data, it is often important to predict a patient's outcome status at a clinically meaningful time point after incorporating the informative censoring due to competing risks. This can be done by adopting a regression model that relates the cumulative incidence probabilities to a set of covariates. To assess the performance of the resulting prediction tool, we propose an estimator of the polytomous discrimination index applicable to competing risks data, which can quantify a prognostic model's ability to discriminate among subjects from different outcome groups. The proposed estimator allows the prediction model to be subject to model misspecification and enjoys desirable asymptotic properties. We also develop an efficient computation algorithm that features a computational complexity of O(n log n). A perturbation resampling scheme is developed to achieve consistent variance estimation. Numerical results suggest that the estimator performs well under realistic sample sizes. We apply the proposed methods to a study of monoclonal gammopathy of undetermined significance.

Backward joint model and dynamic prediction of survival with multivariate longitudinal data

An important approach to dynamic prediction of time-to-event outcomes using longitudinal data is based on modeling the joint distribution of longitudinal and time-to-event data. The widely used joint model for this purpose is the shared random effect model. Presumably, adding more longitudinal predictors improves the predictive accuracy. However, the shared random effect model can be computationally difficult or prohibitive when a large number of longitudinal variables are used. In this paper, we study an alternative way of modeling the joint distribution of longitudinal and time-to-event data. Under this formulation, the log-likelihood involves no more than one-dimensional integration, regardless of the number of longitudinal variables in the model. Therefore, this model is particularly suitable in dynamic prediction problems with large number of longitudinal predictors. The model fitting can be implemented with tractable and stable computation by using a combination of pseudo maximum likelihood estimation, Expectation-Maximization algorithm, and convex optimization. We evaluate the proposed methodology and its predictive accuracy with varying number of longitudinal variables using simulations and data from a primary biliary cirrhosis study.

Functional survival forests for multivariate longitudinal outcomes: Dynamic prediction of Alzheimer's disease progression

The random survival forest (RSF) is a non-parametric alternative to the Cox proportional hazards model in modeling time-to-event data. In this article, we developed a modeling framework to incorporate multivariate longitudinal data in the model building process to enhance the predictive performance of RSF. To extract the essential features of the multivariate longitudinal outcomes, two methods were adopted and compared: multivariate functional principal component analysis and multivariate fast covariance estimation for sparse functional data. These resulting features, which capture the trajectories of the multiple longitudinal outcomes, are then included as time-independent predictors in the subsequent RSF model. This non-parametric modeling framework, denoted as functional survival forests, is better at capturing the various trends in both the longitudinal outcomes and the survival model which may be difficult to model using only parametric approaches. These advantages are demonstrated through simulations and applications to the Alzheimer's Disease Neuroimaging Initiative.

Dynamic prediction of competing risk events using landmark sub-distribution hazard model with multiple longitudinal biomarkers

The cause-specific cumulative incidence function quantifies the subject-specific disease risk with competing risk outcome. With longitudinally collected biomarker data, it is of interest to dynamically update the predicted cumulative incidence function by incorporating the most recent biomarker as well as the cumulating longitudinal history. Motivated by a longitudinal cohort study of chronic kidney disease, we propose a framework for dynamic prediction of end stage renal disease using multivariate longitudinal biomarkers, accounting for the competing risk of death. The proposed framework extends the local estimation-based landmark survival modeling to competing risks data, and implies that a distinct sub-distribution hazard regression model is defined at each biomarker measurement time. The model parameters, prediction horizon, longitudinal history and at-risk population are allowed to vary over the landmark time. When the measurement times of biomarkers are irregularly spaced, the predictor variable may not be observed at the time of prediction. Local polynomial is used to estimate the model parameters without explicitly imputing the predictor or modeling its longitudinal trajectory. The proposed model leads to simple interpretation of the regression coefficients and closed-form calculation of the predicted cumulative incidence function. The estimation and prediction can be implemented through standard statistical software with tractable computation. We conducted simulations to evaluate the performance of the estimation procedure and predictive accuracy. The methodology is illustrated with data from the African American Study of Kidney Disease and Hypertension.

Development and validation of a risk assessment tool for BKPyV Replication in allogeneic stem cell transplant recipients

BACKGROUND

BK polymavirus (BKPyV), a member of the family Polyomaviridae, is associated with increased morbidity and mortality in allogeneic stem cell transplant recipients.

METHODS

In our previous retrospective study of 2477 stem cell transplant patients, BKPyV replication independently predicted chronic kidney disease and poor survival. In this study, using the same cohort, we derived and validated a risk grading system to identify patients at risk of BKPyV replication after transplantation in a user-friendly modality. We used 3 baseline variables (conditioning regimen, HLA match status, and underlying cancer diagnosis) that significantly predicted BKPyV replication in our initial study in a subdistribution hazard model with death as a competing risk. We also developed a nomogram of the hazard model as a visual aid. The AUC of the ROC of the risk-score-only model was 0.65. We further stratified the patients on the basis of risk score into low-, moderate-, and high-risk groups.

RESULTS

The total risk score was significantly associated with BKPyV replication (P < .0001). At 30 days after transplantation, the low-risk (score ≤ 0) patients had a 9% chance of developing symptomatic BKPyV replication, while the high-risk (score ≥ 8) of the population had 56% of developing BKPyV replication. We validated the risk score using a separate cohort of 1478 patients. The AUC of the ROC of the risk-score-only model was 0.59. Both the total risk score and 3-level risk variable were significantly associated with BKPyV replication in this cohort (P < .0001).

CONCLUSIONS

This grading system for the risk of symptomatic BKPyV replication may help in early monitoring and intervention to prevent BKPyV-associated morbidity, mortality, and kidney function decline.

Evaluating the time-dependent predictive accuracy for event-to-time outcome with a cure fraction

In medical studies, it is often observed that a portion of subjects will never experience the event of interest and thus can be treated as cured or long-term survivors. Many populations of early-stage cancer patients contain both uncured and cured individuals that should be modeled using cure models. In prognostic studies, the cure status (uncure or cure) is an issue of interest for medical practitioners, and the disease status (death or alive) of an individual is not a fixed characteristic and it varies along the time. These statuses are usually predicted by a prognostic risk score. The time-dependent receiver operating characteristic (ROC) curve is a powerful tool to evaluate these predicting performances dynamically. In the context with a cure fraction, quantifying and estimating the predictive performances of the risk score is a challenge since the disease status and cure status are both unknown among individuals who are censored. In this paper, to assess the predictive accuracy for the survival outcome with a cure fraction, we propose a time-dependent ROC curve semi-parametric estimator based on the sieve maximum likelihood (ML) estimation under the mixture cure model. We also apply a Bernstein-based smoothing method in the estimation procedure, and this estimator can lead to substantial gain in efficiency. In addition, we derive the time-dependent area under the ROC curve (AUC) to summarize the discriminatory capacity of the risk score globally. Finally, we evaluate the finite sample performance of the proposed methods by extensive simulations and illustrate the estimation using two real data sets, one from a melanoma study and the other from stomach cancer.