Concordance measure and discriminatory accuracy in transformation cure models

The estimation of long and short term survival time and associated factors of HIV patients using mixture cure rate models

BACKGROUND

HIV is one of the deadliest epidemics and one of the most critical global public health issues. Some are susceptible to die among people living with HIV and some survive longer. The aim of the present study is to use mixture cure models to estimate factors affecting short- and long-term survival of HIV patients.

METHODS

The total sample size was 2170 HIV-infected people referred to the disease counseling centers in Kermanshah Province, in the west of Iran, from 1998 to 2019. A Semiparametric PH mixture cure model and a mixture cure frailty model were fitted to the data. Also, a comparison between these two models was performed.

RESULTS

Based on the results of the mixture cure frailty model, antiretroviral therapy, tuberculosis infection, history of imprisonment, and mode of HIV transmission influenced short-term survival time (p-value < 0.05). On the other hand, prison history, antiretroviral therapy, mode of HIV transmission, age, marital status, gender, and education were significantly associated with long-term survival (p-value < 0.05). The concordance criteria (K-index) value for the mixture cure frailty model was 0.65 whereas for the semiparametric PH mixture cure model was 0.62.

CONCLUSION

This study showed that the frailty mixture cure models is more suitable in the situation where the studied population consisted of two groups, susceptible and non-susceptible to the event of death. The people with a prison history, who received ART treatment, and contracted HIV through injection drug users survive longer. Health professionals should pay more attention to these findings in HIV prevention and treatment.

Measures of explained variation under the mixture cure model for survival data

Explained variation is well understood under linear regression models and has been extended to models for survival data. In this article, we consider the mixture cure models. We propose two approaches to define explained variation under the mixture cure models, one based on the Kullback-Leibler information gain and the other based on residual sum of squares. We show that the proposed measures have desired properties as measures of explained variation, similar to those under other regression models. A simulation study is conducted to demonstrate the properties of the proposed measures. They are also applied to real data analyses to illustrate the use of explained variation.

The ROC of Cox proportional hazards cure models with application in cancer studies

With recent advancement in cancer screening and treatment, many patients with cancers are identified at early stage and clinically cured. Importantly, uncured patients should be treated timely before the cancer progresses to advanced stages for which therapeutic options are rather limited. It is also crucial to identify uncured subjects among patients with early-stage cancers for clinical trials to develop effective adjuvant therapies. Thus, it is of interest to develop statistical predictive models with as high accuracy as possible in predicting the latent cure status. The receiver operating characteristic curve (ROC) and the area under the ROC curve (AUC) are among the most widely used statistical metrics for assessing predictive accuracy or discriminatory power for a dichotomous outcome (cured/uncured). Yet the conventional AUC cannot be directly used due to incompletely observed cure status. In this article, we proposed new estimates of the ROC curve and its AUC for predicting latent cure status in Cox proportional hazards (PH) cure models and transformation cure models. We developed explicit formulas to estimate sensitivity, specificity, the ROC and its AUC without requiring to know the patient cure status. We also developed EM type estimates to approximate sensitivity, specificity, ROC and AUC conditional on observed data. Numerical studies were used to assess their finite-sample performance of the proposed methods. Both methods are consistent and have similar efficiency as shown in our numerical studies. A melanoma dataset was used to demonstrate the utility of the proposed estimates of the ROC curve for the latent cure status. We also have developed an [Formula: see text] package called [Formula: see text] to efficiently compute the proposed estimates.

[Subgroup identification based on accelerated failure time model combined with adaptive elastic net]

OBJECTIVE

To solve the problem of identifying subgroup in a randomized clinical trial with respect to survival time, we present a strategy based on accelerated failure time model to identify the subgroup with an enhanced treatment effect.

OBJECTIVE

We fitted and compared univariate accelerated failure time (AFT) models and penalized AFT models regularized by adaptive elastic net to identify the candidate covariates. Based on these covariates, we utilized change-point algorithm to classify the patient subgroups. A two-stage adaptive design was adopted to verify the treatment effect in certain subgroups. Simulations were conducted across different scenarios to evaluate the performance of the models.

OBJECTIVE

As the correlation between covariates differed, the power of the models with an adaptive design was stable. In the two-stage adaptive design, the power of the models was the highest when the two significance levels (α1 and α2) were allocated to be 0.035 and 0.015, respectively. A better effect of the responder subgroup was associated with a higher power of the models. For a fixed sample size, the power decreased as the covariate number to sample size ratio increased, but the power showed a stable trend when the ratio was above 1. The univariate models showed different distribution patterns of the parameters for different survival time, while their distribution was relatively stable in the penalized AFT models.

OBJECTIVE

The correlation between the covariates does not affect the performance of univariate AFT models and penalized AFT models. (0.035, 0.015) can be used as a reference for the significance level of an adaptive design. For small differences in the treatment effect between the responder and the non-responder, the penalized AFT model including the main effect of covariate (Penalized, Eq_in) outperforms the univariate AFT model excluding the main effect of covariate (Univariate, Eq_ex). Univariate, Eq_ex performs better when the covariate number to sample size ratio is less than 1, but is outperformed by Penalized, Eq_in when the ratio is above 1. The parameter distribution of survival time has a greater impact on the univariate models but a smaller impact on the penalized models.

Network-based survival analysis to discover target genes for developing cancer immunotherapies and predicting patient survival

Recently, cancer immunotherapies have been life-savers, however, only a fraction of treated patients have durable responses. Consequently, statistical methods that enable the discovery of target genes for developing new treatments and predicting patient survival are of importance. This paper introduced a network-based survival analysis method and applied it to identify candidate genes as possible targets for developing new treatments. RNA-seq data from a mouse study was used to select differentially expressed genes, which were then translated to those in humans. We constructed a gene network and identified gene clusters using a training set of 310 human gliomas. Then we conducted gene set enrichment analysis to select the gene clusters with significant biological function. A penalized Cox model was built to identify a small set of candidate genes to predict survival. An independent set of 690 human glioma samples was used to evaluate predictive accuracy of the survival model. The areas under time-dependent ROC curves in both the training and validation sets are more than 90%, indicating strong association between selected genes and patient survival. Consequently, potential biomedical interventions targeting these genes might be able to alter their expressions and prolong patient survival.

Measuring the temporal prognostic utility of a baseline risk score

In the time-to-event setting, the concordance probability assesses the relative level of agreement between a model-based risk score and the survival time of a patient. While it provides a measure of discrimination over the entire follow-up period of a study, the probability does not provide information on the longitudinal durability of a baseline risk score. It is possible that a baseline risk model is able to segregate short-term from long-term survivors but unable to maintain its discriminatory strength later in the follow-up period. As a consequence, this would motivate clinicians to re-evaluate the risk score longitudinally. This longitudinal re-evaluation may not, however, be feasible in many scenarios since a single baseline evaluation may be the only data collectible due to treatment or other clinical or ethical reasons. In these scenarios, an attenuation of the discriminatory power of the patient risk score over time would indicate decreased clinical utility and call into question whether this score should remain a prognostic tool at later time points. Working within the concordance probability paradigm, we propose a method to address this clinical scenario and evaluate the discriminatory power of a baseline derived risk score over time. The methodology is illustrated with two examples: a baseline risk score in colorectal cancer defined at the time of tumor resection, and for circulating tumor cells in metastatic prostate cancer.

Assessing cure status prediction from survival data using receiver operating characteristic curves

Survival analysis relies on the hypothesis that, if the follow-up is long enough, the event of interest will eventually be observed for all observations. This assumption, however, is often not realistic. The survival data then contain a cure fraction. A common approach to modelling and analysing this type of data consists in using cure models. Two types of information can therefore be obtained: the survival at a given time and the cure status, both possibly modelled as a function of the covariates. The cure status is often of interest to medical practitioners, and one is usually interested in predicting it based on markers. Receiver operating characteristic, Roc, curves are one way to evaluate the predicted performance; however, the classical Roc curve method is not appropriate since the cure status is partially unobserved due to the presence of censoring in survival data. We propose a Roc curve estimator that aims to evaluate the cured/noncured status classification performance from cure survival data. This estimator, which handles the presence of censoring, decomposes sensitivity and specificity by means of the definition of conditional probability, and estimates these two quantities by means of weighted empirical distribution functions. The mixture cure model is used to calculate the weights. Based on simulations, we demonstrate good performance of the proposed method, and compare it with the classical Roc curve nonparametric estimator that would be obtained if the cure status was fully observed. We also compare our proposed method with the Roc curves of Heagerty et al. (2000) for classical survival analysis. Finally, we illustrate the methodology on a breast cancer dataset.

A numerical strategy to evaluate performance of predictive scores via a copula-based approach

Assessing and comparing the performance of correlated predictive scores are of current interest in precision medicine. Given the limitations of available theoretical approaches for assessing and comparing the predictive accuracy, numerical methods are highly desired which, however, have not been systematically developed due to technical challenges. The main challenges include the lack of a general strategy on effectively simulating many kinds of correlated predictive scores each with some given level of predictive accuracy in either concordance index or the area under a receiver operating characteristic curve area under the curves (AUC). To fill in this important knowledge gap, this paper is to provide a general copula-based numeric framework for assessing and comparing predictive performance of correlated predictive or risk scores. The new algorithms are designed to effectively simulate correlated predictive scores with given levels of predictive accuracy as measured in terms of concordance indices or time-dependent AUC for predicting survival outcomes. The copula-based numerical strategy is convenient for numerically evaluating and comparing multiple measures of predictive accuracy of correlated risk scores and for investigating finite-sample properties of test statistics and confidence intervals as well as assessing for optimism of given performance measures using cross-validation or bootstrap.

Introduction to clinical research based on modern epidemiology

Over the past 20 years, recent advances in science technologies have dramatically changed the styles of clinical research. Currently, it has become more popular to use recent modern epidemiological techniques, such as propensity score, instrumental variable, competing risks, marginal structural modeling, mixed effects modeling, bootstrapping, and missing data analyses, than before. These advanced techniques, also known as modern epidemiology, may be strong tools for performing good clinical research, especially in large-scale observational studies, along with relevant research questions, good databases, and the passion of researchers. However, to use these methods effectively, we need to understand the basic assumptions behind them. Here, I will briefly introduce the concepts of these techniques and their implementation. In addition, I would like to emphasize that various types of clinical studies, not only large database studies but also small studies on rare and intractable diseases, are equally important because clinicians always do their best to take care of many kinds of patients who suffer from various kidney diseases and this is our most important mission.

[Subgroup identification based on an accelerated failure time model combined with adaptive elastic net]

OBJECTIVE

We propose a strategy for identifying subgroups with the treatment effect from the survival data of a randomized clinical trial based on accelerated failure time (AFT) model.

METHODS

We applied adaptive elastic net to the AFT model (designated as the penalized model) and identified the candidate covariates based on covariate-treatment interactions. To classify the patient subgroups, we utilized a likelihood-based change-point algorithm to determine the threshold cutoff point. A two-stage adaptive design was adopted to verify if the treatment effect existed within the identified subgroups.

RESULTS

The penalized model with the main effect of the covariates considerably outperformed the univariate model without the main effect for the trial data with a small sample size, a high censoring rate, a small subgroup size, or a sample size that did not exceed the number of covariates; in other scenarios, the latter model showed better performances. Compared with the traditional design, the adaptive design improved the power for detecting the treatment effect where subgroup effect exists with a well-controlled type Ⅰ error.

CONCLUSIONS

The penalized AFT model with the main effect of the covariates has advantages in subgroup identification from the survival data of clinical trials. Compared with the traditional design, the two-stage adaptive design has better performance in evaluation of the treatment effect when a subgroup effect exists.

Comparison of Long-term Survival Benefits in Trials of Immune Checkpoint Inhibitor vs Non-Immune Checkpoint Inhibitor Anticancer Agents Using ASCO Value Framework and ESMO Magnitude of Clinical Benefit Scale

IMPORTANCE

Recently, anticancer agents have generated excitement owing to their capacity to preserve long-term durable survival in some patients who are represented by a tail of the survival curve. However, because traditional measures of clinical benefit may not accurately capture durable survival, amendments to various valuation frameworks have been proposed to capture this benefit.

OBJECTIVES

To determine how frequently immune checkpoint inhibitor (ICI) anticancer agents vs non-ICI anticancer agents displayed trends of long-term durable survival, as defined by the American Society of Clinical Oncology Value Framework version 2 (ASCO-VF v2) and European Society of Medical Oncology Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1), as well as to further analyze the degree of agreement between ASCO and ESMO frameworks.

DESIGN, SETTING, AND PARTICIPANTS

In this cohort study, anticancer agents from phase 2 or 3 randomized clinical trials (RCTs) cited for clinical efficacy evidence in drug approval by the US Food and Drug Administration between January 2011 and March 2018 were identified. Data required for the ASCO-VF v2 tail-of-the-curve bonus and the ESMO-MCBS v1.1 immunotherapy-triggered long-term plateau adjustments were extracted from relevant RCTs. Frequency and difference in proportions were calculated to determine how often survival benefits were awarded to anticancer agents overall and to ICI and non-ICI anticancer agents individually.

MAIN OUTCOMES AND MEASURES

American Society of Clinical Oncology Value Framework v2 tail-of-the-curve bonuses and ESMO-MCBS v1.1 immunotherapy-triggered long-term plateau adjustments.

RESULTS

In total, 247 RCTs were identified, and 100 RCTs involving 57 164 patients were included, with 14 examining ICI agents (1 ipilimumab, 5 pembrolizumab, 5 nivolumab, 2 atezolizumab, and 1 durvalumab) and 86 examining non-ICI agents (74 targeted therapy, 8 chemotherapy, 3 hormone therapy, and 1 radiopharmaceutical). Randomized clinical trials were awarded ASCO-VF v2 tail-of-the-curve bonuses more often than ESMO-MCBS v1.1 immunotherapy-triggered long-term plateau adjustments (ASCO-VF v2, 45.0% [8 of 14 ICI RCTs and 37 of 86 non-ICI RCTs] vs ESMO-MCBS v1.1, 2.6% [1 of 12 ICI RCTs and 1 of 66 non-ICI RCTs). Randomized clinical trials for ICIs were not more likely to receive an ASCO-VF v2 bonus or ESMO-MCBS v1.1 adjustment than non-ICI RCTs (ASCO-VF: risk difference, 0.14; 95% CI, -0.14 to 0.42; P = .32; ESMO-MCBS: risk difference, 0.07; 95% CI, -0.09 to 0.23; P = .40). Poor agreement was found between the framework algorithms in identifying long-term survival benefits from RCTs (κ = 0.01; 95% CI, -0.23 to 0.22; P = .50).

CONCLUSIONS AND RELEVANCE

The ASCO-VF v2 and ESMO-MCBS v1.1 may require additional refinement to accurately capture the benefit of durable long-term survival, or ICI agents may not preserve long-term survival as conventionally thought.

Association between Ki-67 expression and clinical outcomes among patients with clinically node-negative, thick primary melanoma who underwent nodal staging

BACKGROUND

Patients with thick primary melanomas (≥4 mm) have highly variable survival outcomes. Cell proliferation marker Ki-67 has been identified as promising biomarker in thick melanoma but has not been evaluated since the wide spread adoption of sentinel lymph node biopsy. We revisit its prognostic relevance in the sentinel node era.

METHODS

We studied patients with thick (≥4 mm) primary melanoma prospectively enrolled in a clinicopathological biospecimen database from 2002 to 2015, and evaluated the prognostic value of Ki-67 expression while controlling for features included in the existing staging criteria.

RESULTS

We analyzed 68 patients who underwent lymph node sampling and who had an available tumor for Ki-67 immunohistochemical (IHC) staining. The median tumor thickness was 6.0 mm; the median follow-up was 2.6 years. In multivariable analysis including nodal status and primary tumor ulceration, Ki-67 expression was an independent predictor of worse recurrence-free survival (HR 2.19, P = 0.024) and overall survival (HR 2.49, P = 0.028). Natural log-transformed tumor thickness (ln [thickness]) was also significantly associated with worse OS (HR 2.39, P = 0.010).

CONCLUSION

We identify Ki-67 and ln (thickness) as potential biomarkers for patients with thick melanoma who have undergone nodal staging. If validated in additional studies, these biomarkers could be integrated into the staging criteria to improve risk-stratification.

Application of Concordance Probability Estimate to Predict Conversion from Mild Cognitive Impairment to Alzheimer's Disease

Subjects with mild cognitive impairment (MCI) have a substantially increased risk of developing dementia due to Alzheimer's disease (AD). Identifying MCI subjects who have high progression risk to AD is important in clinical management. Existing risk prediction models of AD among MCI subjects generally use either the AUC or Harrell's C-statistic to evaluate predictive accuracy. AUC is aimed at binary outcome and Harrell's C-statistic depends on the unknown censoring distribution. Gönen & Heller's K-index, also known as concordance probability estimate (CPE), is another measure of overall predictive accuracy for Cox proportional hazards (PH) models, which does not depend on censoring distribution. As a comprehensive example, using Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, we built a Cox PH model to predict the conversion from MCI to AD where the prognostic accuracy was evaluated using K-index.