Boosting the discriminatory power of sparse survival models via optimization of the concordance index and stability selection

Machine learning algorithms for identifying predictive variables of mortality risk following dementia diagnosis: a longitudinal cohort study

Machine learning (ML) could have advantages over traditional statistical models in identifying risk factors. Using ML algorithms, our objective was to identify the most important variables associated with mortality after dementia diagnosis in the Swedish Registry for Cognitive/Dementia Disorders (SveDem). From SveDem, a longitudinal cohort of 28,023 dementia-diagnosed patients was selected for this study. Sixty variables were considered as potential predictors of mortality risk, such as age at dementia diagnosis, dementia type, sex, body mass index (BMI), mini-mental state examination (MMSE) score, time from referral to initiation of work-up, time from initiation of work-up to diagnosis, dementia medications, comorbidities, and some specific medications for chronic comorbidities (e.g., cardiovascular disease). We applied sparsity-inducing penalties for three ML algorithms and identified twenty important variables for the binary classification task in mortality risk prediction and fifteen variables to predict time to death. Area-under-ROC curve (AUC) measure was used to evaluate the classification algorithms. Then, an unsupervised clustering algorithm was applied on the set of twenty-selected variables to find two main clusters which accurately matched surviving and dead patient clusters. A support-vector-machines with an appropriate sparsity penalty provided the classification of mortality risk with accuracy = 0.7077, AUROC = 0.7375, sensitivity = 0.6436, and specificity = 0.740. Across three ML algorithms, the majority of the identified twenty variables were compatible with literature and with our previous studies on SveDem. We also found new variables which were not previously reported in literature as associated with mortality in dementia. Performance of basic dementia diagnostic work-up, time from referral to initiation of work-up, and time from initiation of work-up to diagnosis were found to be elements of the diagnostic process identified by the ML algorithms. The median follow-up time was 1053 (IQR = 516-1771) days in surviving and 1125 (IQR = 605-1770) days in dead patients. For prediction of time to death, the CoxBoost model identified 15 variables and classified them in order of importance. These highly important variables were age at diagnosis, MMSE score, sex, BMI, and Charlson Comorbidity Index with selection scores of 23%, 15%, 14%, 12% and 10%, respectively. This study demonstrates the potential of sparsity-inducing ML algorithms in improving our understanding of mortality risk factors in dementia patients and their application in clinical settings. Moreover, ML methods can be used as a complement to traditional statistical methods.

A boosting first-hitting-time model for survival analysis in high-dimensional settings

In this paper we propose a boosting algorithm to extend the applicability of a first hitting time model to high-dimensional frameworks. Based on an underlying stochastic process, first hitting time models do not require the proportional hazards assumption, hardly verifiable in the high-dimensional context, and represent a valid parametric alternative to the Cox model for modelling time-to-event responses. First hitting time models also offer a natural way to integrate low-dimensional clinical and high-dimensional molecular information in a prediction model, that avoids complicated weighting schemes typical of current methods. The performance of our novel boosting algorithm is illustrated in three real data examples.

Mid-Arm Muscle Circumference or Body Weight-Standardized Hand Grip Strength in the GLIM Superiorly Predicts Survival in Chinese Colorectal Cancer Patients

Our objective was to identify the optimal method to assess reduced muscle mass (RMM) using the Global Leadership Initiative on Malnutrition (GLIM) approach and investigate the roles of the GLIM approach in nutrition assessment and survival prediction in colorectal cancer (CRC) patients. During a median follow-up period of 4.2 (4.0, 4.4) years, a development cohort of 3612 CRC patients with a mean age of 64.09 ± 12.45 years was observed, as well as an external validation cohort of 875 CRC patients. Kaplan−Meier curves and multivariate Cox regression were adopted to analyze the association between GLIM-diagnosed malnutrition and the overall survival (OS) of CRC patients. A nomogram predicting individualized survival was constructed based on independent prognostic predictors. The concordance index, calibration curve, and decision curve were applied to appraise the discrimination, accuracy, and clinical efficacy of the nomogram, respectively. Patients diagnosed with severe malnutrition based on either the mid-arm muscle circumference (MAMC) or body weight-standardized hand grip strength (HGS/W) method had the highest mortality hazard ratio (HR, 1.51; 95% CI, 1.34−1.70; p < 0.001). GLIM-defined malnutrition was diagnosed in 47.6% of patients. Severe malnutrition was an independent mortality risk factor for OS (HR, 1.25; 95% CI, 1.10−1.42; p < 0.001). The GLIM nomogram showed good performance in predicting the survival of CRC patients and was clinically beneficial. Our findings support the effectiveness of GLIM in diagnosing malnutrition and predicting OS in CRC patients.

Deselection of base-learners for statistical boosting-with an application to distributional regression

We present a new procedure for enhanced variable selection for component-wise gradient boosting. Statistical boosting is a computational approach that emerged from machine learning, which allows to fit regression models in the presence of high-dimensional data. Furthermore, the algorithm can lead to data-driven variable selection. In practice, however, the final models typically tend to include too many variables in some situations. This occurs particularly for low-dimensional data (p<n), where we observe a slow overfitting behavior of boosting. As a result, more variables get included into the final model without altering the prediction accuracy. Many of these false positives are incorporated with a small coefficient and therefore have a small impact, but lead to a larger model. We try to overcome this issue by giving the algorithm the chance to deselect base-learners with minor importance. We analyze the impact of the new approach on variable selection and prediction performance in comparison to alternative methods including boosting with earlier stopping as well as twin boosting. We illustrate our approach with data of an ongoing cohort study for chronic kidney disease patients, where the most influential predictors for the health-related quality of life measure are selected in a distributional regression approach based on beta regression.

Randomized boosting with multivariable base-learners for high-dimensional variable selection and prediction

BACKGROUND

Statistical boosting is a computational approach to select and estimate interpretable prediction models for high-dimensional biomedical data, leading to implicit regularization and variable selection when combined with early stopping. Traditionally, the set of base-learners is fixed for all iterations and consists of simple regression learners including only one predictor variable at a time. Furthermore, the number of iterations is typically tuned by optimizing the predictive performance, leading to models which often include unnecessarily large numbers of noise variables.

RESULTS

We propose three consecutive extensions of classical component-wise gradient boosting. In the first extension, called Subspace Boosting (SubBoost), base-learners can consist of several variables, allowing for multivariable updates in a single iteration. To compensate for the larger flexibility, the ultimate selection of base-learners is based on information criteria leading to an automatic stopping of the algorithm. As the second extension, Random Subspace Boosting (RSubBoost) additionally includes a random preselection of base-learners in each iteration, enabling the scalability to high-dimensional data. In a third extension, called Adaptive Subspace Boosting (AdaSubBoost), an adaptive random preselection of base-learners is considered, focusing on base-learners which have proven to be predictive in previous iterations. Simulation results show that the multivariable updates in the three subspace algorithms are particularly beneficial in cases of high correlations among signal covariates. In several biomedical applications the proposed algorithms tend to yield sparser models than classical statistical boosting, while showing a very competitive predictive performance also compared to penalized regression approaches like the (relaxed) lasso and the elastic net.

CONCLUSIONS

The proposed randomized boosting approaches with multivariable base-learners are promising extensions of statistical boosting, particularly suited for highly-correlated and sparse high-dimensional settings. The incorporated selection of base-learners via information criteria induces automatic stopping of the algorithms, promoting sparser and more interpretable prediction models.

Effect of timing, technique and molecular features on brain control with local therapies in oncogene-driven lung cancer

BACKGROUND

The improved efficacy of tyrosine kinase inhibitors (TKI) mandates reappraisal of local therapy (LT) for brain metastases (BM) of oncogene-driven non-small-cell lung cancer (NSCLC).

PATIENTS AND METHODS

This study included all epidermal growth factor receptor-mutated (EGFR⁺, n = 108) and anaplastic lymphoma kinase-rearranged (ALK⁺, n = 33) TKI-naive NSCLC patients diagnosed with BM in the Thoraxklinik Heidelberg between 2009 and 2019. Eighty-seven patients (62%) received early LT, while 54 (38%) received delayed (n = 34; 24%) or no LT (n = 20; 14%). LT comprised stereotactic (SRT; n = 40; 34%) or whole-brain radiotherapy (WBRT; n = 77; 66%), while neurosurgical resection was carried out in 19 cases.

RESULTS

Median overall survival (OS) was 49.1 months for ALK⁺ and 19.5 months for EGFR⁺ patients (P = 0.001), with similar median intracranial progression-free survival (icPFS) (15.7 versus 14.0 months, respectively; P = 0.80). Despite the larger and more symptomatic BM (P < 0.001) of patients undergoing early LT, these experienced longer icPFS [hazard ratio (HR) 0.52; P = 0.024], but not OS (HR 1.63; P = 0.12), regardless of the radiotherapy technique (SRT versus WBRT) and number of lesions. High-risk oncogene variants, i.e. non-del19 EGFR mutations and 'short' EML4-ALK fusions (mainly variant 3, E6:A20), were associated with earlier intracranial progression (HR 2.97; P = 0.001). The longer icPFS with early LT was also evident in separate analyses of the EGFR⁺ and ALK⁺ subsets.

CONCLUSIONS

Despite preferential use for cases with poor prognostic factors, early LT prolongs the icPFS, but not OS, in TKI-treated EGFR⁺/ALK⁺ NSCLC. Considering the lack of survival benefit, and the neurocognitive effects of WBRT, patients presenting with polytopic BM may benefit from delaying radiotherapy, or from radiosurgery of multiple or selected lesions. For SRT candidates, the improved tumor control with earlier radiotherapy should be weighed against the potential toxicity and the enhanced intracranial activity of newer TKI. High-risk EGFR/ALK variants are associated with earlier intracranial failure and identify patients who could benefit from more aggressive management.

Development of a novel lipid metabolism-based risk score model in hepatocellular carcinoma patients

Background

Liver cancer is one of the most common malignancies worldwide. HCC (hepatocellular carcinoma) is the predominant pathological type of liver cancer, accounting for approximately 75–85 % of all liver cancers. Lipid metabolic reprogramming has emerged as an important feature of HCC. However, the influence of lipid metabolism-related gene expression in HCC patient prognosis remains unknown. In this study, we performed a comprehensive analysis of HCC gene expression data from TCGA (The Cancer Genome Atlas) to acquire further insight into the role of lipid metabolism-related genes in HCC patient prognosis.

Methods

We analyzed the mRNA expression profiles of 424 HCC patients from the TCGA database. GSEA(Gene Set Enrichment Analysis) was performed to identify lipid metabolism-related gene sets associated with HCC. We performed univariate Cox regression and LASSO(least absolute shrinkage and selection operator) regression analyses to identify genes with prognostic value and develop a prognostic model, which was tested in a validation cohort. We performed Kaplan-Meier survival and ROC (receiver operating characteristic) analyses to evaluate the performance of the model.

Results

We identified three lipid metabolism-related genes (ME1, MED10, MED22) with prognostic value in HCC and used them to calculate a risk score for each HCC patient. High-risk HCC patients exhibited a significantly lower survival rate than low-risk patients. Multivariate Cox regression analysis revealed that the 3-gene signature was an independent prognostic factor in HCC. Furthermore, the signature provided a highly accurate prediction of HCC patient prognosis.

Conclusions

We identified three lipid-metabolism-related genes that are upregulated in HCC tissues and established a 3-gene signature-based risk model that can accurately predict HCC patient prognosis. Our findings support the strong links between lipid metabolism and HCC and may facilitate the development of new metabolism-targeted treatment approaches for HCC.

Radiomics analysis using stability selection supervised component analysis for right-censored survival data

Radiomics is a newly emerging field that involves the extraction of massive quantitative features from biomedical images by using data-characterization algorithms. Distinctive imaging features identified from biomedical images can be used for prognosis and therapeutic response prediction, and they can provide a noninvasive approach for personalized therapy. So far, many of the published radiomics studies utilize existing out of the box algorithms to identify the prognostic markers from biomedical images that are not specific to radiomics data. To better utilize biomedical images, we propose a novel machine learning approach, stability selection supervised principal component analysis (SSSuperPCA) that identifies stable features from radiomics big data coupled with dimension reduction for right-censored survival outcomes. The proposed approach allows us to identify a set of stable features that are highly associated with the survival outcomes in a simple yet meaningful manner, while controlling the per-family error rate. We evaluate the performance of SSSuperPCA using simulations and real data sets for non-small cell lung cancer and head and neck cancer, and compare it with other machine learning algorithms. The results demonstrate that our method has a competitive edge over other existing methods in identifying the prognostic markers from biomedical imaging data for the prediction of right-censored survival outcomes.

Characterization of a five-microRNA signature as a prognostic biomarker for esophageal squamous cell carcinoma

This study aims to identify a miRNAs signature for predicting overall survival (OS) in esophageal squamous cell carcinoma (ESCC) patients. MiRNA expression profiles and corresponding clinical information of 119 ESCC patients were obtained from NCBI GEO and used as the training set. Differentially expressed miRNAs (DEmiRNAs) were screened between early-stage and late-stage samples. Cox regression analysis, recursive feature elimination (RFE)-support vector machine (SVM) algorithm, and LASSO Cox regression model were used to identify prognostic miRNAs and consequently build a prognostic scoring model. Moreover, promising target genes of these prognostic miRNAs were predicted followed by construction of miRNA-target gene networks. Functional relevance of predicted target genes of these prognostic miRNAs in ESCC was analyzed by performing function enrichment analyses. There were 46 DEmiRNAs between early-stage and late-stage samples in the training set. A risk score model based on five miRNAs was built. The five-miRNA risk score could classify the training set into a high-risk group and a low-risk group with significantly different OS time. Risk stratification ability of the five-miRNA risk score was successfully validated on an independent set from the Cancer Genome Atlas (TCGA). Various biological processes and pathways were identified to be related to these miRNAs, such as Wnt signaling pathway, inflammatory mediator regulation of TRP channels pathway, and estrogen signaling pathway. The present study suggests a pathological stage-related five-miRNA signature that may have clinical implications in predicting prognosis of ESCC patients.

Tree-based classification system incorporating the HVTT-PVTT score for personalized management of hepatocellular carcinoma patients with macroscopic vascular invasion

Purpose: To develop a decision tree algorithm-based classification system for personalized management of hepatocellular carcinoma (HCC) patients with macroscopic vascular invasion.

Results: The HVTT-PVTT score could differentiate two groups of patients (< 3 and ≥ 3 points) with different survival outcomes (7.4 vs 4.6 months, P < 0.001) and surgical proportion (24.4% vs 3.6%, P < 0.001). Using the Cox regression model and classification and regression tree (CART) algorithm, patients in the training set were automatically separated into three subgroups with different prognosis (10.3 vs 6.1 vs 3.3 months). The predictive accuracy was verified in the validation group (12.3 vs 6.9 vs 5.6 months) and was better than other commonly used staging systems.

Conclusions: Our study proposed a new classification system for HCC patients with macroscopic vascular invasion that could be meaningful for personalized management of these patients.

Methods: A total of 869 HCC patients initially diagnosed with macroscopic vascular invasion were randomly divided into training and validation sets. A comprehensive and simplified HVTT-PVTT score was set up for subdivision of vascular invasion according to the patients’ survival outcome. Then, a decision tree algorithm-based classification system was used to establish the refined subdivision system incorporating all independent prognostic factors.

Significance Tests for Boosted Location and Scale Models with Linear Base-Learners

Generalized additive models for location scale and shape (GAMLSS) offer very flexible solutions to a wide range of statistical analysis problems, but can be challenging in terms of proper model specification. This complex task can be simplified using regularization techniques such as gradient boosting algorithms, but the estimates derived from such models are shrunken towards zero and it is consequently not straightforward to calculate proper confidence intervals or test statistics. In this article, we propose two strategies to obtain p-values for linear effect estimates for Gaussian location and scale models based on permutation tests and a parametric bootstrap approach. These procedures can provide a solution for one of the remaining problems in the application of gradient boosting algorithms for distributional regression in biostatistical data analyses. Results from extensive simulations indicate that in low-dimensional data both suggested approaches are able to hold the type-I error threshold and provide reasonable test power comparable to the Wald-type test for maximum likelihood inference. In high-dimensional data, when gradient boosting is the only feasible inference for this model class, the power decreases but the type-I error is still under control. In addition, we demonstrate the application of both tests in an epidemiological study to analyse the impact of physical exercise on both average and the stability of the lung function of elderly people in Germany.

A Refined Staging Model for Resectable Pancreatic Ductal Adenocarcinoma Incorporating Examined Lymph Nodes, Location of Tumor and Positive Lymph Nodes Ratio

Background: Nodal status and tumor site are prognostic factors for resectable pancreatic ductal adenocarcinoma (PDAC). Parameters for nodal status are diverse, and the number of examined lymph nodes (eNs) needed for good prognosis are uncertain. We try to modify staging system of resectable PDAC with parameters mentioned above by recursive partitioning analysis.

Methods: Patients from the Surveillance, Epidemiology, and End Results (SEER) database were divided into training cohort and internal validation cohort, randomly. PDAC patients from Sun Yat-sen University Cancer Center were regarded as external validation cohort. The training cohort was used to refine staging model by recursive partitioning analysis, while the internal validation cohort and the external validation cohort were applied to assess discriminatory capacity of staging model. For parameters included in the modified model, their effects were studied.

Results: The number of eNs, tumor site and tumor size were risk factors for positive nodal status. Lymph nodes ratio (LNR), tumor site, eNs and T stages of 8^th the American Joint Committee on Cancer (AJCC) were selected to develop a refined model, dividing patients into 5 groups of different outcomes, preceding 8^th AJCC classification. Besides, we found that (1) for small PDAC (diameter < 1cm), lymph node metastasis was rarely found; (2) enough eNs were needed to ensure better prognosis of node-negative patients; (3) tumors in the head of pancreas were prone to lymph nodes metastasis; (4) for node-positive patients, LNR was a better nodal parameter compared to positive lymph nodes (pNs).

Conclusion: Our improved staging system helps to illuminate the interactions among tumor site, size and eNs.

Boosting for statistical modelling-A non-technical introduction

Boosting algorithms were originally developed for machine learning but were later adapted to estimate statistical models—offering various practical advantages such as automated variable selection and implicit regularization of effect estimates. The interpretation of the resulting models, however, remains the same as if they had been fitted by classical methods. Boosting, hence, allows to use an advanced machine learning scheme to estimate various types of statistical models. This tutorial aims to highlight how boosting can be used for semi-parametric modelling, what practical implications follow from the design of the algorithm and what kind of drawbacks data analysts have to expect. We illustrate the application of boosting in the analysis of a stunting score from children in India and a high-dimensional dataset of tumour DNA to develop a biomarker for the occurrence of metastases in breast cancer patients.

An Update on Statistical Boosting in Biomedicine

Statistical boosting algorithms have triggered a lot of research during the last decade. They combine a powerful machine learning approach with classical statistical modelling, offering various practical advantages like automated variable selection and implicit regularization of effect estimates. They are extremely flexible, as the underlying base-learners (regression functions defining the type of effect for the explanatory variables) can be combined with any kind of loss function (target function to be optimized, defining the type of regression setting). In this review article, we highlight the most recent methodological developments on statistical boosting regarding variable selection, functional regression, and advanced time-to-event modelling. Additionally, we provide a short overview on relevant applications of statistical boosting in biomedicine.

Probing for Sparse and Fast Variable Selection with Model-Based Boosting

We present a new variable selection method based on model-based gradient boosting and randomly permuted variables. Model-based boosting is a tool to fit a statistical model while performing variable selection at the same time. A drawback of the fitting lies in the need of multiple model fits on slightly altered data (e.g., cross-validation or bootstrap) to find the optimal number of boosting iterations and prevent overfitting. In our proposed approach, we augment the data set with randomly permuted versions of the true variables, so-called shadow variables, and stop the stepwise fitting as soon as such a variable would be added to the model. This allows variable selection in a single fit of the model without requiring further parameter tuning. We show that our probing approach can compete with state-of-the-art selection methods like stability selection in a high-dimensional classification benchmark and apply it on three gene expression data sets.