Comparison of three prognostic models for predicting cancer-specific survival among patients with gastrointestinal stromal tumors

StrokeClassifier: ischemic stroke etiology classification by ensemble consensus modeling using electronic health records

Determining acute ischemic stroke (AIS) etiology is fundamental to secondary stroke prevention efforts but can be diagnostically challenging. We trained and validated an automated classification tool, StrokeClassifier, using electronic health record (EHR) text from 2039 non-cryptogenic AIS patients at 2 academic hospitals to predict the 4-level outcome of stroke etiology adjudicated by agreement of at least 2 board-certified vascular neurologists' review of the EHR. StrokeClassifier is an ensemble consensus meta-model of 9 machine learning classifiers applied to features extracted from discharge summary texts by natural language processing. StrokeClassifier was externally validated in 406 discharge summaries from the MIMIC-III dataset reviewed by a vascular neurologist to ascertain stroke etiology. Compared with vascular neurologists' diagnoses, StrokeClassifier achieved the mean cross-validated accuracy of 0.74 and weighted F1 of 0.74 for multi-class classification. In MIMIC-III, its accuracy and weighted F1 were 0.70 and 0.71, respectively. In binary classification, the two metrics ranged from 0.77 to 0.96. The top 5 features contributing to stroke etiology prediction were atrial fibrillation, age, middle cerebral artery occlusion, internal carotid artery occlusion, and frontal stroke location. We designed a certainty heuristic to grade the confidence of StrokeClassifier's diagnosis as non-cryptogenic by the degree of consensus among the 9 classifiers and applied it to 788 cryptogenic patients, reducing cryptogenic diagnoses from 25.2% to 7.2%. StrokeClassifier is a validated artificial intelligence tool that rivals the performance of vascular neurologists in classifying ischemic stroke etiology. With further training, StrokeClassifier may have downstream applications including its use as a clinical decision support system.

StrokeClassifier: Ischemic Stroke Etiology Classification by Ensemble Consensus Modeling Using Electronic Health Records

Determining the etiology of an acute ischemic stroke (AIS) is fundamental to secondary stroke prevention efforts but can be diagnostically challenging. We trained and validated an automated classification machine intelligence tool, StrokeClassifier, using electronic health record (EHR) text data from 2,039 non-cryptogenic AIS patients at 2 academic hospitals to predict the 4-level outcome of stroke etiology determined by agreement of at least 2 board-certified vascular neurologists' review of the stroke hospitalization EHR. StrokeClassifier is an ensemble consensus meta-model of 9 machine learning classifiers applied to features extracted from discharge summary texts by natural language processing. StrokeClassifier was externally validated in 406 discharge summaries from the MIMIC-III dataset reviewed by a vascular neurologist to ascertain stroke etiology. Compared with stroke etiologies adjudicated by vascular neurologists, StrokeClassifier achieved the mean cross-validated accuracy of 0.74 (±0.01) and weighted F1 of 0.74 (±0.01). In the MIMIC-III cohort, the accuracy and weighted F1 of StrokeClassifier were 0.70 and 0.71, respectively. SHapley Additive exPlanation analysis elucidated that the top 5 features contributing to stroke etiology prediction were atrial fibrillation, age, middle cerebral artery occlusion, internal carotid artery occlusion, and frontal stroke location. We then designed a certainty heuristic to deem a StrokeClassifier diagnosis as confidently non-cryptogenic by the degree of consensus among the 9 classifiers, and applied it to 788 cryptogenic patients. This reduced the percentage of the cryptogenic strokes from 25.2% to 7.2% of all ischemic strokes. StrokeClassifier is a validated artificial intelligence tool that rivals the performance of vascular neurologists in classifying ischemic stroke etiology for individual patients. With further training, StrokeClassifier may have downstream applications including its use as a clinical decision support system.

Construction and validation of prognostic models in critically Ill patients with sepsis-associated acute kidney injury: interpretable machine learning approach

BACKGROUND

Acute kidney injury (AKI) is a common complication in critically ill patients with sepsis and is often associated with a poor prognosis. We aimed to construct and validate an interpretable prognostic prediction model for patients with sepsis-associated AKI (S-AKI) using machine learning (ML) methods.

METHODS

Data on the training cohort were collected from the Medical Information Mart for Intensive Care IV database version 2.2 to build the model, and data of patients were extracted from Hangzhou First People's Hospital Affiliated to Zhejiang University School of Medicine for external validation of model. Predictors of mortality were identified using Recursive Feature Elimination (RFE). Then, random forest, extreme gradient boosting (XGBoost), multilayer perceptron classifier, support vector classifier, and logistic regression were used to establish a prognosis prediction model for 7, 14, and 28 days after intensive care unit (ICU) admission, respectively. Prediction performance was assessed using the receiver operating characteristic (ROC) curve and decision curve analysis (DCA). SHapley Additive exPlanations (SHAP) were used to interpret the ML models.

RESULTS

In total, 2599 patients with S-AKI were included in the analysis. Forty variables were selected for the model development. According to the areas under the ROC curve (AUC) and DCA results for the training cohort, XGBoost model exhibited excellent performance with F1 Score of 0.847, 0.715, 0.765 and AUC (95% CI) of 0.91 (0.90, 0.92), 0.78 (0.76, 0.80), and 0.83 (0.81, 0.85) in 7 days, 14 days and 28 days group, respectively. It also demonstrated excellent discrimination in the external validation cohort. Its AUC (95% CI) was 0.81 (0.79, 0.83), 0.75 (0.73, 0.77), 0.79 (0.77, 0.81) in 7 days, 14 days and 28 days group, respectively. SHAP-based summary plot and force plot were used to interpret the XGBoost model globally and locally.

CONCLUSIONS

ML is a reliable tool for predicting the prognosis of patients with S-AKI. SHAP methods were used to explain intrinsic information of the XGBoost model, which may prove clinically useful and help clinicians tailor precise management.

Predicting the Prognosis of Patients in the Coronary Care Unit: A Novel Multi-Category Machine Learning Model Using XGBoost

Background

Early prediction and classification of prognosis is essential for patients in the coronary care unit (CCU). We applied a machine learning (ML) model using the eXtreme Gradient Boosting (XGBoost) algorithm to prognosticate CCU patients and compared XGBoost with traditional classification models.

Methods

CCU patients' data were extracted from the MIMIC-III v1.4 clinical database, and divided into four groups based on the time to death: <30 days, 30 days−1 year, 1–5 years, and ≥5 years. Four classification models, including XGBoost, naïve Bayes (NB), logistic regression (LR), and support vector machine (SVM) were constructed using the Python software. These four models were tested and compared for accuracy, F1 score, Matthews correlation coefficient (MCC), and area under the curve (AUC) of the receiver operating characteristic curves. Subsequently, Local Interpretable Model-Agnostic Explanations method was performed to improve XGBoost model interpretability. We also constructed sub-models of each model based on the different categories of death time and compared the differences by decision curve analysis. The optimal model was further analyzed using a clinical impact curve. At last, feature ablation curves of the XGBoost model were conducted to obtain the simplified model.

Results

Overall, 5360 CCU patients were included. Compared to NB, LR, and SVM, the XGBoost model showed better accuracy (0.663, 0.605, 0.632, and 0.622), micro-AUCs (0.873, 0.811, 0.841, and 0.818), and MCC (0.337, 0.317, 0.250, and 0.182). In subgroup analysis, the XGBoost model had a better predictive performance in acute myocardial infarction subgroup. The decision curve and clinical impact curve analyses verified the clinical utility of the XGBoost model for different categories of patients. Finally, we obtained a simplified model with thirty features.

Conclusions

For CCU physicians, the ML technique by XGBoost is a potential predictive tool in patients with different conditions, and it may contribute to improvements in prognosis.

Monitoring Circulating Tumor DNA During Surgical Treatment in Patients with Gastrointestinal Stromal Tumors

The majority of patients diagnosed with advanced gastrointestinal stromal tumors (GISTs) are successfully treated with a combination of surgery and tyrosine kinase inhibitors (TKIs). However, it remains challenging to monitor treatment efficacy and identify relapse early. Here, we utilized a sequencing strategy based on molecular barcodes and developed a GIST-specific panel to monitor tumor-specific and TKI resistance mutations in cell-free DNA and applied the approach to patients undergoing surgical treatment. Thirty-two patients with GISTs were included, and 161 blood plasma samples were collected and analyzed at routine visits before and after surgery and at the beginning, during, and after surgery. Patients were included regardless of their risk category. Our GIST-specific sequencing approach allowed detection of tumor-specific mutations and TKI resistance mutations with mutant allele frequency < 0.1%. Circulating tumor DNA (ctDNA) was detected in at least one timepoint in nine of 32 patients, ranging from 0.04% to 93% in mutant allele frequency. High-risk patients were more often ctDNA positive than other risk groups (P < 0.05). Patients with detectable ctDNA also displayed higher tumor cell proliferation rates (P < 0.01) and larger tumor sizes (P < 0.01). All patients who were ctDNA positive during surgery became negative after surgery. Finally, in two patients who progressed on TKI treatment, we detected multiple resistance mutations. Our data show that ctDNA may become a clinically useful biomarker in monitoring treatment efficacy in patients with high-risk GISTs and can assist in treatment decision making.

Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost

Background

Sepsis is a significant cause of mortality in-hospital, especially in ICU patients. Early prediction of sepsis is essential, as prompt and appropriate treatment can improve survival outcomes. Machine learning methods are flexible prediction algorithms with potential advantages over conventional regression and scoring system. The aims of this study were to develop a machine learning approach using XGboost to predict the 30-days mortality for MIMIC-III Patients with sepsis-3 and to determine whether such model performs better than traditional prediction models.

Methods

Using the MIMIC-III v1.4, we identified patients with sepsis-3. The data was split into two groups based on death or survival within 30 days and variables, selected based on clinical significance and availability by stepwise analysis, were displayed and compared between groups. Three predictive models including conventional logistic regression model, SAPS-II score prediction model and XGBoost algorithm model were constructed by R software. Then, the performances of the three models were tested and compared by AUCs of the receiver operating characteristic curves and decision curve analysis. At last, nomogram and clinical impact curve were used to validate the model.

Results

A total of 4559 sepsis-3 patients are included in the study, in which, 889 patients were death and 3670 survival within 30 days, respectively. According to the results of AUCs (0.819 [95% CI 0.800–0.838], 0.797 [95% CI 0.781–0.813] and 0.857 [95% CI 0.839–0.876]) and decision curve analysis for the three models, the XGboost model performs best. The risk nomogram and clinical impact curve verify that the XGboost model possesses significant predictive value.

Conclusions

Using machine learning technique by XGboost, more significant prediction model can be built. This XGboost model may prove clinically useful and assist clinicians in tailoring precise management and therapy for the patients with sepsis-3.

Identification of low-density lipoprotein receptor class A domain containing 4 (LDLRAD4) as a prognostic indicator in primary gastrointestinal stromal tumors

BACKGROUND

There is an urgent clinical need to select the patients with resectable gastrointestinal stromal tumors (GISTs) who can benefit from adjuvant treatment after complete resection based on disease recurrence risk stratification. We hypothesized that integrating biomarkers into available risk assessment tools may improve the precision of GIST prognostic predictions.

METHODS

Candidate genes that may cause GIST progression were identified using the Gene Expression Omnibus dataset GSE20708. Quantitative Real-time was used to confirm the prognostic value of the candidate genes for recurrence-free survival (RFS) in a cohort of 94 patients.

RESULTS

Thirty-seven differentially expressed genes between localized tumors and metastatic primary tumors were found; 14 (37.8%) were upregulated and 23 (62.2%) were downregulated in the latter tumors. Low-density lipoprotein receptor class A domain containing 4 (LDLRAD4) was selected for further prognostic analysis. Although LDLRAD4 mRNA expression was not associated with recurrence risk grades as determined by the revised NIH consensus criteria, multivariate Cox regression analysis showed that LDLRAD4 expression (hazard ratio [HR] = 4.403, 95% confidence interval [CI]: 1.822-10.641, P = 0.001), tumor size (HR = 1.174, 95% CI: 1.027-1.342, P = 0.019) and tumor location (HR = 6.291, 95% CI: 1.128-35.080, P = 0.036) were independent prognostic factors for RFS in patients with resectable GISTs. Moreover, the RFS model constructed by these three factors may effectively predict GIST prognosis within the first 2 postsurgical years.

CONCLUSION

Our study identifies LDLRAD4 as a suitable prognostic marker for GISTs. The integration of biomarkers into risk assessment tools may improve the precision of GIST prognostic predictions.

Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis

Background

Sepsis is a common critical condition caused by the body’s overwhelming response to certain infective agents. Many biomarkers, including the serum lactate level, have been used for sepsis diagnosis and guiding treatment. Recently, the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) recommended the Sequential Organ Failure Assessment (SOFA) and the quick SOFA (qSOFA) rather than lactate for screening sepsis and assess prognosis. Here, we aim to explore and compare the prognostic accuracy of the lactate level, the SOFA score and the qSOFA score for mortality in septic patients using the public Medical Information Mart for Intensive Care III database (MIMIC III).

Methods

The baseline characteristics, laboratory test results and outcomes for sepsis patients were retrieved from MIMIC III. Survival was analysed by the Kaplan-Meier method. Univariate and multivariate analysis was performed to identify predictors of prognosis. Receiver operating characteristic curve (ROC) analysis was conducted to compare lactate with SOFA and qSOFA scores.

Results

A total of 3713 cases were initially identified. The analysis cohort included 1865 patients. The 24-h average lactate levels and the worst scores during the first 24 h of ICU admission were collected. Patients in the higher lactate group had higher mortality than those in the lower lactate group. Lactate was an independent predictor of sepsis prognosis. The AUROC of lactate (AUROC, 0.664 [95% CI, 0.639–0.689]) was significantly higher than that of qSOFA (AUROC, 0.547 [95% CI, 0.521–0.574]), and it was similar to the AUROC of SOFA (AUROC, 0.686 [95% CI, 0.661–0.710]). But the timing of lactate relative to SOFA and qSOFA scores was inconsistent.

Conclusion

Lactate is an independent prognostic predictor of mortality for patients with sepsis. It has superior discriminative power to qSOFA, and shows discriminative ability similar to that of SOFA.

Electronic supplementary material

The online version of this article (10.1186/s13049-019-0609-3) contains supplementary material, which is available to authorized users.

Colonic Gastrointestinal Stromal Tumor: A Population-Based Analysis of Incidence and Survival

Objectives

The incidence of gastrointestinal stromal tumors (GISTs) located in the colon is rare. Current studies mainly focus on case reports for colonic GISTs. Therefore, a population-based analysis was useful to guide the clinical treatment strategy.

Methods

The patients were selected from 2000 to 2015 based on Surveillance, Epidemiology, and End Results (SEER) database. Patients' demographics, tumor characteristics, incidence, treatment, and survival were retrieved for analysis.

Results

249 cases of colonic GISTs were collected. The male-female ratio was close to 1 : 1 (male 51.41%, female 48.59%). Most cases were Caucasians (70.28%), and African Americans accounted for 19.68%. Age of diagnosis ranged from 21 to 93 years with a median (mean) age of 67.5 (65.56). The incidence was rare, only 0.018 per 100,000. It had an annual percentage change (APC = -0.7728) without statistical significance (P = 0.5127) while the incidence of other GISTs increased from 2000 to 2015, with an annual percentage change of 3.9% (P = 0.0001). Surgery was associated with better prognosis whereas chemotherapy did not impact the survival rate.

Conclusion

Colonic GIST is a rare solid tumor, and the incidence is stable. The entity has a poorer prognosis than other GISTs. Surgery improved the survival rate, while chemotherapy did not.