Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD Statement

Machine Learning Model Identifies Increased Operative Time and Greater BMI as Predictors for Overnight Admission After Outpatient Hip Arthroscopy

Purpose

The purposes of this study were to identify patient characteristics and risk factors for overnight admission following outpatient hip arthroscopy and to develop a machine learning algorithm that can effectively identify patients requiring admission following elective hip arthroscopy.

Methods

A retrospective review of a prospectively collected national surgical outcomes database was performed to identify patients who underwent elective outpatient hip arthroscopy from 2006 to 2018. Patients admitted overnight postoperatively were identified as those with length of stay of 1 or more days. Models were generated using random forest (RF), extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), elastic net penalized logistic regression, and an additional model was produced as a weighted ensemble of the four final algorithms.

Results

Overall, 1,276 patients were included. The median age was 43 years, and 64.2% (819) were female. Of the included patients, 109 (8.5%) required an overnight stay following elective outpatient hip arthroscopy. The most important factors for inpatient admission were increasing operative time, general anesthesia, age extremes, male gender, greater body mass index (BMI), American Society of Anesthesiologists classification >1, and the following preoperative lab values outside of normal ranges: sodium, platelet count, hematocrit, and leukocyte count. The ensemble model achieved the best performance based on discrimination assessed via internal validation (area under the curve = .71), calibration, and decision curve analysis. The model was integrated into a Web-based open-access application able to provide both personalized predictions and explanations.

Conclusion

A machine learning algorithm developed based on preoperative features identified increasing operative time, age extremes, greater BMI, sodium, hematocrit, platelets, and leukocyte count as the most important variables associated with inpatient admission with fair validity.

Impact of Intraoperative Data on Risk Prediction for Mortality After Intra-Abdominal Surgery

BACKGROUND

Risk prediction models for postoperative mortality after intra-abdominal surgery have typically been developed using preoperative variables. It is unclear if intraoperative data add significant value to these risk prediction models.

METHODS

With IRB approval, an institutional retrospective cohort of intra-abdominal surgery patients in the 2005 to 2015 American College of Surgeons National Surgical Quality Improvement Program was identified. Intraoperative data were obtained from the electronic health record. The primary outcome was 30-day mortality. We evaluated the performance of machine learning algorithms to predict 30-day mortality using: 1) baseline variables and 2) baseline + intraoperative variables. Algorithms evaluated were: 1) logistic regression with elastic net selection, 2) random forest (RF), 3) gradient boosting machine (GBM), 4) support vector machine (SVM), and 5) convolutional neural networks (CNNs). Model performance was evaluated using the area under the receiver operator characteristic curve (AUROC). The sample was randomly divided into a training/testing split with 80%/20% probabilities. Repeated 10-fold cross-validation identified the optimal model hyperparameters in the training dataset for each model, which were then applied to the entire training dataset to train the model. Trained models were applied to the test cohort to evaluate model performance. Statistical significance was evaluated using P < .05.

RESULTS

The training and testing cohorts contained 4322 and 1079 patients, respectively, with 62 (1.4%) and 15 (1.4%) experiencing 30-day mortality, respectively. When using only baseline variables to predict mortality, all algorithms except SVM (area under the receiver operator characteristic curve [AUROC], 0.83 [95% confidence interval {CI}, 0.69-0.97]) had AUROC >0.9: GBM (AUROC, 0.96 [0.94-1.0]), RF (AUROC, 0.96 [0.92-1.0]), CNN (AUROC, 0.96 [0.92-0.99]), and logistic regression (AUROC, 0.95 [0.91-0.99]). AUROC significantly increased with intraoperative variables with CNN (AUROC, 0.97 [0.96-0.99]; P = .047 versus baseline), but there was no improvement with GBM (AUROC, 0.97 [0.95-0.99]; P = .3 versus baseline), RF (AUROC, 0.96 [0.93-1.0]; P = .5 versus baseline), and logistic regression (AUROC, 0.94 [0.90-0.99]; P = .6 versus baseline).

CONCLUSIONS

Postoperative mortality is predicted with excellent discrimination in intra-abdominal surgery patients using only preoperative variables in various machine learning algorithms. The addition of intraoperative data to preoperative data also resulted in models with excellent discrimination, but model performance did not improve.

Physiologic signatures within six hours of hospitalization identify acute illness phenotypes

During the early stages of hospital admission, clinicians use limited information to make decisions as patient acuity evolves. We hypothesized that clustering analysis of vital signs measured within six hours of hospital admission would reveal distinct patient phenotypes with unique pathophysiological signatures and clinical outcomes. We created a longitudinal electronic health record dataset for 75,762 adult patient admissions to a tertiary care center in 2014-2016 lasting six hours or longer. Physiotypes were derived via unsupervised machine learning in a training cohort of 41,502 patients applying consensus k-means clustering to six vital signs measured within six hours of admission. Reproducibility and correlation with clinical biomarkers and outcomes were assessed in validation cohort of 17,415 patients and testing cohort of 16,845 patients. Training, validation, and testing cohorts had similar age (54-55 years) and sex (55% female), distributions. There were four distinct clusters. Physiotype A had physiologic signals consistent with early vasoplegia, hypothermia, and low-grade inflammation and favorable short-and long-term clinical outcomes despite early, severe illness. Physiotype B exhibited early tachycardia, tachypnea, and hypoxemia followed by the highest incidence of prolonged respiratory insufficiency, sepsis, acute kidney injury, and short- and long-term mortality. Physiotype C had minimal early physiological derangement and favorable clinical outcomes. Physiotype D had the greatest prevalence of chronic cardiovascular and kidney disease, presented with severely elevated blood pressure, and had good short-term outcomes but suffered increased 3-year mortality. Comparing sequential organ failure assessment (SOFA) scores across physiotypes demonstrated that clustering did not simply recapitulate previously established acuity assessments. In a heterogeneous cohort of hospitalized patients, unsupervised machine learning techniques applied to routine, early vital sign data identified physiotypes with unique disease categories and distinct clinical outcomes. This approach has the potential to augment understanding of pathophysiology by distilling thousands of disease states into a few physiological signatures.

Potential benefits, unintended consequences, and future roles of artificial intelligence in orthopaedic surgery research : a call to emphasize data quality and indications

Artificial intelligence and machine-learning analytics have gained extensive popularity in recent years due to their clinically relevant applications. A wide range of proof-of-concept studies have demonstrated the ability of these analyses to personalize risk prediction, detect implant specifics from imaging, and monitor and assess patient movement and recovery. Though these applications are exciting and could potentially influence practice, it is imperative to understand when these analyses are indicated and where the data are derived from, prior to investing resources and confidence into the results and conclusions. In this article, we review the current benefits and potential limitations of machine-learning for the orthopaedic surgeon with a specific emphasis on data quality.

Development and Internal Validation of a Model for Predicting Internet Gaming Disorder Risk in Adolescents and Children

BACKGROUND

The high prevalence of Internet gaming disorder among children and adolescents and its severe psychological, health, and social consequences have become a public emergency. A high efficiency and cost-effective early recognition method are urgently needed.

OBJECTIVE

We aim to develop and internally validate a nomogram model for predicting Internet gaming disorder (IGD) risk in Chinese adolescents and children.

METHODS

Through an online survey, 780 children and adolescents aged 7-18 years who participated in the survey from June to August 2021 were selected. The least absolute shrinkage and selection operator regression model was used to filter the factors. Multivariate logistic regression analysis was used to establish the prediction model and generate nomograms and a website calculator. The area under the receiver operating characteristic curve, calibration plot, and decision curve analysis were used to evaluate the model's discrimination, calibration, and clinical utility. Bootstrapping validation was used to verify the model internally.

RESULTS

Male sex and experience of game consumption were the two most important predictors. Both models exhibited good discrimination, with an area under the curve >0.80. The calibration plots were both close to the diagonal line (45°). Decision curve analyses revealed that two nomograms were clinically useful when the threshold probability for the intervention was set to 5-75%.

CONCLUSION

Two prediction models appear to be reliable tools for Internet gaming disorder screening in children and adolescents, which can also help clinicians to personalize treatment plans. Moreover, from the standpoint of simplification and cost, Model 2 appears to be a better alternative.

Clinical prediction system of complications among patients with COVID-19: A development and validation retrospective multicentre study during first wave of the pandemic

Clinical evidence suggests that some patients diagnosed with coronavirus disease 2019 (COVID-19) experience a variety of complications associated with significant morbidity, especially in severe cases during the initial spread of the pandemic. To support early interventions, we propose a machine learning system that predicts the risk of developing multiple complications. We processed data collected from 3,352 patient encounters admitted to 18 facilities between April 1 and April 30, 2020, in Abu Dhabi (AD), United Arab Emirates. Using data collected during the first 24 h of admission, we trained machine learning models to predict the risk of developing any of three complications after 24 h of admission. The complications include Secondary Bacterial Infection (SBI), Acute Kidney Injury (AKI), and Acute Respiratory Distress Syndrome (ARDS). The hospitals were grouped based on geographical proximity to assess the proposed system's learning generalizability, AD Middle region and AD Western & Eastern regions, A and B, respectively. The overall system includes a data filtering criterion, hyperparameter tuning, and model selection. In test set A, consisting of 587 patient encounters (mean age: 45.5), the system achieved a good area under the receiver operating curve (AUROC) for the prediction of SBI (0.902 AUROC), AKI (0.906 AUROC), and ARDS (0.854 AUROC). Similarly, in test set B, consisting of 225 patient encounters (mean age: 42.7), the system performed well for the prediction of SBI (0.859 AUROC), AKI (0.891 AUROC), and ARDS (0.827 AUROC). The performance results and feature importance analysis highlight the system's generalizability and interpretability. The findings illustrate how machine learning models can achieve a strong performance even when using a limited set of routine input variables. Since our proposed system is data-driven, we believe it can be easily repurposed for different outcomes considering the changes in COVID-19 variants over time.

Using Iterative Pairwise External Validation to Contextualize Prediction Model Performance: A Use Case Predicting 1-Year Heart Failure Risk in Patients with Diabetes Across Five Data Sources

INTRODUCTION

External validation of prediction models is increasingly being seen as a minimum requirement for acceptance in clinical practice. However, the lack of interoperability of healthcare databases has been the biggest barrier to this occurring on a large scale. Recent improvements in database interoperability enable a standardized analytical framework for model development and external validation. External validation of a model in a new database lacks context, whereby the external validation can be compared with a benchmark in this database. Iterative pairwise external validation (IPEV) is a framework that uses a rotating model development and validation approach to contextualize the assessment of performance across a network of databases. As a use case, we predicted 1-year risk of heart failure in patients with type 2 diabetes mellitus.

METHODS

The method follows a two-step process involving (1) development of baseline and data-driven models in each database according to best practices and (2) validation of these models across the remaining databases. We introduce a heatmap visualization that supports the assessment of the internal and external model performance in all available databases. As a use case, we developed and validated models to predict 1-year risk of heart failure in patients initializing a second pharmacological intervention for type 2 diabetes mellitus. We leveraged the power of the Observational Medical Outcomes Partnership common data model to create an open-source software package to increase the consistency, speed, and transparency of this process.

RESULTS

A total of 403,187 patients from five databases were included in the study. We developed five models that, when assessed internally, had a discriminative performance ranging from 0.73 to 0.81 area under the receiver operating characteristic curve with acceptable calibration. When we externally validated these models in a new database, three models achieved consistent performance and in context often performed similarly to models developed in the database itself. The visualization of IPEV provided valuable insights. From this, we identified the model developed in the Commercial Claims and Encounters (CCAE) database as the best performing model overall.

CONCLUSION

Using IPEV lends weight to the model development process. The rotation of development through multiple databases provides context to model assessment, leading to improved understanding of transportability and generalizability. The inclusion of a baseline model in all modelling steps provides further context to the performance gains of increasing model complexity. The CCAE model was identified as a candidate for clinical use. The use case demonstrates that IPEV provides a huge opportunity in a new era of standardised data and analytics to improve insight into and trust in prediction models at an unprecedented scale.

Development and optimisation of a machine-learning prediction model for acute desquamation following breast radiotherapy in the multi-centre REQUITE cohort

Purpose

Some patients with breast cancer treated by surgery and radiation therapy experience clinically significant toxicity, which may adversely affect cosmesis and quality of life. There is a paucity of validated clinical prediction models for radiation toxicity. We used machine learning (ML) algorithms to develop and optimise a clinical prediction model for acute breast desquamation after whole breast external beam radiation therapy in the prospective multicenter REQUITE cohort study.

Methods and Materials

Using demographic and treatment-related features (m = 122) from patients (n = 2058) at 26 centers, we trained 8 ML algorithms with 10-fold cross-validation in a 50:50 random-split data set with class stratification to predict acute breast desquamation. Based on performance in the validation data set, the logistic model tree, random forest, and naïve Bayes models were taken forward to cost-sensitive learning optimisation.

Results

One hundred and ninety-two patients experienced acute desquamation. Resampling and cost-sensitive learning optimisation facilitated an improvement in classification performance. Based on maximising sensitivity (true positives), the “hero” model was the cost-sensitive random forest algorithm with a false-negative: false-positive misclassification penalty of 90:1 containing m = 114 predictive features. Model sensitivity and specificity were 0.77 and 0.66, respectively, with an area under the curve of 0.77 in the validation cohort.

Conclusions

ML algorithms with resampling and cost-sensitive learning generated clinically valid prediction models for acute desquamation using patient demographic and treatment features. Further external validation and inclusion of genomic markers in ML prediction models are worthwhile, to identify patients at increased risk of toxicity who may benefit from supportive intervention or even a change in treatment plan.

Application and research progress of machine learning in the diagnosis and treatment of neurodevelopmental disorders in children

The prevalence of neurodevelopment disorders (NDDs) among children has been on the rise. This has affected the health and social life of children. This condition has also imposed a huge economic burden on families and health care systems. Currently, it is difficult to perform early diagnosis of NDDs, which results in delayed intervention. For this reason, patients with NDDs have a prognosis. In recent years, machine learning (ML) technology, which integrates artificial intelligence technology and medicine, has been applied in the early detection and prediction of diseases based on data mining. This paper reviews the progress made in the application of ML in the diagnosis and treatment of NDDs in children based on supervised and unsupervised learning tools. The data reviewed here provide new perspectives on early diagnosis and treatment of NDDs.

Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble

Driven by recent innovations and technological progress, the increasing quality and amount of biomedical data coupled with the advances in computing power allowed for much progress in artificial intelligence (AI) approaches for health and biomedical research. In interventional cardiology, the hope is for AI to provide automated analysis and deeper interpretation of data from electrocardiography, computed tomography, magnetic resonance imaging, and electronic health records, among others. Furthermore, high-performance predictive models supporting decision-making hold the potential to improve safety, diagnostic and prognostic prediction in patients undergoing interventional cardiology procedures. These applications include robotic-assisted percutaneous coronary intervention procedures and automatic assessment of coronary stenosis during diagnostic coronary angiograms. Machine learning (ML) has been used in these innovations that have improved the field of interventional cardiology, and more recently, deep Learning (DL) has emerged as one of the most successful branches of ML in many applications. It remains to be seen if DL approaches will have a major impact on current and future practice. DL-based predictive systems also have several limitations, including lack of interpretability and lack of generalizability due to cohort heterogeneity and low sample sizes. There are also challenges for the clinical implementation of these systems, such as ethical limits and data privacy. This review is intended to bring the attention of health practitioners and interventional cardiologists to the broad and helpful applications of ML and DL algorithms to date in the field. Their implementation challenges in daily practice and future applications in the field of interventional cardiology are also discussed.

Development and Validation of a Simplified Prehospital Triage Model Using Neural Network to Predict Mortality in Trauma Patients: The Ability to Follow Commands, Age, Pulse Rate, Systolic Blood Pressure and Peripheral Oxygen Saturation (CAPSO) Model

Objective: Most trauma scoring systems with high accuracy are difficult to use quickly in field triage, especially in the case of mass casualty events. We aimed to develop a machine learning model for trauma mortality prediction using variables easy to obtain in the prehospital setting.

Methods: This was a retrospective prognostic study using the National Trauma Data Bank (NTDB). Data from 2013 to 2016 were used for model training and internal testing, and data from 2017 were used for validation. A neural network model (NN-CAPSO) was developed using the ability to follow commands (whether GCS-motor was <6), age, pulse rate, systolic blood pressure (SBP) and peripheral oxygen saturation, and a new score (the CAPSO score) was developed based on logistic regression. To achieve further simplification, a neural network model with the SBP variable removed (NN-CAPO) was also developed. The discrimination ability of different models and scores was compared based on the area under the receiver operating characteristic curve (AUROC). Furthermore, a reclassification table with three defined risk groups was used to compare NN-CAPSO and other models or scores.

Results: The NN-CAPSO had an AUROC of 0.911(95% confidence interval 0.909 to 0.913) in the validation set, which was higher than the other trauma scores available for prehospital settings (all p < 0.001). The NN-CAPO and CAPSO score both reached the AUROC of 0.904 (95% confidence interval 0.902 to 0.906), and were no worse than other prehospital trauma scores. Compared with the NN-CAPO, CAPSO score, and the other trauma scores in reclassification tables, NN-CAPSO was found to more accurately classify patients to the right risk groups.

Conclusions: The newly developed CAPSO system simplifies the method of consciousness assessment and has the potential to accurately predict trauma patient mortality in the prehospital setting.

Inclusion of interleukin-6 improved performance of postoperative delirium prediction for patients undergoing coronary artery bypass graft (POD-CABG): A derivation and validation study

BACKGROUND

Patients undergoing coronary artery bypass graft (CABG) are at high risk for developing postoperative delirium (POD). A simple prediction rule may benefit patients from early identification of POD followed by adequate preventive strategies. The purpose of the current study was to develop and validate a POD prediction rule for patients undergoing CABG (POD-CABG), by considering all possible perioperative factors.

METHODS

In this prospective cohort study, patients who underwent first elective isolated CABG were continuously enrolled from May 2014 to November 2015 in a tertiary hospital. Delirium was assessed using the Confusion Assessment Method for Intensive Care Unit. Patients' perioperative risk factors were collected through interviews and review of medical records. The area under receiver-operating characteristic curve (AUC) was used to assess the overall performance of the predictive rule.

RESULTS

A total of 242 and 148 patients were enrolled in the derivation and validation cohorts, respectively. Multiple logistic regression analysis identified seven variables that were independently associated with POD: age (≥65 years), gender (female), history of myocardial infarction and diabetes mellitus, postoperative atrial fibrillation, the use of intra-aortic balloon pump, and serum interleukin-6 ≥478 pg/ml at 18 hours after surgery. The AUC of the POD-CABG was 0.84 (95% CI, 0.79-0.90) in the derivation cohort, and was 0.86 (95% CI, 0.80-0.91) after bootstrap resampling. The AUC was 0.81 (95% CI, 0.73-0.88) after the POD-CABG was applied to the validation cohort.

CONCLUSIONS

The POD-CABG with inclusion of interleukin-6 demonstrated good performance in predicting POD.

Understanding Anterior Shoulder Instability Through Machine Learning: New Models That Predict Recurrence, Progression to Surgery, and Development of Arthritis

Background

Management of anterior shoulder instability (ASI) aims to reduce risk of future recurrence and prevent complications via nonoperative and surgical management. Machine learning may be able to reliably provide predictions to improve decision making for this condition.

Purpose

To develop and internally validate a machine-learning model to predict the following outcomes after ASI: (1) recurrent instability, (2) progression to surgery, and (3) the development of symptomatic osteoarthritis (OA) over long-term follow-up.

Study Design

Cohort study (prognosis); Level of evidence, 2.

Methods

An established geographic database of >500,000 patients was used to identify 654 patients aged <40 years with an initial diagnosis of ASI between 1994 and 2016; the mean follow-up was 11.1 years. Medical records were reviewed to obtain patient information, and models were generated to predict the outcomes of interest. Five candidate algorithms were trained in the development of each of the models, as well as an additional ensemble of the algorithms. Performance of the algorithms was assessed using discrimination, calibration, and decision curve analysis.

Results

Of the 654 included patients, 443 (67.7%) experienced multiple instability events, 228 (34.9%) underwent surgery, and 39 (5.9%) developed symptomatic OA. The ensemble gradient-boosted machines achieved the best performances based on discrimination (via area under the receiver operating characteristic curve [AUC]: AUC_recurrence = 0.86), AUC_surgery = 0.76, AUC_OA = 0.78), calibration, decision curve analysis, and Brier score (Brier_recurrence = 0.138, Brier_surgery = 0.185, Brier_OA = 0.05). For demonstration purposes, models were integrated into a single web-based open-access application able to provide predictions and explanations for practitioners and researchers.

Conclusion

After identification of key features, including time from initial instability, age at initial instability, sports involvement, and radiographic findings, machine-learning models were developed that effectively and reliably predicted recurrent instability, progression to surgery, and the development of OA in patients with ASI. After careful external validation, these models can be incorporated into open-access digital applications to inform patients, clinicians, and researchers regarding quantifiable risks of relevant outcomes in the clinic.

Key Aspects of Prognostic Model Development and Interpretation From a Clinical Perspective

Importance

Prognostication is an important aspect of clinical decision-making, but it is often challenging. Previous studies show that both patients and physicians tend to overestimate survival chances. Prediction models may assist in estimating and quantifying prognosis. However, insufficient understanding of the development, possibilities, and limitations of such models can lead to misinterpretations. Although many excellent books and comprehensive methodological articles on prognostic model research are published, they may not be accessible enough for the clinical audience. Our aim is to provide an overview on the main issues regarding prediction research for health care professionals to achieve better interpretation and increase the use of prognostic models in daily clinical practice.

Observations

The first steps of model development include coding of predictors, model specification, and estimation. Next, we discuss the assessment of the performance of a prediction model, including discrimination and calibration aspects, followed by approaches to internal and external validation and updating. Finally, model reporting, presentation, and steps toward clinical implementation are presented.

Conclusions and Relevance

After thorough consideration of the research question, data inspection, and coding of predictors, one can start with the specification of a prediction model. The number of candidate predictors should be kept limited, in view of the number of events in the data, to prevent overfitting. Calibration and discrimination are 2 aspects of model performance that complement each other and should be assessed preferably at external validation. Model development should be accompanied by qualitative research among patients and physicians to facilitate the development of a valuable tool and maximize possibilities for successful implementation. After model presentation is optimized, impact studies are required to assess the clinical value of a prediction model.

Automatic Classification Between COVID-19 and Non-COVID-19 Pneumonia Using Symptoms, Comorbidities, and Laboratory Findings: The Khorshid COVID Cohort Study

Coronavirus disease-2019, also known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was a disaster in 2020. Accurate and early diagnosis of coronavirus disease-2019 (COVID-19) is still essential for health policymaking. Reverse transcriptase-polymerase chain reaction (RT-PCR) has been performed as the operational gold standard for COVID-19 diagnosis. We aimed to design and implement a reliable COVID-19 diagnosis method to provide the risk of infection using demographics, symptoms and signs, blood markers, and family history of diseases to have excellent agreement with the results obtained by the RT-PCR and CT-scan. Our study primarily used sample data from a 1-year hospital-based prospective COVID-19 open-cohort, the Khorshid COVID Cohort (KCC) study. A sample of 634 patients with COVID-19 and 118 patients with pneumonia with similar characteristics whose RT-PCR and chest CT scan were negative (as the control group) (dataset 1) was used to design the system and for internal validation. Two other online datasets, namely, some symptoms (dataset 2) and blood tests (dataset 3), were also analyzed. A combination of one-hot encoding, stability feature selection, over-sampling, and an ensemble classifier was used. Ten-fold stratified cross-validation was performed. In addition to gender and symptom duration, signs and symptoms, blood biomarkers, and comorbidities were selected. Performance indices of the cross-validated confusion matrix for dataset 1 were as follows: sensitivity of 96% [confidence interval, CI, 95%: 94–98], specificity of 95% [90–99], positive predictive value (PPV) of 99% [98–100], negative predictive value (NPV) of 82% [76–89], diagnostic odds ratio (DOR) of 496 [198–1,245], area under the ROC (AUC) of 0.96 [0.94–0.97], Matthews Correlation Coefficient (MCC) of 0.87 [0.85–0.88], accuracy of 96% [94–98], and Cohen's Kappa of 0.86 [0.81–0.91]. The proposed algorithm showed excellent diagnosis accuracy and class-labeling agreement, and fair discriminant power. The AUC on the datasets 2 and 3 was 0.97 [0.96–0.98] and 0.92 [0.91–0.94], respectively. The most important feature was white blood cell count, shortness of breath, and C-reactive protein for datasets 1, 2, and 3, respectively. The proposed algorithm is, thus, a promising COVID-19 diagnosis method, which could be an amendment to simple blood tests and screening of symptoms. However, the RT-PCR and chest CT-scan, performed as the gold standard, are not 100% accurate.

Ancestry may confound genetic machine learning: Candidate-gene prediction of opioid use disorder as an example

BACKGROUND

Machine learning (ML) models are beginning to proliferate in psychiatry, however machine learning models in psychiatric genetics have not always accounted for ancestry. Using an empirical example of a proposed genetic test for OUD, and exploring a similar test for tobacco dependence and a simulated binary phenotype, we show that genetic prediction using ML is vulnerable to ancestral confounding.

METHODS

We utilize five ML algorithms trained with 16 brain reward-derived "candidate" SNPs proposed for commercial use and examine their ability to predict OUD vs. ancestry in an out-of-sample test set (N = 1000, stratified into equal groups of n = 250 cases and controls each of European and African ancestry). We rerun analyses with 8 random sets of allele-frequency matched SNPs. We contrast findings with 11 genome-wide significant variants for tobacco smoking. To document generalizability, we generate and test a random phenotype.

RESULTS

None of the 5 ML algorithms predict OUD better than chance when ancestry was balanced but were confounded with ancestry in an out-of-sample test. In addition, the algorithms preferentially predicted admixed subpopulations. Random sets of variants matched to the candidate SNPs by allele frequency produced similar bias. Genome-wide significant tobacco smoking variants were also confounded by ancestry. Finally, random SNPs predicting a random simulated phenotype show that the bias attributable to ancestral confounding could impact any ML-based genetic prediction.

CONCLUSIONS

Researchers and clinicians are encouraged to be skeptical of claims of high prediction accuracy from ML-derived genetic algorithms for polygenic traits like addiction, particularly when using candidate variants.

Artificial Intelligence Predicts Cost After Ambulatory Anterior Cruciate Ligament Reconstruction

Purpose

To develop and internally validate a machine-learning algorithm to reliably predict cost after anterior cruciate ligament reconstruction (ACLR).

Methods

A retrospective review of the New York State Ambulatory Surgery and Services database was performed to identify patients who underwent elective ACLR from 2015 to 2016. Features included in initial models consisted of patient characteristics (age, sex, insurance status, income, medical comorbidities as classified by the Clinical Classifications Software diagnosis code) as well as intraoperative variables (type of anesthesia and procedure-specific factors). Models were generated to predict total charges using 4 algorithms: random forest, extreme gradient boost, elastic net penalized regression, and support vector machines with radial kernels. Training was performed with 10-fold cross-validation followed by internal validation via 0.632 bootstrapping. Model discriminative performance was assessed by area under the receiver operating characteristic curve, calibration, and the Brier score. Decision curve analysis was performed to demonstrate the net benefit of using the final model in practice.

Results

In total, 7,311 patients undergoing ambulatory ACLR were included. The random forest model demonstrated the best performance assessed via internal validation (area under the curve = 0.85), calibration, and the Brier score (0.208). Cost incurred was influenced by anesthesia type, operating room time, and number of chronic comorbidities. Decision curve analysis revealed a net benefit for use of the random forest model and the model was integrated into a web-based open-access application.

Conclusions

The random forest model predicted cost after ambulatory ACLR using a large, statewide database with good performance. The top variables found to predict increased charges were general anesthesia, operating room time, meniscal repair, self-pay insurance, patient neighborhood characteristics, and number of chronic conditions.

Level of Evidence

III, retrospective cohort study.

Development of a novel nomogram-based online tool to predict axillary status after neoadjuvant chemotherapy in cN+ breast cancer: A multicentre study on 1,950 patients

BACKGROUND

Type of axillary surgery in breast cancer (BC) patients who convert from cN + to ycN0 after neoadjuvant chemotherapy (NAC) is still debated. The aim of the present study was to develop and validate a preoperative predictive nomogram to select those patients with a low risk of residual axillary disease after NAC, in whom axillary surgery could be minimized.

PATIENTS AND METHODS

1950 clinically node-positive BC patients from 11 Breast Units, treated by NAC and subsequent surgery, were included from 2005 to 2020. Patients were divided in two groups: those who achieved nodal pCR vs. those with residual nodal disease after NAC. The cohort was divided into training and validation set with a geographic separation criterion. The outcome was to identify independent predictors of axillary pathologic complete response (pCR).

RESULTS

Independent predictive factors associated to nodal pCR were axillary clinical complete response (cCR) after NAC (OR 3.11, p < 0.0001), ER-/HER2+ (OR 3.26, p < 0.0001) or ER+/HER2+ (OR 2.26, p = 0.0002) or ER-/HER2- (OR 1.89, p = 0.009) BC, breast cCR (OR 2.48, p < 0.0001), Ki67 > 14% (OR 0.52, p = 0.0005), and tumor grading G2 (OR 0.35, p = 0.002) or G3 (OR 0.29, p = 0.0003). The nomogram showed a sensitivity of 71% and a specificity of 73% (AUC 0.77, 95%CI 0.75-0.80). After external validation the accuracy of the nomogram was confirmed.

CONCLUSION

The accuracy makes this freely-available, nomogram-based online tool useful to predict nodal pCR after NAC, translating the concept of tailored axillary surgery also in this setting of patients.

Central nervous system infection in the intensive care unit: Development and validation of a multi-parameter diagnostic prediction tool to identify suspected patients

Background

Central nervous system infections (CNSI) are diseases with high morbidity and mortality, and their diagnosis in the intensive care environment can be challenging. Objective: To develop and validate a diagnostic model to quickly screen intensive care patients with suspected CNSI using readily available clinical data.

Methods

Derivation cohort: 783 patients admitted to an infectious diseases intensive care unit (ICU) in Oswaldo Cruz Foundation, Rio de Janeiro RJ, Brazil, for any reason, between 01/01/2012 and 06/30/2019, with a prevalence of 97 (12.4%) CNSI cases. Validation cohort 1: 163 patients prospectively collected, between 07/01/2019 and 07/01/2020, from the same ICU, with 15 (9.2%) CNSI cases. Validation cohort 2: 7,270 patients with 88 CNSI (1.21%) admitted to a neuro ICU in Chicago, IL, USA between 01/01/2014 and 06/30/2019. Prediction model: Multivariate logistic regression analysis was performed to construct the model, and Receiver Operating Characteristic (ROC) curve analysis was used for model validation. Eight predictors—age <56 years old, cerebrospinal fluid white blood cell count >2 cells/mm³, fever (≥38°C/100.4°F), focal neurologic deficit, Glasgow Coma Scale <14 points, AIDS/HIV, and seizure—were included in the development diagnostic model (P<0.05).

Results

The pool data’s model had an Area Under the Receiver Operating Characteristics (AUC) curve of 0.892 (95% confidence interval 0.864–0.921, P<0.0001).

Conclusions

A promising and straightforward screening tool for central nervous system infections, with few and readily available clinical variables, was developed and had good accuracy, with internal and external validity.

Growth prediction model for abdominal aortic aneurysms

BACKGROUND

The most relevant determinant in scheduling monitoring intervals for abdominal aortic aneurysms (AAAs) is maximum diameter. The aim of the study was to develop a statistical model that takes into account specific characteristics of AAA growth distributions such as between-patient variability as well as within-patient variability across time, and allows probabilistic statements to be made regarding expected AAA growth.

METHODS

CT angiography (CTA) data from patients monitored at 6-month intervals with maximum AAA diameters at baseline between 30 and 66 mm were used to develop the model. By extending the model of geometric Brownian motion with a log-normal random effect, a stochastic growth model was developed. An additional set of ultrasound-based growth data was used for external validation.

RESULTS

The study data included 363 CTAs from 87 patients, and the external validation set comprised 390 patients. Internal and external cross-validation showed that the stochastic growth model allowed accurate description of the distribution of aneurysm growth. Median relative growth within 1 year was 4.1 (5-95 per cent quantile 0.5-13.3) per cent. Model calculations further resulted in relative 1-year growth of 7.0 (1.0-16.4) per cent for patients with previously observed rapid 1-year growth of 10 per cent, and 2.6 (0.3-8.3) per cent for those with previously observed slow growth of 1 per cent. The probability of exceeding a threshold of 55 mm was calculated to be 1.78 per cent at most when adhering to the current RESCAN guidelines for rescreening intervals. An online calculator based on the fitted model was made available.

CONCLUSION

The stochastic growth model was found to provide a reliable tool for predicting AAA growth.

Lung Ultrasound, Clinical and Analytic Scoring Systems as Prognostic Tools in SARS-CoV-2 Pneumonia: A Validating Cohort

At the moment, several COVID-19 scoring systems have been developed. It is necessary to determine which one better predicts a poor outcome of the disease. We conducted a single-center prospective cohort study to validate four COVID-19 prognosis scores in adult patients with confirmed infection at ward. These are National Early Warning Score (NEWS) 2, Lung Ultrasound Score (LUS), COVID-19 Worsening Score (COWS), and Spanish Society of Infectious Diseases and Clinical Microbiology score (SEIMC Score). Our outcomes were the combined variable “poor outcome” (non-invasive mechanical ventilation, intubation, intensive care unit admission, and death at 28 days) and death at 28 days. Scores were analysed using univariate logistic regression models, receiver operating characteristic curves, and areas under the curve. Eighty-one patients were included, from which 21 had a poor outcome, and 9 died. We found a statistically significant correlation between poor outcome and NEWS2, LUS > 15, and COWS. Death at 28 days was statistically correlated with NEWS2 and SEIMC Score although COWS also performs well. NEWS2, LUS, and COWS accurately predict poor outcome; and NEWS2, SEIMC Score, and COWS are useful for anticipating death at 28 days. Lung ultrasound is a diagnostic tool that should be included in COVID-19 patients evaluation.

Development and validation of a prognostic tool: Pulmonary embolism short-term clinical outcomes risk estimation (PE-SCORE)

OBJECTIVE

Develop and validate a prognostic model for clinical deterioration or death within days of pulmonary embolism (PE) diagnosis using point-of-care criteria.

METHODS

We used prospective registry data from six emergency departments. The primary composite outcome was death or deterioration (respiratory failure, cardiac arrest, new dysrhythmia, sustained hypotension, and rescue reperfusion intervention) within 5 days. Candidate predictors included laboratory and imaging right ventricle (RV) assessments. The prognostic model was developed from 935 PE patients. Univariable analysis of 138 candidate variables was followed by penalized and standard logistic regression on 26 retained variables, and then tested with a validation database (N = 801).

RESULTS

Logistic regression yielded a nine-variable model, then simplified to a nine-point tool (PE-SCORE): one point each for abnormal RV by echocardiography, abnormal RV by computed tomography, systolic blood pressure < 100 mmHg, dysrhythmia, suspected/confirmed systemic infection, syncope, medico-social admission reason, abnormal heart rate, and two points for creatinine greater than 2.0 mg/dL. In the development database, 22.4% had the primary outcome. Prognostic accuracy of logistic regression model versus PE-SCORE model: 0.83 (0.80, 0.86) vs. 0.78 (0.75, 0.82) using area under the curve (AUC) and 0.61 (0.57, 0.64) vs. 0.50 (0.39, 0.60) using precision-recall curve (AUCpr). In the validation database, 26.6% had the primary outcome. PE-SCORE had AUC 0.77 (0.73, 0.81) and AUCpr 0.63 (0.43, 0.81). As points increased, outcome proportions increased: a score of zero had 2% outcome, whereas scores of six and above had ≥ 69.6% outcomes. In the validation dataset, PE-SCORE zero had 8% outcome [no deaths], whereas all patients with PE-SCORE of six and above had the primary outcome.

CONCLUSIONS

PE-SCORE model identifies PE patients at low- and high-risk for deterioration and may help guide decisions about early outpatient management versus need for hospital-based monitoring.

Individualized risk trajectories for iron-related adverse outcomes in repeat blood donors

BACKGROUND

Despite a fingerstick hemoglobin requirement and 56-day minimum donation interval, repeat blood donation continues to cause and exacerbate iron deficiency.

STUDY DESIGN AND METHODS

Using data from the REDS-II Donor Iron Status Evaluation study, we developed multiclass prediction models to estimate the competing risk of hemoglobin deferral and collecting blood from a donor with sufficient hemoglobin but low or absent underlying iron stores. We compared models developed with and without two biomarkers not routinely measured in most blood centers: ferritin and soluble transferrin receptor. We generated and analyzed "individual risk trajectories": estimates of how each donors' risk developed as a function of the time interval until their next donation attempt.

RESULTS

With standard biomarkers, the top model had a multiclass area under the receiver operator characteristic curve (AUC) of 77.6% (95% CI [77.3%-77.8%]). With extra biomarkers, multiclass AUC increased to 82.8% (95% CI [82.5%-83.1%]). In the extra biomarkers model, ferritin was the single most important variable, followed by the donation interval. We identified three risk archetypes: "fast recoverers" (<10% risk of any adverse outcome on post-donation day 56), "slow recoverers" (>60% adverse outcome risk on day 56 that declines to <35% by day 250), and "chronic high-risk" (>85% risk of the adverse outcome on day 250).

DISCUSSION

A longer donation interval reduced the estimated risk of iron-related adverse outcomesfor most donors, but risk remained high for some. Tailoring safeguards to individual risk estimates could reduce blood collections from donors with low or absent iron stores.

Prediction of recovery in trauma patients using Latent Markov models

PURPOSE

Patients' expectations during recovery after a trauma can affect the recovery. The aim of the present study was to identify different physical recovery trajectories based on Latent Markov Models (LMMs) and predict these recovery states based on individual patient characteristics.

METHODS

The data of a cohort of adult trauma patients until the age of 75 years with a length of hospital stay of 3 days and more were derived from the Brabant Injury Outcome Surveillance (BIOS) study. The EuroQol-5D 3-level version and the Health Utilities Index were used 1 week, and 1, 3, 6, 12, and 24 months after injury. Four prediction models, for mobility, pain, self-care, and daily activity, were developed using LMMs with ordinal latent states and patient characteristics as predictors for the latent states.

RESULTS

In total, 1107 patients were included. Four models with three ordinal latent states were developed, with different covariates in each model. The prediction of the (ordinal) latent states in the LMMs yielded pseudo-R² values between 40 and 53% and between 21 and 41% (depending of the type R² used) and classification errors between 24 and 40%. Most patients seem to recover fast as only about a quarter of the patients remain with severe problems after 1 month.

CONCLUSION

The use of LMMs to model the development of physical function post-injury is a promising way to obtain a prediction of the physical recovery. The step-by-step prediction fits well with the outpatient follow-up and it can be used to inform the patients more tailor-made to manage the expectations.

Clinical Scores in Veterinary Medicine: What Are the Pitfalls of Score Construction, Reliability, and Validation? A General Methodological Approach Applied in Cattle

Simple Summary

Clinical scores are practical tools that can be used in the daily management of cattle. Score building and validation are a challenge involving various methodological and statistical issues. This article provides a specific framework for clinical score building where the target condition can be assessed directly or indirectly. Practical examples are given throughout the manuscript in order to build new scores or to assess score robustness.

Abstract

Clinical scores are commonly used for cattle. They generally contain a mix of categorical and numerical variables that need to be assessed by scorers, such as farmers, animal caretakers, scientists, and veterinarians. This article examines the key concepts that need to be accounted for when developing the test for optimal outcomes. First, the target condition or construct that the scale is supposed to measure should be defined, and if possible, an adequate proxy used for classification should be determined. Then, items (e.g., clinical signs) of interest that are either caused by the target condition (reflective items) or that caused the target condition (formative items) are listed, and reliable items (inter and intra-rater reliability) are kept for the next step. A model is then developed to determine the relative weight of the items associated with the target condition. A scale is then built after validating the model and determining the optimal threshold in terms of sensitivity (ability to detect the target condition) and specificity (ability to detect the absence of the target condition). Its robustness to various scenarios of the target condition prevalence and the impact of the relative cost of false negatives to false positives can also be assessed to tailor the scale used based on specific application conditions.

Early postoperative arterial lactate concentrations to stratify risk of post-hepatectomy liver failure

BACKGROUND

Post-hepatectomy liver failure (PHLF) represents the major determinant for death after liver resection. Early recognition is essential. Perioperative lactate dynamics for risk assessment of PHLF and associated morbidity were evaluated.

METHODS

This was a multicentre observational study of patients undergoing hepatectomy with validation in international high-volume units. Receiver operating characteristics analysis and cut-off calculation for the predictive value of lactate for clinically relevant International Study Group of Liver Surgery grade B/C PHLF (clinically relevant PHLF (CR-PHLF)) were performed. Lactate and other perioperative factors were assessed in a multivariable CR-PHLF regression model.

RESULTS

The exploratory cohort comprised 509 patients. CR-PHLF, death, overall morbidity and severe morbidity occurred in 7.7, 3.3, 40.9 and 29.3 per cent of patients respectively. The areas under the curve (AUCs) regarding CR-PHLF were 0.829 (95 per cent c.i. 0.770 to 0.888) for maximum lactate within 24 h (Lactate_Max) and 0.870 (95 per cent c.i. 0.818 to 0.922) for postoperative day 1 levels (Lactate_POD1). The respective AUCs in the validation cohort (482 patients) were 0.812 and 0.751 and optimal Lactate_Max cut-offs were identical in both cohorts. Exploration cohort patients with Lactate_Max 50 mg/dl or greater more often developed CR-PHLF (50.0 per cent) than those with Lactate_Max between 20 and 49.9 mg/dl (7.4 per cent) or less than 20 mg/dl (0.5 per cent; P < 0.001). This also applied to death (18.4, 2.7 and 1.4 per cent), severe morbidity (71.1, 35.7 and 14.1 per cent) and associated complications such as acute kidney injury (26.3, 3.1 and 2.3 per cent) and haemorrhage (15.8, 3.1 and 1.4 per cent). These results were confirmed in the validation group. Combining Lactate_Max with Lactate_POD1 further increased AUC (ΔAUC = 0.053) utilizing lactate dynamics for risk assessment. Lactate_Max, major resections, age, cirrhosis and chronic kidney disease were independent risk factors for CR-PHLF. A freely available calculator facilitates clinical risk stratification (www.liver-calculator.com).

CONCLUSION

Early postoperative lactate values are powerful, readily available markers for CR-PHLF and associated complications after hepatectomy with potential for guiding postoperative care.Presented in part as an oral video abstract at the 2020 online Congress of the European Society for Surgical Research and the 2021 Congress of the Austrian Surgical Society.

Pre-operative radiomics model for prognostication in resectable pancreatic adenocarcinoma with external validation

OBJECTIVES

In resectable pancreatic ductal adenocarcinoma (PDAC), few pre-operative prognostic biomarkers are available. Radiomics has demonstrated potential but lacks external validation. We aimed to develop and externally validate a pre-operative clinical-radiomic prognostic model.

METHODS

Retrospective international, multi-center study in resectable PDAC. The training cohort included 352 patients (pre-operative CTs from five Canadian hospitals). Cox models incorporated (a) pre-operative clinical variables (clinical), (b) clinical plus CT-radiomics, and (c) post-operative TNM model, which served as the reference. Outcomes were overall (OS)/disease-free survival (DFS). Models were assessed in the validation cohort from Ireland (n = 215, CTs from 34 hospitals), using C-statistic, calibration, and decision curve analyses.

RESULTS

The radiomic signature was predictive of OS/DFS in the validation cohort, with adjusted hazard ratios (HR) 2.87 (95% CI: 1.40-5.87, p < 0.001)/5.28 (95% CI 2.35-11.86, p < 0.001), respectively, along with age 1.02 (1.01-1.04, p = 0.01)/1.02 (1.00-1.04, p = 0.03). In the validation cohort, median OS was 22.9/37 months (p = 0.0092) and DFS 14.2/29.8 (p = 0.0023) for high-/low-risk groups and calibration was moderate (mean absolute errors 7%/13% for OS at 3/5 years). The clinical-radiomic model discrimination (C = 0.545, 95%: 0.543-0.546) was higher than the clinical model alone (C = 0.497, 95% CI 0.496-0.499, p < 0.001) or TNM (C = 0.525, 95% CI: 0.524-0.526, p < 0.001). Despite superior net benefit compared to the clinical model, the clinical-radiomic model was not clinically useful for most threshold probabilities.

CONCLUSION

A multi-institutional pre-operative clinical-radiomic model for resectable PDAC prognostication demonstrated superior net benefit compared to a clinical model but limited clinical utility at external validation. This reflects inherent limitations of radiomics for PDAC prognostication, when deployed in real-world settings.

KEY POINTS

• At external validation, a pre-operative clinical-radiomics prognostic model for pancreatic ductal adenocarcinoma (PDAC) outperformed pre-operative clinical variables alone or pathological TNM staging. • Discrimination and clinical utility of the clinical-radiomic model for treatment decisions remained low, likely due to heterogeneity of CT acquisition parameters. • Despite small improvements, prognosis in PDAC using state-of-the-art radiomics methodology remains challenging, mostly owing to its low discriminative ability. Future research should focus on standardization of CT protocols and acquisition parameters.

A Radiomics Nomogram for Preoperative Prediction of Clinical Occult Lymph Node Metastasis in cT1-2N0M0 Solid Lung Adenocarcinoma

Background

Clinical occult lymph node metastasis (cOLNM) means that the lymph node is negatively diagnosed by preoperative computed tomography (CT), but has been proven to be positive by postoperative pathology. The aim of this study was to establish and validate a nomogram based on radiomics features for the preoperative prediction of cOLNM in early-stage solid lung adenocarcinoma patients.

Methods

A total of 244 patients with clinical T1-2N0M0 solid lung adenocarcinoma who underwent preoperative contrast-enhanced chest CT were divided into a primary group (n = 160) and an independent validation group from another hospital (n = 84). The records of 851 radiomics features of each primary tumor were extracted. LASSO analysis was used to reduce the data dimensionality and select features. Multivariable logistic regression was utilized to identify independent predictors of cOLNM and develop a predictive nomogram. The performance of the predictive model was assessed by its calibration and discrimination. Decision curve analysis (DCA) was performed to estimate the clinical usefulness of the nomogram.

Results

The predictive model consisted of a clinical factor (CT-reported tumor size) and a radiomics feature (Rad-score). The nomogram presented good discrimination, with a C-index of 0.782 (95% CI, 0.768–0.796) in the primary cohort and 0.813 (95% CI, 0.787–0.839) in the validation cohort, and good calibration. DCA showed that the radiomics nomogram was clinically useful.

Conclusion

This study develops and validates a nomogram that incorporates clinical and radiomics factors. It can be tailored for the individualized preoperative prediction of cOLNM in early-stage solid lung adenocarcinoma patients.

National time trends in mortality and graft survival following liver transplantation from circulatory death or brainstem death donors

BACKGROUND

Despite high waiting list mortality rates, concern still exists on the appropriateness of using livers donated after circulatory death (DCD). We compared mortality and graft loss in recipients of livers donated after circulatory or brainstem death (DBD) across two successive time periods.

METHODS

Observational multinational data from the United Kingdom and Ireland were partitioned into two time periods (2008-2011 and 2012-2016). Cox regression methods were used to estimate hazard ratios (HRs) comparing the impact of periods on post-transplant mortality and graft failure.

RESULTS

A total of 1176 DCD recipients and 3749 DBD recipients were included. Three-year patient mortality rates decreased markedly from 19.6 per cent in time period 1 to 10.4 per cent in time period 2 (adjusted HR 0.43, 95 per cent c.i. 0.30 to 0.62; P < 0.001) for DCD recipients but only decreased from 12.8 to 11.3 per cent (adjusted HR 0.96, 95 per cent c.i. 0.78 to 1.19; P = 0.732) in DBD recipients (P for interaction = 0.001). No time period-specific improvements in 3-year graft failure were observed for DCD (adjusted HR 0.80, 95% c.i. 0.61 to 1.05; P = 0.116) or DBD recipients (adjusted HR 0.95, 95% c.i. 0.79 to 1.14; P = 0.607). A slight increase in retransplantation rates occurred between time period 1 and 2 in those who received a DCD liver (from 7.3 to 11.8 per cent; P = 0.042), but there was no change in those receiving a DBD liver (from 4.9 to 4.5 per cent; P = 0.365). In time period 2, no difference in mortality rates between those receiving a DCD liver and those receiving a DBD liver was observed (adjusted HR 0.78, 95% c.i. 0.56 to 1.09; P = 0.142).

CONCLUSION

Mortality rates more than halved in recipients of a DCD liver over a decade and eventually compared similarly to mortality rates in recipients of a DBD liver. Regions with high waiting list mortality may mitigate this by use of DCD livers.

Development and Validation of a Deep Learning Model for Earlier Detection of Cognitive Decline From Clinical Notes in Electronic Health Records

IMPORTANCE

Detecting cognitive decline earlier among older adults can facilitate enrollment in clinical trials and early interventions. Clinical notes in longitudinal electronic health records (EHRs) provide opportunities to detect cognitive decline earlier than it is noted in structured EHR fields as formal diagnoses.

OBJECTIVE

To develop and validate a deep learning model to detect evidence of cognitive decline from clinical notes in the EHR.

DESIGN, SETTING, AND PARTICIPANTS

Notes documented 4 years preceding the initial mild cognitive impairment (MCI) diagnosis were extracted from Mass General Brigham's Enterprise Data Warehouse for patients aged 50 years or older and with initial MCI diagnosis during 2019. The study was conducted from March 1, 2020, to June 30, 2021. Sections of notes for cognitive decline were labeled manually and 2 reference data sets were created. Data set I contained a random sample of 4950 note sections filtered by a list of keywords related to cognitive functions and was used for model training and testing. Data set II contained 2000 randomly selected sections without keyword filtering for assessing whether the model performance was dependent on specific keywords.

MAIN OUTCOMES AND MEASURES

A deep learning model and 4 baseline models were developed and their performance was compared using the area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC).

RESULTS

Data set I represented 1969 patients (1046 [53.1%] women; mean [SD] age, 76.0 [13.3] years). Data set II comprised 1161 patients (619 [53.3%] women; mean [SD] age, 76.5 [10.2] years). With some overlap of patients deleted, the unique population was 2166. Cognitive decline was noted in 1453 sections (29.4%) in data set I and 69 sections (3.45%) in data set II. Compared with the 4 baseline models, the deep learning model achieved the best performance in both data sets, with AUROC of 0.971 (95% CI, 0.967-0.976) and AUPRC of 0.933 (95% CI, 0.921-0.944) for data set I and AUROC of 0.997 (95% CI, 0.994-0.999) and AUPRC of 0.929 (95% CI, 0.870-0.969) for data set II.

CONCLUSIONS AND RELEVANCE

In this diagnostic study, a deep learning model accurately detected cognitive decline from clinical notes preceding MCI diagnosis and had better performance than keyword-based search and other machine learning models. These results suggest that a deep learning model could be used for earlier detection of cognitive decline in the EHRs.

Identification and validation of a multivariable prediction model based on blood plasma and serum metabolomics for the distinction of chronic pancreatitis subjects from non-pancreas disease control subjects

OBJECTIVE

Chronic pancreatitis (CP) is a fibroinflammatory syndrome leading to organ dysfunction, chronic pain, an increased risk for pancreatic cancer and considerable morbidity. Due to a lack of specific biomarkers, diagnosis is based on symptoms and specific but insensitive imaging features, preventing an early diagnosis and appropriate management.

DESIGN

We conducted a type 3 study for multivariable prediction for individual prognosis according to the TRIPOD guidelines. A signature to distinguish CP from controls (n=160) was identified using gas chromatography-mass spectrometry and liquid chromatography-tandem mass spectrometry on ethylenediaminetetraacetic acid (EDTA)-plasma and validated in independent cohorts.

RESULTS

A Naive Bayes algorithm identified eight metabolites of six ontology classes. After algorithm training and computation of optimal cut-offs, classification according to the metabolic signature detected CP with an area under the curve (AUC) of 0.85 ((95% CI 0.79 to 0.91). External validation in two independent cohorts (total n=502) resulted in similar accuracy for detection of CP compared with non-pancreatic controls in EDTA-plasma (AUC 0.85 (95% CI 0.81 to 0.89)) and serum (AUC 0.87 (95% CI 0.81 to 0.95)).

CONCLUSIONS

This is the first study that identifies and independently validates a metabolomic signature in plasma and serum for the diagnosis of CP in large, prospective cohorts. The results could provide the basis for the development of the first routine laboratory test for CP.

Machine Learning Prediction Models for Mortality in Intensive Care Unit Patients with Lactic Acidosis

Background: Lactic acidosis is the most common cause of anion gap metabolic acidosis in the intensive care unit (ICU), associated with poor outcomes including mortality. We sought to compare machine learning (ML) approaches versus logistic regression analysis for prediction of mortality in lactic acidosis patients admitted to the ICU. Methods: We used the Medical Information Mart for Intensive Care (MIMIC-III) database to identify ICU adult patients with lactic acidosis (serum lactate ≥4 mmol/L). The outcome of interest was hospital mortality. We developed prediction models using four ML approaches consisting of random forest (RF), decision tree (DT), extreme gradient boosting (XGBoost), artificial neural network (ANN), and statistical modeling with forward stepwise logistic regression using the testing dataset. We then assessed model performance using area under the receiver operating characteristic curve (AUROC), accuracy, precision, error rate, Matthews correlation coefficient (MCC), F1 score, and assessed model calibration using the Brier score, in the independent testing dataset. Results: Of 1919 lactic acidosis ICU patients, 1535 and 384 were included in the training and testing dataset, respectively. Hospital mortality was 30%. RF had the highest AUROC at 0.83, followed by logistic regression 0.81, XGBoost 0.81, ANN 0.79, and DT 0.71. In addition, RF also had the highest accuracy (0.79), MCC (0.45), F1 score (0.56), and lowest error rate (21.4%). The RF model was the most well-calibrated. The Brier score for RF, DT, XGBoost, ANN, and multivariable logistic regression was 0.15, 0.19, 0.18, 0.19, and 0.16, respectively. The RF model outperformed multivariable logistic regression model, SOFA score (AUROC 0.74), SAP II score (AUROC 0.77), and Charlson score (AUROC 0.69). Conclusion: The ML prediction model using RF algorithm provided the highest predictive performance for hospital mortality among ICU patient with lactic acidosis.

Application of Machine Learning Algorithms to Predict Clinically Meaningful Improvement After Arthroscopic Anterior Cruciate Ligament Reconstruction

Background:

Understanding specific risk profiles for each patient and their propensity to experience clinically meaningful improvement after anterior cruciate ligament reconstruction (ACLR) is important for preoperative patient counseling and management of expectations.

Purpose:

To develop machine learning algorithms to predict achievement of the minimal clinically important difference (MCID) on the International Knee Documentation Committee (IKDC) score at a minimum 2-year follow-up after ACLR.

Study Design:

Case-control study; Level of evidence, 3.

Methods:

An ACLR registry of patients from 27 fellowship-trained sports medicine surgeons at a large academic institution was retrospectively analyzed. Thirty-six variables were tested for predictive value. The study population was randomly partitioned into training and independent testing sets using a 70:30 split. Six machine learning algorithms (stochastic gradient boosting, random forest, neural network, support vector machine, adaptive gradient boosting, and elastic-net penalized logistic regression [ENPLR]) were trained using 10-fold cross-validation 3 times and internally validated on the independent set of patients. Algorithm performance was assessed using discrimination, calibration, Brier score, and decision-curve analysis.

Results:

A total of 442 patients, of whom 39 (8.8%) did not achieve the MCID, were included. The 5 most predictive features of achieving the MCID were body mass index ≤27.4, grade 0 medial collateral ligament examination (compared with other grades), intratunnel femoral tunnel fixation (compared with suspensory), no history of previous contralateral knee surgery, and achieving full knee extension preoperatively. The ENPLR algorithm had the best relative performance (C-statistic, 0.82; calibration intercept, 0.10; calibration slope, 1.15; Brier score, 0.068), demonstrating excellent predictive ability in the study’s data set.

Conclusion:

Machine learning, specifically the ENPLR algorithm, demonstrated good performance for predicting a patient’s propensity to achieve the MCID for the IKDC score after ACLR based on preoperative and intraoperative factors. The femoral tunnel fixation method was the only significant intraoperative variable. Range of motion and medial collateral ligament integrity were found to be important physical examination parameters. Increased body mass index and prior contralateral surgery were also significantly predictive of outcome.

Machine Learning Model to Identify Sepsis Patients in the Emergency Department: Algorithm Development and Validation

Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.

Interpretable machine learning for early neurological deterioration prediction in atrial fibrillation-related stroke

We aimed to develop a novel prediction model for early neurological deterioration (END) based on an interpretable machine learning (ML) algorithm for atrial fibrillation (AF)-related stroke and to evaluate the prediction accuracy and feature importance of ML models. Data from multicenter prospective stroke registries in South Korea were collected. After stepwise data preprocessing, we utilized logistic regression, support vector machine, extreme gradient boosting, light gradient boosting machine (LightGBM), and multilayer perceptron models. We used the Shapley additive explanation (SHAP) method to evaluate feature importance. Of the 3,213 stroke patients, the 2,363 who had arrived at the hospital within 24 h of symptom onset and had available information regarding END were included. Of these, 318 (13.5%) had END. The LightGBM model showed the highest area under the receiver operating characteristic curve (0.772; 95% confidence interval, 0.715–0.829). The feature importance analysis revealed that fasting glucose level and the National Institute of Health Stroke Scale score were the most influential factors. Among ML algorithms, the LightGBM model was particularly useful for predicting END, as it revealed new and diverse predictors. Additionally, the effects of the features on the predictive power of the model were individualized using the SHAP method.

External validation of the Toronto hepatocellular carcinoma risk index in a Swedish population

Background & Aims

The Toronto hepatocellular carcinoma (HCC) risk index (THRI) is a predictive model to determine the risk of HCC in patients with cirrhosis. This study aimed to externally validate the THRI in a Swedish setting to investigate whether it could identify patients not requiring HCC surveillance.

Methods

From 2004-2017, 2,491 patients with cirrhosis at the Karolinska University Hospital were evaluated. Patients were classified into low-, intermediate- and high-risk groups for future HCC according to the THRI. Harrell’s C-index, calibration-in-the-large, calibration slope and goodness-of-fit estimates were calculated to assess model discrimination and calibration. Cox proportional hazards regression was used to determine the risk of HCC.

Results

Most patients were male (n = 1,638, 66%). The most common etiologies of cirrhosis were steatohepatitis (n = 1,182, 48%) followed by viral hepatitis (n = 987, 40%). In all, 131 patients (5.3%) were designated as low risk for HCC. Harrell’s C-index was 0.69. Calibration-in-the-large (0.11), calibration slope (1.24, not different from 1, p = 0.66) and goodness-of-fit showed good model calibration. Patients in the high-risk group had a 7.1-fold (95% CI 2.9–17.2) higher risk of HCC and patients in the intermediate-risk group had a 2.5-fold (95% CI 1.0–6.3) higher risk compared to the low-risk group.

Conclusions

In a Swedish setting, the THRI could differentiate between low- and high-risk of HCC development. However, because the low-risk group was relatively small (5.3%), the clinical applicability of the THRI could be limited.

Lay summary

The Toronto hepatocellular carcinoma (HCC) risk index (THRI) is a novel prediction model used to stratify patients with cirrhosis based on future risk of HCC. In this study, the THRI was validated in an external cohort using the TRIPOD guidance. Few patients were identified as low-risk, and the THRI had a modest discriminative ability, limiting its clinical applicability.

•

The THRI is a simple and non-invasive method to estimate 5- and 10-year HCC risk.

•

This was the largest validation of the THRI to date.

•

The THRI had a modest discriminative ability and was well-calibrated.

•

However, the THRI could only identify few patients at low risk of HCC, limiting its clinical use.

Correlations between bone marrow radiation dose and hematologic toxicity in locally advanced cervical cancer patients receiving chemoradiation with cisplatin: a systematic review

Patients with locally advanced cervical cancer (LACC) treated with chemoradiation often experience hematologic toxicity (HT), as chemoradiation can induce bone marrow (BM) suppression. Studies on the relationship between BM dosimetric parameters and clinically significant HT might provide relevant indices for developing BM sparing (BMS) radiotherapy techniques. This systematic review studied the relationship between BM dose and HT in patients with LACC treated with primary cisplatin-based chemoradiation. A systematic search was conducted in Embase, Medline, and Web of Science. Eligibility criteria were treatment of LACC-patients with cisplatin-based chemoradiation and report of HT or complete blood cell count (CBC). The search identified 1346 papers, which were screened on title and abstract before two reviewers independently evaluated the full-text. 17 articles were included and scored according to a selection of the TRIPOD criteria. The mean TRIPOD score was 12.1 out of 29. Fourteen studies defining BM as the whole pelvic bone contour (PB) detected significant associations with V10 (3/14), V20 (6/14), and V40 (4/11). Recommended cut-off values were V10 > 95-75%, V20 > 80-65%, and V40 > 37-28%. The studies using lower density marrow spaces (PBM) or active bone marrow (ABM) as a proxy for BM only found limited associations with HT. Our study was the first literature review providing an overview of articles evaluating the correlation between BM and HT for patients with LACC undergoing cisplatin-based chemoradiation. There is a scarcity of studies independently validating developed prediction models between BM dose and HT. Future studies may use PB contouring to develop normal tissue complication probability models.

FLAIR and ADC Image-Based Radiomics Features as Predictive Biomarkers of Unfavorable Outcome in Patients With Acute Ischemic Stroke

At present, it is still challenging to predict the clinical outcome of acute ischemic stroke (AIS). In this retrospective study, we explored whether radiomics features extracted from fluid-attenuated inversion recovery (FLAIR) and apparent diffusion coefficient (ADC) images can predict clinical outcome of patients with AIS. Patients with AIS were divided into a training (n = 110) and an external validation (n = 80) sets. A total of 753 radiomics features were extracted from each FLAIR and ADC image of the 190 patients. Interquartile range (IQR), Wilcoxon rank sum test, and least absolute shrinkage and selection operator (LASSO) were used to reduce the feature dimension. The six strongest radiomics features were related to an unfavorable outcome of AIS. A logistic regression analysis was employed for selection of potential predominating clinical and conventional magnetic resonance imaging (MRI) factors. Subsequently, we developed several models based on clinical and conventional MRI factors and radiomics features to predict the outcome of AIS patients. For predicting unfavorable outcome [modified Rankin scale (mRS) > 2] in the training set, the area under the receiver operating characteristic curve (AUC) of ADC radiomics model was 0.772, FLAIR radiomics model 0.731, ADC and FLAIR radiomics model 0.815, clinical model 0.791, and clinical and conventional MRI model 0.782. In the external validation set, the AUCs for the prediction with ADC radiomics model was 0.792, FLAIR radiomics model 0.707, ADC and FLAIR radiomics model 0.825, clinical model 0.763, and clinical and conventional MRI model 0.751. When adding radiomics features to the combined model, the AUCs for predicting unfavorable outcome in the training and external validation sets were 0.926 and 0.864, respectively. Our results indicate that the radiomics features extracted from FLAIR and ADC can be instrumental biomarkers to predict unfavorable clinical outcome of AIS and would additionally improve predictive performance when adding to combined model.

Prediction models in gynaecology: Transparent reporting needed for clinical application

The clinical application of prediction models is increasing within the field of gynaecology and obstetrics. This is mostly due to the fact that clinicians and patients prefer individualized counselling and person specific, more objective outcome assessment. To prevent using inadequate models, it is important to construct and perform prediction model studies correctly. Therefore, the TRIPOD statement (the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) was developed. The aim of this review is to obtain an overview of the existing published prediction models for benign gynaecology and to investigate to what extent these studies meet the TRIPOD criteria. We performed a literature search in the databases PubMed, Embase and Cochrane Library from inception to August 2020. Searching the cross-references of the relevant studies within our search identified additional articles. Publications were included if the aim of the study was to develop a multivariable prediction model within the field of benign gynaecology. Two independent reviewers extracted the data. Analysis of the studies was performed by using a checklist derived from the TRIPOD criteria. Based on our search, 2487 studies were selected, including potential duplications. Eventually, a total of twenty-two studies were selected. 91% of these studies handled their predictors by univariable analysis before developing a multivariable prediction model. Fifteen studies described having missing data, but not all of them (9%) handled these missing data. Four different internal validation methods were used in twenty studies. Fifteen studies (68%) had prediction models with a C-index ≥ 0.7, which indicates a good model. Half of the studies (50%) did not measure the calibration, overall performance was described in two studies (9%). External validation was performed in 9% of the studies. The correct development of a prediction model within benign gynaecology and subsequent transparent reporting of the model development is important to facilitate clinical use. Without transparent reporting, wrong assumptions can be made leading to incorrect application of a specific prediction model. This overview shows that excepting carrying out an external validation, only one article met all the criteria. Therefore, we strongly recommend use of the TRIPOD criteria for developing and validating a prediction model (study). In addition, prior to publication, content experts should critically and statistically review the prediction model. If too many criteria are not met, refusing publication should be considered.

Clinically relevant deep learning for detection and quantification of geographic atrophy from optical coherence tomography: a model development and external validation study

BACKGROUND

Geographic atrophy is a major vision-threatening manifestation of age-related macular degeneration, one of the leading causes of blindness globally. Geographic atrophy has no proven treatment or method for easy detection. Rapid, reliable, and objective detection and quantification of geographic atrophy from optical coherence tomography (OCT) retinal scans is necessary for disease monitoring, prognostic research, and to serve as clinical endpoints for therapy development. To this end, we aimed to develop and validate a fully automated method to detect and quantify geographic atrophy from OCT.

METHODS

We did a deep-learning model development and external validation study on OCT retinal scans at Moorfields Eye Hospital Reading Centre and Clinical AI Hub (London, UK). A modified U-Net architecture was used to develop four distinct deep-learning models for segmentation of geographic atrophy and its constituent retinal features from OCT scans acquired with Heidelberg Spectralis. A manually segmented clinical dataset for model development comprised 5049 B-scans from 984 OCT volumes selected randomly from 399 eyes of 200 patients with geographic atrophy secondary to age-related macular degeneration, enrolled in a prospective, multicentre, phase 2 clinical trial for the treatment of geographic atrophy (FILLY study). Performance was externally validated on an independently recruited dataset from patients receiving routine care at Moorfields Eye Hospital (London, UK). The primary outcome was segmentation and classification agreement between deep-learning model geographic atrophy prediction and consensus of two independent expert graders on the external validation dataset.

FINDINGS

The external validation cohort included 884 B-scans from 192 OCT volumes taken from 192 eyes of 110 patients as part of real-life clinical care at Moorfields Eye Hospital between Jan 1, 2016, and Dec, 31, 2019 (mean age 78·3 years [SD 11·1], 58 [53%] women). The resultant geographic atrophy deep-learning model produced predictions similar to consensus human specialist grading on the external validation dataset (median Dice similarity coefficient [DSC] 0·96 [IQR 0·10]; intraclass correlation coefficient [ICC] 0·93) and outperformed agreement between human graders (DSC 0·80 [0·28]; ICC 0·79). Similarly, the three independent feature-specific deep-learning models could accurately segment each of the three constituent features of geographic atrophy: retinal pigment epithelium loss (median DSC 0·95 [IQR 0·15]), overlying photoreceptor degeneration (0·96 [0·12]), and hypertransmission (0·97 [0·07]) in the external validation dataset versus consensus grading.

INTERPRETATION

We present a fully developed and validated deep-learning composite model for segmentation of geographic atrophy and its subtypes that achieves performance at a similar level to manual specialist assessment. Fully automated analysis of retinal OCT from routine clinical practice could provide a promising horizon for diagnosis and prognosis in both research and real-life patient care, following further clinical validation FUNDING: Apellis Pharmaceuticals.

Developing and validating a prediction model for lymphedema detection in breast cancer survivors

PURPOSE

Early detection and intervention of lymphedema is essential for improving the quality of life of breast cancer survivors. Previous studies have shown that patients have symptoms such as arm tightness and arm heaviness before experiencing obvious limb swelling. Thus, this study aimed to develop a symptom-warning model for the early detection of breast cancer-related lymphedema.

METHODS

A cross-sectional study was conducted at a tertiary hospital in Beijing between April 2017 and December 2018. A total of 24 lymphedema-associated symptoms were identified as candidate predictors. Circumferential measurements were used to diagnose lymphedema. The data were randomly split into training and validation sets with a 7:3 ratio to derive and evaluate six machine learning models. Both the discrimination and calibration of each model were assessed on the validation set.

RESULTS

A total of 533 patients were included in the study. The logistic regression model showed the best performance for early detection of lymphedema, with AUC = 0.889 (0.840-0.938), sensitivity = 0.771, specificity = 0.883, accuracy = 0.825, and Brier scores = 0.141. Calibration was also acceptable. It has been deployed as an open-access web application, allowing users to estimate the probability of lymphedema individually in real time. The application can be found at https://apredictiontoolforlymphedema.shinyapps.io/dynnomapp/.

CONCLUSION

The symptom-warning model developed by logistic regression performed well in the early detection of lymphedema. Integrating this model into an open-access web application is beneficial to patients and healthcare providers to monitor lymphedema status in real-time.

Machine learning can reliably identify patients at risk of overnight hospital admission following anterior cruciate ligament reconstruction

PURPOSE

Overnight admission following anterior cruciate ligament reconstruction has implications on clinical outcomes as well as cost benefit, yet there are few validated risk calculators for reliable identification of appropriate candidates. The purpose of this study is to develop and validate a machine learning algorithm that can effectively identify patients requiring admission following elective anterior cruciate ligament (ACL) reconstruction.

METHODS

A retrospective review of a national surgical outcomes database was performed to identify patients who underwent elective ACL reconstruction from 2006 to 2018. Patients admitted overnight postoperatively were identified as those with length of stay of 1 or more days. Models were generated using random forest (RF), extreme gradient boosting (XGBoost), linear discriminant classifier (LDA), and adaptive boosting algorithms (AdaBoost), and an additional model was produced as a weighted ensemble of the four final algorithms.

RESULTS

Overall, of the 4,709 patients included, 531 patients (11.3%) required at least one overnight stay following ACL reconstruction. The factors determined most important for identification of candidates for inpatient admission were operative time, anesthesia type, age, gender, and BMI. Smoking history, history of COPD, and history of coagulopathy were identified as less important variables. The following factors supported overnight admission: operative time > 200 min, age < 35.8 or > 53.5 years, male gender, BMI < 25 or > 31.2 kg/m², positive smoking history, history of COPD and the presence of preoperative coagulopathy. The ensemble model achieved the best performance based on discrimination assessed via internal validation (AUC = 0.76), calibration, and decision curve analysis. The model was integrated into a web-based open-access application able to provide both predictions and explanations.

CONCLUSION

Modifiable risk factors identified by the model such as increased BMI, operative time, anesthesia type, and comorbidities can help clinicians optimize preoperative status to prevent costs associated with unnecessary admissions. If externally validated in independent populations, this algorithm could use these inputs to guide preoperative screening and risk stratification to identify patients requiring overnight admission for observation following ACL reconstruction.

LEVEL OF EVIDENCE

IV.

Biomarkers of Inflammation and Inflammation-Related Indexes upon Emergency Department Admission Are Predictive for the Risk of Intensive Care Unit Hospitalization and Mortality in Acute Poisoning: A 6-Year Prospective Observational Study

Patients poisoned with drugs and nonpharmaceutical substances are frequently admitted from the emergency department (ED) to a medical or ICU department. We hypothesized that biomarkers of inflammation and inflammation-related indexes based on the complete blood cell (CBC) count can identify acutely poisoned patients at increased risk for ICU hospitalization and death. We performed a 6-year prospective cohort study on 1548 adult patients. The demographic data, the levels of hs-CRP (high-sensitivity C-reactive protein), CBC, and inflammation-related indexes based on CBC counts were collected upon admission and compared between survivors and nonsurvivors, based on the poison involved. Both a multivariate logistic regression model with only significant univariate predictors and a model including univariate predictors plus each log-transformed inflammation-related indexes for mortality were constructed. The importance of the variables for mortality was graphically represented using the nomogram. hs-CRP (odds ratio (OR), 1.38; 95% CI, 1.16–1.65, p < 0.001 for log-transformed hs-CRP), red cell distribution width (RDW), neutrophil-lymphocyte ratio (NLR), and platelet-lymphocyte ratio (PLR) were significantly associated with the risk of ICU hospitalization, after multivariable adjustment. Only RDW, NLR, and monocyte-lymphocyte ratio (MLR) were significantly associated with mortality. The predictive accuracy for mortality of the models which included either NLR (AUC 0.917, 95% CI 0.886-0.948) or MLR (AUC 0.916, 95% CI 0.884-0.948) showed a high ability for prognostic detection. The use of hs-CRP, RDW, NLR, and MLR upon ED admission are promising screening tools for predicting the outcomes of patients acutely intoxicated with undifferentiated poisons.

Development and validation of an improved classification and risk stratification system for carotid body tumors: Multinational collaborative cohort study

BACKGROUND

This study aims to develop and validate a new classification system that better predicts combined risk of neurological and neurovascular complications following CBT surgery, crucial for treatment decision-making.

METHODS

Multinational retrospective cohort study with 199 consecutive cases. A cohort of 132 CBT cases was used to develop the new classification. To undertake external validation, assessment was made between the actual complication rate and predicted risk by the model on an independent cohort (n = 67).

RESULTS

Univariate analyses showed statistically significant associations between developing a complication and the following factors: craniocaudal dimension, volume, Shamblin classification, and Mehanna types. In the multivariate prognostic model, only Mehanna type remained as a significant risk predictor. The risk of developing complications increases with increasing Mehanna type.

CONCLUSIONS

We have developed and then validated a new classification and risk stratification system for CBTs, which demonstrated better prognostic power for the risk of developing neurovascular complications after surgery.

Machine Learning in Aging: An Example of Developing Prediction Models for Serious Fall Injury in Older Adults

BACKGROUND

Advances in computational algorithms and the availability of large datasets with clinically relevant characteristics provide an opportunity to develop machine learning prediction models to aid in diagnosis, prognosis, and treatment of older adults. Some studies have employed machine learning methods for prediction modeling, but skepticism of these methods remains due to lack of reproducibility and difficulty in understanding the complex algorithms that underlie models. We aim to provide an overview of two common machine learning methods: decision tree and random forest. We focus on these methods because they provide a high degree of interpretability.

METHOD

We discuss the underlying algorithms of decision tree and random forest methods and present a tutorial for developing prediction models for serious fall injury using data from the Lifestyle Interventions and Independence for Elders (LIFE) study.

RESULTS

Decision tree is a machine learning method that produces a model resembling a flow chart. Random forest consists of a collection of many decision trees whose results are aggregated. In the tutorial example, we discuss evaluation metrics and interpretation for these models. Illustrated using data from the LIFE study, prediction models for serious fall injury were moderate at best (area under the receiver operating curve of 0.54 for decision tree and 0.66 for random forest).

CONCLUSIONS

Machine learning methods offer an alternative to traditional approaches for modeling outcomes in aging, but their use should be justified and output should be carefully described. Models should be assessed by clinical experts to ensure compatibility with clinical practice.

A logistic analysis prediction model of TMJ condylar erosion in patients with TMJ arthralgia

BACKGROUND

In terms of diagnostic and therapeutic management, clinicians should adequately address the frequent aspects of temporomandibular joint (TMJ) osteoarthritis (OA) associated with disk displacement. Condylar erosion (CE) is considered an inflammatory subset of OA and is regarded as a sign of progressive OA changes potentially contributing to changes in dentofacial morphology or limited mandibular growth. The purpose of this study was to establish a risk prediction model of CE by a multivariate logistic regression analysis to predict the individual risk of CE in TMJ arthralgia. It was hypothesized that there was a closer association between CE and magnetic resonance imaging (MRI) indicators.

METHODS

This retrospective paired-design study enrolled 124 consecutive TMJ pain patients and analyzed the clinical and TMJ-related MRI data in predicting CE. TMJ pain patients were categorized according to the research diagnostic criteria for temporomandibular disorders (RDC/TMD) Axis I protocol. Each patient underwent MRI examination of both TMJs, 1-7 days following clinical examination.

RESULTS

In the univariate analysis analyses, 9 influencing factors were related to CE, of which the following 4 as predictors determined the binary multivariate logistic regression model: missing posterior teeth (odds ratio [OR] = 1.42; P = 0.018), RDC/TMD of arthralgia coexistant with disk displacement without reduction with limited opening (DDwoR/wLO) (OR = 3.30, P = 0.007), MRI finding of disk displacement without reduction (OR = 10.96, P < 0.001), and MRI finding of bone marrow edema (OR = 11.97, P < 0.001). The model had statistical significance (chi-square = 148.239, Nagelkerke R square = 0.612, P < 0.001). Out of the TMJs, 83.9% were correctly predicted to be CE cases or Non-CE cases with a sensitivity of 81.4% and a specificity of 85.2%. The area under the receiver operating characteristic curve was 0.916.

CONCLUSION

The established prediction model using the risk factors of TMJ arthralgia may be useful for predicting the risk of CE. The data suggest MRI indicators as dominant factors in the definition of CE. Further research is needed to improve the model, and confirm the validity and reliability of the model.

Predicting pathological lymph node status in clinical stage IA peripheral lung adenocarcinoma

OBJECTIVES

Even with current diagnostic technology, it is difficult to accurately predict pathological lymph node status (PLNS). This study aimed to develop a prediction model of PLNS in peripheral adenocarcinoma with a dominant solid component, based on clinical and radiological factors on thin-section computed tomography, to identify patients to whom wedge resection or other local therapies could be applied.

METHODS

Of 811 patients enrolled in a prospective multi-institutional study (JCOG0201), 420 patients with clinical stage IA peripheral lung adenocarcinoma having a dominant solid component were included. Multivariable logistic regression was performed to develop a model based on clinical and centrally reviewed radiological factors. Leave-one-out cross-validation and external validation analyses were performed, using independent data from 221 patients. Sensitivity, specificity and concordance statistics were calculated to evaluate diagnostic performance.

RESULTS

The formula for calculating the probability of pathological lymph node metastasis included the following variables: tumour diameter (including ground-glass opacity), consolidation-to-tumour ratio and density of solid component. The concordance statistic was 0.8041. When the cut-off value associated with the risk of incorrectly predicting negative pathological lymph node metastasis (pN-) was 4.9%, diagnostic sensitivity and specificity in predicting PLNS were 95.7% and 46.0%, respectively. The concordance statistic for the external validation set was 0.7972, and diagnostic sensitivity and specificity in predicting PLNS were 95.4% and 40.5%, respectively.

CONCLUSIONS

The proposed model is clinically useful and successfully predicts pN- in patients with clinical stage IA peripheral lung adenocarcinoma with a dominant solid component.

A Machine Learning Algorithm for Quantitatively Diagnosing Oxidative Stress Risks in Healthy Adult Individuals Based on Health Space Methodology: A Proof-of-Concept Study Using Korean Cross-Sectional Cohort Data

Oxidative stress aggravates the progression of lifestyle-related chronic diseases. However, knowledge and practices that enable quantifying oxidative stress are still lacking. Here, we performed a proof-of-concept study to predict the oxidative stress status in a healthy population using retrospective cohort data from Boramae medical center in Korea (n = 1328). To obtain binary performance measures, we selected healthy controls versus oxidative disease cases based on the "health space" statistical methodology. We then developed a machine learning algorithm for discrimination of oxidative stress status using least absolute shrinkage and selection operator (LASSO)/elastic net regression with 10-fold cross-validation. A proposed fine-tune model included 16 features out of the full spectrum of diverse and complex data. The predictive performance was externally evaluated by generating receiver operating characteristic curves with area under the curve of 0.949 (CI 0.925 to 0.974), sensitivity of 0.923 (CI 0.879 to 0.967), and specificity of 0.855 (CI 0.795 to 0.915). Moreover, the discrimination power was confirmed by applying the proposed diagnostic model to the full dataset consisting of subjects with various degrees of oxidative stress. The results provide a feasible approach for stratifying the oxidative stress risks in the healthy population and selecting appropriate strategies for individual subjects toward implementing data-driven precision nutrition.