Artificial Neural Network and Cox Regression Models for Predicting Mortality after Hip Fracture Surgery: A Population-Based Comparison. ACTA ACUST UNITED AC. 2020;56. [PMID: 32438724 PMCID: PMC7279348 DOI: 10.3390/medicina56050243]

Systematic review of machine-learning models in orthopaedic trauma

Aims

Machine-learning (ML) prediction models in orthopaedic trauma hold great promise in assisting clinicians in various tasks, such as personalized risk stratification. However, an overview of current applications and critical appraisal to peer-reviewed guidelines is lacking. The objectives of this study are to 1) provide an overview of current ML prediction models in orthopaedic trauma; 2) evaluate the completeness of reporting following the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement; and 3) assess the risk of bias following the Prediction model Risk Of Bias Assessment Tool (PROBAST) tool.

Methods

A systematic search screening 3,252 studies identified 45 ML-based prediction models in orthopaedic trauma up to January 2023. The TRIPOD statement assessed transparent reporting and the PROBAST tool the risk of bias.

Results

A total of 40 studies reported on training and internal validation; four studies performed both development and external validation, and one study performed only external validation. The most commonly reported outcomes were mortality (33%, 15/45) and length of hospital stay (9%, 4/45), and the majority of prediction models were developed in the hip fracture population (60%, 27/45). The overall median completeness for the TRIPOD statement was 62% (interquartile range 30 to 81%). The overall risk of bias in the PROBAST tool was low in 24% (11/45), high in 69% (31/45), and unclear in 7% (3/45) of the studies. High risk of bias was mainly due to analysis domain concerns including small datasets with low number of outcomes, complete-case analysis in case of missing data, and no reporting of performance measures.

Conclusion

The results of this study showed that despite a myriad of potential clinically useful applications, a substantial part of ML studies in orthopaedic trauma lack transparent reporting, and are at high risk of bias. These problems must be resolved by following established guidelines to instil confidence in ML models among patients and clinicians. Otherwise, there will remain a sizeable gap between the development of ML prediction models and their clinical application in our day-to-day orthopaedic trauma practice.

Using Machine Learning to Identify Predictors of Sexually Transmitted Infections Over Time Among Young People Living With or at Risk for HIV Who Participated in ATN Protocols 147, 148, and 149

BACKGROUND

Sexually transmitted infections (STIs) among youth aged 12 to 24 years have doubled in the last 13 years, accounting for 50% of STIs nationally. We need to identify predictors of STI among youth in urban HIV epicenters.

METHODS

Sexual and gender minority (gay, bisexual, transgender, gender-diverse) and other youth with multiple life stressors (homelessness, incarceration, substance use, mental health disorders) were recruited from 13 sites in Los Angeles and New Orleans (N = 1482). Self-reports and rapid diagnostic tests for STI, HIV, and drug use were conducted at 4-month intervals for up to 24 months. Machine learning was used to identify predictors of time until new STI (including a new HIV diagnosis).

RESULTS

At recruitment, 23.9% of youth had a current or past STI. Over 24 months, 19.3% tested positive for a new STI. Heterosexual males had the lowest STI rate (12%); African American youth were 23% more likely to acquire an STI compared with peers of other ethnicities. Time to STI was best predicted by attending group sex venues or parties, moderate but not high dating app use, and past STI and HIV seropositive status.

CONCLUSIONS

Sexually transmitted infections are concentrated among a subset of young people at highest risk. The best predictors of youth's risk are their sexual environments and networks. Machine learning will allow the next generation of research on predictive patterns of risk to be more robust.

Machine learning applications for the prediction of extended length of stay in geriatric hip fracture patients

BACKGROUND

Geriatric hip fractures are one of the most common fractures in elderly individuals, and prolonged hospital stays increase the risk of death and complications. Machine learning (ML) has become prevalent in clinical data processing and predictive models. This study aims to develop ML models for predicting extended length of stay (eLOS) among geriatric patients with hip fractures and to identify the associated risk factors.

AIM

To develop ML models for predicting the eLOS among geriatric patients with hip fractures, identify associated risk factors, and compare the performance of each model.

METHODS

A retrospective study was conducted at a single orthopaedic trauma centre, enrolling all patients who underwent hip fracture surgery between January 2018 and December 2022. The study collected various patient characteristics, encompassing demographic data, general health status, injury-related data, laboratory examinations, surgery-related data, and length of stay. Features that exhibited significant differences in univariate analysis were integrated into the ML model establishment and subsequently cross-verified. The study compared the performance of the ML models and determined the risk factors for eLOS.

RESULTS

The study included 763 patients, with 380 experiencing eLOS. Among the models, the decision tree, random forest, and extreme Gradient Boosting models demonstrated the most robust performance. Notably, the artificial neural network model also exhibited impressive results. After cross-validation, the support vector machine and logistic regression models demonstrated superior performance. Predictors for eLOS included delayed surgery, D-dimer level, American Society of Anaesthesiologists (ASA) classification, type of surgery, and sex.

CONCLUSION

ML proved to be highly accurate in predicting the eLOS for geriatric patients with hip fractures. The identified key risk factors were delayed surgery, D-dimer level, ASA classification, type of surgery, and sex. This valuable information can aid clinicians in allocating resources more efficiently to meet patient demand effectively.

Survival prediction and prognostic factors in colorectal cancer after curative surgery: insights from cox regression and neural networks

Medical research frequently relies on Cox regression to analyze the survival distribution of cancer patients. Nonetheless, in specific scenarios, neural networks hold the potential to serve as a robust alternative. In this study, we aim to scrutinize the effectiveness of Cox regression and neural network models in assessing the survival outcomes of patients who have undergone treatment for colorectal cancer. We conducted a retrospective study on 284 colorectal cancer patients who underwent surgery at Imam Khomeini clinic in Hamadan between 2001 and 2017. The data was used to train both Cox regression and neural network models, and their predictive accuracy was compared using diagnostic measures such as sensitivity, specificity, positive predictive value, accuracy, negative predictive value, and area under the receiver operating characteristic curve. The analyses were performed using STATA 17 and R4.0.4 software. The study revealed that the best neural network model had a sensitivity of 74.5% (95% CI 61.0-85.0), specificity of 83.3% (65.3-94.4), positive predictive value of 89.1% (76.4-96.4), negative predictive value of 64.1% (47.2-78.8), AUC of 0.79 (0.70-0.88), and accuracy of 0.776 for death prediction. For recurrence, the best neural network model had a sensitivity of 88.1% (74.4-96.0%), specificity of 83.7% (69.3-93.2%), positive predictive value of 84.1% (69.9-93.4%), negative predictive value of 87.8% (73.8-95.9%), AUC of 0.86 (0.78-0.93), and accuracy of 0.859. The Cox model had comparable results, with a sensitivity of 73.6% (64.8-81.2) and 85.5% (78.3-91.0), specificity of 89.6% (83.8-93.8) and 98.0% (94.4-99.6), positive predictive value of 84.0% (75.6-90.4) and 97.4% (92.6-99.5), negative predictive value of 82.0% (75.6-90.4) and 88.8% (0.83-93.1), AUC of 0.82 (0.77-0.86) and 0.92 (0.89-0.95), and accuracy of 0.88 and 0.92 for death and recurrence prediction, respectively. In conclusion, the study found that both Cox regression and neural network models are effective in predicting early recurrence and death in patients with colorectal cancer after curative surgery. The neural network model showed slightly better sensitivity and negative predictive value for death, while the Cox model had better specificity and positive predictive value for recurrence. Overall, both models demonstrated high accuracy and AUC, indicating their usefulness in predicting these outcomes.

Determinants of Inadequate Empiric Antimicrobial Therapy in ICU Sepsis Patients in Al-Madinah Al-Munawwarah, Saudi Arabia: A Comparison of Artificial Neural Network and Regression Analysis

In the management of sepsis, providing adequate empiric antimicrobial therapy is one of the most important pillars of sepsis management. Therefore, it is important to evaluate the adequacy of empiric antimicrobial therapy (EAMT) in sepsis patients admitted to intensive care units (ICU) and to identify the determinants of inadequate EAMT. The aim of this study was to evaluate the adequacy of empiric antimicrobial therapy in patients admitted to the ICU with sepsis or septic shock, and the determinants of inadequate EAMT. The data of patients admitted to the ICU units due to sepsis or septic shock in two tertiary healthcare facilities in Al-Madinah Al-Munawwarah were retrospectively reviewed. The current study used logistic regression analysis and artificial neural network (ANN) analysis to identify determinants of inadequate empiric antimicrobial therapy, and evaluated the performance of these two approaches in predicting the inadequacy of EAMT. The findings of this study showed that fifty-three per cent of patients received inadequate EAMT. Determinants for inadequate EAMT were APACHE II score, multidrug-resistance organism (MDRO) infections, surgical history (lower limb amputation), and comorbidity (coronary artery disease). ANN performed as well as or better than logistic regression in predicating inadequate EAMT, as the receiver operating characteristic area under the curve (ROC-AUC) of the ANN model was higher when compared with the logistic regression model (LRM): 0.895 vs. 0.854. In addition, the ANN model performed better than LRM in predicting inadequate EAMT in terms of classification accuracy. In addition, ANN analysis revealed that the most important determinants of EAMT adequacy were the APACHE II score and MDRO. In conclusion, more than half of the patients received inadequate EAMT. Determinants of inadequate EAMT were APACHE II score, MDRO infections, comorbidity, and surgical history. This provides valuable inputs to improve the prescription of empiric antimicrobials in Saudi Arabia going forward. In addition, our study demonstrated the potential utility of applying artificial neural network analysis in the prediction of outcomes in healthcare research.

Deep Learning Approaches for Glioblastoma Prognosis in Resource-Limited Settings: A Study Using Basic Patient Demographic, Clinical, and Surgical Inputs

BACKGROUND

Glioblastoma (GBM) is the most common brain tumor in the United States, with an annual incidence rate of 3.21 per 100,000. It is the most aggressive type of diffuse glioma and has a median survival of months after treatment. This study aims to assess the accuracy of different novel deep learning models trained on a set of simple clinical, demographic, and surgical variables to assist in clinical practice, even in areas with constrained health care infrastructure.

METHODS

Our study included 37,095 patients with GBM from the SEER (Surveillance Epidemiology and End Results) database. All predictors were based on demographic, clinicopathologic, and treatment information of the cases. Our outcomes of interest were months of survival and vital status. Concordance index (C-index) and integrated Brier scores (IBS) were used to evaluate the performance of the models.

RESULTS

The patient characteristics and the statistical analyses were consistent with the epidemiologic literature. The models C-index and IBS ranged from 0.6743 to 0.6918 and from 0.0934 to 0.1034, respectively. Probabilistic matrix factorization (0.6918), multitask logistic regression (0.6916), and logistic hazard (0.6916) had the highest C-index scores. The models with the lowest IBS were the probabilistic matrix factorization (0.0934), multitask logistic regression (0.0935), and logistic hazard (0.0936). These models had an accuracy (1-IBS) of 90.66%; 90.65%, and 90.64%, respectively. The deep learning algorithms were deployed on an interactive Web-based tool for practical use available via https://glioblastoma-survanalysis.herokuapp.com/.

CONCLUSIONS

Novel deep learning algorithms can better predict GBM prognosis than do baseline methods and can lead to more personalized patient care regardless of extensive electronic health record availability.

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction: A Systematic Review and Meta-analysis

IMPORTANCE

Artificial intelligence (AI) enables powerful models for establishment of clinical diagnostic and prognostic tools for hip fractures; however the performance and potential impact of these newly developed algorithms are currently unknown.

OBJECTIVE

To evaluate the performance of AI algorithms designed to diagnose hip fractures on radiographs and predict postoperative clinical outcomes following hip fracture surgery relative to current practices.

DATA SOURCES

A systematic review of the literature was performed using the MEDLINE, Embase, and Cochrane Library databases for all articles published from database inception to January 23, 2023. A manual reference search of included articles was also undertaken to identify any additional relevant articles.

STUDY SELECTION

Studies developing machine learning (ML) models for the diagnosis of hip fractures from hip or pelvic radiographs or to predict any postoperative patient outcome following hip fracture surgery were included.

DATA EXTRACTION AND SYNTHESIS

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses and was registered with PROSPERO. Eligible full-text articles were evaluated and relevant data extracted independently using a template data extraction form. For studies that predicted postoperative outcomes, the performance of traditional predictive statistical models, either multivariable logistic or linear regression, was recorded and compared with the performance of the best ML model on the same out-of-sample data set.

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy of AI models was compared with the diagnostic accuracy of expert clinicians using odds ratios (ORs) with 95% CIs. Areas under the curve for postoperative outcome prediction between traditional statistical models (multivariable linear or logistic regression) and ML models were compared.

RESULTS

Of 39 studies that met all criteria and were included in this analysis, 18 (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A total of 39 598 plain radiographs and 714 939 hip fractures were used for training, validating, and testing ML models specific to diagnosis and postoperative outcome prediction, respectively. Mortality and length of hospital stay were the most predicted outcomes. On pooled data analysis, compared with clinicians, the OR for diagnostic error of ML models was 0.79 (95% CI, 0.48-1.31; P = .36; I2 = 60%) for hip fracture radiographs. For the ML models, the mean (SD) sensitivity was 89.3% (8.5%), specificity was 87.5% (9.9%), and F1 score was 0.90 (0.06). The mean area under the curve for mortality prediction was 0.84 with ML models compared with 0.79 for alternative controls (P = .09).

CONCLUSIONS AND RELEVANCE

The findings of this systematic review and meta-analysis suggest that the potential applications of AI to aid with diagnosis from hip radiographs are promising. The performance of AI in diagnosing hip fractures was comparable with that of expert radiologists and surgeons. However, current implementations of AI for outcome prediction do not seem to provide substantial benefit over traditional multivariable predictive statistics.

Use of artificial neural networks in the prognosis of musculoskeletal diseases-a scoping review

To determine the current evidence on artificial neural network (ANN) in prognostic studies of musculoskeletal diseases (MSD) and to assess the accuracy of ANN in predicting the prognosis of patients with MSD. The scoping review was reported under the Preferred Items for Systematic Reviews and the Meta-Analyses extension for Scope Reviews (PRISMA-ScR). Cochrane Library, Embase, Pubmed, and Web of science core collection were searched from inception to January 2023. Studies were eligible if they used ANN to make predictions about MSD prognosis. Variables, model prediction accuracy, and disease type used in the ANN model were extracted and charted, then presented as a table along with narrative synthesis. Eighteen Studies were included in this scoping review, with 16 different types of musculoskeletal diseases. The accuracy of the ANN model predictions ranged from 0.542 to 0.947. ANN models were more accurate compared to traditional logistic regression models. This scoping review suggests that ANN can predict the prognosis of musculoskeletal diseases, which has the potential to be applied to different types of MSD.

Development and internal validation of a machine-learning-developed model for predicting 1-year mortality after fragility hip fracture

Background

Fragility hip fracture increases morbidity and mortality in older adult patients, especially within the first year. Identification of patients at high risk of death facilitates modification of associated perioperative factors that can reduce mortality. Various machine learning algorithms have been developed and are widely used in healthcare research, particularly for mortality prediction. This study aimed to develop and internally validate 7 machine learning models to predict 1-year mortality after fragility hip fracture.

Methods

This retrospective study included patients with fragility hip fractures from a single center (Siriraj Hospital, Bangkok, Thailand) from July 2016 to October 2018. A total of 492 patients were enrolled. They were randomly categorized into a training group (344 cases, 70%) or a testing group (148 cases, 30%). Various machine learning techniques were used: the Gradient Boosting Classifier (GB), Random Forests Classifier (RF), Artificial Neural Network Classifier (ANN), Logistic Regression Classifier (LR), Naive Bayes Classifier (NB), Support Vector Machine Classifier (SVM), and K-Nearest Neighbors Classifier (KNN). All models were internally validated by evaluating their performance and the area under a receiver operating characteristic curve (AUC).

Results

For the testing dataset, the accuracies were GB model = 0.93, RF model = 0.95, ANN model = 0.94, LR model = 0.91, NB model = 0.89, SVM model = 0.90, and KNN model = 0.90. All models achieved high AUCs that ranged between 0.81 and 0.99. The RF model also provided a negative predictive value of 0.96, a positive predictive value of 0.93, a specificity of 0.99, and a sensitivity of 0.68.

Conclusions

Our machine learning approach facilitated the successful development of an accurate model to predict 1-year mortality after fragility hip fracture. Several machine learning algorithms (eg, Gradient Boosting and Random Forest) had the potential to provide high predictive performance based on the clinical parameters of each patient. The web application is available at www.hipprediction.com. External validation in a larger group of patients or in different hospital settings is warranted to evaluate the clinical utility of this tool.

Trial registration

Thai Clinical Trials Registry (22 February 2021; reg. no. TCTR20210222003).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03152-x.

Implementation of a machine learning application in preoperative risk assessment for hip repair surgery

Background

This study aims to develop a machine learning-based application in a real-world medical domain to assist anesthesiologists in assessing the risk of complications in patients after a hip surgery.

Methods

Data from adult patients who underwent hip repair surgery at Chi-Mei Medical Center and its 2 branch hospitals from January 1, 2013, to March 31, 2020, were analyzed. Patients with incomplete data were excluded. A total of 22 features were included in the algorithms, including demographics, comorbidities, and major preoperative laboratory data from the database. The primary outcome was a composite of adverse events (in-hospital mortality, acute myocardial infarction, stroke, respiratory, hepatic and renal failure, and sepsis). Secondary outcomes were intensive care unit (ICU) admission and prolonged length of stay (PLOS). The data obtained were imported into 7 machine learning algorithms to predict the risk of adverse outcomes. Seventy percent of the data were randomly selected for training, leaving 30% for testing. The performances of the models were evaluated by the area under the receiver operating characteristic curve (AUROC). The optimal algorithm with the highest AUROC was used to build a web-based application, then integrated into the hospital information system (HIS) for clinical use.

Results

Data from 4,448 patients were analyzed; 102 (2.3%), 160 (3.6%), and 401 (9.0%) patients had primary composite adverse outcomes, ICU admission, and PLOS, respectively. Our optimal model had a superior performance (AUROC by DeLong test) than that of ASA-PS in predicting the primary composite outcomes (0.810 vs. 0.629, p < 0.01), ICU admission (0.835 vs. 0.692, p < 0.01), and PLOS (0.832 vs. 0.618, p < 0.01).

Conclusions

The hospital-specific machine learning model outperformed the ASA-PS in risk assessment. This web-based application gained high satisfaction from anesthesiologists after online use.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12871-022-01648-y.

Classification Performance of Neural Networks Versus Logistic Regression Models: Evidence From Healthcare Practice

Machine learning encompasses statistical approaches such as logistic regression (LR) through to more computationally complex models such as neural networks (NN). The aim of this study is to review current published evidence for performance from studies directly comparing logistic regression, and neural network classification approaches in medicine.

A literature review was carried out to identify primary research studies which provided information regarding comparative area under the curve (AUC) values for the overall performance of both LR and NN for a defined clinical healthcare-related problem. Following an initial search, articles were reviewed to remove those that did not meet the criteria and performance metrics were extracted from the included articles. Teh initial search revealed 114 articles; 21 studies were included in the study. In 13/21 (62%) of cases, NN had a greater AUC compared to LR, but in most the difference was small and unlikely to be of clinical significance; (unweighted mean difference in AUC 0.03 (95% CI 0-0.06) in favour of NN versus LR.

In the majority of cases examined across a range of clinical settings, LR models provide reasonable performance that is only marginally improved using more complex methods such as NN. In many circumstances, the use of a relatively simple LR model is likely to be adequate for real-world needs but in specific circumstances in which large amounts of data are available, and where even small increases in performance would provide significant management value, the application of advanced analytic tools such as NNs may be indicated.

Status quo and future prospects of artificial neural network from the perspective of gastroenterologists

Artificial neural networks (ANNs) are one of the primary types of artificial intelligence and have been rapidly developed and used in many fields. In recent years, there has been a sharp increase in research concerning ANNs in gastrointestinal (GI) diseases. This state-of-the-art technique exhibits excellent performance in diagnosis, prognostic prediction, and treatment. Competitions between ANNs and GI experts suggest that efficiency and accuracy might be compatible in virtue of technique advancements. However, the shortcomings of ANNs are not negligible and may induce alterations in many aspects of medical practice. In this review, we introduce basic knowledge about ANNs and summarize the current achievements of ANNs in GI diseases from the perspective of gastroenterologists. Existing limitations and future directions are also proposed to optimize ANN’s clinical potential. In consideration of barriers to interdisciplinary knowledge, sophisticated concepts are discussed using plain words and metaphors to make this review more easily understood by medical practitioners and the general public.

Predictive Values of Preoperative Characteristics for 30-Day Mortality in Traumatic Hip Fracture Patients

Hip fracture patients have a high risk of mortality after surgery, with 30-day postoperative rates as high as 10%. This study aimed to explore the predictive ability of preoperative characteristics in traumatic hip fracture patients as they relate to 30-day postoperative mortality using readily available variables in clinical practice. All adult patients who underwent primary emergency hip fracture surgery in Sweden between 2008 and 2017 were included in the analysis. Associations between the possible predictors and 30-day mortality was performed using a multivariate logistic regression (LR) model; the bidirectional stepwise method was used for variable selection. An LR model and convolutional neural network (CNN) were then fitted for prediction. The relative importance of individual predictors was evaluated using the permutation importance and Gini importance. A total of 134,915 traumatic hip fracture patients were included in the study. The CNN and LR models displayed an acceptable predictive ability for predicting 30-day postoperative mortality using a test dataset, displaying an area under the ROC curve (AUC) of as high as 0.76. The variables with the highest importance in prediction were age, sex, hypertension, dementia, American Society of Anesthesiologists (ASA) classification, and the Revised Cardiac Risk Index (RCRI). Both the CNN and LR models achieved an acceptable performance in identifying patients at risk of mortality 30 days after hip fracture surgery. The most important variables for prediction, based on the variables used in the current study are age, hypertension, dementia, sex, ASA classification, and RCRI.