Comparison of Deep Learning and Classical Machine Learning Algorithms to Predict Postoperative Outcomes for Anterior Cervical Discectomy and Fusion Procedures With State-of-the-art Performance

Machine learning models on a web application to predict short-term postoperative outcomes following anterior cervical discectomy and fusion

BACKGROUND

The frequency of anterior cervical discectomy and fusion (ACDF) has increased up to 400% since 2011, underscoring the need to preoperatively anticipate adverse postoperative outcomes given the procedure's expanding use. Our study aims to accomplish two goals: firstly, to develop a suite of explainable machine learning (ML) models capable of predicting adverse postoperative outcomes following ACDF surgery, and secondly, to embed these models in a user-friendly web application, demonstrating their potential utility.

METHODS

We utilized data from the National Surgical Quality Improvement Program database to identify patients who underwent ACDF surgery. The outcomes of interest were four short-term postoperative adverse outcomes: prolonged length of stay (LOS), non-home discharges, 30-day readmissions, and major complications. We utilized five ML algorithms - TabPFN, TabNET, XGBoost, LightGBM, and Random Forest - coupled with the Optuna optimization library for hyperparameter tuning. To bolster the interpretability of our models, we employed SHapley Additive exPlanations (SHAP) for evaluating predictor variables' relative importance and used partial dependence plots to illustrate the impact of individual variables on the predictions generated by our top-performing models. We visualized model performance using receiver operating characteristic (ROC) curves and precision-recall curves (PRC). Quantitative metrics calculated were the area under the ROC curve (AUROC), balanced accuracy, weighted area under the PRC (AUPRC), weighted precision, and weighted recall. Models with the highest AUROC values were selected for inclusion in a web application.

RESULTS

The analysis included 57,760 patients for prolonged LOS [11.1% with prolonged LOS], 57,780 for non-home discharges [3.3% non-home discharges], 57,790 for 30-day readmissions [2.9% readmitted], and 57,800 for major complications [1.4% with major complications]. The top-performing models, which were the ones built with the Random Forest algorithm, yielded mean AUROCs of 0.776, 0.846, 0.775, and 0.747 for predicting prolonged LOS, non-home discharges, readmissions, and complications, respectively.

CONCLUSIONS

Our study employs advanced ML methodologies to enhance the prediction of adverse postoperative outcomes following ACDF. We designed an accessible web application to integrate these models into clinical practice. Our findings affirm that ML tools serve as vital supplements in risk stratification, facilitating the prediction of diverse outcomes and enhancing patient counseling for ACDF.

Deep Learning Prediction of Cervical Spine Surgery Revision Outcomes Using Standard Laboratory and Operative Variables

BACKGROUND

Cervical spine procedures represent a major proportion of all spine surgery. Mitigating the revision rate following cervical procedures requires careful patient selection. While complication risk has successfully been predicted, revision risk has proven more challenging. This is likely due to the absence of granular variables in claims databases. The objective of this study was to develop a state-of-the-art model of revision prediction of cervical spine surgery using laboratory and operative variables.

METHODS

Using the Stanford Research Repository, patients undergoing a cervical spine procedure between 2016 and 2022 were identified (N = 3151), and recent laboratory values were collected. Patients were classified into separate cohorts by revision outcome and time frame. Machine and deep learning models were trained to predict each revision outcome from laboratory and operative variables.

RESULTS

Red blood cell count, hemoglobin, hematocrit, mean corpuscular hemoglobin concentration, red blood cell distribution width, platelet count, carbon dioxide, anion gap, and calcium all were significantly associated with ≥1 revision cohorts. For the prediction of 3-month revision, the deep neural network achieved an area under the receiver operating characteristic curve of 0.833. The model demonstrated increased performance for anterior versus posterior and arthrodesis versus decompression procedures.

CONCLUSIONS

Our deep learning approach successfully predicted 3-month revision outcomes from demographic variables, standard laboratory values, and operative variables in a cervical spine surgery cohort. This work used standard laboratory values and operative codes as meaningful predictive variables for revision outcome prediction. The increased performance on certain procedures evidences the need for careful development and validation of one-size-fits-all risk scores for spine procedures.

Integrating Artificial Intelligence for Advancing Multiple-Cancer Early Detection via Serum Biomarkers: A Narrative Review

The concept and policies of multicancer early detection (MCED) have gained significant attention from governments worldwide in recent years. In the era of burgeoning artificial intelligence (AI) technology, the integration of MCED with AI has become a prevailing trend, giving rise to a plethora of MCED AI products. However, due to the heterogeneity of both the detection targets and the AI technologies, the overall diversity of MCED AI products remains considerable. The types of detection targets encompass protein biomarkers, cell-free DNA, or combinations of these biomarkers. In the development of AI models, different model training approaches are employed, including datasets of case-control studies or real-world cancer screening datasets. Various validation techniques, such as cross-validation, location-wise validation, and time-wise validation, are used. All of the factors show significant impacts on the predictive efficacy of MCED AIs. After the completion of AI model development, deploying the MCED AIs in clinical practice presents numerous challenges, including presenting the predictive reports, identifying the potential locations and types of tumors, and addressing cancer-related information, such as clinical follow-up and treatment. This study reviews several mature MCED AI products currently available in the market, detecting their composing factors from serum biomarker detection, MCED AI training/validation, and the clinical application. This review illuminates the challenges encountered by existing MCED AI products across these stages, offering insights into the continued development and obstacles within the field of MCED AI.

Evaluating Computer Vision, Large Language, and Genome-Wide Association Models in a Limited Sized Patient Cohort for Pre-Operative Risk Stratification in Adult Spinal Deformity Surgery

Background: Adult spinal deformities (ASD) are varied spinal abnormalities, often necessitating surgical intervention when associated with pain, worsening deformity, or worsening function. Predicting post-operative complications and revision surgery is critical for surgical planning and patient counseling. Due to the relatively small number of cases of ASD surgery, machine learning applications have been limited to traditional models (e.g., logistic regression or standard neural networks) and coarse clinical variables. We present the novel application of advanced models (CNN, LLM, GWAS) using complex data types (radiographs, clinical notes, genomics) for ASD outcome prediction. Methods: We developed a CNN trained on 209 ASD patients (1549 radiographs) from the Stanford Research Repository, a CNN pre-trained on VinDr-SpineXR (10,468 spine radiographs), and an LLM using free-text clinical notes from the same 209 patients, trained via Gatortron. Additionally, we conducted a GWAS using the UK Biobank, contrasting 540 surgical ASD patients with 7355 non-surgical ASD patients. Results: The LLM notably outperformed the CNN in predicting pulmonary complications (F1: 0.545 vs. 0.2881), neurological complications (F1: 0.250 vs. 0.224), and sepsis (F1: 0.382 vs. 0.132). The pre-trained CNN showed improved sepsis prediction (AUC: 0.638 vs. 0.534) but reduced performance for neurological complication prediction (AUC: 0.545 vs. 0.619). The LLM demonstrated high specificity (0.946) and positive predictive value (0.467) for neurological complications. The GWAS identified 21 significant (p < 10-5) SNPs associated with ASD surgery risk (OR: mean: 3.17, SD: 1.92, median: 2.78), with the highest odds ratio (8.06) for the LDB2 gene, which is implicated in ectoderm differentiation. Conclusions: This study exemplifies the innovative application of cutting-edge models to forecast outcomes in ASD, underscoring the utility of complex data in outcome prediction for neurosurgical conditions. It demonstrates the promise of genetic models when identifying surgical risks and supports the integration of complex machine learning tools for informed surgical decision-making in ASD.

Exploring the Landscape of Machine Learning Applications in Neurosurgery: A Bibliometric Analysis and Narrative Review of Trends and Future Directions

BACKGROUND

The field of neurosurgery has consistently represented an area of innovation and integration of technology since its inception. As such, machine learning (ML) has found its way into applications within neurosurgery relatively rapidly. Through this bibliometric review and cluster analysis, we seek to identify trends and emerging applications of ML within neurosurgery.

METHODS

A bibliometric analysis was carried out in the Web of Science database on publications from January 2000 to March 2023. The full data set of the 200 most cited publications including title, author information, journal, citation count, keywords, and abstracts for each publication was evaluated in CiteSpace. CiteSpace was used to elucidate publication characteristics, trends, and topic clusters via collaborate network analysis using the Kamada-Kawai algorithm.

RESULTS

The 25 most cited titles were included in our analysis. Harvard University and its affiliates represented the top institution, contributing nearly 25% of publications in the literature. WORLD NEUROSURGERY was the journal with the highest net citation count of 747 (29%). Collaborative network analysis generated 12 unique clusters, the largest of which was machine learning, followed by feature importance and deep brain stimulation.

CONCLUSION

This review highlights the most impactful articles pertaining to ML in the field of neurosurgery. ML has been applied into several sub-specialties within neurosurgery to optimize patient care, with special attention to outcome predictors, patient selection, and surgical decision making.

Machine Learning in Neurosurgery: Toward Complex Inputs, Actionable Predictions, and Generalizable Translations

Machine learning can predict neurosurgical diagnosis and outcomes, power imaging analysis, and perform robotic navigation and tumor labeling. State-of-the-art models can reconstruct and generate images, predict surgical events from video, and assist in intraoperative decision-making. In this review, we will detail the neurosurgical applications of machine learning, ranging from simple to advanced models, and their potential to transform patient care. As machine learning techniques, outputs, and methods become increasingly complex, their performance is often more impactful yet increasingly difficult to evaluate. We aim to introduce these advancements to the neurosurgical audience while suggesting major potential roadblocks to their safe and effective translation. Unlike the previous generation of machine learning in neurosurgery, the safe translation of recent advancements will be contingent on neurosurgeons' involvement in model development and validation.

Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review

Clinical prediction models for spine surgery applications are on the rise, with an increasing reliance on machine learning (ML) and deep learning (DL). Many of the predicted outcomes are uncommon; therefore, to ensure the models' effectiveness in clinical practice it is crucial to properly evaluate them. This systematic review aims to identify and evaluate current research-based ML and DL models applied for spine surgery, specifically those predicting binary outcomes with a focus on their evaluation metrics. Overall, 60 papers were included, and the findings were reported according to the PRISMA guidelines. A total of 13 papers focused on lengths of stay (LOS), 12 on readmissions, 12 on non-home discharge, 6 on mortality, and 5 on reoperations. The target outcomes exhibited data imbalances ranging from 0.44% to 42.4%. A total of 59 papers reported the model's area under the receiver operating characteristic (AUROC), 28 mentioned accuracies, 33 provided sensitivity, 29 discussed specificity, 28 addressed positive predictive value (PPV), 24 included the negative predictive value (NPV), 25 indicated the Brier score with 10 providing a null model Brier, and 8 detailed the F1 score. Additionally, data visualization varied among the included papers. This review discusses the use of appropriate evaluation schemes in ML and identifies several common errors and potential bias sources in the literature. Embracing these recommendations as the field advances may facilitate the integration of reliable and effective ML models in clinical settings.

Using Machine Learning and Deep Learning Algorithms to Predict Postoperative Outcomes Following Anterior Cervical Discectomy and Fusion

STUDY DESIGN

A retrospective cohort study from a multisite academic medical center.

OBJECTIVE

To construct, evaluate, and interpret a series of machine learning models to predict outcomes related to inpatient health care resource utilization for patients undergoing anterior cervical discectomy and fusion (ACDF).

SUMMARY OF BACKGROUND DATA

Reducing postoperative health care utilization is an important goal for improving the delivery of surgical care and serves as a metric for quality assessment. Recent data has shown marked hospital resource utilization after ACDF surgery, including readmissions, and ED visits. The burden of postoperative health care use presents a potential application of machine learning techniques, which may be capable of accurately identifying at-risk patients using patient-specific predictors.

METHODS

Patients 18-88 years old who underwent ACDF from 2011 to 2021 at a multisite academic center and had preoperative lab values within 3 months of surgery were included. Outcomes analyzed included 90-day readmissions, postoperative length of stay, and nonhome discharge. Four machine learning models-Extreme Gradient Boosted Trees, Balanced Random Forest, Elastic-Net Penalized Logistic Regression, and a Neural Network-were trained and evaluated through the Area Under the Curve estimates. Feature importance scores were computed for the highest-performing model per outcome through model-specific metrics.

RESULTS

A total of 1026 cases were included in the analysis cohort. All machine learning models were predictive for outcomes of interest, with the Random Forest algorithm consistently demonstrating the strongest average area under the curve performance, with a peak performance of 0.84 for nonhome discharge. Important features varied per outcome, though age, body mass index, American Society of Anesthesiologists classification >2, and medical comorbidities were highly weighted in the studied outcomes.

CONCLUSIONS

Machine learning models were successfully applied and predictive of postoperative health utilization after ACDF. Deployment of these tools can assist clinicians in determining high-risk patients.

LEVEL OF EVIDENCE

III.