Predictive Models for Length of Stay and Discharge Disposition in Elective Spine Surgery: Development, Validation, and Comparison to the ACS NSQIP Risk Calculator

The Evolution of Risk Assessment in Spine Surgery: A Narrative Review

BACKGROUND

Risk assessment is critically important in elective and high-risk interventions, particularly spine surgery. This narrative review describes the evolution of risk assessment from the earliest instruments focused on general surgical risk stratification, to more accurate and spine-specific risk calculators that quantified risk, to the current era of big data.

METHODS

The PubMed and SCOPUS databases were queried on October 11, 2023 using search terms to identify risk assessment tools (RATs) in spine surgery. A total of 108 manuscripts were included after screening with full-text review using the following inclusion criteria: 1) study population of adult spine surgical patients, 2) studies describing validation and subsequent performance of preoperative RATs, and 3) studies published in English.

RESULTS

Early RATs provided stratified patients into broad categories and allowed for improved communication between physicians. Subsequent risk calculators attempted to quantify risk by estimating general outcomes such as mortality, but then evolved to estimate spine-specific surgical complications. The integration of novel concepts such as invasiveness, frailty, genetic biomarkers, and sarcopenia led to the development of more sophisticated predictive models that estimate the risk of spine-specific complications and long-term outcomes.

CONCLUSIONS

RATs have undergone a transformative shift from generalized risk stratification to quantitative predictive models. The next generation of tools will likely involve integration of radiographic and genetic biomarkers, machine learning, and artificial intelligence to improve the accuracy of these models and better inform patients, surgeons, and payers.

Development and external validation of a predictive model for prolonged length of hospital stay in elderly patients undergoing lumbar fusion surgery: comparison of three predictive models

PURPOSE

This study aimed to develop a predictive model for prolonged length of hospital stay (pLOS) in elderly patients undergoing lumbar fusion surgery, utilizing multivariate logistic regression, single classification and regression tree (hereafter, "classification tree") and random forest machine-learning algorithms.

METHODS

This study was a retrospective review of a prospective Geriatric Lumbar Disease Database. The primary outcome measure was pLOS, which was defined as the LOS greater than the 75th percentile. All patients were grouped as pLOS group and non-pLOS. Three models (including logistic regression, single-classification tree and random forest algorithms) for predicting pLOS were developed using training dataset and internal validation using testing dataset. Finally, online tool based on our model was developed to assess its validity in the clinical setting (external validation).

RESULTS

The development set included 1025 patients (mean [SD] age, 72.8 [5.6] years; 632 [61.7%] female), and the external validation set included 175 patients (73.2 [5.9] years; 97[55.4%] female). Multivariate logistic analyses revealed that older age (odds ratio [OR] 1.06, p < 0.001), higher BMI (OR 1.08, p = 0.002), number of fused segments (OR 1.41, p < 0.001), longer operative time (OR 1.02, p < 0.001), and diabetes (OR 1.05, p = 0.046) were independent risk factors for pLOS in elderly patients undergoing lumbar fusion surgery. The single-classification tree revealed that operative time ≥ 232 min, delayed ambulation, and BMI ≥ 30 kg/m2 as particularly influential predictors for pLOS. A random forest model was developed using the remaining 14 variables. Intraoperative EBL, operative time, delayed ambulation, age, number of fused segments, BMI, and RBC count were the most significant variables in the final model. The predictive ability of our three models was comparable, with no significant differences in AUC (0.73 vs. 0.71 vs. 0.70, respectively). The logistic regression model had a higher net benefit for clinical intervention than the other models. The nomogram was developed, and the C-index of external validation for PLOS was 0.69 (95% CI, 0.65-0.76).

CONCLUSION

This investigation produced three predictive models for pLOS in elderly patients undergoing lumbar fusion surgery. The predictive ability of our three models was comparable. Logistic regression model had a higher net benefit for clinical intervention than the other models. Our predictive model could inform physicians about elderly patients with a high risk of pLOS after surgery.

Development and external validation of a nomogram for predicting postoperative adverse events in elderly patients undergoing lumbar fusion surgery: comparison of three predictive models

BACKGROUND

The burden of lumbar degenerative diseases (LDD) has increased substantially with the unprecedented aging population. Identifying elderly patients with high risk of postoperative adverse events (AEs) and establishing individualized perioperative management is critical to mitigate added costs and optimize cost-effectiveness to the healthcare system. We aimed to develop a predictive tool for AEs in elderly patients with transforaminal lumbar interbody fusion (TLIF), utilizing multivariate logistic regression, single classification and regression tree (hereafter, "classification tree"), and random forest machine learning algorithms.

METHODS

This study was a retrospective review of a prospective Geriatric Lumbar Disease Database (age ≥ 65). Our outcome measure was postoperative AEs, including prolonged hospital stays, postoperative complications, readmission, and reoperation within 90 days. Patients were grouped as either having at least one adverse event (AEs group) or not (No-AEs group). Three models for predicting postoperative AEs were developed using training dataset and internal validation using testing dataset. Finally, online tool was developed to assess its validity in the clinical setting (external validation).

RESULTS

The development set included 1025 patients (mean [SD] age, 72.8 [5.6] years; 632 [61.7%] female), and the external validation set included 175 patients (73.2 [5.9] years; 97 [55.4%] female). The predictive ability of our three models was comparable, with no significant differences in AUC (0.73 vs. 0.72 vs. 0.70, respectively). The logistic regression model had a higher net benefit for clinical intervention than the other models. A nomogram based on logistic regression was developed, and the C-index of external validation for AEs was 0.69 (95% CI 0.65-0.76).

CONCLUSION

The predictive ability of our three models was comparable. Logistic regression model had a higher net benefit for clinical intervention than the other models. Our nomogram and online tool ( https://xuanwumodel.shinyapps.io/Model_for_AEs/ ) could inform physicians about elderly patients with a high risk of AEs within the 90 days after TLIF surgery.

Machine Learning in Spine Surgery: A Narrative Review

Artificial intelligence and machine learning (ML) can offer revolutionary advances in their application to the field of spine surgery. Within the past 5 years, novel applications of ML have assisted in surgical decision-making, intraoperative imaging and navigation, and optimization of clinical outcomes. ML has the capacity to address many different clinical needs and improve diagnostic and surgical techniques. This review will discuss current applications of ML in the context of spine surgery by breaking down its implementation preoperatively, intraoperatively, and postoperatively. Ethical considerations to ML and challenges in ML implementation must be addressed to maximally benefit patients, spine surgeons, and the healthcare system. Areas for future research in augmented reality and mixed reality, along with limitations in generalizability and bias, will also be highlighted.

Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review

Clinical prediction models for spine surgery applications are on the rise, with an increasing reliance on machine learning (ML) and deep learning (DL). Many of the predicted outcomes are uncommon; therefore, to ensure the models' effectiveness in clinical practice it is crucial to properly evaluate them. This systematic review aims to identify and evaluate current research-based ML and DL models applied for spine surgery, specifically those predicting binary outcomes with a focus on their evaluation metrics. Overall, 60 papers were included, and the findings were reported according to the PRISMA guidelines. A total of 13 papers focused on lengths of stay (LOS), 12 on readmissions, 12 on non-home discharge, 6 on mortality, and 5 on reoperations. The target outcomes exhibited data imbalances ranging from 0.44% to 42.4%. A total of 59 papers reported the model's area under the receiver operating characteristic (AUROC), 28 mentioned accuracies, 33 provided sensitivity, 29 discussed specificity, 28 addressed positive predictive value (PPV), 24 included the negative predictive value (NPV), 25 indicated the Brier score with 10 providing a null model Brier, and 8 detailed the F1 score. Additionally, data visualization varied among the included papers. This review discusses the use of appropriate evaluation schemes in ML and identifies several common errors and potential bias sources in the literature. Embracing these recommendations as the field advances may facilitate the integration of reliable and effective ML models in clinical settings.