Utility of Surgical Risk Calculators in Spine Surgery in Patients Aged Over 80 Years: Analysis of SpineSage and ACS NSQIP

The Evolution of Risk Assessment in Spine Surgery: A Narrative Review

BACKGROUND

Risk assessment is critically important in elective and high-risk interventions, particularly spine surgery. This narrative review describes the evolution of risk assessment from the earliest instruments focused on general surgical risk stratification, to more accurate and spine-specific risk calculators that quantified risk, to the current era of big data.

METHODS

The PubMed and SCOPUS databases were queried on October 11, 2023 using search terms to identify risk assessment tools (RATs) in spine surgery. A total of 108 manuscripts were included after screening with full-text review using the following inclusion criteria: 1) study population of adult spine surgical patients, 2) studies describing validation and subsequent performance of preoperative RATs, and 3) studies published in English.

RESULTS

Early RATs provided stratified patients into broad categories and allowed for improved communication between physicians. Subsequent risk calculators attempted to quantify risk by estimating general outcomes such as mortality, but then evolved to estimate spine-specific surgical complications. The integration of novel concepts such as invasiveness, frailty, genetic biomarkers, and sarcopenia led to the development of more sophisticated predictive models that estimate the risk of spine-specific complications and long-term outcomes.

CONCLUSIONS

RATs have undergone a transformative shift from generalized risk stratification to quantitative predictive models. The next generation of tools will likely involve integration of radiographic and genetic biomarkers, machine learning, and artificial intelligence to improve the accuracy of these models and better inform patients, surgeons, and payers.

American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator as a Predictor of Postoperative Outcomes After Adult Spinal Deformity Surgery: A Retrospective Cohort Analysis

BACKGROUND AND OBJECTIVES

In recent years, there has been an outpouring of scoring systems that were built to predict outcomes after various surgical procedures; however, research validating these studies in spinal surgery is quite limited. In this study, we evaluated the predictability of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator (ACS NSQIP SRC) for various postoperative outcomes after spinal deformity surgery.

METHODS

A retrospective chart review was conducted to identify patients who underwent spinal deformity surgery at our hospital between January 1, 2014, and December 31, 2022. Demographic and clinical data necessary to use the ACS NSQIP SRC and postoperative outcomes were collected for these patients. Predictability was analyzed using the area under the curve (AUC) of receiver operating characteristic curves and Brier scores.

RESULTS

Among the 159 study patients, the mean age was 64.5 ± 9.5 years, mean body mass index was 31.9 ± 6.6, and 95 (59.7%) patients were women. The outcome most accurately predicted by the ACS NSQIP SRC was postoperative pneumonia (observed = 5.0% vs predicted = 3.2%, AUC = 0.75, Brier score = 0.05), but its predictability still fell below the acceptable threshold. Other outcomes that were underpredicted by the ACS NSQIP SRC were readmission within 30 days (observed = 13.8% vs predicted = 9.0%, AUC = 0.63, Brier score = 0.12), rate of discharge to nursing home or rehabilitation facilities (observed = 56.0% vs predicted = 46.6%, AUC = 0.59, Brier = 0.26), reoperation (observed 11.9% vs predicted 5.4%, AUC = 0.60, Brier = 0.11), surgical site infection (observed 9.4% vs predicted 3.5%, AUC = 0.61, Brier = 0.05), and any complication (observed 33.3% vs 19%, AUC = 0.65, Brier = 0.23). Predicted and observed length of stay were not significantly associated (β = 0.132, P = .47).

CONCLUSION

The ACS NSQIP SRC is a poor predictor of outcomes after spinal deformity surgery.

The Accuracy of the NSQIP Universal Surgical Risk Calculator Compared to Operation-Specific Calculators

Objective

To compare the performance of the ACS NSQIP "universal" risk calculator (N-RC) to operation-specific RCs.

Background

Resources have been directed toward building operation-specific RCs because of an implicit belief that they would provide more accurate risk estimates than the N-RC. However, operation-specific calculators may not provide sufficient improvements in accuracy to justify the costs in development, maintenance, and access.

Methods

For the N-RC, a cohort of 5,020,713 NSQIP patient records were randomly divided into 80% for machine learning algorithm training and 20% for validation. Operation-specific risk calculators (OS-RC) and OS-RCs with operation-specific predictors (OSP-RC) were independently developed for each of 6 operative groups (colectomy, whipple pancreatectomy, thyroidectomy, abdominal aortic aneurysm (open), hysterectomy/myomectomy, and total knee arthroplasty) and 14 outcomes using the same 80%/20% rule applied to the appropriate subsets of the 5M records. Predictive accuracy was evaluated using the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and Hosmer-Lemeshow (H-L) P values, for 13 binary outcomes, and mean squared error for the length of stay outcome.

Results

The N-RC was found to have greater AUROC (P = 0.002) and greater AUPRC (P < 0.001) compared to the OS-RC. No other statistically significant differences in accuracy, across the 3 risk calculator types, were found. There was an inverse relationship between the operation group sample size and magnitude of the difference in AUROC (r = -0.278; P = 0.014) and in AUPRC (r = -0.425; P < 0.001) between N-RC and OS-RC. The smaller the sample size, the greater the superiority of the N-RC.

Conclusions

While operation-specific RCs might be assumed to have advantages over a universal RC, their reliance on smaller datasets may reduce their ability to accurately estimate predictor effects. In the present study, this tradeoff between operation specificity and accuracy, in estimating the effects of predictor variables, favors the N-R, though the clinical impact is likely to be negligible.