1
|
Ton A, Wishart D, Ball JR, Shah I, Murakami K, Ordon MP, Alluri RK, Hah R, Safaee MM. The Evolution of Risk Assessment in Spine Surgery: A Narrative Review. World Neurosurg 2024; 188:1-14. [PMID: 38677646 DOI: 10.1016/j.wneu.2024.04.117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Revised: 04/17/2024] [Accepted: 04/18/2024] [Indexed: 04/29/2024]
Abstract
BACKGROUND Risk assessment is critically important in elective and high-risk interventions, particularly spine surgery. This narrative review describes the evolution of risk assessment from the earliest instruments focused on general surgical risk stratification, to more accurate and spine-specific risk calculators that quantified risk, to the current era of big data. METHODS The PubMed and SCOPUS databases were queried on October 11, 2023 using search terms to identify risk assessment tools (RATs) in spine surgery. A total of 108 manuscripts were included after screening with full-text review using the following inclusion criteria: 1) study population of adult spine surgical patients, 2) studies describing validation and subsequent performance of preoperative RATs, and 3) studies published in English. RESULTS Early RATs provided stratified patients into broad categories and allowed for improved communication between physicians. Subsequent risk calculators attempted to quantify risk by estimating general outcomes such as mortality, but then evolved to estimate spine-specific surgical complications. The integration of novel concepts such as invasiveness, frailty, genetic biomarkers, and sarcopenia led to the development of more sophisticated predictive models that estimate the risk of spine-specific complications and long-term outcomes. CONCLUSIONS RATs have undergone a transformative shift from generalized risk stratification to quantitative predictive models. The next generation of tools will likely involve integration of radiographic and genetic biomarkers, machine learning, and artificial intelligence to improve the accuracy of these models and better inform patients, surgeons, and payers.
Collapse
Affiliation(s)
- Andy Ton
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Danielle Wishart
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Jacob R Ball
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Ishan Shah
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Kiley Murakami
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Matthew P Ordon
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - R Kiran Alluri
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Raymond Hah
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Michael M Safaee
- Department of Neurological Surgery, Keck School of MedicineUniversity of Southern California, Los Angeles, California, USA.
| |
Collapse
|
2
|
Im J, Soliman MAR, Aguirre AO, Quiceno E, Burns E, Khan AMA, Kuo CC, Baig RA, Khan A, Hess RM, Pollina J, Mullin JP. American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator as a Predictor of Postoperative Outcomes After Adult Spinal Deformity Surgery: A Retrospective Cohort Analysis. Neurosurgery 2024:00006123-990000000-01249. [PMID: 38934614 DOI: 10.1227/neu.0000000000003066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 05/01/2024] [Indexed: 06/28/2024] Open
Abstract
BACKGROUND AND OBJECTIVES In recent years, there has been an outpouring of scoring systems that were built to predict outcomes after various surgical procedures; however, research validating these studies in spinal surgery is quite limited. In this study, we evaluated the predictability of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator (ACS NSQIP SRC) for various postoperative outcomes after spinal deformity surgery. METHODS A retrospective chart review was conducted to identify patients who underwent spinal deformity surgery at our hospital between January 1, 2014, and December 31, 2022. Demographic and clinical data necessary to use the ACS NSQIP SRC and postoperative outcomes were collected for these patients. Predictability was analyzed using the area under the curve (AUC) of receiver operating characteristic curves and Brier scores. RESULTS Among the 159 study patients, the mean age was 64.5 ± 9.5 years, mean body mass index was 31.9 ± 6.6, and 95 (59.7%) patients were women. The outcome most accurately predicted by the ACS NSQIP SRC was postoperative pneumonia (observed = 5.0% vs predicted = 3.2%, AUC = 0.75, Brier score = 0.05), but its predictability still fell below the acceptable threshold. Other outcomes that were underpredicted by the ACS NSQIP SRC were readmission within 30 days (observed = 13.8% vs predicted = 9.0%, AUC = 0.63, Brier score = 0.12), rate of discharge to nursing home or rehabilitation facilities (observed = 56.0% vs predicted = 46.6%, AUC = 0.59, Brier = 0.26), reoperation (observed 11.9% vs predicted 5.4%, AUC = 0.60, Brier = 0.11), surgical site infection (observed 9.4% vs predicted 3.5%, AUC = 0.61, Brier = 0.05), and any complication (observed 33.3% vs 19%, AUC = 0.65, Brier = 0.23). Predicted and observed length of stay were not significantly associated (β = 0.132, P = .47). CONCLUSION The ACS NSQIP SRC is a poor predictor of outcomes after spinal deformity surgery.
Collapse
Affiliation(s)
- Justin Im
- Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
| | - Mohamed A R Soliman
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
- Department of Neurosurgery, Faculty of Medicine, Cairo University, Cairo, Egypt
| | - Alexander O Aguirre
- Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
| | - Esteban Quiceno
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
| | - Evan Burns
- Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
| | - Ali M A Khan
- Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
| | - Cathleen C Kuo
- Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
| | - Rehman A Baig
- Current Affiliation: Department of Neurosurgery, Imperial College, London, UK
| | - Asham Khan
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
| | - Ryan M Hess
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
| | - John Pollina
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
| | - Jeffrey P Mullin
- Department of Neurosurgery, Jacobs School of Medicine and Biomedical Sciences at University at Buffalo, Buffalo, New York, USA
- Department of Neurosurgery, Buffalo General Medical Center, Kaleida Health, Buffalo, New York, USA
| |
Collapse
|
3
|
Cohen ME, Liu Y, Hall BL, Ko CY. The Accuracy of the NSQIP Universal Surgical Risk Calculator Compared to Operation-Specific Calculators. ANNALS OF SURGERY OPEN 2023; 4:e358. [PMID: 38144509 PMCID: PMC10735075 DOI: 10.1097/as9.0000000000000358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 10/09/2023] [Indexed: 12/26/2023] Open
Abstract
Objective To compare the performance of the ACS NSQIP "universal" risk calculator (N-RC) to operation-specific RCs. Background Resources have been directed toward building operation-specific RCs because of an implicit belief that they would provide more accurate risk estimates than the N-RC. However, operation-specific calculators may not provide sufficient improvements in accuracy to justify the costs in development, maintenance, and access. Methods For the N-RC, a cohort of 5,020,713 NSQIP patient records were randomly divided into 80% for machine learning algorithm training and 20% for validation. Operation-specific risk calculators (OS-RC) and OS-RCs with operation-specific predictors (OSP-RC) were independently developed for each of 6 operative groups (colectomy, whipple pancreatectomy, thyroidectomy, abdominal aortic aneurysm (open), hysterectomy/myomectomy, and total knee arthroplasty) and 14 outcomes using the same 80%/20% rule applied to the appropriate subsets of the 5M records. Predictive accuracy was evaluated using the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and Hosmer-Lemeshow (H-L) P values, for 13 binary outcomes, and mean squared error for the length of stay outcome. Results The N-RC was found to have greater AUROC (P = 0.002) and greater AUPRC (P < 0.001) compared to the OS-RC. No other statistically significant differences in accuracy, across the 3 risk calculator types, were found. There was an inverse relationship between the operation group sample size and magnitude of the difference in AUROC (r = -0.278; P = 0.014) and in AUPRC (r = -0.425; P < 0.001) between N-RC and OS-RC. The smaller the sample size, the greater the superiority of the N-RC. Conclusions While operation-specific RCs might be assumed to have advantages over a universal RC, their reliance on smaller datasets may reduce their ability to accurately estimate predictor effects. In the present study, this tradeoff between operation specificity and accuracy, in estimating the effects of predictor variables, favors the N-R, though the clinical impact is likely to be negligible.
Collapse
Affiliation(s)
- Mark E. Cohen
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL
| | - Yaoming Liu
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL
| | - Bruce L. Hall
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL
- Department of Surgery, Washington University in St. Louis, Center for Health Policy and the Olin Business School at Washington University in St Louis, John Cochran Veterans Affairs Medical Center; and BJC Healthcare, St. Louis, MO
| | - Clifford Y. Ko
- From the Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL
- Department of Surgery, University of California Los Angeles David Geffen School of Medicine and the VA Greater Los Angeles Healthcare System, Los Angeles
| |
Collapse
|