Examining the validity of the ACS-NSQIP Risk Calculator in plastic surgery: lack of input specificity, outcome variability and imprecise risk calculations

Performance and usability of pre-operative prediction models for 30-day peri-operative mortality risk: a systematic review

Estimating pre-operative mortality risk may inform clinical decision-making for peri-operative care. However, pre-operative mortality risk prediction models are rarely implemented in routine clinical practice. High predictive accuracy and clinical usability are essential for acceptance and clinical implementation. In this systematic review, we identified and appraised prediction models for 30-day postoperative mortality in non-cardiac surgical cohorts. PubMed and Embase were searched up to December 2022 for studies investigating pre-operative prediction models for 30-day mortality. We assessed predictive performance in terms of discrimination and calibration. Risk of bias was evaluated using a tool to assess the risk of bias and applicability of prediction model studies. To further inform potential adoption, we also assessed clinical usability for selected models. In all, 15 studies evaluating 10 prediction models were included. Discrimination ranged from a c-statistic of 0.82 (MySurgeryRisk) to 0.96 (extreme gradient boosting machine learning model). Calibration was reported in only six studies. Model performance was highest for the surgical outcome risk tool (SORT) and its external validations. Clinical usability was highest for the surgical risk pre-operative assessment system. The SORT and risk quantification index also scored high on clinical usability. We found unclear or high risk of bias in the development of all models. The SORT showed the best combination of predictive performance and clinical usability and has been externally validated in several heterogeneous cohorts. To improve clinical uptake, full integration of reliable models with sufficient face validity within the electronic health record is imperative.

Predictive Models for Length of Stay and Discharge Disposition in Elective Spine Surgery: Development, Validation, and Comparison to the ACS NSQIP Risk Calculator

STUDY DESIGN

A retrospective study at a single academic institution.

OBJECTIVE

The purpose of this study is to utilize machine learning to predict hospital length of stay (LOS) and discharge disposition following adult elective spine surgery, and to compare performance metrics of machine learning models to the American College of Surgeon's National Surgical Quality Improvement Program's (ACS NSQIP) prediction calculator.

SUMMARY OF BACKGROUND DATA

A total of 3678 adult patients undergoing elective spine surgery between 2014 and 2019, acquired from the electronic health record.

METHODS

Patients were divided into three stratified cohorts: cervical degenerative, lumbar degenerative, and adult spinal deformity groups. Predictive variables included demographics, body mass index, surgical region, surgical invasiveness, surgical approach, and comorbidities. Regression, classification trees, and least absolute shrinkage and selection operator (LASSO) were used to build predictive models. Validation of the models was conducted on 16% of patients (N=587), using area under the receiver operator curve (AUROC), sensitivity, specificity, and correlation. Patient data were manually entered into the ACS NSQIP online risk calculator to compare performance. Outcome variables were discharge disposition (home vs. rehabilitation) and LOS (days).

RESULTS

Of 3678 patients analyzed, 51.4% were male (n=1890) and 48.6% were female (n=1788). The average LOS was 3.66 days. In all, 78% were discharged home and 22% discharged to rehabilitation. Compared with NSQIP (Pearson R2 =0.16), the predictions of poisson regression ( R2 =0.29) and LASSO ( R2 =0.29) models were significantly more correlated with observed LOS ( P =0.025 and 0.004, respectively). Of the models generated to predict discharge location, logistic regression yielded an AUROC of 0.79, which was statistically equivalent to the AUROC of 0.75 for NSQIP ( P =0.135).

CONCLUSION

The predictive models developed in this study can enable accurate preoperative estimation of LOS and risk of rehabilitation discharge for adult patients undergoing elective spine surgery. The demonstrated models exhibited better performance than NSQIP for prediction of LOS and equivalent performance to NSQIP for prediction of discharge location.

Evaluating the Inaccuracy of the National Surgical Quality Improvement Project Surgical Risk Calculator in Plastic Surgery: A Meta-analysis of Short-Term Predicted Complications

BACKGROUND

Preoperative surgical risk assessment is a major component of clinical decision making. The ability to provide accurate, individualized risk estimates has become critical because of growing emphasis on quality metrics benchmarks. The American College of Surgeons National Surgical Quality Improvement Project (NSQIP) Surgical Risk Calculator (SRC) was designed to quantify patient-specific risk across various surgeries. Its applicability to plastic surgery is unclear, however, with multiple studies reporting inaccuracies among certain patient populations. This study uses meta-analysis to evaluate the NSQIP SRC's ability to predict complications among patients having plastic surgery.

METHODS

OVID MEDLINE and PubMed were searched for all studies evaluating the predictive accuracy of the NSQIP SRC in plastic surgery, including oncologic reconstruction, ventral hernia repair, and body contouring. Only studies directly comparing SCR predicted to observed complication rates were included. The primary measure of SRC prediction accuracy, area under the curve (AUC), was assessed for each complication via DerSimonian and Laird random-effects analytic model. The I2 statistic, indicating heterogeneity, was judged low (I2 < 50%) or borderline/unacceptably high (I2 > 50%). All analyses were conducted in StataSE 16.1 (StataCorp LP, College Station, Tex).

RESULTS

Ten of the 296 studies screened met criteria for inclusion (2416 patients). Studies were classified as follows: (head and neck: n = 5, breast: n = 1, extremity: n = 1), open ventral hernia repair (n = 2), and panniculectomy (n = 1). Predictive accuracy was poor for medical and surgical complications (medical: pulmonary AUC = 0.67 [0.48-0.87], cardiac AUC = 0.66 [0.20-0.99], venous thromboembolism AUC = 0.55 [0.47-0.63]), (surgical: surgical site infection AUC = 0.55 [0.46-0.63], reoperation AUC = 0.54 [0.49-0.58], serious complication AUC = 0.58 [0.43-0.73], and any complication AUC = 0.60 [0.57-0.64]). Although mortality was accurately predicted in 2 studies (AUC = 0.87 [0.54-0.99]), heterogeneity was high with I2 = 68%. Otherwise, heterogeneity was minimal (I2 = 0%) or acceptably low (I2 < 50%) for all other outcomes.

CONCLUSIONS

The NSQIP Universal SRC, aimed at offering individualized quantifiable risk estimates for surgical complications, consistently demonstrated poor risk discrimination in this plastic surgery-focused meta-analysis. The limitations of the SRC are perhaps most pronounced where complex, multidisciplinary reconstructions are needed. Future efforts should identify targets for improving SRC reliability to better counsel patients in the perioperative setting and guide appropriate healthcare resource allocation.

Effective Communication of Personalized Risks and Patient Preferences During Surgical Informed Consent Using Data Visualization: Qualitative Semistructured Interview Study With Patients After Surgery

Background

There is no consensus on which risks to communicate to a prospective surgical patient during informed consent or how. Complicating the process, patient preferences may diverge from clinical assumptions and are often not considered for discussion. Such discrepancies can lead to confusion and resentment, raising the potential for legal action. To overcome these issues, we propose a visual consent tool that incorporates patient preferences and communicates personalized risks to patients using data visualization. We used this platform to identify key effective visual elements to communicate personalized surgical risks.

Objective

Our main focus is to understand how to best communicate personalized risks using data visualization. To contextualize patient responses to the main question, we examine how patients perceive risks before surgery (research question 1), how suitably the visual consent tool is able to present personalized surgical risks (research question 2), how well our visualizations convey those personalized surgical risks (research question 3), and how the visual consent tool could improve the informed consent process and how it can be used (research question 4).

Methods

We designed a visual consent tool to meet the objectives of our study. To calculate and list personalized surgical risks, we used the American College of Surgeons risk calculator. We created multiple visualization mock-ups using visual elements previously determined to be well-received for risk communication. Semistructured interviews were conducted with patients after surgery, and each of the mock-ups was presented and evaluated independently and in the context of our visual consent tool design. The interviews were transcribed, and thematic analysis was performed to identify major themes. We also applied a quantitative approach to the analysis to assess the prevalence of different perceptions of the visualizations presented in our tool.

Results

In total, 20 patients were interviewed, with a median age of 59 (range 29-87) years. Thematic analysis revealed factors that influenced the perception of risk (the surgical procedure, the cognitive capacity of the patient, and the timing of consent; research question 1); factors that influenced the perceived value of risk visualizations (preference for rare event communication, preference for risk visualization, and usefulness of comparison with the average; research question 3); and perceived usefulness and use cases of the visual consent tool (research questions 2 and 4). Most importantly, we found that patients preferred the visual consent tool to current text-based documents and had no unified preferences for risk visualization. Furthermore, our findings suggest that patient concerns were not often represented in existing risk calculators.

Conclusions

We identified key elements that influence effective visual risk communication in the perioperative setting and pointed out the limitations of the existing calculators in addressing patient concerns. Patient preference is highly variable and should influence choices regarding risk presentation and visualization.

Evaluating Discrimination of ACS-NSQIP Surgical Risk Calculator in Thyroidectomy Patients

BACKGROUND

The ACS-NSQIP surgical risk calculator (SRC) often guides preoperative counseling, but the rarity of complications in certain populations causes class imbalance, complicating risk prediction. We aimed to compare the performance of the ACS-NSQIP SRC to other classical machine learning algorithms trained on NSQIP data, and to demonstrate challenges and strategies in predicting such rare events.

METHODS

Data from the NSQIP thyroidectomy module ys 2016 - 2018 were used to train logistic regression, Ridge regression and Random Forest classifiers for predicting 2 different composite outcomes of surgical risk (systemic and thyroidectomy-specific). We implemented techniques to address imbalanced class sizes and reported the area under the receiver operating characteristic (AUC) for each classifier including the ACS-NSQIP SRC, along with sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) at a 5% - 15% predicted risk threshold.

RESULTS

Of 18,078 included patients, 405 (2.24%) patients suffered systemic complications and 1670 (9.24%) thyroidectomy-specific complications. Logistic regression performed best for predicting systemic complication risk (AUC 0.723 [0.658 - 0.778]); Random Forest with RUSBoost performed best for predicting thyroidectomy-specific complication risk (0.702; 0.674 - 0.726). The addition of optimizations for class imbalance improved performance for all classifiers.

CONCLUSIONS

Complications are rare after thyroidectomy even when considered as composite outcomes, and class imbalance poses a challenge in surgical risk prediction. Using the SRC as a classifier where intervention occurs above a certain validated threshold, rather than citing the numeric estimates of complication risk, should be considered in low-risk patients.

Predicted versus actual complications in Australian women undergoing post-mastectomy breast reconstruction: a retrospective cohort study using the BRA Score tool

INTRODUCTION

The Breast Reconstruction Risk Assessment (BRA) Score tool is a risk calculator developed to predict the risk of complications in individual patients undergoing breast reconstruction. It was developed in a North American population exclusively undergoing immediate breast reconstruction. This study sought to assess the predictions of the BRA Score tool against the measured outcomes of surgery for an Australian public hospital population, including both immediate and delayed reconstructions.

METHOD

This was a retrospective cohort study of data from women at a single Australian public teaching hospital unit. Data from the Flinders Breast Reconstruction Database was retrieved and compared to BRA Scores calculated for each patient. Receiver operating curve area under the curve analysis was performed as well as Brier scores to compare predicted versus observed complications.

RESULTS

BRA Score predictions were reasonable or good (C-statistic >0.7, Brier score <0.09) for the complications of overall surgical complications, surgical site infection (SSI) and seroma at 30 days, and similarly accurate for prediction of the same complications for implant reconstructions at 12 months. There were similar findings between delayed and immediate reconstructions.

CONCLUSION

The BRA Score risk calculator is valid to detect some risks in both patients undergoing immediate and delayed breast reconstruction in an Australian public hospital setting. SSI is the best predicted complication and is well-predicted across both autologous and prosthetic reconstruction types.

A Critical Examination of Length of Stay in Autologous Breast Reconstruction: A National Surgical Quality Improvement Program Analysis

BACKGROUND

This study aims to use the National Surgical Quality Improvement Program database to identify factors associated with extended postoperative length of stay after breast reconstruction with free tissue transfer.

METHODS

Consecutive cases of breast reconstruction with free tissue transfer were retrieved from the National Surgical Quality Improvement Program (2005 to 2017) database using CPT code 19364. Extended length of stay (dependent variable) was defined as greater than 5 days.

RESULTS

Nine thousand six hundred eighty-six cases were analyzed; extended length of stay was noted in 34 percent. On regression, patient factors independently associated with extended length of stay were body mass index (OR, 1.5; 95 percent CI, 1.2 to 1.9; p < 0.001), diabetes (OR, 1.3; 95 percent CI, 1.1 to 1.6; p = 0.003), and malignancy history (OR, 1.9; 95 percent CI, 1.22 to 3.02; p = 0.005). Operation time greater than 500 minutes (OR, 3; 95 percent CI, 2.73 to 3.28; p < 0.001) and immediate postmastectomy reconstruction (OR, 1.7; 95 percent CI, 1.16 to 2.48; p < 0.001) conferred risk for extended length of stay. Bilateral free tissue transfer was not significant. Operations performed in 2017 were at lower risk (OR, 0.2; 95 percent CI, 0.06 to 0.81; p = 0.02) for extended length of stay. Reoperation is more likely following operative transfusion and bilateral free tissue transfers, but less likely following concurrent alloplasty. Given a known operation time (minutes), postoperative length of stay (days) can be calculated using the following equation: length of stay = 2.559 + 0.003 × operation time.

CONCLUSIONS

This study characterizes the risks for extended length of stay after free tissue transfer breast reconstruction using a prospective multicenter national database. The result of this study can be used to risk-stratify patients during surgical planning to optimize perioperative decision-making.

CLINICAL QUESTION/LEVEL OF EVIDENCE

Risk, III.

Assessing the Performance of the De Novo Postoperative Stress Urinary Incontinence Calculator

OBJECTIVE

The aim of the study was to determine the performance of a previously published stress urinary incontinence (SUI) risk calculator in women undergoing minimally invasive or transvaginal apical suspensions.

METHODS

Using a database of stress-continent women who underwent minimally invasive or transvaginal apical suspensions, we calculated 2 prediction risks for development of SUI within 12 months based on inclusion of a "prophylactic" midurethral sling at the time of prolapse surgery. Observed subjective and objective continence status was abstracted from medical records. Regression models were created for the outcome of de novo SUI to generate receiver operating curves. Concordance (c) indices were estimated for the overall and procedure subgroups to determine the calculator's ability to discriminate between SUI outcomes.

RESULTS

Analyses included 502 women. De novo SUI was observed in 23.5% of women. The mean ± SD calculated risk of de novo SUI if a sling was performed was 18.9% ± 13.9 at 12 months compared with 36.4% ± 8.3 without sling. The calculator's discriminative ability for those with a planned sling was moderate (c-index = 0.55, P = 0.037). The calculator failed to discriminate continence outcomes when a sling was not planned in the overall group (c-index = 0.50, P = 0.799) and individual apical procedures.

CONCLUSIONS

The SUI risk calculator is significantly limited in its ability to predict de novo SUI in our population of women planning minimally invasive apical suspensions. Refinements to the calculator model are needed to improve its utility in clinical practice.

Performance of the American College of Surgeons NSQIP Surgical Risk Calculator for Total Gastrectomy

BACKGROUND

To encourage implementation of the American College of Surgeons (ACS) NSQIP Risk Calculator for total gastrectomy for gastric cancer, its predictive performance for this specific procedure should be validated. We assessed its discriminatory accuracy and goodness of fit for predicting 12 adverse outcomes.

STUDY DESIGN

Data were collected on all patients with gastric cancer who underwent total gastrectomy with curative intent at Memorial Sloan Kettering Cancer Center between 2002 and 2017. Preoperative risk factors from the electronic medical record were manually inserted into the ACS-NSQIP Risk Calculator. Predictions for adverse outcomes were compared with observed outcomes by Brier scores, c-statistics, and Hosmer-Lemeshow p value.

RESULTS

In a total of 452 patients, the predicted rate of all complications (29%) was lower than the observed rate (45%). Brier scores varied between 0.017 for death and 0.272 for any complication. C-statistics were moderate (0.7-0.8) for death and renal failure, good (0.8-0.9) for cardiac complication, and excellent (≥0.9) for discharge to nursing or rehabilitation facility. Hosmer-Lemeshow p value found poor goodness of fit for pneumonia only.

CONCLUSIONS

For adverse outcomes after total gastrectomy with curative intent in gastric cancer patients, performance of the ACS-NSQIP Risk Calculator is variable. Its predictive performance is best for cardiac complications, renal failure, death, and discharge to nursing or rehabilitation facility.

Predicting Postoperative Complications after Acute Care Surgery: How Accurate is the ACS NSQIP Surgical Risk Calculator?

The ACS NSQIP Surgical Risk Calculator (SRC) is an evidence-based clinical tool commonly used for evaluating postoperative risk. The goal of this study was to validate SRC-predicted complications by comparing them with observed outcomes in the acute care surgical setting. In this study, pre- and postoperative data from 1693 acute care surgeries (hernia repair, enterolysis, intestinal incision/excision and enterectomy, gastrectomy, debridement, colectomy, appendectomy, cholecystectomy, gastrorrhaphy, and incision and drainage of soft tissue, breast abscesses, and removal of foreign bodies) performed at a Level I trauma center over a five-year time period were abstracted. Predictions for any and serious complications were based on SRC were compared with observed outcomes using various measures of diagnostic. When evaluated as one group, the SRC had good discriminative power for predicting any and serious complications after acute care surgeries (Area Under the Curve (AUC) 0.79, 0.81). In addition, the SRC met Brier score requirements for an informative model overall. However, the predictive accuracy of the SRC varied for various procedures within the acute care patient population. For serious complications, the diagnostic measures ranged from an AUC of 0.61 and negative likelihood ratio of 0.716 for incision & drainage soft tissue to AUC of 0.91 and negative likelihood ratio of 0.064 for gastrorrhaphy. Length of stay was significantly underestimated by the SRC overall (8.56 days, P < 0.01) and for individual procedures. The SRC performs well at predicting complications after acute care surgeries overall; however, there is great variability in performance between procedure types. Further refinements in risk stratification may improve SRC predictions.

A Comparative Analysis of Surgical Wound Infection Methods: Predictive Values of the CDC, ASEPSIS, and Southampton Scoring Systems in Evaluating Breast Reconstruction Surgical Site Infections

Infection is the most significant complication in breast reconstruction surgery. While the Center for Disease Control and Prevention (CDC) is the most prevalent tool for surgical site infection (SSI) diagnosis, ASEPSIS and Southampton scoring methods have been speculated to be more sensitive. The ASEPSIS scoring system previously demonstrated much better interrater reliability than the CDC. We sought to assess the predictive value of various wound scoring methods in breast reconstruction SSIs. A retrospective analysis of all single-institution breast reconstruction infections from January 2013 to June 2016 was performed. Patients' postoperative wound-related complications were collected. Southampton, CDC, and modified ASEPSIS scores-extended to 30 postoperative days-were calculated. Relative predictive values for implant-based reconstruction were evaluated. Among the 22 reviewed cases, ASEPSIS scores greater than 30 resulted in a more than 50% rate of implant-based breast reconstruction failure. There was a significant positive correlation between ASEPSIS score and failure rate (P = .022). A Southampton classification of B-minor complication (60% failure)-had a greater associative risk of reconstruction failure than a classification of C-major complication (23% failure)-or classification of D-major hematoma (0% failure). The CDC score had no predictive value of success versus failure of reconstruction. While the CDC criteria and Southampton scoring systems demonstrated little clinical use, the ASEPSIS scoring system shows substantial predictive value for breast reconstruction SSIs. New procedure protocols should be implemented to require detailed surgical notes including the proportion of the wounds affected by inflammatory responses to allow for easier wound score calculation by these alternate scoring systems.

Scoping Review of the National Surgical Quality Improvement Program in Plastic Surgery Research

Background

The National Surgical Quality Improvement Program (NSQIP) is a robust, high-quality surgical outcomes database that measures risk-adjusted 30-day outcomes of surgical interventions. The purpose of this scoping review is to describe how the NSQIP is being used in plastic surgery research.

Methods

A comprehensive electronic literature search was completed in PubMed, Embase, MEDLINE, and CINAHL. Two reviewers independently reviewed articles to determine their relevance using predefined inclusion criteria. Articles were included if they utilized NSQIP data to conduct research in a domain of plastic surgery or analyzed surgical procedures completed by plastic surgeons. Extracted information included the domain of plastic surgery, country of origin, journal, and year of publication.

Results

A total of 106 articles met the inclusion criteria. The most common domain of plastic surgery was breast reconstruction representing 35% of the articles. Of the 106 articles, 95% were published within the last 5 years. The Plastic and Reconstructive Surgery journal published most of the (59%) NSQIP-related articles. All of the studies were retrospective. Of note, there were no articles on burns and only one study on trauma as the domain of plastic surgery.

Conclusion

This scoping review describes how NSQIP data are being used to analyze plastic surgery interventions and outcomes in order to guide quality improvement in 106 articles. It demonstrates the utility of NSQIP in the literature, however also identifies some limitations of the program as it applies to plastic surgery.

Early Postoperative Morbidity After Chronic Subdural Hematoma: Predictive Usefulness of the Physiological and Operative Severity Score for Enumeration of Mortality and Morbidity, American College of Surgeons National Surgical Quality Improvement Program, and American Society of Anesthesiologists Grade in a Prospective Cohort

BACKGROUND

Although chronic subdural hematoma (CSDH) is generally benign, long-term survival (LTS) after CSDH is poor in a significant subgroup. This dichotomy has been compared to fractured neck of femur. However, although early postoperative mortality (within 30 days of CSDH) is well recorded with CSDH and similar to fractured neck of femur (4%-8%), scant accurate data exist regarding early postoperative morbidity (POMB). POMB, which prolongs length of stay (LOS) after major nonneurosurgery, is associated with decreased LTS. One recent CSDH study suggested a POMB standard of 10% i.e., notably less than with fractured neck of femur (45%).

METHODS

POMB was recorded in a novel prospective single-center cohort after CSDH. The POSSUM (Physiological and Operative Severity Score for Enumeration of Mortality and Morbidity), American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) score, and American Society of Anesthesiologists (ASA) grade were assessed as tools for potentially predicting POMB. Receiver operating characteristic (ROC) curves were calculated.

RESULTS

Early postoperative mortality (within 30 days of CSDH) occurred in 3 of 114 patients (3%). Seventy-one POMB events occurred in 54 of 114 patients (47%), with 27 of 54 (50%) having a Clavien-Dindo grade ≥2. Most POMB was neurologic (n = 47/71, 66%). Age (P = 0.01), Glasgow Coma Scale (GCS) score (P = 0.001), Markwalder grade (P = 0.01), hypertension (P = 0.047), and/or ≥1 preexistent comorbidity (P = 0.041) were predictive. LOS (P = 0.01) and discharge modified Rankin Scale score (P < 0.001) were significantly associated. Predicted and observed POMB with POSSUM were significantly disparate (χ² = 15.23; P = 0.001): POSSUM area under ROC (AUROC = 0.611) was also nondiscriminatory. ACS-NSQIP (χ² = 18.51; P < 0.001; AUROC = 0.629) and ASA grades (P = 0.25) were also nonpredictive.

CONCLUSIONS

POMB was frequently disabling, mostly neurologic, and as frequent and diverse as with fractured neck of femur. POMB was significantly correlated with LOS and discharge modified Rankin Scale score. Surprisingly, POSSUM, ACS-NSQIP, and ASA grades were not predictive and would not aid consent. Simple parameters (age, Glasgow Coma Scale, Markwalder grade, hypertension, and/or ≥1 other comorbidity) were instead predictive. Longitudinal follow-up will determine whether POMB affects LTS. CSDH, like fractured neck of femur, is distinct.

Beyond 30 Days: A Risk Calculator for Longer Term Outcomes of Prosthetic Breast Reconstruction

Background

Despite growing use of surgical risk calculators, many are limited to 30-day outcomes due to the constraints of their underlying datasets. Because complications of breast reconstruction can occur well beyond 30 days after surgery, we endeavored to expand the Breast Reconstruction Risk Assessment (BRA) Score to prediction of 1-year complications after primary prosthetic breast reconstruction.

Methods

We examined our prospective intrainstitutional database of prosthetic breast reconstructions from 2004 to 2015. Patients without 1-year follow-up were excluded. Pertinent patient variables include those enumerated in past iterations of the BRA Score. Outcomes of interest include seroma, surgical site infection (SSI), implant exposure, and explantation occurring within 1 year of tissue expander placement. Risk calculators were developed for each outcome using multivariate logistic regression models and made available online at www.BRAScore.org. Internal validity was assessed using C-statistic, Hosmer-Lemeshow test, and Brier score.

Results

Nine-hundred three patients met inclusion criteria. Within 1-year, 3.0% of patients experienced seroma, 6.9% infection, 7.1% implant exposure, and 13.2% explantation. Thirty-day, 90-day, and 180-day windows captured 17.6%, 39.5%, and 59.7% of explantations, respectively. One-year risk calculators were developed for each complication of interest, and all demonstrated good internal validity: C-statistics for the 5 models ranged from 0.674 to 0.739, Hosmer-Lemeshow tests were uniformly nonsignificant, and Brier scores ranged from 0.027 to 0.154.

Conclusions

Clinically significant complications of prosthetic breast reconstruction usually occur beyond the 30-day window following tissue expander placement. To better reflect long-term patient experiences, the BRA Score was enhanced with individualized risk models that predicted 1-year complications after prosthetic reconstruction (BRA Score XL). All models performed as well as, if not better than, the original BRA Score models and other popular risk calculators such as the CHA₂DS₂VASc Score. The patient-friendly BRA Score XL risk calculator is available at www.brascore.org to facilitate operative decision-making and heighten the informed consent process for patients.

The National Surgical Quality Improvement Program 30-Day Challenge: Microsurgical Breast Reconstruction Outcomes Reporting Reliability

Supplemental Digital Content is available in the text.

Background:

The aim was to assess reliability of the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) 30-day perioperative outcomes and complications for immediate, free-tissue transfer breast reconstruction by direct comparisons with our 30-day and overall institutional data, and assessing those that occur after 30 days.

Methods:

Data were retrieved for consecutive immediate, free-tissue transfer breast reconstruction patients from a single-institution database (2010–2015) and the ACS-NSQIP (2011–2014). Multiple logistic regressions were performed to compare adjusted outcomes between the 2 datasets.

Results:

For institutional versus ACS-NSQIP outcomes, there were no significant differences in surgical-site infection (SSI; 30-day, 3.6% versus 4.1%, P = 0.818; overall, 5.3% versus 4.1%, P = 0.198), wound disruption (WD; 30-day, 1.3% versus 1.5%, P = 0.526; overall, 2.3% versus 1.5%, P = 0.560), or unplanned readmission (URA; 30-day, 2.3% versus 3.3%, P = 0.714; overall, 4.6% versus 3.3%, P = 0.061). However, the ACS-NSQIP reported a significantly higher unplanned reoperation (URO) rate (30-day, 3.6% versus 9.5%, P < 0.001; overall, 5.3% versus 9.5%, P = 0.025). Institutional complications consisted of 5.3% SSI, 2.3% WD, 5.3% URO, and 4.6% URA, of which 25.0% SSI, 28.6% WD, 12.5% URO, and 7.1% URA occurred at 30–60 days, and 6.3% SSI, 14.3% WD, 18.8% URO, and 42.9% URA occurred after 60 days.

Conclusion:

For immediate, free-tissue breast reconstruction, the ACS-NSQIP may be reliable for monitoring and comparing SSI, WD, URO, and URA rates. However, clinicians may find it useful to understand limitations of the ACS-NSQIP for complications and risk factors, as it may underreport complications occurring beyond 30 days.

External validation of the breast reconstruction risk assessment calculator

INTRODUCTION

The Breast reconstruction Risk Assessment (BRA) Score estimates patient-specific risk for postsurgical complications using an individual's unique combination of preoperative variables. In this report, we externally validate the BRA Score models for surgical site infection, seroma, and explantation in a large sample of intra-institutional patients who underwent prosthetic breast reconstruction.

METHODS

We reviewed all initiated tissue expander/implant reconstructions by the senior authors from January 2004 to December 2015. BRA Score risk estimates were computed for each patient and compared against observed rates of complications. Hosmer-Lemeshow goodness-of-fit test, concordance statistic, and Brier score were used to assess the calibration, discrimination, and accuracy of the models, respectively.

RESULTS

Of the 1152 patients (1743 breasts) reviewed, 855 patients (1333 breasts) had complete data for BRA-score calculations and were included for analysis. Hosmer-Lemeshow tests for calibration demonstrated a good agreement between observed and predicted outcomes for surgical site infection (SSI) and seroma models (P-values of 0.33 and 0.16, respectively). In contrast, predicted rates of explantation deviated from observed rates (Hosmer-Lemeshow P-value of 0.04). C statistics demonstrated good discrimination for SSI, seroma, and explantation (0.73, 0.69, and 0.78, respectively).

CONCLUSIONS

In this external validation study, the BRA Score tissue expander/implant reconstruction models performed with generally good calibration, discrimination, and accuracy. Some weaknesses in certain models were identified as targets for future improvement. Taken together, these analyses validate the clinical utility of the BRA score risk models in predicting 30-day outcomes.