Assessment of attitudes towards future implementation of the "Surgical Risk Preoperative Assessment System" (SURPAS) tool: a pilot survey among patients, surgeons, and hospital administrators

Surgeon views regarding the adoption of a novel surgical innovation into clinical practice: systematic review

BACKGROUND

The haphazard adoption of new surgical technologies into practice has the potential to cause patient harm and there are many misconceptions in the decision-making behind the adoption of new innovations. The aim of this study was to synthesize factors affecting a surgeon's decision to adopt a novel surgical innovation into clinical practice.

METHODS

A systematic literature search was performed to obtain all studies where surgeon views on the adoption of a novel surgical innovation into clinical practice have been collected. The databases screened were MEDLINE, Embase, Science Direct, Scopus, the Web of Science, and the Cochrane Library of Systematic Reviews (last accessed October 2022). Innovations covered multiple specialties, including cardiac, general, urology, and orthopaedics. The quality of the papers was assessed using a 10-question Critical Appraisal Skills Programme (CASP) tool for qualitative research.

RESULTS

A total of 26 studies (including 1112 participants, of which 694 were surgeons) from nine countries satisfied the inclusion and exclusion criteria. Types of study included semi-structured interviews and focus groups, for example. Themes and sub-themes that emerged after a thematic synthesis were categorized using five causal factors (structural, organizational, patient-level, provider-level, and innovation-based). These themes were further split into facilitators and barriers. Key facilitators to adoption of an innovation include improved clinical outcomes, cost-effectiveness, and support from internal and external stakeholders. Barriers to adoption include lack of organizational support and views of senior surgeons.

CONCLUSION

There are multiple complex factors that dynamically interact, affecting the adoption of a novel surgical innovation into clinical practice. There is a need to further investigate surgeon and other stakeholder views regarding the strength of clinical evidence required to support the widespread adoption of a surgical innovation into clinical practice.

An Individualized Postoperative Pain Risk Communication Tool for Use in Pediatric Surgery: Co-Design and Usability Evaluation

BACKGROUND

Risk identification and communication tools have the potential to improve health care by supporting clinician-patient or family discussion of treatment risks and benefits and helping patients make more informed decisions; however, they have yet to be tailored to pediatric surgery. User-centered design principles can help to ensure the successful development and uptake of health care tools.

OBJECTIVE

We aimed to develop and evaluate the usability of an easy-to-use tool to communicate a child's risk of postoperative pain to improve informed and collaborative preoperative decision-making between clinicians and families.

METHODS

With research ethics board approval, we conducted web-based co-design sessions with clinicians and family participants (people with lived surgical experience and parents of children who had recently undergone a surgical or medical procedure) at a tertiary pediatric hospital. Qualitative data from these sessions were analyzed thematically using NVivo (Lumivero) to identify design requirements to inform the iterative redesign of an existing prototype. We then evaluated the usability of our final prototype in one-to-one sessions with a new group of participants, in which we measured mental workload with the National Aeronautics and Space Administration (NASA) Task Load Index (TLX) and user satisfaction with the Post-Study System Usability Questionnaire (PSSUQ).

RESULTS

A total of 12 participants (8 clinicians and 4 family participants) attended 5 co-design sessions. The 5 requirements were identified: (A) present risk severity descriptively and visually; (B) ensure appearance and navigation are user-friendly; (C) frame risk identification and mitigation strategies in positive terms; (D) categorize and describe risks clearly; and (E) emphasize collaboration and effective communication. A total of 12 new participants (7 clinicians and 5 family participants) completed a usability evaluation. Tasks were completed quickly (range 5-17 s) and accurately (range 11/12, 92% to 12/12, 100%), needing only 2 requests for assistance. The median (IQR) NASA TLX performance score of 78 (66-89) indicated that participants felt able to perform the required tasks, and an overall PSSUQ score of 2.1 (IQR 1.5-2.7) suggested acceptable user satisfaction with the tool.

CONCLUSIONS

The key design requirements were identified, and that guided the prototype redesign, which was positively evaluated during usability testing. Implementing a personalized risk communication tool into pediatric surgery can enhance the care process and improve informed and collaborative presurgical preparation and decision-making between clinicians and families of pediatric patients.

Development and validation of a multivariable preoperative prediction model for postoperative length of stay in a broad inpatient surgical population

BACKGROUND

Postoperative length of stay is a meaningful patient-centered outcome and an important determinant of healthcare costs. The Surgical Risk Preoperative Assessment System preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict postoperative length of stay has not been assessed. We aimed to determine whether the Surgical Risk Preoperative Assessment System variables could accurately predict postoperative length of stay up to 30 days in a broad inpatient surgical population.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database from 2012 to 2018. A model using the Surgical Risk Preoperative Assessment System variables and a 28-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables, were fit to the analytical cohort (2012-2018) using multiple linear regression and compared using model performance metrics. Internal chronological validation of the Surgical Risk Preoperative Assessment System model was conducted using training (2012-2017) and test (2018) datasets.

RESULTS

We analyzed 3,295,028 procedures. The adjusted R2 for the Surgical Risk Preoperative Assessment System model fit to this cohort was 93.3% of that for the full model (0.347 vs 0.372). In the internal chronological validation of the Surgical Risk Preoperative Assessment System model, the adjusted R2 for the test dataset was 97.1% of that for the training dataset (0.3389 vs 0.3489).

CONCLUSION

The parsimonious Surgical Risk Preoperative Assessment System model can preoperatively predict postoperative length of stay up to 30 days for inpatient surgical procedures almost as accurately as a model using all 28 American College of Surgeons' National Surgical Quality Improvement Program preoperative nonlaboratory variables and has shown acceptable internal chronological validation.

Current use of the National Surgical Quality Improvement Program surgical risk calculator in academic surgery: a mixed-methods study

Background

This study aims to quantitatively assess use of the NSQIP surgical risk calculator (NSRC) in contemporary surgical practice and to identify barriers to use and potential interventions that might increase use.

Materials and methods

We performed a cross-sectional study of surgeons at seven institutions. The primary outcomes were self-reported application of the calculator in general clinical practice and specific clinical scenarios as well as reported barriers to use.

Results

In our sample of 99 surgeons (49.7% response rate), 73.7% reported use of the NSRC in the past month. Approximately half (51.9%) of respondents reported infrequent NSRC use (<20% of preoperative discussions), while 14.3% used it in ≥40% of preoperative assessments. Reported use was higher in nonelective cases (30.2% vs 11.1%) and in patients who were ≥65 years old (37.1% vs 13.0%), functionally dependent (41.2% vs 6.6%), or with surrogate consent (39.9% vs 20.4%). NSRC use was not associated with training status or years in practice. Respondents identified a lack of influence on the decision to pursue surgery as well as concerns regarding the calculator's accuracy as barriers to use. Surgeons suggested improving integration to workflow and better education as strategies to increase NSRC use.

Conclusions

Many surgeons reported use of the NSRC, but few used it frequently. Surgeons reported more frequent use in nonelective cases and frail patients, suggesting the calculator is of greater utility for high-risk patients. Surgeons raised concerns about perceived accuracy and suggested additional education as well as integration of the calculator into the electronic health record.

Preoperative Prediction of Unplanned Reoperation in a Broad Surgical Population

INTRODUCTION

Unplanned reoperation is an undesirable outcome with considerable risks and an increasingly assessed quality of care metric. There are no preoperative prediction models for reoperation after an index surgery in a broad surgical population in the literature. The Surgical Risk Preoperative Assessment System (SURPAS) preoperatively predicts 12 postoperative adverse events using 8 preoperative variables, but its ability to predict unplanned reoperation has not been assessed. This study's objective was to determine whether the SURPAS model could accurately predict unplanned reoperation.

METHODS

This was a retrospective analysis of the American College of Surgeons' National Surgical Quality Improvement Program adult database, 2012-2018. An unplanned reoperation was defined as any unintended operation within 30 d of an initial scheduled operation. The 8-variable SURPAS model and a 29-variable "full" model, incorporating all available American College of Surgeons' National Surgical Quality Improvement Program nonlaboratory preoperative variables, were developed using multiple logistic regression and compared using discrimination and calibration metrics: C-indices (C), Hosmer-Lemeshow observed-to-expected plots, and Brier scores (BSs). The internal chronological validation of the SURPAS model was conducted using "training" (2012-2017) and "test" (2018) datasets.

RESULTS

Of 5,777,108 patients, 162,387 (2.81%) underwent an unplanned reoperation. The SURPAS model's C-index of 0.748 was 99.20% of that for the full model (C = 0.754). Hosmer-Lemeshow plots showed good calibration for both models and BSs were similar (BS = 0.0264, full; BS = 0.0265, SURPAS). Internal chronological validation results were similar for the training (C = 0.749, BS = 0.0268) and test (C = 0.748, BS = 0.0250) datasets.

CONCLUSIONS

The SURPAS model accurately predicted unplanned reoperation and was internally validated. Unplanned reoperation can be integrated into the SURPAS tool to provide preoperative risk assessment of this outcome, which could aid patient risk education.

Development and validation of a prediction model for conversion of outpatient to inpatient surgery

BACKGROUND

Unplanned hospital admission after intended outpatient surgery is an undesirable outcome. We aimed to develop a prediction model that estimates a patient's risk of conversion from outpatient surgery to inpatient hospitalization.

METHODS

This was a retrospective analysis using the American College of Surgeons National Surgical Quality Improvement Program database, 2005 to 2018. Conversion from outpatient to inpatient surgery was defined as having outpatient surgery and >1 day hospital stay. The Surgical Risk Preoperative Assessment System was developed using multiple logistic regression on a training dataset (2005-2016) and compared to a model using the 26 relevant variables in the American College of Surgeons National Surgical Quality Improvement Program. The Surgical Risk Preoperative Assessment System was validated using a testing dataset (2017-2018). Performance statistics and Hosmer-Lemeshow plots were compared. Two high-risk definitions were compared: (1) the maximum Youden index, and (2) the cohort above the tenth decile of risk on the Hosmer-Lemeshow plot. The sensitivities, specificities, positive predictive values, negative predictive values, and accuracies were compared.

RESULTS

In all, 2,822,379 patients were included; 3.6% of patients unexpectedly converted to inpatient. The 6-variable Surgical Risk Preoperative Assessment System model performed comparably to the 26-variable American College of Surgeons National Surgical Quality Improvement Program model (c-indices = 0.818 vs. 0.823; Brier scores = 0.0308 vs 0.0306, respectively). The Surgical Risk Preoperative Assessment System performed well on internal validation (c-index = 0.818, Brier score = 0.0341). The tenth decile of risk definition had higher specificity, positive predictive values, and accuracy than the maximum Youden index definition, while having lower sensitivity.

CONCLUSION

The Surgical Risk Preoperative Assessment System accurately predicted a patient's risk of unplanned outpatient-to-inpatient conversion. Patients at higher risk should be considered for inpatient surgery, while lower risk patients could safely undergo operations at ambulatory surgery centers.

Predicting Elective Surgical Patient Outcome Destination Based on the Preoperative Modified Frailty Index and Laboratory Values

INTRODUCTION

To determine the accuracy of preoperative modified frailty index (mFI) with or without laboratory values (mFI-labs or labs-continuous) in predicting postoperative discharge destination. Discharge destination is important to providers and patients. The ability to accurately predict discharge destination preoperatively can improve hospital resource utilization and help set patient and family expectations.

METHODS

Cohort analysis of the 2018 American College of Surgeon National Surgical Quality Improvement Project (ACS-NSQIP) Participant Use File of patients undergoing operations with complete data point sets: age, sex, operation work relative-value units; mFI-clinical based on 12 clinical findings, mFI-labs based on seven laboratory values. The nine hierarchical destinations: home, home with assistance, multi-level community, unskilled-care facility, rehabilitation facility, skilled-nursing facility, acute care hospital, hospice, or death, from best to worst outcome. Data were analyzed using univariate analysis, multiple logistic regression and supervised learning artificial neural networks.

RESULTS

Univariate and multivariate in general showed that patients with higher mFI-clinical and mFI-lab scores, as well as older age and more complex operations were more likely to be discharged to facilities other than home. However, these statistical techniques could not predict the exact destination. An artificial neural network analysis demonstrated perfect location prediction in 64.9% of cases and within one level of prefect prediction is 87.4%.

CONCLUSIONS

Using a limited number of preoperative factors, combining the mFI-clinical with laboratory values significantly improves the destination prediction performance significantly better than using the values separately. Preoperative knowledge of the likely discharge destination can benefit postoperative care planning and delivery.

Attitudes about use of preoperative risk assessment tools: a survey of surgeons and surgical residents in an academic health system

Background

Formal surgical risk assessment tools have been developed to predict risk of adverse postoperative patient outcomes. Such tools accurately predict common postoperative complications, inform patients and providers of likely perioperative outcomes, guide decision making, and improve patient care. However, these are underutilized. We studied the attitudes towards and techniques of how surgeons preoperatively assess risk.

Methods

Surgeons at a large academic tertiary referral hospital and affiliate community hospitals were emailed a 16-question survey via REDCap (Research Electronic Data Capture) between 8/2019-6/2020. Reminder emails were sent once weekly for three weeks. All completed surveys by surgical residents and attendings were included; incomplete surveys were excluded. Surveys were analyzed using descriptive statistics (frequency distributions and percentages for categorical variables, means, and standard deviations for continuous variables), and Fisher’s exact test and unpaired t-tests comparing responses by surgical attendings vs. residents.

Results

A total of 108 surgical faculty, 95 surgical residents, and 58 affiliate surgeons were emailed the survey. Overall response rates were 50.0% for faculty surgeons, 47.4% for residents, and 36.2% for affiliate surgeons. Only 20.8% of surgeons used risk calculators most or all of the time. Attending surgeons were more likely to use prior experience and current literature while residents used risk calculators more frequently. Risk assessment tools were more likely to be used when predicting major complications and death in older patients with significant risk factors. Greatest barriers for use of risk assessment tools included time, inaccessibility, and trust in accuracy.

Conclusions

A small percentage of surgeons use surgical risk calculators as part of their routine practice. Time, inaccessibility, and trust in accuracy were the most significant barriers to use.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13037-022-00320-1.

Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons Versus a Standardized Surgical Risk Prediction Tool

Considerable variability exists between surgeons' assessments of a patient's individual pre-operative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes versus a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program (NSQIP) database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs. 5.1%, p=<0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs. 69.8%, p<0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs. 44.3%, p<0.001). C-indices for SURPAS vs. surgeons were 0.84 vs. 0.76 (p=0.3) for morbidity and 0.98 vs. 0.85 (p=0.001) for mortality. Brier scores for SURPAS vs. surgeons were 0.1579 vs. 0.1986 for morbidity (p=0.03) and 0.0409 vs. 0.0543 for mortality (p=0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC=0.654) and mortality (ICC=0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.

A non-linear ensemble model-based surgical risk calculator for mixed data from multiple surgical fields

BACKGROUND

The misestimation of surgical risk is a serious threat to the lives of patients when implementing surgical risk calculator. Improving the accuracy of postoperative risk prediction has received much attention and many methods have been proposed to cope with this problem in the past decades. However, those linear approaches are inable to capture the non-linear interactions between risk factors, which have been proved to play an important role in the complex physiology of the human body, and thus may attenuate the performance of surgical risk calculators.

METHODS

In this paper, we presented a new surgical risk calculator based on a non-linear ensemble algorithm named Gradient Boosting Decision Tree (GBDT) model, and explored the corresponding pipeline to support it. In order to improve the practicability of our approach, we designed three different modes to deal with different data situations. Meanwhile, considering that one of the obstacles to clinical acceptance of surgical risk calculators was that the model was too complex to be used in practice, we reduced the number of input risk factors according to the importance of them in GBDT. In addition, we also built some baseline models and similar models to compare with our approach.

RESULTS

The data we used was three-year clinical data from Surgical Outcome Monitoring and Improvement Program (SOMIP) launched by the Hospital Authority of Hong Kong. In all experiments our approach shows excellent performance, among which the best result of area under curve (AUC), Hosmer-Lemeshow test ([Formula: see text]) and brier score (BS) can reach 0.902, 7.398 and 0.047 respectively. After feature reduction, the best result of AUC, [Formula: see text] and BS of our approach can still be maintained at 0.894, 7.638 and 0.060, respectively. In addition, we also performed multiple groups of comparative experiments. The results show that our approach has a stable advantage in each evaluation indicator.

CONCLUSIONS

The experimental results demonstrate that NL-SRC can not only improve the accuracy of predicting the surgical risk of patients, but also effectively capture important risk factors and their interactions. Meanwhile, it also has excellent performance on the mixed data from multiple surgical fields.

Preoperative Nomogram Predicts Non-home Discharge in Patients Undergoing Pancreatoduodenectomy

BACKGROUND

In patients undergoing pancreatoduodenectomy, non-home discharge is common and often results in an unnecessary delay in hospital discharge. This study aimed to develop and validate a preoperative prediction model to identify patients with a high likelihood of non-home discharge following pancreatoduodenectomy.

METHODS

Patients undergoing pancreatoduodenectomy from 2013 to 2018 were identified using an institutional database. Patients were categorized according to discharge location (home vs. non-home). Preoperative risk factors, including social determinants of health associated with non-home discharge, were identified using Pearson's chi-squared test and then included in a multiple logistic regression model. A training cohort composed of 80% of the sampled patients was used to create the prediction model, and validation carried out using the remaining 20%. Statistical significance was defined as P < 0.05.

RESULTS

Seven hundred sixty-six pancreatoduodenectomy patients met the study criteria for inclusion in the analysis (non-home, 126; home, 640). Independent predictors of non-home discharge on multivariable analysis were age, marital status, mental health diagnosis, functional health status, dyspnea, and chronic obstructive pulmonary disease. The prediction model was then used to generate a nomogram to predict likelihood of non-home discharge. The training and validation cohorts demonstrated comparable performances with an identical area under the curve (0.81) and an accuracy of 84%.

CONCLUSION

A prediction model to reliably assess the likelihood of non-home discharge after pancreatoduodenectomy was developed and validated in the present study.

Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas

BACKGROUND

The universal Surgical Risk Preoperative Assessment System prediction models for postoperative adverse outcomes have good accuracy for estimating risk in broad surgical populations and for surgical specialties. The accuracy in individual operations has not yet been assessed. The objective of this study was to evaluate the Surgical Risk Preoperative Assessment System in predicting adverse outcomes for selected individual operations.

METHODS

The Surgical Risk Preoperative Assessment System models were applied to the top 2 most frequent common procedural terminology codes in 9 surgical specialties and 5 additional common general surgical operations in the 2009 to 2018 database of the American College of Surgeons National Surgical Quality Improvement Program. Goodness of fit statistics were estimated, including c-indices for discrimination, Hosmer-Lemeshow graphs and P values for calibration, overall observed versus expected event rates, and Brier scores.

RESULTS

The total sample size was 2,020,172, which represented 29% of the 6.9 million operations in the American College of Surgeons National Surgical Quality Improvement Program database. Average c-indices across 12 outcomes were acceptable (≥0.70) for 13 (56.5%) of the 23 operations. Overall observed-to-expected rates were similar for mortality and overall morbidity across the 23 operations. Hosmer-Lemeshow graphs over quintiles of risk comparing observed-to-expected rates of mortality and overall morbidity were similar for 52% and 70% of operations, respectively. Model performance was better in less complex operations and those done in patients with lower preoperative risk.

CONCLUSION

Surgical Risk Preoperative Assessment System displayed accuracy in estimating postoperative adverse events for some of the 23 operations studied, but not all. In the procedures where Surgical Risk Preoperative Assessment System was not accurate, developing disease or operation-specific risk models might be appropriate.

The preoperative risk tool SURPAS accurately predicts outcomes in emergency surgery

BACKGROUND

The Surgical Risk Preoperative Assessment System (SURPAS) uses eight variables to accurately predict postoperative complications but has not been sufficiently studied in emergency surgery. We evaluated SURPAS in emergency surgery, comparing it to the Emergency Surgery Score (ESS).

METHODS

SURPAS and ESS estimates of 30-day mortality and overall morbidity were calculated for emergency operations in the 2009-2018 ACS-NSQIP database and compared using observed-to-expected plots and rates, c-indices, and Brier scores. Cases with incomplete data were excluded.

RESULTS

In 205,318 emergency patients, SURPAS underestimated (8.1%; 35.9%) while ESS overestimated (10.1%; 43.8%) observed mortality and morbidity (8.9%; 38.8%). Each showed good calibration on observed-to-expected plots. SURPAS had better c-indices (0.855 vs 0.848 mortality; 0.802 vs 0.755 morbidity), while the Brier score was better for ESS for mortality (0.0666 vs. 0.0684) and for SURPAS for morbidity (0.1772 vs. 0.1950).

CONCLUSIONS

SURPAS accurately predicted mortality and morbidity in emergency surgery using eight predictor variables.

External Validation of Surgical Risk Preoperative Assessment System in Pulmonary Resection

BACKGROUND

Accurate preoperative risk assessment is necessary for informed decision making for patients and surgeons. Several preoperative risk calculators are available but few have been examined in the general thoracic surgical patient population. The Surgical Risk Preoperative Assessment System (SURPAS), a risk-assessment tool applicable to a wide spectrum of surgical procedures, was developed to predict the risks of common adverse postoperative outcomes using a parsimonious set of preoperative input variables. We sought to externally validate the performance of SURPAS for postoperative complications in patients undergoing pulmonary resection.

METHODS

Between January 2016 and December 2018, 2514 patients underwent pulmonary resection at our center. Using data from our institution's prospectively maintained database, we calculated the predicted risks of 12 categories of postoperative outcomes using the latest version of SURPAS. Performance of SURPAS against observed patient outcomes was assessed by discrimination (concordance index) and calibration (calibration curves).

RESULTS

The discrimination ability of SURPAS was moderate across all outcomes (concordance indices, 0.640 to 0.788). Calibration curves indicated good calibration for all outcomes except infectious and cardiac complications, discharge to a location other than home, and mortality (all overestimated by SURPAS).

CONCLUSIONS

SURPAS demonstrates outcomes for pulmonary resections with reasonable predictive ability. Discretion should be applied when assessing risk for postoperative infectious and cardiac complications, discharge to a location other than home, and mortality. Although the parsimonious nature of SURPAS is one of its strengths, its performance might be improved by including additional factors known to influence outcomes after pulmonary resection, such as sex and pulmonary function.

Performance Comparison Between SURPAS and ACS NSQIP Surgical Risk Calculator in Pulmonary Resection

BACKGROUND

Accurate preoperative risk assessment is critical for informed decision making. The Surgical Risk Preoperative Assessment System (SURPAS) and the National Surgical Quality Improvement Program (NSQIP) Surgical Risk Calculator (SRC) predict risks of common postoperative complications. This study compares observed and predicted outcomes after pulmonary resection between SURPAS and NSQIP SRC.

METHODS

Between January 2016 and December 2018, 2514 patients underwent pulmonary resection and were included. We entered the requisite patient demographics, preoperative risk factors, and procedural details into the online NSQIP SRC and SURPAS formulas. Performance of the prediction models was assessed by discrimination and calibration.

RESULTS

No statistically significant differences were found between the 2 models in discrimination performance for 30-day mortality, urinary tract infection, readmission, and discharge to a nursing or rehabilitation facility. The ability to discriminate between a patient who will develop a complication and a patient who will not was statistically indistinguishable between NSQIP and SURPAS, except for renal failure. With a C index closer to 1.0, the NSQIP performed significantly better than the SURPAS SRC in discriminating risk of renal failure (C index, 0.798 vs 0.694; P = .003). The calibration curves of predicted and observed risk for each model demonstrate similar performance with a tendency toward overestimation of risk, apart from renal failure.

CONCLUSIONS

Overall, SURPAS and NSQIP SRC performed similarly in predicting outcomes for pulmonary resections in this large, single-center validation study with moderate to good discrimination of outcomes. Notably, SURPAS uses a smaller set of input variables to generate the preoperative risk assessment. The addition of thoracic-specific input variables may improve performance.

Systematic Review of Preoperative Risk Discussion in Practice

BACKGROUND

Informed consent is an ethical imperative of surgical practice. This requires effective communication of procedural risks to patients and is learned during residency. No systematic review has yet examined current risk disclosure. This systematic review aims to use existing published information to assess preoperative provision of risk information by surgeons.

METHODS

Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses as a guide, a standardized search in Ovid MEDLINE, Embase, CINHAL, and PubMed was performed. Three reviewers performed the study screening, with 2-reviewer consensus required at each stage. Studies containing objective information concerning preoperative risk provision in adult surgical patients were selected for inclusion. Studies exclusively addressing interventions for pediatric patients or trauma were excluded, as were studies addressing risks of anesthesia.

RESULTS

The initial search returned 12,988 papers after deduplication, 33 of which met inclusion criteria. These studies primarily evaluated consent through surveys of providers, record reviews and consent recordings. The most ubiquitous finding of all study types was high levels of intra-surgeon variation in what risk information is provided to patients preoperatively. Studies recording consents found the lowest rates of risk disclosure. Studies using multiple forms of investigation corroborated this, finding disparity between verbally provided information vs chart documentation.

CONCLUSIONS

The wide variance in what information is provided to patients preoperatively inhibits the realization of the ethical and practical components of informed consent. The findings of this review indicate that significant opportunities exist for practice improvement. Future development of surgical communication tools and techniques should emphasize standardizing what risks are shared with patients.

The Surgical Risk Preoperative Assessment System: Determining which predictor variables can be automatically obtained from the electronic health record

Introduction

The Surgical Risk Preoperative Assessment System is a parsimonious, universal surgical risk calculator integrated into our local electronic health record. We determined how many of its eight preoperative risk predictor variables could be automatically obtained from the electronic health record. This has implications for the usability and adoption of Surgical Risk Preoperative Assessment System, serving as an example of use of electronic health record data for populating clinical decision support tools.

Methods

We quantified the availability and accuracy in the electronic health record of the eight Surgical Risk Preoperative Assessment System predictor variables (patient age, American Society of Anesthesiology physical status classification, functional health status, sepsis, work Relative Value Unit, in-/outpatient operation, surgeon specialty, emergency status) at the patient’s preoperative encounter of 5205 patients entered into the American College of Surgeons National Surgical Quality Improvement Program. Accuracy was determined by comparing the electronic health record data to the same patient’s National Surgical Quality Improvement Program data, used as the “gold standard.” Acceptable accuracy was defined as a Kappa statistic or Pearson correlation coefficient ≥0.8 when comparing electronic health record and National Surgical Quality Improvement Program data. Acceptable availability was defined as presence of the variable in the electronic health record at the preoperative encounter ≥95% of the time.

Results

Of the eight predictor variables, six had acceptable accuracy. Only preoperative sepsis and functional health status had Kappa statistics <0.8. However, only patient age and surgeon specialty were ≥95% available in the electronic health record at the preoperative visit.

Conclusions

Processes need to be developed to populate more of the Surgical Risk Preoperative Assessment System preoperative predictor variables in the patient’s electronic health record prior to the preoperative visit to lessen the burden on the busy surgeon and encourage more widespread use of Surgical Risk Preoperative Assessment System.

Preoperative predictors of postoperative complications after gastric cancer resection

Risk management is becoming an increasingly important healthcare issue. Gastrectomy with lymphadenectomy is still the mainstay of treatment for localized gastric cancer, but it is sometimes associated with postoperative complications that compromise the patient’s quality of life, tolerability of adjuvant treatment, and prognosis. Parameters based exclusively on preoperative factors can identify patients most at risk of postoperative complications, whereby surgeons can provide the patient with precise informed consent information and optimal perioperative management. Ultimately, these predictive tools can also help minimize medical costs. In this context, many studies have identified factors that predict postoperative complications, including indicators based on body constitution, nutrition, inflammation, organ function and hypercoagulation. This review presents our current understanding and discusses some future perspectives of preoperatively identified factors predictive of complications after resection for gastric cancer.

Refining the predictive variables in the "Surgical Risk Preoperative Assessment System" (SURPAS): a descriptive analysis

Background

The Surgical Risk Preoperative Assessment System (SURPAS) is a parsimonious set of models providing accurate preoperative prediction of common adverse outcomes for individual patients. However, focus groups with surgeons and patients have developed a list of questions about and recommendations for how to further improve SURPAS’s usability and usefulness. Eight issues were systematically evaluated to improve SURPAS.

Methods

The eight issues were divided into three groups: concerns to be addressed through further analysis of the database; addition of features to the SURPAS tool; and the collection of additional outcomes. Standard multiple logistic regression analysis was performed using the 2005–2015 American College of Surgeons National Surgical Quality Improvement Participant Use File (ACS NSQIP PUF) to refine models: substitution of the preoperative sepsis variable with a procedure-related risk variable; testing of an indicator variable for multiple concurrent procedure codes in complex operations; and addition of outcomes to increase clinical applicability. Automated risk documentation in the electronic health record and a patient handout and supporting documentation were developed. Long term functional outcomes were considered.

Results

Model discrimination and calibration improved when preoperative sepsis was replaced with a procedure-related risk variable. Addition of an indicator variable for multiple concurrent procedures did not significantly improve the models. Models were developed for a revised set of eleven adverse postoperative outcomes that separated bleeding/transfusion from the cardiac outcomes, UTI from the other infection outcomes, and added a predictive model for unplanned readmission. Automated documentation of risk assessment in the electronic health record, visual displays of risk for providers and patients and an “About” section describing the development of the tool were developed and implemented. Long term functional outcomes were considered to be beyond the scope of the current SURPAS tool.

Conclusion

Refinements to SURPAS were successful in improving the accuracy of the models, while reducing manual entry to five of the eight variables. Adding a predictor variable to indicate a complex operation with multiple current procedure codes did not improve the accuracy of the models. We developed graphical displays of risk for providers and patients, including a take-home handout and automated documentation of risk in the electronic health record. These improvements should facilitate easier implementation of SURPAS.

Electronic supplementary material

The online version of this article (10.1186/s13037-019-0208-2) contains supplementary material, which is available to authorized users.

Use of the consolidated framework for implementation research to guide dissemination and implementation of new technologies in surgery

Background

Improving surgical outcomes is important to patients, providers, and healthcare systems. Understanding best methods to ensure evidence based practices are successfully implemented and sustained in clinical practices leads to improved care. Dissemination and implementation (D&I) science facilitates the successful pathway from clinical trials to sustained implementation.

Methods

We describe D&I science, introduce the consolidated framework for implementation research (CFIR), a D&I framework, and provide an example of how CFIR was utilized to facilitate the translational process from design adaptations to implementation, broad utilization by clinicians, and sustainability of the SUrgical Risk Preoperative Assessment System (SURPAS) tool into regular clinical practice. SURPAS creates data-driven individualized risk assessments of common adverse postoperative outcomes to enhance the informed consent process, shared decision making, and consequently improved surgical outcomes. The CFIR provided a structured systematic way to identify constructs influencing the D&I of SURPAS, including adaptations for the process and tool.

Results

We identified three domains, each with specific constructs, that participants believed would strongly influence effectiveness of SURPAS implementation efforts: the importance of patients' perspectives (outer setting); the quality of SURPAS (intervention characteristic); and integration of SURPAS into the electronic health record (inner setting). Additionally, providers' positive attitudes toward and support of SURPAS (characteristics of individuals); and the ease of integration of SURPAS into the workflow (process), were also identified. Tension emerged between patients' preference of the provision of risk information and providers' concern about additional clinic time required for formal risk discussion with low-risk patients.

Conclusions

Systematically identifying constructs from the beginning of the design through the implementation process can guide design of a multi-component strategy for future large-scale implementation by assessing the relative impact of factors on implementation using the CFIR framework. In the example studied, this allows key stakeholders to ensure success of D&I of SURPAS at multiple levels and times, continuously optimizing the process.