An anaesthetic pre-operative assessment clinic reduces pre-operative inpatient stay in patients requiring major vascular surgery

Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning

BACKGROUND

Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization.

METHODS AND RESULTS

The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure.

CONCLUSIONS

Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.

Predicting outcomes following lower extremity open revascularization using machine learning

Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.

Using machine learning to predict outcomes following carotid endarterectomy

OBJECTIVE

Prediction of outcomes following carotid endarterectomy (CEA) remains challenging, with a lack of standardized tools to guide perioperative management. We used machine learning (ML) to develop automated algorithms that predict outcomes following CEA.

METHODS

The Vascular Quality Initiative (VQI) database was used to identify patients who underwent CEA between 2003 and 2022. We identified 71 potential predictor variables (features) from the index hospitalization (43 preoperative [demographic/clinical], 21 intraoperative [procedural], and 7 postoperative [in-hospital complications]). The primary outcome was stroke or death at 1 year following CEA. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, insurance status, symptom status, and urgency of surgery.

RESULTS

Overall, 166,369 patients underwent CEA during the study period. In total, 7749 patients (4.7%) had the primary outcome of stroke or death at 1 year. Patients with an outcome were older with more comorbidities, had poorer functional status, and demonstrated higher risk anatomic features. They were also more likely to undergo intraoperative surgical re-exploration and have in-hospital complications. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.90 (95% confidence interval [CI], 0.89-0.91). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. Our XGBoost models maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.90 (95% CI, 0.89-0.91) and 0.94 (95% CI, 0.93-0.95), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.15 (preoperative), 0.14 (intraoperative), and 0.11 (postoperative). Of the top 10 predictors, eight were preoperative features, including comorbidities, functional status, and previous procedures. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

We developed ML models that accurately predict outcomes following CEA. Our algorithms perform better than logistic regression and existing tools, and therefore, have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes.

Pre-anesthetic clinic internship: new teaching method of pre-anesthesia evaluation for undergraduates

Background

This study aimed to observe the effect of internship in a pre-anesthetic clinic on the teaching quality of pre-anesthesia evaluation for undergraduates.

Methods

A total of 120 undergraduates from July 2017 to July 2018 in the anesthesia department of our hospital were randomly divided into two groups: pre-anesthetic clinic internship teaching group (n = 60) and traditional teaching group (n = 60). The knowledge in the pre-anesthesia evaluation teaching chapters was evaluated between the two groups of undergraduates.

Results

There were no significant differences in the demographic information between the two groups. The scores in the case analysis and theoretical knowledge test in the pre-anesthetic clinic internship teaching group were significantly higher than those in the traditional teaching group. In addition, the students' satisfaction with the curriculum design was significantly higher in the pre-anesthetic clinic internship teaching group than in the traditional teaching group.

Conclusion

Pre-anesthetic clinic internships can improve the quality of pre-anesthesia assessment teaching for undergraduates.

Optimisation of perioperative investigations among elective orthopaedic patients in a Dublin-based teaching hospital

Background

The current National Institute for Health and Care Excellence guidelines, in accordance with the Association of Anaesthetists of Great Britain and Ireland guidelines, recommend the following haematological investigations for all patients undergoing major elective surgery: full blood count, renal profile and coagulation screen if clinically indicated. However, the guidelines fail to specify a time-interval for which normal blood results remain valid. Currently all patients in Ireland undergoing substantial elective surgery requiring general or regional anaesthetic have a preoperative assessment prior to the surgery. Patients have phlebotomy performed as part of this assessment. Patients admitted for elective surgery often have these bloods repeated on the morning of surgery.

Objectives

To determine if blood investigations taken over a one-year period prior to surgery can be used as a baseline for clinically stable patients undergoing elective surgery.

Study design and methods

All consecutive day of surgery admission patients >18 years of age undergoing elective orthopaedic surgery in Tallaght Hospital between 1 December 2014 and 1 December 2015 were identified using hospital records. Their blood results in the one-year period prior to surgery were compared to the blood results on the morning of surgery, using a McNemar’s test. A further clinical analysis was performed.

Results

There was no statistically significant change between blood results from three months prior to the surgery and the morning of surgery (P < 0.05). Furthermore, the blood results remained largely unchanged in the one year prior to surgery. No patient had the operation deferred due to aberrant blood results, following previously normal results prior to surgery. The potential cost-saving of omitting bloods is enormous.

Conclusions

There appears to be neither a statistical nor clinical benefit to repeating blood tests on the morning of surgery, following normal bloods <3 months in a clinically stable individual.

Measuring and Improving the Quality of Preprocedural Assessments

BACKGROUND

Preprocedural assessments are used by anesthesia providers to optimize perioperative care for patients undergoing invasive procedures. When these assessments are performed in advance by providers who are not caring for the patient during the procedure, there is an additional layer of complexity in ensuring that the workup meets the needs of the primary anesthesia care team. In this study, anesthesia providers were asked to rate the quality of preprocedural assessments prepared by other providers to evaluate anesthesia care team satisfaction.

METHODS

Quality ratings for preprocedural assessments were collected from anesthesia providers on the day of surgery using an electronic quality assurance tool from January 9, 2014 to October 21, 2014. Users could rate assessments as "exemplary," "satisfactory," or "unsatisfactory." Free text comments could be entered for any of the quality ratings chosen. A reviewer trained in clinical anesthesia categorized all comments as "positive," "constructive," or "neutral" and conducted in-depth chart reviews triggered by 67 "constructive" comments submitted during the first 3 months of data collection to further subcategorize perceived deficiencies in the preprocedural assessments. In May 2014, providers were asked to participate in a midpoint survey and provide general feedback about the preprocedural process and evaluations.

RESULTS

37,611 procedures requiring anesthesia were analyzed. Of the 17,522 (46.6%) cases with a rated preprocedural assessment, anesthesia providers rated 3828 (21.8%) as "exemplary," 13,454 (76.8%) as "satisfactory," and 240 (1.4%) as "unsatisfactory." The monthly proportion of "unsatisfactory" ratings ranged from 3.1% to 0% over the study period, whereas the midpoint survey showed that anesthesia providers estimated that the number of unsatisfactory evaluations was 11.5%. Preprocedural evaluations performed on inpatients received significantly better ratings than evaluations performed on outpatients by the preadmission testing clinic or phone program (P < .0001). The most common reason given for "unsatisfactory" ratings was a perception of "missing information" (49.2%). Chart reviews revealed that inadequate documentation was in reality the most common deficiency in preprocedural evaluations (35 of 67 reviews, 52.2%).

CONCLUSIONS

The overwhelming majority of preprocedural assessments performed at our institution were considered satisfactory or exemplary by day-of-surgery anesthesia providers. This was demonstrated by both the case-by-case ratings and midpoint survey. However, the perceived frequency of "unsatisfactory" evaluations was worse when providers were asked to reflect on the quality of preprocedural evaluations generally versus rate them individually. Analysis of comments left by providers allowed us to identify specific and actionable areas for improvement. This method can be used by other institutions to identify systemic deficiencies in the preprocedural evaluation process.