A new decision support model for preanesthetic evaluation

Machine learning approach for predicting post-intubation hemodynamic instability (PIHI) index values: towards enhanced perioperative anesthesia quality and safety

BACKGROUND

Adequate preoperative evaluation of the post-intubation hemodynamic instability (PIHI) is crucial for accurate risk assessment and efficient anesthesia management. However, the incorporation of this evaluation within a predictive framework have been insufficiently addressed and executed. This study aims to developed a machine learning approach for preoperatively and precisely predicting the PIHI index values.

METHODS

In this retrospective study, the valid features were collected from 23,305 adult surgical patients at Peking Union Medical College Hospital between 2012 and 2020. Three hemodynamic response sequences including systolic pressure, diastolic pressure and heart rate, were utilized to design the post-intubation hemodynamic instability (PIHI) index by computing the integrated coefficient of variation (ICV) values. Different types of machine learning models were constructed to predict the ICV values, leveraging preoperative patient information and initiatory drug infusion. The models were trained and cross-validated based on balanced data using the SMOTETomek technique, and their performance was evaluated according to the mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE) and R-squared index (R2).

RESULTS

The ICV values were proved to be consistent with the anesthetists' ratings with Spearman correlation coefficient of 0.877 (P < 0.001), affirming its capability to effectively capture the PIHI variations. The extra tree regression model outperformed the other models in predicting the ICV values with the smallest MAE (0.0512, 95% CI: 0.0511-0.0513), RMSE (0.0792, 95% CI: 0.0790-0.0794), and MAPE (0.2086, 95% CI: 0.2077-0.2095) and the largest R2 (0.9047, 95% CI: 0.9043-0.9052). It was found that the features of age and preoperative hemodynamic status were the most important features for accurately predicting the ICV values.

CONCLUSIONS

Our results demonstrate the potential of the machine learning approach in predicting PIHI index values, thereby preoperatively informing anesthetists the possible anesthetic risk and enabling the implementation of individualized and precise anesthesia interventions.

Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools

BACKGROUND

The utilization of artificial intelligence and machine learning as diagnostic and predictive tools in perioperative medicine holds great promise. Indeed, many studies have been performed in recent years to explore the potential. The purpose of this systematic review is to assess the current state of machine learning in perioperative medicine, its utility in prediction of complications and prognostication, and limitations related to bias and validation.

METHODS

A multidisciplinary team of clinicians and engineers conducted a systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol. Multiple databases were searched, including Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, PubMed, Medline, Embase, and Web of Science. The systematic review focused on study design, type of machine learning model used, validation techniques applied, and reported model performance on prediction of complications and prognostication. This review further classified outcomes and machine learning applications using an ad hoc classification system. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used to assess risk of bias and applicability of the studies.

RESULTS

A total of 103 studies were identified. The models reported in the literature were primarily based on single-center validations (75%), with only 13% being externally validated across multiple centers. Most of the mortality models demonstrated a limited ability to discriminate and classify effectively. The PROBAST assessment indicated a high risk of systematic errors in predicted outcomes and artificial intelligence or machine learning applications.

CONCLUSIONS

The findings indicate that the development of this field is still in its early stages. This systematic review indicates that application of machine learning in perioperative medicine is still at an early stage. While many studies suggest potential utility, several key challenges must be first overcome before their introduction into clinical practice.

EDITOR’S PERSPECTIVE

Machine Learning Algorithm to Perform the American Society of Anesthesiologists Physical Status Classification

OBJECTIVE

The American Society of Anesthesiologists (ASA) Physical Status (PS) Classification System defines perioperative patient scores ranging from 1 to 6 (healthy to brain dead, respectively). The scoring is performed and used by physician anesthesiologists and providers to classify surgical patients based on co-morbidities and various clinical characteristics. There is potentially a variability in scoring stemming from individual biases. The biases impact the prediction of operating times, length of stay in the hospital, anesthetic management, and billing. This study's purpose was to develop an automated system to achieve reproducible scoring.

METHODS

A machine learning (ML) model was trained on already assigned ASA PS scores of 12,064 patients. The ML algorithm was automatically selected by Wolfram Mathematica (Wolfram Research, Champaign, IL) and tested with retrospective records not used in training. Manual scoring was performed by the anesthesiologist as part of the standard preoperative evaluation. Intraclass correlation coefficient (ICC) in R (version 4.2.2; R Development Core Team, Vienna, Austria) was calculated to assess the consistency of scoring.

RESULTS

An ML model was trained on the data corresponding to 12,064 patients. Logistic regression was chosen automatically, with an accuracy of 70.3±1.0% against the training dataset. The accuracy against 1,999 patients (the test dataset) was 69.6±1.0%. The ICC for the comparison between ML and the anesthesiologists' ASA PS scores was greater than 0.4 ("fair to good").

CONCLUSIONS

We have shown the feasibility of applying ML to assess the ASA PS score within an oncology patient population. Though our accuracy was not very good, we feel that, as more data are mined, a valid foundation for refinement to ML will emerge.

Improving accuracy of American Society of Anesthesiologists Physical Status using audit and feedback and artificial intelligence: a time-series analysis

BACKGROUND

While the American Society of Anesthesiologists (ASA) Physical Status (PS) is used to adjust for greater mortality risk with higher ASA PS classification, inaccurate classification can lead to an inaccurate comparison of institutions.

OBJECTIVE

The purpose of this study was to assess the effect of audit and feedback with a rule-based artificial intelligence algorithm on the accuracy of ASA PS classification.

METHODS

We reviewed 78 121 anesthetic records from 1 January 2017 to 19 February 2020. The first intervention entailed audit and feedback emphasizing accurately documenting ASA PS classification using body mass index (BMI), while the second intervention consisted of implementing a rule-based artificial intelligence algorithm. If a patient with a BMI ≥40 kg/m2 had a documented ASA PS classification of 1 or 2, the provider was alerted to change the ASA PS classification to 3 or above. The primary outcome was the overall proportion of patients with inaccurate ASA PS classification based on BMI per month. Secondary outcomes included the proportion of patients with a BMI ≥40 or a BMI 30-39.9 who had inaccurate ASA PS classification and the proportion of patients documented as having ASA 3-5. Data were analyzed using interrupted time-series analysis.

RESULTS

For the primary outcome, the slope for ASA PS classification inaccurately incorporating BMI was unchanging before the first intervention (parameter coefficient 0.002, 95% CI -0.034 to 0.038; P = 0.911). Following the first intervention, there was an immediate level change (parameter coefficient -0.821, 95% CI -1.236 to -0.0406; P < 0.001) without significant change in slope (parameter coefficient -0.048, 95% CI -0.100 to 0.004; P = 0.067). The post-intervention slope was negative (parameter coefficient -0.046, 95% CI -0.083 to -0.009; P = 0.017). Following the second intervention, there was no level change (parameter coefficient 0.203, 95% CI -0.380 to 0.463; P = 0.839) and no significant change in slope (parameter coefficient 0.013, 95% CI -0.043 to 0.043; P = 0.641). The post-intervention slope was not significant (parameter coefficient -0.034, 95% CI -0.078 to 0.010; P = 0.121). The proportion of patients whose ASA PS classification inaccurately incorporated BMI at the first and final timepoint of the study was 2.6% and 0.8%, respectively.

CONCLUSIONS

Our quality improvement efforts successfully modified clinician behavior to accurately incorporate BMI into the ASA PS classification. By combining audit and feedback methodology with a rule-based artificial intelligence algorithm, we created a process that resulted in immediate and sustained effects. Improving ASA PS classification accuracy is important because it affects quality metrics, research design, resource allocation and workflow processes.

Artificial Intelligence and Machine Learning in Anesthesiology

Commercial applications of artificial intelligence and machine learning have made remarkable progress recently, particularly in areas such as image recognition, natural speech processing, language translation, textual analysis, and self-learning. Progress had historically languished in these areas, such that these skills had come to seem ineffably bound to intelligence. However, these commercial advances have performed best at single-task applications in which imperfect outputs and occasional frank errors can be tolerated.The practice of anesthesiology is different. It embodies a requirement for high reliability, and a pressured cycle of interpretation, physical action, and response rather than any single cognitive act. This review covers the basics of what is meant by artificial intelligence and machine learning for the practicing anesthesiologist, describing how decision-making behaviors can emerge from simple equations. Relevant clinical questions are introduced to illustrate how machine learning might help solve them-perhaps bringing anesthesiology into an era of machine-assisted discovery.

Artificial Intelligence: A New Tool in Operating Room Management. Role of Machine Learning Models in Operating Room Optimization

We conducted a systematic review of literature to better understand the role of new technologies in the perioperative period; in particular we focus on the administrative and managerial Operating Room (OR) perspective. Studies conducted on adult (≥ 18 years) patients between 2015 and February 2019 were deemed eligible. A total of 19 papers were included. Our review suggests that the use of Machine Learning (ML) in the field of OR organization has many potentials. Predictions of the surgical case duration were obtain with a good performance; their use could therefore allow a more precise scheduling, limiting waste of resources. ML is able to support even more complex models, which can coordinate multiple spaces simultaneously, as in the case of the post-anesthesia care unit and operating rooms. Types of Artificial Intelligence could also be used to limit another organizational problem, which has important economic repercussions: cancellation. Random Forest has proven effective in identifing surgeries with high risks of cancellation, allowing to plan preventive measures to reduce the cancellation rate accordingly. In conclusion, although data in literature are still limited, we believe that ML has great potential in the field of OR organization; however, further studies are needed to assess the effective role of these new technologies in the perioperative medicine.

PARS, a system combining semantic technologies with multiple criteria decision aiding for supporting antibiotic prescriptions

OBJECTIVE

Motivated by the well documented worldwide spread of adverse drug events, as well as the increased danger of antibiotic resistance (caused mainly by inappropriate prescribing and overuse), we propose a novel recommendation system for antibiotic prescription (PARS).

METHOD

Our approach is based on the combination of semantic technologies with MCDA (Multiple Criteria Decision Aiding) that allowed us to build a two level decision support model. Given a specific domain, the approach assesses the adequacy of an alternative/action (prescription of antibiotic) for a specific subject (patient) with an issue (bacterial infection) in a given context (medical). The goal of the first level of the decision support model is to select the set of alternatives which have the potential to be suitable. Then the second level sorts the alternatives into categories according to their adequacy using an MCDA sorting method (MR-Sort with Veto) and a structured set of description logic queries.

RESULTS

We applied this approach in the domain of antibiotic prescriptions, working closely with the EpiCura Hospital Center (BE). Its performance was compared to the EpiCura recommendation guidelines which are currently in use. The results showed that the proposed system is more consistent in its recommendations when compared with the static EpiCura guidelines. Moreover, with PARS the antibiotic prescribing workflow becomes more flexible. PARS allows the user (physician) to update incrementally and dynamically a patient's profile with more information, or to input knowledge modifications that accommodate the decision context (like the introduction of new side effects and antibiotics, the development of germs that are resistant, etc). At the end of our evaluation, we detail a number of limitations of the current version of PARS and discuss future perspectives.

Cardiac biomarkers improve prediction performance of the combination of American Society of Anesthesiologists physical status classification and Americal College of Surgeons National Surgical Quality Improvement Program calculator for postoperative mortality in elderly patients: a pilot study

BACKGROUND

Our previous research has shown American Society of Anaesthesiologists physical status classification (ASA) score and Americal College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) calculator to have the most accuracy in the prediction of postoperative mortality.

AIMS

The aim of our research was to define the most reliable combination of cardiac biomarkers with ASA and ACS NSQIP.

METHODS

We have included a total of 78 patients. ASA score has been determined in standard fashion, while we used the available interactive calculator for the ACS NSQIP score. Biomarkers BIRC5, H-FABP, and hsCRP have been measured in specialized laboratories.

RESULTS

All of the deceased patients had survivin (BIRC5) > 4.00 pg/ml, higher values of H-FABP and hsCRP and higher estimated levels of ASA and ACS NSQIP (P = 0.0001). ASA and ACS NSQIP alone had AUC of, respectively, 0.669 and 0.813. The combination of ASA and ACS NSQIP had AUC = 0.841. Combination of hsCRP with the two risk scores had AUC = 0.926 (95% CI 0.853-1.000, P < 0.0001). If we add three cardiac biomarkers to this model, we get AUC as high as 0.941 (95% CI 0.876-1.000, P < 0.0001). The correction of statistical models with comorbidities (CIRS-G score) did not change the accuracy of prediction models that we have provided.

DISCUSSION

Addition of ACS NSQIP and biomarkers adds to the accuracy of ASA score, which has already been proved by other authors.

CONCLUSION

Cardiac biomarker hsCRP can be used as the most reliable cardiac biomarker; however, the "multimarker approach" adds the most to the accuracy of the combination of clinical risk scores.