MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery

Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery

BACKGROUND

Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG.

HYOTHESIS

ML algorithms can significantly improve mortality prediction after CABG.

METHODS

Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models.

RESULTS

Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients.

CONCLUSIONS

All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).

Performance and usability of pre-operative prediction models for 30-day peri-operative mortality risk: a systematic review

Estimating pre-operative mortality risk may inform clinical decision-making for peri-operative care. However, pre-operative mortality risk prediction models are rarely implemented in routine clinical practice. High predictive accuracy and clinical usability are essential for acceptance and clinical implementation. In this systematic review, we identified and appraised prediction models for 30-day postoperative mortality in non-cardiac surgical cohorts. PubMed and Embase were searched up to December 2022 for studies investigating pre-operative prediction models for 30-day mortality. We assessed predictive performance in terms of discrimination and calibration. Risk of bias was evaluated using a tool to assess the risk of bias and applicability of prediction model studies. To further inform potential adoption, we also assessed clinical usability for selected models. In all, 15 studies evaluating 10 prediction models were included. Discrimination ranged from a c-statistic of 0.82 (MySurgeryRisk) to 0.96 (extreme gradient boosting machine learning model). Calibration was reported in only six studies. Model performance was highest for the surgical outcome risk tool (SORT) and its external validations. Clinical usability was highest for the surgical risk pre-operative assessment system. The SORT and risk quantification index also scored high on clinical usability. We found unclear or high risk of bias in the development of all models. The SORT showed the best combination of predictive performance and clinical usability and has been externally validated in several heterogeneous cohorts. To improve clinical uptake, full integration of reliable models with sufficient face validity within the electronic health record is imperative.

Building an automated, machine learning-enabled platform for predicting post-operative complications

Objective. In 2019, the University of Florida College of Medicine launched theMySurgeryRiskalgorithm to predict eight major post-operative complications using automatically extracted data from the electronic health record.Approach. This project was developed in parallel with our Intelligent Critical Care Center and represents a culmination of efforts to build an efficient and accurate model for data processing and predictive analytics.Main Results and Significance. This paper discusses how our model was constructed and improved upon. We highlight the consolidation of the database, processing of fixed and time-series physiologic measurements, development and training of predictive models, and expansion of those models into different aspects of patient assessment and treatment. We end by discussing future directions of the model.

Overtriage, Undertriage, and Value of Care after Major Surgery: An Automated, Explainable Deep Learning-Enabled Classification System

BACKGROUND

In single-institution studies, overtriaging low-risk postoperative patients to ICUs has been associated with a low value of care; undertriaging high-risk postoperative patients to general wards has been associated with increased mortality and morbidity. This study tested the reproducibility of an automated postoperative triage classification system to generating an actionable, explainable decision support system.

STUDY DESIGN

This longitudinal cohort study included adults undergoing inpatient surgery at two university hospitals. Triage classifications were generated by an explainable deep learning model using preoperative and intraoperative electronic health record features. Nearest neighbor algorithms identified risk-matched controls. Primary outcomes were mortality, morbidity, and value of care (inverted risk-adjusted mortality/total direct costs).

RESULTS

Among 4,669 ICU admissions, 237 (5.1%) were overtriaged. Compared with 1,021 control ward admissions, overtriaged admissions had similar outcomes but higher costs ($15.9K [interquartile range $9.8K to $22.3K] vs $10.7K [$7.0K to $17.6K], p < 0.001) and lower value of care (0.2 [0.1 to 0.3] vs 1.5 [0.9 to 2.2], p < 0.001). Among 8,594 ward admissions, 1,029 (12.0%) were undertriaged. Compared with 2,498 control ICU admissions, undertriaged admissions had longer hospital length-of-stays (6.4 [3.4 to 12.4] vs 5.4 [2.6 to 10.4] days, p < 0.001); greater incidence of hospital mortality (1.7% vs 0.7%, p = 0.03), cardiac arrest (1.4% vs 0.5%, p = 0.04), and persistent acute kidney injury without renal recovery (5.2% vs 2.8%, p = 0.002); similar costs ($21.8K [$13.3K to $34.9K] vs $21.9K [$13.1K to $36.3K]); and lower value of care (0.8 [0.5 to 1.3] vs 1.2 [0.7 to 2.0], p < 0.001).

CONCLUSIONS

Overtriage was associated with low value of care; undertriage was associated with both low value of care and increased mortality and morbidity. The proposed framework for generating automated postoperative triage classifications is reproducible.

Postoperative Overtriage to an Intensive Care Unit Is Associated With Low Value of Care

OBJECTIVE

We test the hypothesis that for low-acuity surgical patients, postoperative intensive care unit (ICU) admission is associated with lower value of care compared with ward admission.

BACKGROUND

Overtriaging low-acuity patients to ICU consumes valuable resources and may not confer better patient outcomes. Associations among postoperative overtriage, patient outcomes, costs, and value of care have not been previously reported.

METHODS

In this longitudinal cohort study, postoperative ICU admissions were classified as overtriaged or appropriately triaged according to machine learning-based patient acuity assessments and requirements for immediate postoperative mechanical ventilation or vasopressor support. The nearest neighbors algorithm identified risk-matched control ward admissions. The primary outcome was value of care, calculated as inverse observed-to-expected mortality ratios divided by total costs.

RESULTS

Acuity assessments had an area under the receiver operating characteristic curve of 0.92 in generating predictions for triage classifications. Of 8592 postoperative ICU admissions, 423 (4.9%) were overtriaged. These were matched with 2155 control ward admissions with similar comorbidities, incidence of emergent surgery, immediate postoperative vital signs, and do not resuscitate order placement and rescindment patterns. Compared with controls, overtraiged admissions did not have a lower incidence of any measured complications. Total costs for admission were $16.4K for overtriage and $15.9K for controls ( P =0.03). Value of care was lower for overtriaged admissions [2.9 (2.0-4.0)] compared with controls [24.2 (14.1-34.5), P <0.001].

CONCLUSIONS

Low-acuity postoperative patients who were overtriaged to ICUs had increased total costs, no improvements in outcomes, and received low-value care.

Dynamic predictions of postoperative complications from explainable, uncertainty-aware, and multi-task deep neural networks

Accurate prediction of postoperative complications can inform shared decisions regarding prognosis, preoperative risk-reduction, and postoperative resource use. We hypothesized that multi-task deep learning models would outperform conventional machine learning models in predicting postoperative complications, and that integrating high-resolution intraoperative physiological time series would result in more granular and personalized health representations that would improve prognostication compared to preoperative predictions. In a longitudinal cohort study of 56,242 patients undergoing 67,481 inpatient surgical procedures at a university medical center, we compared deep learning models with random forests and XGBoost for predicting nine common postoperative complications using preoperative, intraoperative, and perioperative patient data. Our study indicated several significant results across experimental settings that suggest the utility of deep learning for capturing more precise representations of patient health for augmented surgical decision support. Multi-task learning improved efficiency by reducing computational resources without compromising predictive performance. Integrated gradients interpretability mechanisms identified potentially modifiable risk factors for each complication. Monte Carlo dropout methods provided a quantitative measure of prediction uncertainty that has the potential to enhance clinical trust. Multi-task learning, interpretability mechanisms, and uncertainty metrics demonstrated potential to facilitate effective clinical implementation.

Validation of Adult Spinal Deformity Surgical Outcome Prediction Tools in Adult Symptomatic Lumbar Scoliosis

STUDY DESIGN

A post hoc analysis.

OBJECTIVE

Advances in machine learning (ML) have led to tools offering individualized outcome predictions for adult spinal deformity (ASD). Our objective is to examine the properties of these ASD models in a cohort of adult symptomatic lumbar scoliosis (ASLS) patients.

SUMMARY OF BACKGROUND DATA

ML algorithms produce patient-specific probabilities of outcomes, including major complication (MC), reoperation (RO), and readmission (RA) in ASD. External validation of these models is needed.

METHODS

Thirty-nine predictive factors (12 demographic, 9 radiographic, 4 health-related quality of life, 14 surgical) were retrieved and entered into web-based prediction models for MC, unplanned RO, and hospital RA. Calculated probabilities were compared with actual event rates. Discrimination and calibration were analyzed using receiver operative characteristic area under the curve (where 0.5=chance, 1=perfect) and calibration curves (Brier scores, where 0.25=chance, 0=perfect). Ninety-five percent confidence intervals are reported.

RESULTS

A total of 169 of 187 (90%) surgical patients completed 2-year follow up. The observed rate of MCs was 41.4% with model predictions ranging from 13% to 68% (mean: 38.7%). RO was 20.7% with model predictions ranging from 9% to 54% (mean: 30.1%). Hospital RA was 17.2% with model predictions ranging from 13% to 50% (mean: 28.5%). Model classification for all three outcome measures was better than chance for all [area under the curve=MC 0.6 (0.5-0.7), RA 0.6 (0.5-0.7), RO 0.6 (0.5-0.7)]. Calibration was better than chance for all, though best for RA and RO (Brier Score=MC 0.22, RA 0.16, RO 0.17).

CONCLUSIONS

ASD prediction models for MC, RA, and RO performed better than chance in a cohort of adult lumbar scoliosis patients, though the homogeneity of ASLS affected calibration and accuracy. Optimization of models require samples with the breadth of outcomes (0%-100%), supporting the need for continued data collection as personalized prediction models may improve decision-making for the patient and surgeon alike.

A Surgeon's Guide to Artificial Intelligence-Driven Predictive Models

Artificial intelligence (AI) focuses on processing and interpreting complex information as well as identifying relationships and patterns among complex data. Artificial intelligence- and machine learning (ML)-driven predictions have shown promising potential in influencing real-time decisions and improving surgical outcomes by facilitating screening, diagnosis, risk assessment, preoperative planning, and shared decision-making. Fundamental understanding of the algorithms, as well as their development and interpretation, is essential for the evolution of AI in surgery. In this article, we provide surgeons with a fundamental understanding of AI-driven predictive models through an overview of common ML and deep learning algorithms, model development, performance metrics and interpretation. This would serve as a basis for understanding ML-based research, while fostering new ideas and innovations for furthering the reach of this emerging discipline.

A Synopsis of Artificial Intelligence and its Applications in Surgery

Artificial intelligence (AI) has made steady in-roads into the healthcare scenario over the last decade. While widespread adoption into clinical practice remains elusive, the outreach of this discipline has progressed beyond the physician scientist, and different facets of this technology have been incorporated into the care of surgical patients. New AI applications are developing at rapid pace, and it is imperative that the general surgeon be aware of the broad utility of AI as applicable in his or her day-to-day practice, so that healthcare continues to remain up-to-date and evidence based. This review provides a broad account of the tip of the AI iceberg and highlights it potential for positively impacting surgical care.

Effect of Present at Time of Surgery on Unadjusted and Risk-Adjusted Postoperative Complication Rate

BACKGROUND

Present at the time of surgery (PATOS) is an important measure to collect in postoperative complication surveillance systems because it may affect a patient's risk of a subsequent complication and the estimation of postoperative complication rates attributed to the healthcare system. The American College of Surgeons (ACS) NSQIP started collecting PATOS data for 8 postoperative complications in 2011, but no one has used these data to quantify how this may affect unadjusted and risk-adjusted postoperative complication rates.

STUDY DESIGN

This study was a retrospective observational study of the ACS NSQIP database from 2012 to 2018. PATOS data were analyzed for the 8 postoperative complications of superficial, deep, and organ space surgical site infection; pneumonia; urinary tract infection; ventilator dependence; sepsis; and septic shock. Unadjusted postoperative complication rates were compared ignoring PATOS vs taking PATOS into account. Observed to expected ratios over time were also compared by calculating expected values using multiple logistic regression analyses with complication as the dependent variable and the 28 nonlaboratory preoperative variables in the ACS NSQIP database as the independent variables.

RESULTS

In 5,777,108 patients, observed event rates for each outcome were reduced by between 6.1% (superficial surgical site infection) and 52.5% (sepsis) when PATOS was taken into account. The observed to expected ratios were similar each year for all outcomes, except for sepsis and septic shock in the early years.

CONCLUSIONS

Taking PATOS into account is important for reporting unadjusted event rates. The effect varied by type of complication-lowest for superficial surgical site infection and highest for sepsis and septic shock. Taking PATOS into account was less important for risk-adjusted outcomes (observed to expected ratios), except for sepsis and septic shock.

Artificial Intelligence and Machine Learning in Prediction of Surgical Complications: Current State, Applications, and Implications

Surgical complications pose significant challenges for surgeons, patients, and health care systems as they may result in patient distress, suboptimal outcomes, and higher health care costs. Artificial intelligence (AI)-driven models have revolutionized the field of surgery by accurately identifying patients at high risk of developing surgical complications and by overcoming several limitations associated with traditional statistics-based risk calculators. This article aims to provide an overview of AI in predicting surgical complications using common machine learning and deep learning algorithms and illustrates how this can be utilized to risk stratify patients preoperatively. This can form the basis for discussions on informed consent based on individualized patient factors in the future.

Artificial intelligence for the prediction of acute kidney injury during the perioperative period: systematic review and Meta-analysis of diagnostic test accuracy

BACKGROUND

Acute kidney injury (AKI) is independently associated with morbidity and mortality in a wide range of surgical settings. Nowadays, with the increasing use of electronic health records (EHR), advances in patient information retrieval, and cost reduction in clinical informatics, artificial intelligence is increasingly being used to improve early recognition and management for perioperative AKI. However, there is no quantitative synthesis of the performance of these methods. We conducted this systematic review and meta-analysis to estimate the sensitivity and specificity of artificial intelligence for the prediction of acute kidney injury during the perioperative period.

METHODS

Pubmed, Embase, and Cochrane Library were searched to 2nd October 2021. Studies presenting diagnostic performance of artificial intelligence in the early detection of perioperative acute kidney injury were included. True positives, false positives, true negatives and false negatives were pooled to collate specificity and sensitivity with 95% CIs and results were portrayed in forest plots. The risk of bias of eligible studies was assessed using the PROBAST tool.

RESULTS

Nineteen studies involving 304,076 patients were included. Quantitative random-effects meta-analysis using the Rutter and Gatsonis hierarchical summary receiver operating characteristics (HSROC) model revealed pooled sensitivity, specificity, and diagnostic odds ratio of 0.77 (95% CI: 0.73 to 0.81),0.75 (95% CI: 0.71 to 0.80), and 10.7 (95% CI 8.5 to 13.5), respectively. Threshold effect was found to be the only source of heterogeneity, and there was no evidence of publication bias.

CONCLUSIONS

Our review demonstrates the promising performance of artificial intelligence for early prediction of perioperative AKI. The limitations of lacking external validation performance and being conducted only at a single center should be overcome.

TRIAL REGISTRATION

This study was not registered with PROSPERO.

Machine learning models for predicting acute kidney injury: a systematic review and critical appraisal

BACKGROUND

The number of studies applying machine learning (ML) to predict acute kidney injury (AKI) has grown steadily over the past decade. We assess and critically appraise the state of the art in ML models for AKI prediction, considering performance, methodological soundness, and applicability.

METHODS

We searched PubMed and ArXiv, extracted data, and critically appraised studies based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD), Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), and Prediction Model Risk of Bias Assessment Tool (PROBAST) guidelines.

RESULTS

Forty-six studies from 3166 titles were included. Thirty-eight studies developed a model, five developed and externally validated one, and three studies externally validated one. Flexible ML methods were used more often than deep learning, although the latter was common with temporal variables and text as predictors. Predictive performance showed an area under receiver operating curves ranging from 0.49 to 0.99. Our critical appraisal identified a high risk of bias in 39 studies. Some studies lacked internal validation, whereas external validation and interpretability of results were rarely considered. Fifteen studies focused on AKI prediction in the intensive care setting, and the US-derived Medical Information Mart for Intensive Care (MIMIC) data set was commonly used. Reproducibility was limited as data and code were usually unavailable.

CONCLUSIONS

Flexible ML methods are popular for the prediction of AKI, although more complex models based on deep learning are emerging. Our critical appraisal identified a high risk of bias in most models: Studies should use calibration measures and external validation more often, improve model interpretability, and share data and code to improve reproducibility.

Risk stratification with explainable machine learning for 30-day procedure-related mortality and 30-day unplanned readmission in patients with peripheral arterial disease

Predicting 30-day procedure-related mortality risk and 30-day unplanned readmission in patients undergoing lower extremity endovascular interventions for peripheral artery disease (PAD) may assist in improving patient outcomes. Risk prediction of 30-day mortality can help clinicians identify treatment plans to reduce the risk of death, and prediction of 30-day unplanned readmission may improve outcomes by identifying patients who may benefit from readmission prevention strategies. The goal of this study is to develop machine learning models to stratify risk of 30-day procedure-related mortality and 30-day unplanned readmission in patients undergoing lower extremity infra-inguinal endovascular interventions. We used a cohort of 14,444 cases from the American College of Surgeons National Surgical Quality Improvement Program database. For each outcome, we developed and evaluated multiple machine learning models, including Support Vector Machines, Multilayer Perceptrons, and Gradient Boosting Machines, and selected a random forest as the best-performing model for both outcomes. Our 30-day procedure-related mortality model achieved an AUC of 0.75 (95% CI: 0.71-0.79) and our 30-day unplanned readmission model achieved an AUC of 0.68 (95% CI: 0.67-0.71). Stratification of the test set by race (white and non-white), sex (male and female), and age (≥65 years and <65 years) and subsequent evaluation of demographic parity by AUC shows that both models perform equally well across race, sex, and age groups. We interpret the model globally and locally using Gini impurity and SHapley Additive exPlanations (SHAP). Using the top five predictors for death and mortality, we demonstrate differences in survival for subgroups stratified by these predictors, which underscores the utility of our model.

Postsurgical Remote Patient Monitoring Outcomes and Perceptions: A Mixed-Methods Assessment

Objective

To determine how postsurgical remote patient monitoring (RPM) influences readmissions and emergency visits within 30 days of discharge after operation and to understand patient and surgeon perspectives on postsurgical RPM.

Patients and Methods

This study was conducted at a US tertiary academic medical center between April 1, 2021, and December 31, 2021. This mixed-methods evaluation included a randomized controlled trial evaluation of RPM after operation and a qualitative assessment of patients' and surgeons' perceptions of RPM's acceptability, feasibility, and effectiveness.

Results

A total of 292 patients participated in the RPM trial, and 147 were assigned to the RPM intervention. Despite a good balance between the groups, results indicated no difference in primary or secondary outcomes between the intervention and control groups. The qualitative component included 11 patients and 9 surgeons. The overarching theme for patients was that the program brought them peace of mind. Other main themes included technological issues and perceived benefits of the RPM platform. The major themes for surgeons included identifying the best patients to receive postsurgical RPM, actionable data collection and use, and improvements in data collection needed.

Conclusion

Although quantitative results indicate no difference between the groups, postsurgical RPM appears well-accepted from the patient's perspective. However, technological issues could eliminate the benefits. Hospitals seeking to implement similar programs should carefully evaluate which populations to use the program in and seek to collect actionable data.

The use and future perspective of Artificial Intelligence-A survey among German surgeons

Purpose

Clinical abundance of artificial intelligence has increased significantly in the last decade. This survey aims to provide an overview of the current state of knowledge and acceptance of AI applications among surgeons in Germany.

Methods

A total of 357 surgeons from German university hospitals, academic teaching hospitals and private practices were contacted by e-mail and asked to participate in the anonymous survey.

Results

A total of 147 physicians completed the survey. The majority of respondents (n = 85, 52.8%) stated that they were familiar with AI applications in medicine. Personal knowledge was self-rated as average (n = 67, 41.6%) or rudimentary (n = 60, 37.3%) by the majority of participants. On the basis of various application scenarios, it became apparent that the respondents have different demands on AI applications in the area of "diagnosis confirmation" as compared to the area of "therapy decision." For the latter category, the requirements in terms of the error level are significantly higher and more respondents view their application in medical practice rather critically. Accordingly, most of the participants hope that AI systems will primarily improve diagnosis confirmation, while they see their ethical and legal problems with regard to liability as the main obstacle to extensive clinical application.

Conclusion

German surgeons are in principle positively disposed toward AI applications. However, many surgeons see a deficit in their own knowledge and in the implementation of AI applications in their own professional environment. Accordingly, medical education programs targeting both medical students and healthcare professionals should convey basic knowledge about the development and clinical implementation process of AI applications in different medical fields, including surgery.

Comparison of Machine Learning Models Including Preoperative, Intraoperative, and Postoperative Data and Mortality After Cardiac Surgery

IMPORTANCE

A variety of perioperative risk factors are associated with postoperative mortality risk. However, the relative contribution of routinely collected intraoperative clinical parameters to short-term and long-term mortality remains understudied.

OBJECTIVE

To examine the performance of multiple machine learning models with data from different perioperative periods to predict 30-day, 1-year, and 5-year mortality and investigate factors that contribute to these predictions.

DESIGN, SETTING, AND PARTICIPANTS

In this prognostic study using prospectively collected data, risk prediction models were developed for short-term and long-term mortality after cardiac surgery. Included participants were adult patients undergoing a first-time valve operation, coronary artery bypass grafting, or a combination of both between 1997 and 2017 in a single center, the University Medical Centre Groningen in the Netherlands. Mortality data were obtained in November 2017. Data analysis took place between February 2020 and August 2021.

EXPOSURE

Cardiac surgery.

MAIN OUTCOMES AND MEASURES

Postoperative mortality rates at 30 days, 1 year, and 5 years were the primary outcomes. The area under the receiver operating characteristic curve (AUROC) was used to assess discrimination. The contribution of all preoperative, intraoperative hemodynamic and temperature, and postoperative factors to mortality was investigated using Shapley additive explanations (SHAP) values.

RESULTS

Data from 9415 patients who underwent cardiac surgery (median [IQR] age, 68 [60-74] years; 2554 [27.1%] women) were included. Overall mortality rates at 30 days, 1 year, and 5 years were 268 patients (2.8%), 420 patients (4.5%), and 612 patients (6.5%), respectively. Models including preoperative, intraoperative, and postoperative data achieved AUROC values of 0.82 (95% CI, 0.78-0.86), 0.81 (95% CI, 0.77-0.85), and 0.80 (95% CI, 0.75-0.84) for 30-day, 1-year, and 5-year mortality, respectively. Models including only postoperative data performed similarly (30 days: 0.78 [95% CI, 0.73-0.82]; 1 year: 0.79 [95% CI, 0.74-0.83]; 5 years: 0.77 [95% CI, 0.73-0.82]). However, models based on all perioperative data provided less clinically usable predictions, with lower detection rates; for example, postoperative models identified a high-risk group with a 2.8-fold increase in risk for 5-year mortality (4.1 [95% CI, 3.3-5.1]) vs an increase of 11.3 (95% CI, 6.8-18.7) for the high-risk group identified by the full perioperative model. Postoperative markers associated with metabolic dysfunction and decreased kidney function were the main factors contributing to mortality risk.

CONCLUSIONS AND RELEVANCE

This study found that the addition of continuous intraoperative hemodynamic and temperature data to postoperative data was not associated with improved machine learning-based identification of patients at increased risk of short-term and long-term mortality after cardiac operations.

Complications and Failure to Rescue After Inpatient Pediatric Surgery

OBJECTIVE

To describe the frequency and patterns of postoperative complications and FTR after inpatient pediatric surgical procedures and to evaluate the association between number of complications and FTR.

SUMMARY AND BACKGROUND

FTR, or a postoperative death after a complication, is currently a nationally endorsed quality measure for adults. Although it is a contributing factor to variation in mortality, relatively little is known about FTR after pediatric surgery.

METHODS

Cohort study of 200,554 patients within the National Surgical Quality Improvement Program-Pediatric database (2012-2016) who underwent a high (≥ 1%) or low (< 1%) mortality risk inpatient surgical procedures. Patients were stratified based on number of postoperative complications (0, 1, 2, or ≥3) and further categorized as having undergone either a low- or high-risk procedure. The association between the number of postoperative complications and FTR was evaluated with multivariable logistic regression.

RESULTS

Among patients who underwent a low- (89.4%) or high-risk (10.6%) procedures, 14.0% and 12.5% had at least 1 postoperative complication, respectively. FTR rates after low- and high-risk procedures demonstrated step-wise increases as the number of complications accrued (eg, low-risk- 9.2% in patients with ≥3 complications; high-risk-36.9% in patients with ≥ 3 complications). Relative to patients who had no complications, there was a dose-response relationship between mortality and the number of complications after low-risk [1 complication - odds ratio (OR) 3.34 (95% CI 2.62-4.27); 2 - OR 10.15 (95% CI 7.40-13.92); ≥3-27.48 (95% CI 19.06-39.62)] and high-risk operations [1 - OR 3.29 (2.61-4.16); 2-7.24 (5.14-10.19); ≥3-20.73 (12.62-34.04)].

CONCLUSIONS

There is a dose-response relationship between the number of postoperative complications after inpatient surgery and FTR, ever after common, "minor" surgical procedures. These findings suggest FTR may be a potential quality measure for pediatric surgical care.

Association of prematurity with complications and failure to rescue in neonatal surgery

BACKGROUND

The majority of failure to rescue (FTR), or death after a postoperative complication, in pediatric surgery occurs among infants and neonates. The purpose of this study is to evaluate the association between gestational age (GA) and FTR in infants and neonates.

METHODS

National cohort study of 46,452 patients < 1 year old within the National Surgical Quality Improvement Program-Pediatric database who underwent inpatient surgery. Patients were categorized as preterm neonates, term neonates, or infants. Neonates were stratified based on GA. Surgical procedures were classified as low- (< 1% mortality) or high-risk (≥ 1%). Multivariable logistic regression and cubic splines were used to evaluate the association between GA and FTR.

RESULTS

Preterm neonates had the highest FTR (28%) rates. Among neonates, FTR increased with decreasing GA (≥ 37 weeks, 12%; 33-36 weeks, 15%; 29-32 weeks, 30%; 25-28 weeks 41%; ≤ 24 weeks, 57%). For both low- and high-risk procedures, FTR significantly (trend test, p < 0.01) increased with decreasing GA. When stratifying preterm neonates by GA, all GAs ≤ 28 weeks were associated with significantly higher odds of FTR for low- (OR 2.47, 95% CI [1.38-4.41]) and high-risk (OR 2.27, 95% CI [1.33-3.87]) procedures. A lone inflection point for FTR was identified at 31-32 weeks with cubic spline analysis.

CONCLUSIONS

The dose-dependent relationship between decreasing GA and FTR as well as the FTR inflection point noted at GA 31-32 weeks can be used by stakeholders in designing quality improvement initiatives and directing perioperative care.

LEVEL OF EVIDENCE

Level IV, Retrospective cohort study.

Prediction and Prevention of Intraoperative Hypotension with the Hypotension Prediction Index: A Narrative Review

Intraoperative hypotension is common and has been associated with adverse events. Although association does not imply causation, predicting and preventing hypotension may improve postoperative outcomes. This review summarizes current evidence on the development and validation of an artificial intelligence predictive algorithm, the Hypotension Prediction (HPI) (formerly known as the Hypotension Probability Indicator). This machine learning model can arguably predict hypotension up to 15 min before its occurrence. Several validation studies, retrospective cohorts, as well as a few prospective randomized trials, have been published in the last years, reporting promising results. Larger trials are needed to definitively assess the usefulness of this algorithm in optimizing postoperative outcomes.

Personalized decision-making for acute cholecystitis: Understanding surgeon judgment

Background

There is sparse high-level evidence to guide treatment decisions for severe, acute cholecystitis (inflammation of the gallbladder). Therefore, treatment decisions depend heavily on individual surgeon judgment, which is highly variable and potentially amenable to personalized, data-driven decision support. We test the hypothesis that surgeons' treatment recommendations misalign with perceived risks and benefits for laparoscopic cholecystectomy (surgical removal) vs. percutaneous cholecystostomy (image-guided drainage).

Methods

Surgery attendings, fellows, and residents applied individual judgement to standardized case scenarios in a live, web-based survey in estimating the quantitative risks and benefits of laparoscopic cholecystectomy vs. percutaneous cholecystostomy for both moderate and severe acute cholecystitis, as well as the likelihood that they would recommend cholecystectomy.

Results

Surgeons predicted similar 30-day morbidity rates for laparoscopic cholecystectomy and percutaneous cholecystostomy. However, a greater proportion of surgeons predicted low (<50%) likelihood of full recovery following percutaneous cholecystostomy compared with cholecystectomy for both moderate (30% vs. 2%, p < 0.001) and severe (62% vs. 38%, p < 0.001) cholecystitis. Ninety-eight percent of all surgeons were likely or very likely to recommend cholecystectomy for moderate cholecystitis; only 32% recommended cholecystectomy for severe cholecystitis (p < 0.001). There were no significant differences in predicted postoperative morbidity when respondents were stratified by academic rank or self-reported ability to predict complications or make treatment recommendations.

Conclusions

Surgeon recommendations for severe cholecystitis were discordant with perceived risks and benefits of treatment options. Surgeons predicted greater functional recovery after cholecystectomy but less than one-third recommended cholecystectomy. These findings suggest opportunities to augment surgical decision-making with personalized, data-driven decision support.

Artificial intelligence and machine learning in precision medicine: A paradigm shift in big data analysis

The integration of artificial intelligence in precision medicine has revolutionized healthcare delivery. Precision medicine identifies the phenotype of particular patients with less-common responses to treatment. Recent studies have demonstrated that translational research exploring the convergence between artificial intelligence and precision medicine will help solve the most difficult challenges facing precision medicine. Here, we discuss different aspects of artificial intelligence in precision medicine that improve healthcare delivery. First, we discuss how artificial intelligence changes the landscape of precision medicine and the evolution of artificial intelligence in precision medicine. Second, we highlight the synergies between artificial intelligence and precision medicine and promises of artificial intelligence and precision medicine in healthcare delivery. Third, we briefly explain the promise of big data analytics and the integration of nanomaterials in precision medicine. Last, we highlight the challenges and opportunities of artificial intelligence in precision medicine.

Prediction of low cardiac output syndrome in patients following cardiac surgery using machine learning

Background

This study aimed to develop machine learning models to predict Low Cardiac Output Syndrome (LCOS) in patients following cardiac surgery using machine learning algorithms.

Methods

The clinical data of cardiac surgery patients in Nanjing First Hospital between June 2019 and November 2020 were retrospectively extracted from the electronic medical records. Six conventional machine learning algorithms, including logistic regression, support vector machine, decision tree, random forest, extreme gradient boosting and light gradient boosting machine, were employed to construct the LCOS predictive models with all predictive features (full models) and selected predictive features (reduced models). The discrimination of these models was evaluated by the area under the receiver operating characteristic curve (AUC) and the calibration of the models was assessed by the calibration curve. Shapley Additive explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) were used to interpret the predictive models.

Results

Data from 1,585 patients [982 (62.0%) were male, aged 18 to 88, 212 (13.4%) with LCOS] were employed to train and validate the LCOS models. Among the full models, the RF model (AUC: 0.909, 95% CI: 0.875–0.943; Sensitivity: 0.849, 95% CI: 0.724–0.933; Specificity: 0.835, 95% CI: 0.796–0.869) and the XGB model (AUC: 0.897, 95% CI: 0.859–0.935; Sensitivity: 0.830, 95% CI: 0.702–0.919; Specificity: 0.809, 95% CI: 0.768–0.845) exhibited well predictive power for LCOS. Eleven predictive features including left ventricular ejection fraction (LVEF), first post-operative blood lactate (Lac), left ventricular diastolic diameter (LVDd), cumulative time of mean artery blood pressure (MABP) lower than 65 mmHg (MABP &lt; 65 time), hypertension history, platelets level (PLT), age, blood creatinine (Cr), total area under curve above threshold central venous pressure (CVP) 12 mmHg and 16 mmHg, and blood loss during operation were used to build the reduced models. Among the reduced models, RF model (AUC: 0.895, 95% CI: 0.857–0.933; Sensitivity: 0.830, 95% CI: 0.702–0.919; Specificity: 0.806, 95% CI: 0.765–0.843) revealed the best performance. SHAP and LIME plot showed that LVEF, Lac, LVDd and MABP &lt; 65 time significantly contributed to the prediction model.

Conclusion

In this study, we successfully developed several machine learning models to predict LCOS after surgery, which may avail to risk stratification, early detection and management of LCOS after cardiac surgery.

Machine learning algorithms for predicting mortality after coronary artery bypass grafting

Background

As the era of big data analytics unfolds, machine learning (ML) might be a promising tool for predicting clinical outcomes. This study aimed to evaluate the predictive ability of ML models for estimating mortality after coronary artery bypass grafting (CABG).

Materials and methods

Various baseline and follow-up features were obtained from the CABG data registry, established in 2005 at Tehran Heart Center. After selecting key variables using the random forest method, prediction models were developed using: Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), K-Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), and Random Forest (RF) algorithms. Area Under the Curve (AUC) and other indices were used to assess the performance.

Results

A total of 16,850 patients with isolated CABG (mean age: 67.34 ± 9.67 years) were included. Among them, 16,620 had one-year follow-up, from which 468 died. Eleven features were chosen to train the models. Total ventilation hours and left ventricular ejection fraction were by far the most predictive factors of mortality. All the models had AUC &gt; 0.7 (acceptable performance) for 1-year mortality. Nonetheless, LR (AUC = 0.811) and XGBoost (AUC = 0.792) outperformed NB (AUC = 0.783), RF (AUC = 0.783), SVM (AUC = 0.738), and KNN (AUC = 0.715). The trend was similar for two-to-five-year mortality, with LR demonstrating the highest predictive ability.

Conclusion

Various ML models showed acceptable performance for estimating CABG mortality, with LR illustrating the highest prediction performance. These models can help clinicians make decisions according to the risk of mortality in patients undergoing CABG.

Communicating Precision Medicine Research: Multidisciplinary Teams and Diverse Communities

INTRODUCTION

Precision medicine research investigates the differences in individuals' genetics, environment, and lifestyle to tailor health prevention and treatment options as part of an emerging model of health care delivery. Advancing precision medicine research will require effective communication across a wide range of scientific and health care disciplines and with research participants who represent diverse segments of the population.

METHODS

A multidisciplinary group convened over the course of a year and developed precision medicine research case examples to facilitate precision medicine research discussions with communities.

RESULTS

A shared definition of precision medicine research as well as six case examples of precision medicine research involving genetic risk, pharmacogenetics, epigenetics, the microbiome, mobile health, and electronic health records were developed.

DISCUSSION/CONCLUSION

The precision medicine research definition and case examples can be used as planning tools to establish a shared understanding of the scope of precision medicine research across multidisciplinary teams and with the diverse communities in which precision medicine research will take place. This shared understanding is vital for successful and equitable progress in precision medicine.

Towards interpretable, medically grounded, EMR-based risk prediction models

Machine-learning based risk prediction models have the potential to improve patient outcomes by assessing risk more accurately than clinicians. Significant additional value lies in these models providing feedback about the factors that amplify an individual patient's risk. Identification of risk factors enables more informed decisions on interventions to mitigate or ameliorate modifiable factors. For these reasons, risk prediction models must be explainable and grounded on medical knowledge. Current machine learning-based risk prediction models are frequently 'black-box' models whose inner workings cannot be understood easily, making it difficult to define risk drivers. Since machine learning models follow patterns in the data rather than looking for medically relevant relationships, possible risk factors identified by these models do not necessarily translate into actionable insights for clinicians. Here, we use the example of risk assessment for postoperative complications to demonstrate how explainable and medically grounded risk prediction models can be developed. Pre- and postoperative risk prediction models are trained based on clinically relevant inputs extracted from electronic medical record data. We show that these models have similar predictive performance as models that incorporate a wider range of inputs and explain the models' decision-making process by visualizing how different model inputs and their values affect the models' predictions.

Translating Data Analytics Into Improved Spine Surgery Outcomes: A Roadmap for Biomedical Informatics Research in 2021

STUDY DESIGN

Narrative review.

OBJECTIVES

There is growing interest in the use of biomedical informatics and data analytics tools in spine surgery. Yet despite the rapid growth in research on these topics, few analytic tools have been implemented in routine spine practice. The purpose of this review is to provide a health information technology (HIT) roadmap to help translate data assets and analytics tools into measurable advances in spine surgical care.

METHODS

We conducted a narrative review of PubMed and Google Scholar to identify publications discussing data assets, analytical approaches, and implementation strategies relevant to spine surgery practice.

RESULTS

A variety of data assets are available for spine research, ranging from commonly used datasets, such as administrative billing data, to emerging resources, such as mobile health and biobanks. Both regression and machine learning techniques are valuable for analyzing these assets, and researchers should recognize the particular strengths and weaknesses of each approach. Few studies have focused on the implementation of HIT, and a variety of methods exist to help translate analytic tools into clinically useful interventions. Finally, a number of HIT-related challenges must be recognized and addressed, including stakeholder acceptance, regulatory oversight, and ethical considerations.

CONCLUSIONS

Biomedical informatics has the potential to support the development of new HIT that can improve spine surgery quality and outcomes. By understanding the development life-cycle that includes identifying an appropriate data asset, selecting an analytic approach, and leveraging an effective implementation strategy, spine researchers can translate this potential into measurable advances in patient care.

IPOscore: An interactive web-based platform for postoperative surgical complications analysis and prediction in the oncology domain

BACKGROUND

The performance of traditional risk score systems to predict (post)-operative outcomes is limited. This weakness reduces confidence in its use to support clinical risk mitigation decisions. However, the rapid growth of health data in the last years offers principles to deal with some of these limitations. In this regard, the data allows the extraction of relevant information for both patients stratification and the rigorous identification of associated risk factors. The patients can then be targeted to specific preoperative optimization programs, thus contributing to the reduction of associated morbidity and mortality.

OBJECTIVES

The main goal of this work is, therefore, to provide a clinical decision support system (CDSS) based on data-driven modeling methods for surgical risk prediction specific for cancer patients in Portugal.

RESULTS

The result is IPOscore, a single web-based platform aimed at being an innovative approach to assist clinical decision-making in the surgical oncology domain. This system includes a database to store/manage the clinical data collected in a structured format, data visualization and analysis tools, and predictive machine learning models to predict postoperative outcomes in cancer patients. IPOscore also includes a pattern mining module based on biclustering to assess the discriminative power of a pattern towards postsurgical outcomes. Additionally, a mobile application is provided to this end.

CONCLUSIONS

The IPOscore platform is a valuable tool for surgical oncologists not only for clinical data management but also as a preventative and predictive healthcare system. Currently, this clinical support tool is being tested at the Portuguese Institute of Oncology (IPO-Porto), and can be accessed online at https://iposcore.org.

Development and Validation of an Explainable Machine Learning Model for Major Complications After Cytoreductive Surgery

Importance

Cytoreductive surgery (CRS) is one of the most complex operations in surgical oncology with significant morbidity, and improved risk prediction tools are critically needed. Machine learning models can potentially overcome the limitations of traditional multiple logistic regression (MLR) models and provide accurate risk estimates.

Objective

To develop and validate an explainable machine learning model for predicting major postoperative complications in patients undergoing CRS.

Design, Setting, and Participants

This prognostic study used patient data from tertiary care hospitals with expertise in CRS included in the US Hyperthermic Intraperitoneal Chemotherapy Collaborative Database between 1998 and 2018. Information from 147 variables was extracted to predict the risk of a major complication. An ensemble-based machine learning (gradient-boosting) model was optimized on 80% of the sample with subsequent validation on a 20% holdout data set. The machine learning model was compared with traditional MLR models. The artificial intelligence SHAP (Shapley additive explanations) method was used for interpretation of patient- and cohort-level risk estimates and interactions to define novel surgical risk phenotypes. Data were analyzed between November 2019 and August 2021.

Exposures

Cytoreductive surgery.

Main Outcomes and Measures

Area under the receiver operating characteristics (AUROC); area under the precision recall curve (AUPRC).

Results

Data from a total 2372 patients were included in model development (mean age, 55 years [range, 11-95 years]; 1366 [57.6%] women). The optimized machine learning model achieved high discrimination (AUROC: mean cross-validation, 0.75 [range, 0.73-0.81]; test, 0.74) and precision (AUPRC: mean cross-validation, 0.50 [range, 0.46-0.58]; test, 0.42). Compared with the optimized machine learning model, the published MLR model performed worse (test AUROC and AUPRC: 0.54 and 0.18, respectively). Higher volume of estimated blood loss, having pelvic peritonectomy, and longer operative time were the top 3 contributors to the high likelihood of major complications. SHAP dependence plots demonstrated insightful nonlinear interactive associations between predictors and major complications. For instance, high estimated blood loss (ie, above 500 mL) was only detrimental when operative time exceeded 9 hours. Unsupervised clustering of patients based on similarity of sources of risk allowed identification of 6 distinct surgical risk phenotypes.

Conclusions and Relevance

In this prognostic study using data from patients undergoing CRS, an optimized machine learning model demonstrated a superior ability to predict individual- and cohort-level risk of major complications vs traditional methods. Using the SHAP method, 6 distinct surgical phenotypes were identified based on sources of risk of major complications.

Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform

IMPORTANCE

Predicting postoperative complications has the potential to inform shared decisions regarding the appropriateness of surgical procedures, targeted risk-reduction strategies, and postoperative resource use. Realizing these advantages requires that accurate real-time predictions be integrated with clinical and digital workflows; artificial intelligence predictive analytic platforms using automated electronic health record (EHR) data inputs offer an intriguing possibility for achieving this, but there is a lack of high-level evidence from prospective studies supporting their use.

OBJECTIVE

To examine whether the MySurgeryRisk artificial intelligence system has stable predictive performance between development and prospective validation phases and whether it is feasible to provide automated outputs directly to surgeons' mobile devices.

DESIGN, SETTING, AND PARTICIPANTS

In this prognostic study, the platform used automated EHR data inputs and machine learning algorithms to predict postoperative complications and provide predictions to surgeons, previously through a web portal and currently through a mobile device application. All patients 18 years or older who were admitted for any type of inpatient surgical procedure (74 417 total procedures involving 58 236 patients) between June 1, 2014, and September 20, 2020, were included. Models were developed using retrospective data from 52 117 inpatient surgical procedures performed between June 1, 2014, and November 27, 2018. Validation was performed using data from 22 300 inpatient surgical procedures collected prospectively from November 28, 2018, to September 20, 2020.

MAIN OUTCOMES AND MEASURES

Algorithms for generalized additive models and random forest models were developed and validated using real-time EHR data. Model predictive performance was evaluated primarily using area under the receiver operating characteristic curve (AUROC) values.

RESULTS

Among 58 236 total adult patients who received 74 417 major inpatient surgical procedures, the mean (SD) age was 57 (17) years; 29 226 patients (50.2%) were male. Results reported in this article focus primarily on the validation cohort. The validation cohort included 22 300 inpatient surgical procedures involving 19 132 patients (mean [SD] age, 58 [17] years; 9672 [50.6%] male). A total of 2765 patients (14.5%) were Black or African American, 14 777 (77.2%) were White, 1235 (6.5%) were of other races (including American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, and multiracial), and 355 (1.9%) were of unknown race because of missing data; 979 patients (5.1%) were Hispanic, 17 663 (92.3%) were non-Hispanic, and 490 (2.6%) were of unknown ethnicity because of missing data. A greater number of input features was associated with stable or improved model performance. For example, the random forest model trained with 135 input features had the highest AUROC values for predicting acute kidney injury (0.82; 95% CI, 0.82-0.83); cardiovascular complications (0.81; 95% CI, 0.81-0.82); neurological complications, including delirium (0.87; 95% CI, 0.87-0.88); prolonged intensive care unit stay (0.89; 95% CI, 0.88-0.89); prolonged mechanical ventilation (0.91; 95% CI, 0.90-0.91); sepsis (0.86; 95% CI, 0.85-0.87); venous thromboembolism (0.82; 95% CI, 0.81-0.83); wound complications (0.78; 95% CI, 0.78-0.79); 30-day mortality (0.84; 95% CI, 0.82-0.86); and 90-day mortality (0.84; 95% CI, 0.82-0.85), with accuracy similar to surgeons' predictions. Compared with the original web portal, the mobile device application allowed efficient fingerprint login access and loaded data approximately 10 times faster. The application output displayed patient information, risk of postoperative complications, top 3 risk factors for each complication, and patterns of complications for individual surgeons compared with their colleagues.

CONCLUSIONS AND RELEVANCE

In this study, automated real-time predictions of postoperative complications with mobile device outputs had good performance in clinical settings with prospective validation, matching surgeons' predictive accuracy.

Surgeons' risk perception in ASD surgery: The value of objective risk assessment on decision making and patient counselling

BACKGROUND

Surgeons often rely on their intuition, experience and published data for surgical decision making and informed consent. Literature provides average values that do not allow for individualized assessments. Accurate validated machine learning (ML) risk calculators for adult spinal deformity (ASD) patients, based on 10 year multicentric prospective data, are currently available. The objective of this study is to assess surgeon ASD risk perception and compare it to validated risk calculator estimates.

METHODS

Nine ASD complete (demographics, HRQL, radiology, surgical plan) preoperative cases were distributed online to 100 surgeons from 22 countries. Surgeons were asked to determine the risk of major complications and reoperations at 72 h, 90 d and 2 years postop, using a 0-100% risk scale. The same preoperative parameters circulated to surgeons were used to obtain ML risk calculator estimates. Concordance between surgeons' responses was analyzed using intraclass correlation coefficients (ICC) (poor < 0.5/excellent > 0.85). Distance between surgeons' and risk calculator predictions was assessed using the mean index of agreement (MIA) (poor < 0.5/excellent > 0.85).

RESULTS

Thirty-nine surgeons (74.4% with > 10 years' experience), from 12 countries answered the survey. Surgeons' risk perception concordance was very low and heterogeneous. ICC ranged from 0.104 (reintervention risk at 72 h) to 0.316 (reintervention risk at 2 years). Distance between calculator and surgeon prediction was very large. MIA ranged from 0.122 to 0.416. Surgeons tended to overestimate the risk of major complications and reintervention in the first 72 h and underestimated the same risks at 2 years postop.

CONCLUSIONS

This study shows that expert surgeon ASD risk perception is heterogeneous and highly discordant. Available validated ML ASD risk calculators can enable surgeons to provide more accurate and objective prognosis to adjust patient expectations, in real time, at the point of care.

Comparison of Electronic Frailty Metrics for Prediction of Adverse Outcomes of Abdominal Surgery

Importance

Electronic frailty metrics have been developed for automated frailty assessment and include the Hospital Frailty Risk Score (HFRS), the Electronic Frailty Index (eFI), the 5-Factor Modified Frailty Index (mFI-5), and the Risk Analysis Index (RAI). Despite substantial differences in their construction, these 4 electronic frailty metrics have not been rigorously compared within a surgical population.

Objective

To characterize the associations between 4 electronic frailty metrics and to measure their predictive value for adverse surgical outcomes.

Design, Setting, and Participants

This retrospective cohort study used electronic health record data from patients who underwent abdominal surgery from January 1, 2010, to December 31, 2020, at 20 medical centers within Kaiser Permanente Northern California (KPNC). Participants included adults older than 50 years who underwent abdominal surgical procedures at KPNC from 2010 to 2020 that were sampled for reporting to the National Surgical Quality Improvement Program.

Main Outcomes and Measures

Pearson correlation coefficients between electronic frailty metrics and area under the receiver operating characteristic curve (AUROC) of univariate models and multivariate preoperative risk models for 30-day mortality, readmission, and morbidity, which was defined as a composite of mortality and major postoperative complications.

Results

Within the cohort of 37 186 patients, mean (SD) age, 67.9 (female, 19 127 [51.4%]), correlations between pairs of metrics ranged from 0.19 (95% CI, 0.18- 0.20) for mFI-5 and RAI 0.69 (95% CI, 0.68-0.70). Only 1085 of 37 186 (2.9%) were classified as frail based on all 4 metrics. In univariate models for morbidity, HFRS demonstrated higher predictive discrimination (AUROC, 0.71; 95% CI, 0.70-0.72) than eFI (AUROC, 0.64; 95% CI, 0.63-0.65), mFI-5 (AUROC, 0.58; 95% CI, 0.57-0.59), and RAI (AUROC, 0.57; 95% CI, 0.57-0.58). The predictive discrimination of multivariate models with age, sex, comorbidity burden, and procedure characteristics for all 3 adverse surgical outcomes improved by including HFRS into the models.

Conclusions and Relevance

In this cohort study, the 4 electronic frailty metrics demonstrated heterogeneous correlation and classified distinct groups of surgical patients as frail. However, HFRS demonstrated the highest predictive value for adverse surgical outcomes.

Gamification for Machine Learning in Surgical Patient Engagement

Patients and their surgeons face a complex and evolving set of choices in the process of shared decision making. The plan of care must be tailored to individual patient risk factors and values, though objective estimates of risk can be elusive, and these risk factors are often modifiable and can alter the plan of care. Machine learning can perform real-time predictions of outcomes, though these technologies are limited by usability and interpretability. Gamification, or the use of game elements in non-game contexts, may be able to incorporate machine learning technology to help patients optimize their pre-operative risks, reduce in-hospital complications, and hasten recovery. This article proposes a theoretical mobile application to help guide decision making and provide evidence-based, tangible goals for patients and surgeons with the goal of achieving the best possible operative outcome that aligns with patient values.

Digitally enabled hemovigilance allows real time response to transfusion reactions

BACKGROUND

Transfusion carries a risk of transfusion reaction that is often underdiagnosed due to reliance on passive reporting. The study investigated the utility of digital methods to identify potential transfusion reactions, thus allowing real-time intervention for affected patients.

METHOD

The hemovigilance unit monitored 3856 patients receiving 43,515 transfusions under the hemovigilance program. Retrospective comparison data included 298,498 transfusions. Transfusion medicine physicians designed and validated algorithms in the electronic health record that analyze discrete data, such as vital sign changes, to assign a risk score during each transfusion. Dedicated hemovigilance nurses remotely monitor all patients and perform real-time chart reviews prioritized by risk score. When a reaction is suspected, a hemovigilance trained licensed clinician responds to manage the patient and ensure data collection. Board-certified transfusion medicine physicians reviewed data and classified transfusion reactions under various categories according to the Centers for Disease Control hemovigilance definitions.

RESULTS

Transfusion medicine physicians diagnosed 564 transfusion reactions (1.3% of transfusions)-a 524% increase compared to the previous passive reporting. The rapid response provider reached the bedside on average at 12.4 min demonstrating logistic feasibility. While febrile reactions were most diagnosed, recognition of transfusion-associated circulatory overload demonstrated the greatest relative increase. Auditing and education programs further enhanced transfusion reaction awareness.

DISCUSSION

The model of digitally-enabled expert real-time review of clinical data that prompts rapid response improved recognition of transfusion reactions. This approach could be applied to other patient deterioration events such as early identification of sepsis.

Artificial intelligence in perioperative medicine - a narrative review

Recent advancements in artificial intelligence (AI) techniques have enabled the development of accurate prediction models using clinical big data. AI models for perioperative risk stratification, intraoperative event prediction, biosignal analyses, and intensive care medicine have been developed in the field of perioperative medicine. Some of these models have been validated using external datasets and randomized controlled trials. Once these models are implemented in electronic health record systems or software medical devices, they could help anesthesiologists improve clinical outcomes by accurately predicting complications and suggesting optimal treatment strategies in real-time. This review provides an overview of the AI techniques used in perioperative medicine and a summary of the studies that have been published using these techniques. Understanding these techniques will aid in their appropriate application in clinical practice.

Emerging early diagnostic methods for acute kidney injury

Many factors such as trauma and COVID-19 cause acute kidney injury (AKI). Late AKI have a very high incidence and mortality rate. Early diagnosis of AKI provides a critical therapeutic time window for AKI treatment to prevent progression to chronic renal failure. However, the current clinical detection based on creatinine and urine output isn't effective in diagnosing early AKI. In recent years, the early diagnosis of AKI has made great progress with the advancement of information technology, nanotechnology, and biomedicine. These emerging methods are mainly divided into two aspects: First, predicting AKI through models construct by machine learning; Second, early diagnosis of AKI through detection of newly-discovered early biomarkers. Currently, these methods have shown great potential and become an attractive tool for the early diagnosis of AKI. Therefore, it is very important to discuss and summarize these methods for the early diagnosis of AKI. In this review, we first systematically summarize the application of machine learning in AKI prediction algorithms and specific scenarios. In addition, we introduce the key role of early biomarkers in the progress of AKI, and then comprehensively summarize the application of emerging detection technologies for early AKI. Finally, we discuss current challenges and prospects of machine learning and biomarker detection. The review is expected to provide new insights for early diagnosis of AKI, and provided important inspiration for the design of early diagnosis of other major diseases.

The derivation of an International Classification of Diseases, Tenth Revision-based trauma-related mortality model using machine learning

BACKGROUND

Existing mortality prediction models have attempted to quantify injury burden following trauma-related admissions with the most notable being the Injury Severity Score (ISS). Although easy to calculate, it requires additional administrative coding. International Classification of Diseases (ICD)-based models such as the Trauma Mortality Prediction Model (TMPM-ICD10) circumvent these limitations, but they use linear modeling, which may not adequately capture the intricate relationships of injuries on mortality. Using ICD-10 coding and machine learning (ML) algorithms, the present study used the National Trauma Data Bank to develop mortality prediction models whose performance was compared with logistic regression, ISS, and TMPM-ICD10.

METHODS

The 2015 to 2017 National Trauma Data Bank was used to identify adults following trauma-related admissions. Of 8,021 ICD-10 codes, injuries were categorized into 1,495 unique variables. The primary outcome was in-hospital mortality. eXtreme Gradient Boosting (XGBoost), a ML technique that uses iterations of decision trees, was used to develop mortality models. Model discrimination was compared with logistic regression, ISS, and TMPM-ICD10 using receiver operating characteristic curve and probabilistic accuracy with calibration curves.

RESULTS

Of 1,611,063 patients, 54,870 (3.41%) experienced in-hospital mortality. Compared with those who survived, those who died more frequently suffered from penetrating trauma and had a greater number of injuries. The XGBoost model exhibited superior receiver operating characteristic curve (0.863 [95% confidence interval (CI), 0.862-0.864]) compared with logistic regression (0.845 [95% CI, 0.844-0.846]), ISS (0.828 [95% CI, 0.827-0.829]), and TMPM-ICD10 (0.861 [95% CI, 0.860-0.862]) (all p < 0.001). Importantly, the ML model also had significantly improved calibration compared with other methodologies (XGBoost, coefficient of determination (R2) = 0.993; logistic regression, R2 = 0.981; ISS, R2 = 0.649; TMPM-ICD10, R2 = 0.830).

CONCLUSION

Machine learning models using XGBoost demonstrated superior performance and calibration compared with logistic regression, ISS, and TMPM-ICD10. Such approaches in quantifying injury severity may improve its utility in mortality prognostication, quality improvement, and trauma research.

LEVEL OF EVIDENCE

Prognostic and Epidemiologic; level III.

Integrated Single-cell and Plasma Proteomic Modeling to Predict Surgical Site Complications: A Prospective Cohort Study

OBJECTIVE

The aim of this study was to determine whether single-cell and plasma proteomic elements of the host's immune response to surgery accurately identify patients who develop a surgical site complication (SSC) after major abdominal surgery.

SUMMARY BACKGROUND DATA

SSCs may occur in up to 25% of patients undergoing bowel resection, resulting in significant morbidity and economic burden. However, the accurate prediction of SSCs remains clinically challenging. Leveraging high-content proteomic technologies to comprehensively profile patients' immune response to surgery is a promising approach to identify predictive biological factors of SSCs.

METHODS

Forty-one patients undergoing non-cancer bowel resection were prospectively enrolled. Blood samples collected before surgery and on postoperative day one (POD1) were analyzed using a combination of single-cell mass cytometry and plasma proteomics. The primary outcome was the occurrence of an SSC, including surgical site infection, anastomotic leak, or wound dehiscence within 30 days of surgery.

RESULTS

A multiomic model integrating the single-cell and plasma proteomic data collected on POD1 accurately differentiated patients with (n = 11) and without (n = 30) an SSC [area under the curve (AUC) = 0.86]. Model features included coregulated proinflammatory (eg, IL-6- and MyD88- signaling responses in myeloid cells) and immunosuppressive (eg, JAK/STAT signaling responses in M-MDSCs and Tregs) events preceding an SSC. Importantly, analysis of the immunological data obtained before surgery also yielded a model accurately predicting SSCs (AUC = 0.82).

CONCLUSIONS

The multiomic analysis of patients' immune response after surgery and immune state before surgery revealed systemic immune signatures preceding the development of SSCs. Our results suggest that integrating immunological data in perioperative risk assessment paradigms is a plausible strategy to guide individualized clinical care.

Artificial intelligence and telemedicine in anesthesia: potential and problems

INTRODUCTION

The application of novel technologies like Artificial Intelligence (AI), Machine Learning (ML) and telemedicine in anesthesiology could play a role in transforming the future of health care. In the present review we discuss the current applications of AI and telemedicine in anesthesiology and perioperative care, exploring their potential influence and the possible hurdles.

EVIDENCE ACQUISITION

AI technologies have the potential to deeply impact all phases of perioperative care from accurate risk prediction to operating room organization, leading to increased cost-effective care quality and better outcomes. Telemedicine is reported as a successful mean within the anaesthetic pathway, including preoperative evaluation, remote patient monitoring, and postoperative care.

EVIDENCE SYNTHESIS

The utilization of AI and telemedicine is promising encouraging results in perioperative management, nevertheless several hurdles remain to be overcome before these tools could be integrated in our daily practice.

CONCLUSIONS

AI models and telemedicine can significantly influence all phases of perioperative care, helping physicians in the development of precision medicine.

Aligning Patient Acuity With Resource Intensity After Major Surgery: A Scoping Review

OBJECTIVE

Develop unifying definitions and paradigms for data-driven methods to augment postoperative resource intensity decisions.

SUMMARY BACKGROUND DATA

Postoperative level-of-care assignments and frequency of vital sign and laboratory measurements (ie, resource intensity) should align with patient acuity. Effective, data-driven decision-support platforms could improve value of care for millions of patients annually, but their development is hindered by the lack of salient definitions and paradigms.

METHODS

Embase, PubMed, and Web of Science were searched for articles describing patient acuity and resource intensity after inpatient surgery. Study quality was assessed using validated tools. Thirty-five studies were included and assimilated according to PRISMA guidelines.

RESULTS

Perioperative patient acuity is accurately represented by combinations of demographic, physiologic, and hospital-system variables as input features in models that capture complex, non-linear relationships. Intraoperative physiologic data enriche these representations. Triaging high-acuity patients to low-intensity care is associated with increased risk for mortality; triaging low-acuity patients to intensive care units (ICUs) has low value and imparts harm when other, valid requests for ICU admission are denied due to resource limitations, increasing their risk for unrecognized decompensation and failure-to-rescue. Providing high-intensity care for low-acuity patients may also confer harm through unnecessary testing and subsequent treatment of incidental findings, but there is insufficient evidence to evaluate this hypothesis. Compared with data-driven models, clinicians exhibit volatile performance in predicting complications and making postoperative resource intensity decisions.

CONCLUSION

To optimize value, postoperative resource intensity decisions should align with precise, data-driven patient acuity assessments augmented by models that accurately represent complex, non-linear relationships among risk factors.

Machine learning to guide clinical decision-making in abdominal surgery-a systematic literature review

PURPOSE

An indication for surgical therapy includes balancing benefits against risk, which remains a key task in all surgical disciplines. Decisions are oftentimes based on clinical experience while guidelines lack evidence-based background. Various medical fields capitalized the application of machine learning (ML), and preliminary research suggests promising implications in surgeons' workflow. Hence, we evaluated ML's contemporary and possible future role in clinical decision-making (CDM) focusing on abdominal surgery.

METHODS

Using the PICO framework, relevant keywords and research questions were identified. Following the PRISMA guidelines, a systemic search strategy in the PubMed database was conducted. Results were filtered by distinct criteria and selected articles were manually full text reviewed.

RESULTS

Literature review revealed 4,396 articles, of which 47 matched the search criteria. The mean number of patients included was 55,843. A total of eight distinct ML techniques were evaluated whereas AUROC was applied by most authors for comparing ML predictions vs. conventional CDM routines. Most authors (N = 30/47, 63.8%) stated ML's superiority in the prediction of benefits and risks of surgery. The identification of highly relevant parameters to be integrated into algorithms allowing a more precise prognosis was emphasized as the main advantage of ML in CDM.

CONCLUSIONS

A potential value of ML for surgical decision-making was demonstrated in several scientific articles. However, the low number of publications with only few collaborative studies between surgeons and computer scientists underpins the early phase of this highly promising field. Interdisciplinary research initiatives combining existing clinical datasets and emerging techniques of data processing may likely improve CDM in abdominal surgery in the future.

Ideal algorithms in healthcare: Explainable, dynamic, precise, autonomous, fair, and reproducible

Established guidelines describe minimum requirements for reporting algorithms in healthcare; it is equally important to objectify the characteristics of ideal algorithms that confer maximum potential benefits to patients, clinicians, and investigators. We propose a framework for ideal algorithms, including 6 desiderata: explainable (convey the relative importance of features in determining outputs), dynamic (capture temporal changes in physiologic signals and clinical events), precise (use high-resolution, multimodal data and aptly complex architecture), autonomous (learn with minimal supervision and execute without human input), fair (evaluate and mitigate implicit bias and social inequity), and reproducible (validated externally and prospectively and shared with academic communities). We present an ideal algorithms checklist and apply it to highly cited algorithms. Strategies and tools such as the predictive, descriptive, relevant (PDR) framework, the Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence (SPIRIT-AI) extension, sparse regression methods, and minimizing concept drift can help healthcare algorithms achieve these objectives, toward ideal algorithms in healthcare.

Machine learning in perioperative medicine: a systematic review

Background

Risk stratification plays a central role in anesthetic evaluation. The use of Big Data and machine learning (ML) offers considerable advantages for collection and evaluation of large amounts of complex health-care data. We conducted a systematic review to understand the role of ML in the development of predictive post-surgical outcome models and risk stratification.

Methods

Following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, we selected the period of the research for studies from 1 January 2015 up to 30 March 2021. A systematic search in Scopus, CINAHL, the Cochrane Library, PubMed, and MeSH databases was performed; the strings of research included different combinations of keywords: “risk prediction,” “surgery,” “machine learning,” “intensive care unit (ICU),” and “anesthesia” “perioperative.” We identified 36 eligible studies. This study evaluates the quality of reporting of prediction models using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist.

Results

The most considered outcomes were mortality risk, systemic complications (pulmonary, cardiovascular, acute kidney injury (AKI), etc.), ICU admission, anesthesiologic risk and prolonged length of hospital stay. Not all the study completely followed the TRIPOD checklist, but the quality was overall acceptable with 75% of studies (Rev #2, comm #minor issue) showing an adherence rate to TRIPOD more than 60%. The most frequently used algorithms were gradient boosting (n = 13), random forest (n = 10), logistic regression (LR; n = 7), artificial neural networks (ANNs; n = 6), and support vector machines (SVM; n = 6). Models with best performance were random forest and gradient boosting, with AUC > 0.90.

Conclusions

The application of ML in medicine appears to have a great potential. From our analysis, depending on the input features considered and on the specific prediction task, ML algorithms seem effective in outcomes prediction more accurately than validated prognostic scores and traditional statistics. Thus, our review encourages the healthcare domain and artificial intelligence (AI) developers to adopt an interdisciplinary and systemic approach to evaluate the overall impact of AI on perioperative risk assessment and on further health care settings as well.

Prediction of intensive care unit admission (>24h) after surgery in elective noncardiac surgical patients using machine learning algorithms

Background

To develop a highly discriminative machine learning model for the prediction of intensive care unit admission (>24h) using the easily available preoperative information from electronic health records. An accurate prediction model for ICU admission after surgery is of great importance for surgical risk assessment and appropriate utilization of ICU resources.

Method

Data were collected retrospectively from a large hospital, comprising 135,442 adult patients who underwent surgery except for cardiac surgery between 1 January 2014, and 31 July 2018 in China. Multiple existing predictive machine learning algorithms were explored to construct the prediction model, including logistic regression, random forest, adaptive boosting, and gradient boosting machine. Four secondary analyses were conducted to improve the interpretability of the results.

Results

A total of 2702 (2.0%) patients were admitted to the intensive care unit postoperatively. The gradient boosting machine model attained the highest area under the receiver operating characteristic curve of 0.90. The machine learning models predicted intensive care unit admission better than the American Society of Anesthesiologists Physical Status (area under the receiver operating characteristic curve: 0.68). The gradient boosting machine recognized several features as highly significant predictors for postoperatively intensive care unit admission. By applying subgroup analysis and secondary analysis, we found that patients with operations on the digestive, respiratory, and vascular systems had higher probabilities for intensive care unit admission.

Conclusion

Compared with conventional American Society of Anesthesiologists Physical Status and logistic regression model, the gradient boosting machine could improve the performance in the prediction of intensive care unit admission. Machine learning models could be used to improve the discrimination and identify the need for intensive care unit admission after surgery in elective noncardiac surgical patients, which could help manage the surgical risk.