Use of Machine Learning to Develop and Evaluate Models Using Preoperative and Intraoperative Data to Identify Risks of Postoperative Complications

Cardiac Comorbidity Risk Score: Zero-Burden Machine Learning to Improve Prediction of Postoperative Major Adverse Cardiac Events in Hip and Knee Arthroplasty

Background In this retrospective, observational study we introduce the Cardiac Comorbidity Risk Score, predicting perioperative major adverse cardiac events (MACE) after elective hip and knee arthroplasty. MACE is a rare but important driver of mortality, and existing tools, eg, the Revised Cardiac Risk Index demonstrate only modest accuracy. We demonstrate an artificial intelligence-based approach to identify patients at high risk of MACE within 4 weeks (primary outcome) of arthroplasty, that imposes zero additional burden of cost/resources. Methods and Results Cardiac Comorbidity Risk Score calculation uses novel machine learning to estimate MACE risk from patient electronic health records, without requiring blood work or access to any demographic data beyond that of sex and age, and accounts for variable/missing/incomplete information across patient records. Validated on a deidentified cohort (age >45 years, n=445 391), performance was evaluated using the area under the receiver operator characteristics curve (AUROC), sensitivity/specificity, positive predictive value, and positive/negative likelihood ratios. In our cohort (age 63.5±10.5 years, 58.2% women, 34.2%/65.8% hip/knee procedures), 0.19% (882) experienced the primary outcome. Cardiac Comorbidity Risk Score achieved area under the receiver operator characteristics curve=80.0±0.4% (95% CI) for women and 80.1±0.5% (95% CI) for males, with 36.4% and 35.1% sensitivities, respectively, at 95% specificity, significantly outperforming Revised Cardiac Risk Index across all studied age-, sex-, risk-, and comorbidity-based subgroups. Conclusions Cardiac Comorbidity Risk Score, a novel artificial intelligence-based screening tool using known and unknown comorbidity patterns, outperforms state-of-the-art in predicting MACE within 4 weeks postarthroplasty, and can identify patients at high risk that do not demonstrate traditional risk factors.

Integration of feature vectors from raw laboratory, medication and procedure names improves the precision and recall of models to predict postoperative mortality and acute kidney injury

Manuscripts that have successfully used machine learning (ML) to predict a variety of perioperative outcomes often use only a limited number of features selected by a clinician. We hypothesized that techniques leveraging a broad set of features for patient laboratory results, medications, and the surgical procedure name would improve performance as compared to a more limited set of features chosen by clinicians. Feature vectors for laboratory results included 702 features total derived from 39 laboratory tests, medications consisted of a binary flag for 126 commonly used medications, procedure name used the Word2Vec package for create a vector of length 100. Nine models were trained: baseline features, one for each of the three types of data Baseline + Each data type, (all features, and then all features with feature reduction algorithm. Across both outcomes the models that contained all features (model 8) (Mortality ROC-AUC 94.32 ± 1.01, PR-AUC 36.80 ± 5.10 AKI ROC-AUC 92.45 ± 0.64, PR-AUC 76.22 ± 1.95) was superior to models with only subsets of features. Featurization techniques leveraging a broad away of clinical data can improve performance of perioperative prediction models.

Towards interpretable, medically grounded, EMR-based risk prediction models

Machine-learning based risk prediction models have the potential to improve patient outcomes by assessing risk more accurately than clinicians. Significant additional value lies in these models providing feedback about the factors that amplify an individual patient's risk. Identification of risk factors enables more informed decisions on interventions to mitigate or ameliorate modifiable factors. For these reasons, risk prediction models must be explainable and grounded on medical knowledge. Current machine learning-based risk prediction models are frequently 'black-box' models whose inner workings cannot be understood easily, making it difficult to define risk drivers. Since machine learning models follow patterns in the data rather than looking for medically relevant relationships, possible risk factors identified by these models do not necessarily translate into actionable insights for clinicians. Here, we use the example of risk assessment for postoperative complications to demonstrate how explainable and medically grounded risk prediction models can be developed. Pre- and postoperative risk prediction models are trained based on clinically relevant inputs extracted from electronic medical record data. We show that these models have similar predictive performance as models that incorporate a wider range of inputs and explain the models' decision-making process by visualizing how different model inputs and their values affect the models' predictions.

Machine learning algorithms as early diagnostic tools for pancreatic fistula following pancreaticoduodenectomy and guide drain removal: A retrospective cohort study

OBJECTIVE

Clinically relevant postoperative pancreatic fistula (CR-POPF) remains the major cause of morbidity following pancreaticoduodenectomy (PD). Several model score systems such as the Fistula Risk Score (FRS) have been developed to predict CR-POPF using preoperative and intraoperative data. Machine learning (ML) algorithms are increasingly applied in the medical field and they could be used to assess the risk of CR-POPF, identify clinically meaningful data and guide drain removal.

METHODS

Data from consecutive patients who underwent PD between January 1, 2010 and March 31, 2021 at a single high-volume center was collected retrospectively in this study. Demographics, clinical features, intraoperative parameters, and laboratory values were used to conduct the ML model. Four different ML algorithms (CatBoost, lightGBM, XGBoost and Random Forest) were used to train this model with cross-validation.

RESULTS

A total of 2421 patients with 62 clinical parameters were enrolled in this ML model. The majority of patients (76.3%) underwent open PD while others underwent robot-assisted PD. CR-POPF occurred in 424 (17.5%) patients. The CatBoost algorithm outperformed other algorithms with a mean area under the receiver operating characteristic curve (AUC) of 0.81 (95% confidence interval: 0.80-0.82) from the 5-fold cross-validation procedure. In the test dataset, the CatBoost algorithm also achieved the best mean-AUC of 0.83. The most important value was mean drain fluid amylase (DFA) in the first seven postoperative days (POD). The performance of models that used only preoperative data and intraoperative data was marginally lower than that of models that used combined data.

CONCLUSION

Our ML algorithms could be applied as early diagnostic tools for CR-POPF in patients who underwent PD. Such real-time clinical decision support tools can identify patients with a high risk of CR-POPF, help in developing the perioperative management plan and guide the optimal timing of drain removal.

Predicting the Risk of Incident Type 2 Diabetes Mellitus in Chinese Elderly Using Machine Learning Techniques

Early identification of individuals at high risk of diabetes is crucial for implementing early intervention strategies. However, algorithms specific to elderly Chinese adults are lacking. The aim of this study is to build effective prediction models based on machine learning (ML) for the risk of type 2 diabetes mellitus (T2DM) in Chinese elderly. A retrospective cohort study was conducted using the health screening data of adults older than 65 years in Wuhan, China from 2018 to 2020. With a strict data filtration, 127,031 records from the eligible participants were utilized. Overall, 8298 participants were diagnosed with incident T2DM during the 2-year follow-up (2019–2020). The dataset was randomly split into training set (n = 101,625) and test set (n = 25,406). We developed prediction models based on four ML algorithms: logistic regression (LR), decision tree (DT), random forest (RF), and extreme gradient boosting (XGBoost). Using LASSO regression, 21 prediction features were selected. The Random under-sampling (RUS) was applied to address the class imbalance, and the Shapley Additive Explanations (SHAP) was used to calculate and visualize feature importance. Model performance was evaluated by the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy. The XGBoost model achieved the best performance (AUC = 0.7805, sensitivity = 0.6452, specificity = 0.7577, accuracy = 0.7503). Fasting plasma glucose (FPG), education, exercise, gender, and waist circumference (WC) were the top five important predictors. This study showed that XGBoost model can be applied to screen individuals at high risk of T2DM in the early phrase, which has the strong potential for intelligent prevention and control of diabetes. The key features could also be useful for developing targeted diabetes prevention interventions.

Optimizing discharge after major surgery using an artificial intelligence-based decision support tool (DESIRE): An external validation study

BACKGROUND

In the DESIRE study (Discharge aftEr Surgery usIng aRtificial intElligence), we have previously developed and validated a machine learning concept in 1,677 gastrointestinal and oncology surgery patients that can predict safe hospital discharge after the second postoperative day. Despite strong model performance (area under the receiver operating characteristics curve of 0.88) in an academic surgical population, it remains unknown whether these findings can be translated to other hospitals and surgical populations. We therefore aimed to determine the generalizability of the previously developed machine learning concept.

METHODS

We externally validated the machine learning concept in gastrointestinal and oncology surgery patients admitted to 3 nonacademic hospitals in The Netherlands between January 2017 and June 2021, who remained admitted 2 days after surgery. Primary outcome was the ability to predict hospital interventions after the second postoperative day, which were defined as unplanned reoperations, radiological interventions, and/or intravenous antibiotics administration. Four forest models were locally trained and evaluated with respect to area under the receiver operating characteristics curve, sensitivity, specificity, positive predictive value, and negative predictive value.

RESULTS

All models were trained on 1,693 epsiodes, of which 731 (29.9%) required a hospital intervention and demonstrated strong performance (area under the receiver operating characteristics curve only varied 4%). The best model achieved an area under the receiver operating characteristics curve of 0.83 (95% confidence interval [0.81-0.85]), sensitivity of 77.9% (0.67-0.87), specificity of 79.2% (0.72-0.85), positive predictive value of 61.6% (0.54-0.69), and negative predictive value of 89.3% (0.85-0.93).

CONCLUSION

This study showed that a previously developed machine learning concept can predict safe discharge in different surgical populations and hospital settings (academic versus nonacademic) by training a model on local patient data. Given its high accuracy, integration of the machine learning concept into the clinical workflow could expedite surgical discharge and aid hospitals in addressing capacity challenges by reducing avoidable bed-days.

Future of Perioperative Precision Medicine: Integration of Molecular Science, Dynamic Health Care Informatics, and Implementation of Predictive Pathways in Real Time

Conceptually, precision medicine is a deep dive to discover disease origin at the molecular or genetic level, thus providing insights that allow clinicians to design corresponding individualized patient therapies. We know that a disease state is created by not only certain molecular derangements but also a biologic milieu promoting the expression of such derangements. These factors together lead to manifested symptoms. At the level of molecular definition, every average, "similar" individual stands to be "dissimilar." Hence, there is the need for customized therapy, moving away from therapy based on aggregate statistics. The perioperative state is a mix of several, simultaneously active molecular mechanisms, surgical insult, drugs, severe inflammatory response, and the body's continuous adaptation to maintain a state of homeostasis. Postoperative outcomes are a net result of several of those rapid genetic and molecular transformations that do or do not ensue. With the advent and advances of artificial intelligence, the translation from identifying these intricate mechanisms to implementing them in clinical practice has made a huge leap. Precision medicine is gaining ground with the help of personalized health recorders and personal devices that identify disease mechanics, patient-reported outcomes, adverse drug reactions, and drug-drug interaction at the individual level in a closed-loop feedback system. This phenomenon is especially true given increasing surgeries in older adults, many of whom are on multiple medications and varyingly frail. In this era of precision medicine, to provide a comprehensive remedy, the perioperative surgical home must expand, incorporating not only clinicians but also basic science experts and data scientists.

Artificial intelligence in perioperative medicine - a narrative review

Recent advancements in artificial intelligence (AI) techniques have enabled the development of accurate prediction models using clinical big data. AI models for perioperative risk stratification, intraoperative event prediction, biosignal analyses, and intensive care medicine have been developed in the field of perioperative medicine. Some of these models have been validated using external datasets and randomized controlled trials. Once these models are implemented in electronic health record systems or software medical devices, they could help anesthesiologists improve clinical outcomes by accurately predicting complications and suggesting optimal treatment strategies in real-time. This review provides an overview of the AI techniques used in perioperative medicine and a summary of the studies that have been published using these techniques. Understanding these techniques will aid in their appropriate application in clinical practice.

Considerations for the implementation of machine learning into acute care settings

INTRODUCTION

Management of patients in the acute care setting requires accurate diagnosis and rapid initiation of validated treatments; therefore, this setting is likely to be an environment in which cognitive augmentation of the clinician's provision of care with technology rooted in artificial intelligence, such as machine learning (ML), is likely to eventuate.

SOURCES OF DATA

PubMed and Google Scholar with search terms that included ML, intensive/critical care unit, electronic health records (EHR), anesthesia information management systems and clinical decision support were the primary sources for this report.

AREAS OF AGREEMENT

Different categories of learning of large clinical datasets, often contained in EHRs, are used for training in ML. Supervised learning uses algorithm-based models, including support vector machines, to pair patients' attributes with an expected outcome. Unsupervised learning uses clustering algorithms to define to which disease grouping a patient's attributes most closely approximates. Reinforcement learning algorithms use ongoing environmental feedback to deterministically pursue likely patient outcome.

AREAS OF CONTROVERSY

Application of ML can result in undesirable outcomes over concerns related to fairness, transparency, privacy and accountability. Whether these ML technologies irrevocably change the healthcare workforce remains unresolved.

GROWING POINTS

Well-resourced Learning Health Systems are likely to exploit ML technology to gain the fullest benefits for their patients. How these clinical advantages can be extended to patients in health systems that are neither well-endowed, nor have the necessary data gathering technologies, needs to be urgently addressed to avoid further disparities in healthcare.

Emerging early diagnostic methods for acute kidney injury

Many factors such as trauma and COVID-19 cause acute kidney injury (AKI). Late AKI have a very high incidence and mortality rate. Early diagnosis of AKI provides a critical therapeutic time window for AKI treatment to prevent progression to chronic renal failure. However, the current clinical detection based on creatinine and urine output isn't effective in diagnosing early AKI. In recent years, the early diagnosis of AKI has made great progress with the advancement of information technology, nanotechnology, and biomedicine. These emerging methods are mainly divided into two aspects: First, predicting AKI through models construct by machine learning; Second, early diagnosis of AKI through detection of newly-discovered early biomarkers. Currently, these methods have shown great potential and become an attractive tool for the early diagnosis of AKI. Therefore, it is very important to discuss and summarize these methods for the early diagnosis of AKI. In this review, we first systematically summarize the application of machine learning in AKI prediction algorithms and specific scenarios. In addition, we introduce the key role of early biomarkers in the progress of AKI, and then comprehensively summarize the application of emerging detection technologies for early AKI. Finally, we discuss current challenges and prospects of machine learning and biomarker detection. The review is expected to provide new insights for early diagnosis of AKI, and provided important inspiration for the design of early diagnosis of other major diseases.

Machine learning in perioperative medicine: a systematic review

Background

Risk stratification plays a central role in anesthetic evaluation. The use of Big Data and machine learning (ML) offers considerable advantages for collection and evaluation of large amounts of complex health-care data. We conducted a systematic review to understand the role of ML in the development of predictive post-surgical outcome models and risk stratification.

Methods

Following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, we selected the period of the research for studies from 1 January 2015 up to 30 March 2021. A systematic search in Scopus, CINAHL, the Cochrane Library, PubMed, and MeSH databases was performed; the strings of research included different combinations of keywords: “risk prediction,” “surgery,” “machine learning,” “intensive care unit (ICU),” and “anesthesia” “perioperative.” We identified 36 eligible studies. This study evaluates the quality of reporting of prediction models using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist.

Results

The most considered outcomes were mortality risk, systemic complications (pulmonary, cardiovascular, acute kidney injury (AKI), etc.), ICU admission, anesthesiologic risk and prolonged length of hospital stay. Not all the study completely followed the TRIPOD checklist, but the quality was overall acceptable with 75% of studies (Rev #2, comm #minor issue) showing an adherence rate to TRIPOD more than 60%. The most frequently used algorithms were gradient boosting (n = 13), random forest (n = 10), logistic regression (LR; n = 7), artificial neural networks (ANNs; n = 6), and support vector machines (SVM; n = 6). Models with best performance were random forest and gradient boosting, with AUC > 0.90.

Conclusions

The application of ML in medicine appears to have a great potential. From our analysis, depending on the input features considered and on the specific prediction task, ML algorithms seem effective in outcomes prediction more accurately than validated prognostic scores and traditional statistics. Thus, our review encourages the healthcare domain and artificial intelligence (AI) developers to adopt an interdisciplinary and systemic approach to evaluate the overall impact of AI on perioperative risk assessment and on further health care settings as well.

Pragmatic Precision Psychiatry-A New Direction for Optimizing Treatment Selection

IMPORTANCE

Clinical trials have identified numerous prescriptive predictors of mental disorder treatment response, ie, predictors of which treatments are best for which patients. However, none of these prescriptive predictors is strong enough alone to guide precision treatment planning. This has prompted growing interest in developing precision treatment rules (PTRs) that combine information across multiple prescriptive predictors, but this work has been much less successful in psychiatry than some other areas of medicine. Study designs and analysis schemes used in research on PTR development in other areas of medicine are reviewed, key challenges for implementing similar studies of mental disorders are highlighted, and recent methodological advances to address these challenges are described here.

OBSERVATIONS

Discovering prescriptive predictors requires large samples. Three approaches have been used in other areas of medicine to do this: conduct very large randomized clinical trials, pool individual-level results across multiple smaller randomized clinical trials, and develop preliminary PTRs in large observational treatment samples that are then tested in smaller randomized clinical trials. The third approach is most feasible for research on mental disorders. This approach requires working with large real-world observational electronic health record databases; carefully selecting samples to emulate trials; extracting information about prescriptive predictors from electronic health records along with other inexpensive data augmentation strategies; estimating preliminary PTRs in the observational data using appropriate methods; implementing pragmatic trials to validate the preliminary PTRs; and iterating between subsequent observational studies and quality improvement pragmatic trials to refine and expand the PTRs. New statistical methods exist to address the methodological challenges of implementing this approach.

CONCLUSIONS AND RELEVANCE

Advances in pragmatic precision psychiatry will require moving beyond the current focus on randomized clinical trials and adopting an iterative discovery-confirmation process that integrates observational and experimental studies in real-world clinical populations.

Application of a Computerized Decision Support System to Develop Care Strategies for Elderly Hemodialysis Patients

In this paper, the strategy of elderly haemodialysis patients' care is analysed by the computer's decision system to conduct an in-depth research machine. Maintenance haemodialysis patients have a high demand for continuation care, and healthcare workers should provide personalized and specialized seamless continuation care services for patients according to patients' needs, by reasonably using the hospital, community, and other health resources and with the help of emerging network technologies, such as information platforms and wearable devices to prolong the survival period of patients and improve their self-management ability and quality of life. The service provision and compensation strategy of the combined healthcare model should be optimized to improve the health protection of the elderly and promote health equity. On the one hand, it should target strengthening the service provision of healthcare integration, guide the elderly to reasonably choose the healthcare integration model, and pay attention to the spiritual and cultural needs and end-of-life care services for the elderly. On the other hand, we should expand the financing channels of medical insurance, optimize the design of compensation mechanisms, explore the role of health risk sharing, and accelerate the development of long-term care insurance, independent of basic medical insurance. The reliability of the scale was found to be 0.916 for the total Cronbach alpha coefficient, 0.798-0.919 for each dimension, and 0.813 for the fold-half reliability of the scale; the validity indicated that the correlation coefficient range of each article day with the total scale score was 0.27-0.72, and the correlation coefficient range of each dimension with the total scale was 0.56-0.72. The validation factor analysis was used to verify the structure of the scale. The validation factor analysis indexes met the fitting criteria after correction. The model fitted better with the actual model after correction, indicating that the scale has good reliability.