1
|
Olang O, Mohseni S, Shahabinezhad A, Hamidianshirazi Y, Goli A, Abolghasemian M, Shafiee MA, Aarabi M, Alavinia M, Shaker P. Artificial Intelligence-Based Models for Prediction of Mortality in ICU Patients: A Scoping Review. J Intensive Care Med 2024:8850666241277134. [PMID: 39150821 DOI: 10.1177/08850666241277134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/18/2024]
Abstract
BACKGROUND AND OBJECTIVE Healthcare professionals may be able to anticipate more accurately a patient's timing of death and assess their possibility of recovery by implementing a real-time clinical decision support system. Using such a tool, the healthcare system can better understand a patient's condition and make more informed judgements about distributing limited resources. This scoping review aimed to analyze various death prediction AI (Artificial Intelligence) algorithms that have been used in ICU (Intensive Care Unit) patient populations. METHODS The search strategy of this study involved keyword combinations of outcome and patient setting such as mortality, survival, ICU, terminal care. These terms were used to perform database searches in MEDLINE, Embase, and PubMed up to July 2022. The variables, characteristics, and performance of the identified predictive models were summarized. The accuracy of the models was compared using their Area Under the Curve (AUC) values. RESULTS Databases search yielded an initial pool of 8271 articles. A two-step screening process was then applied: first, titles and abstracts were reviewed for relevance, reducing the pool to 429 articles. Next, a full-text review was conducted, further narrowing down the selection to 400 key studies. Out of 400 studies on different tools or models for prediction of mortality in ICUs, 16 papers focused on AI-based models which were ultimately included in this study that have deployed different AI-based and machine learning models to make a prediction about negative patient outcome. The accuracy and performance of the different models varied depending on the patient populations and medical conditions. It was found that AI models compared with traditional tools like SAP3 or APACHE IV score were more accurate in death prediction, with some models achieving an AUC of up to 92.9%. The overall mortality rate ranged from 5% to more than 60% in different studies. CONCLUSION We found that AI-based models exhibit varying performance across different patient populations. To enhance the accuracy of mortality prediction, we recommend customizing models for specific patient groups and medical contexts. By doing so, healthcare professionals may more effectively assess mortality risk and tailor treatments accordingly. Additionally, incorporating additional variables-such as genetic information-into new models can further improve their accuracy.
Collapse
Affiliation(s)
- Orkideh Olang
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Sana Mohseni
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Ali Shahabinezhad
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Yasaman Hamidianshirazi
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Amireza Goli
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mansour Abolghasemian
- Division of Orthopedic Surgery, Department of Surgery, University of Alberta, Room 404 Community Service Centre, Royal Alexandra Hospital, 10240 Kingsway Avenue, Edmonton, Alberta, Canada, T5H 3V9
| | - Mohammad Ali Shafiee
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mehdi Aarabi
- Division of General Internal Medicine, Department of Medicine, University Health Network, Toronto General Hospital, 200 Elizabeth Street, 14 EN-208, Toronto, ON, Canada, M5G 2C4
| | - Mohammad Alavinia
- KITE, Toronto Rehabilitation Institute, University Health Network, 550 University Ave, Toronto, ON, Canada, M5G 2A2
| | - Pouyan Shaker
- Kansas City University, College of Osteopathic Medicine, Kansas City, MO, USA, 64106
| |
Collapse
|
2
|
Sulague RM, Beloy FJ, Medina JR, Mortalla ED, Cartojano TD, Macapagal S, Kpodonu J. Artificial intelligence in cardiac surgery: A systematic review. World J Surg 2024. [PMID: 39019775 DOI: 10.1002/wjs.12265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 06/14/2024] [Indexed: 07/19/2024]
Abstract
BACKGROUND Artificial intelligence (AI) has emerged as a tool to potentially increase the efficiency and efficacy of cardiovascular care and improve clinical outcomes. This study aims to provide an overview of applications of AI in cardiac surgery. METHODS A systematic literature search on AI applications in cardiac surgery from inception to February 2024 was conducted. Articles were then filtered based on the inclusion and exclusion criteria and the risk of bias was assessed. Key findings were then summarized. RESULTS A total of 81 studies were found that reported on AI applications in cardiac surgery. There is a rapid rise in studies since 2020. The most popular machine learning technique was random forest (n = 48), followed by support vector machine (n = 33), logistic regression (n = 32), and eXtreme Gradient Boosting (n = 31). Most of the studies were on adult patients, conducted in China, and involved procedures such as valvular surgery (24.7%), heart transplant (9.4%), coronary revascularization (11.8%), congenital heart disease surgery (3.5%), and aortic dissection repair (2.4%). Regarding evaluation outcomes, 35 studies examined the performance, 26 studies examined clinician outcomes, and 20 studies examined patient outcomes. CONCLUSION AI was mainly used to predict complications following cardiac surgeries and improve clinicians' decision-making by providing better preoperative risk assessment, stratification, and prognostication. While the application of AI in cardiac surgery has greatly progressed in the last decade, further studies need to be conducted to verify accuracy and ensure safety before use in clinical practice.
Collapse
Affiliation(s)
- Ralf Martz Sulague
- Graduate School of Arts and Sciences, Georgetown University, Washington, District of Columbia, USA
- Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | | | | | | | | | | | - Jacques Kpodonu
- Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| |
Collapse
|
3
|
Tadege M, Tegegne AS, Dessie ZG. Cardiac patients' surgery outcome and associated factors in Ethiopia: application of machine learning. BMC Pediatr 2024; 24:395. [PMID: 38886745 PMCID: PMC11184771 DOI: 10.1186/s12887-024-04870-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 06/03/2024] [Indexed: 06/20/2024] Open
Abstract
INTRODUCTION Cardiovascular diseases are a class of heart and blood vessel-related illnesses. In Sub-Saharan Africa, including Ethiopia, preventable heart disease continues to be a significant factor, contrasting with its presence in developed nations. Therefore, the objective of the study was to assess the prevalence of death due to cardiac disease and its risk factors among heart patients in Ethiopia. METHODS The current investigation included all cardiac patients who had cardiac surgery in the country between 2012 and 2023. A total of 1520 individuals were participated in the study. Data collection took place between February 2022 and January 2023. The study design was a retrospective cohort since the study track back patients' chart since 2012. Machine learning algorithms were applied for data analysis. For machine learning algorithms comparison, lift and AUC was applied. RESULTS From all possible algorithms, logistic algorithm at 90%/10% was the best fit since it produces the maximum AUC value. In addition, based on the lift value of 3.33, it can be concluded that the logistic regression algorithm was performing well and providing substantial improvement over random selection. From the logistic regression machine learning algorithms, age, saturated oxygen, ejection fraction, duration of cardiac center stays after surgery, waiting time to surgery, hemoglobin, and creatinine were significant predictors of death. CONCLUSION Some of the predictors for the death of cardiac disease patients are identified as such special attention should be given to aged patients, for patients waiting for long periods of time to get surgery, lower saturated oxygen, higher creatinine value, lower ejection fraction and for patients with lower hemoglobin values.
Collapse
Affiliation(s)
- Melaku Tadege
- College of Science, Bahir Dar University, Bahir Dar, Ethiopia.
- Department of Statistics, Injibara University, Injibara, Amhara, Ethiopia.
- Regional Data Management Center for Health (RDMC), Amhara Public Health Institute (APHI), Bahir Dar, Ethiopia.
| | | | - Zelalem G Dessie
- College of Science, Bahir Dar University, Bahir Dar, Ethiopia
- School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal, Durban, South Africa
| |
Collapse
|
4
|
Garcés-Jiménez A, Polo-Luque ML, Gómez-Pulido JA, Rodríguez-Puyol D, Gómez-Pulido JM. Predictive health monitoring: Leveraging artificial intelligence for early detection of infectious diseases in nursing home residents through discontinuous vital signs analysis. Comput Biol Med 2024; 174:108469. [PMID: 38636331 DOI: 10.1016/j.compbiomed.2024.108469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 04/08/2024] [Accepted: 04/09/2024] [Indexed: 04/20/2024]
Abstract
This research addresses the problem of detecting acute respiratory, urinary tract, and other infectious diseases in elderly nursing home residents using machine learning algorithms. The study analyzes data extracted from multiple vital signs and other contextual information for diagnostic purposes. The daily data collection process encounters sampling constraints due to weekends, holidays, shift changes, staff turnover, and equipment breakdowns, resulting in numerous nulls, repeated readings, outliers, and meaningless values. The short time series generated also pose a challenge to analysis, preventing the extraction of seasonal information or consistent trends. Blind data collection results in most of the data coming from periods when residents are healthy, resulting in excessively imbalanced data. This study proposes a data cleaning process and then builds a mechanism that reproduces the basal activity of the residents to improve the classification of the disease. The results show that the proposed basal module-assisted machine learning techniques allow anticipating diagnostics 2, 3 or 4 days before doctors decide to start treatment with antibiotics, achieving a performance measured by the area-under-the-curve metric of 0.857. The contributions of this work are: (1) a new data cleaning process; (2) the analysis of contextual information to improve data quality; (3) the generation of a baseline measure for relative comparison; and (4) the use of either binary (disease/no disease) or multiclass classification, differentiating among types of infections and showing the advantages of multiclass versus binary classification. From a medical point of view, the anticipated detection of infectious diseases in institutionalized individuals is brand new.
Collapse
Affiliation(s)
- Alberto Garcés-Jiménez
- Department of Computer Science, Universidad de Alcalá, Politechnic School, Alcala de Henares, 28805, Spain
| | - María-Luz Polo-Luque
- Department of Nursing and Physiotherapy, Universidad de Alcalá, Faculty of Medicine and Health Sciences, Alcala de Henares, 28805, Spain
| | - Juan A Gómez-Pulido
- Department of Technologies of Computers and Communications, Universidad de Extremadura, School of Technology, Cáceres, 10003, Spain.
| | - Diego Rodríguez-Puyol
- Department of Medicine and Medical Specialties, Research Foundation of the University Hospital Príncipe de Asturias, Campus Científico Tecnológico, Alcala de Henares, 28805, Spain
| | - José M Gómez-Pulido
- Department of Computer Science, Universidad de Alcalá, Politechnic School, Alcala de Henares, 28805, Spain
| |
Collapse
|
5
|
Tadege M, Tegegne AS, Dessie ZG. Post-surgery survival and associated factors for cardiac patients in Ethiopia: applications of machine learning, semi-parametric and parametric modelling. BMC Med Inform Decis Mak 2024; 24:91. [PMID: 38553701 PMCID: PMC10979627 DOI: 10.1186/s12911-024-02480-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 03/11/2024] [Indexed: 04/02/2024] Open
Abstract
INTRODUCTION Living in poverty, especially in low-income countries, are more affected by cardiovascular disease. Unlike the developed countries, it remains a significant cause of preventable heart disease in the Sub-Saharan region, including Ethiopia. According to the Ethiopian Ministry of Health statement, around 40,000 cardiac patients have been waiting for surgery in Ethiopia since September 2020. There is insufficient information about long-term cardiac patients' post-survival after cardiac surgery in Ethiopia. Therefore, the main objective of the current study was to determine the long-term post-cardiac surgery patients' survival status in Ethiopia. METHODS All patients attended from 2012 to 2023 throughout the country were included in the current study. The total number of participants was 1520 heart disease patients. The data collection procedure was conducted from February 2022- January 2023. Machine learning algorithms were applied. Gompertz regression was used also for the multivariable analysis report. RESULTS From possible machine learning models, random survival forest were preferred. It emphasizes, the most important variable for clinical prediction was SPO2, Age, time to surgery waiting time, and creatinine value and it accounts, 42.55%, 25.17%,11.82%, and 12.19% respectively. From the Gompertz regression, lower saturated oxygen, higher age, lower ejection fraction, short period of cardiac center stays after surgery, prolonged waiting time to surgery, and creating value were statistically significant predictors of death outcome for post-cardiac surgery patients' survival in Ethiopia. CONCLUSION Some of the risk factors for the death of post-cardiac surgery patients are identified in the current investigation. Particular attention should be given to patients with prolonged waiting times and aged patients. Since there were only two fully active cardiac centers in Ethiopia it is far from an adequate number of centers for more than 120 million population, therefore, the study highly recommended to increase the number of cardiac centers that serve as cardiac surgery in Ethiopia.
Collapse
Affiliation(s)
- Melaku Tadege
- College of Science, Bahir Dar University, Bahir Dar, Ethiopia.
- Department of Statistics, Injibara University, Injibara, Amhara, Ethiopia.
- Regional Data Management Center for Health (RDMC), Amhara Public Health Institute (APHI), Bahir Dar, Ethiopia.
| | | | - Zelalem G Dessie
- College of Science, Bahir Dar University, Bahir Dar, Ethiopia
- School of Mathematics, Statistics and Computer Science, University of KwaZulu- Natal, Durban, South Africa
| |
Collapse
|
6
|
Notenboom ML, Rhellab R, Etnel JRG, Huygens SA, Hjortnaes J, Kluin J, Takkenberg JJM, Veen KM. How microsimulation translates outcome estimates to patient lifetime event occurrence in the setting of heart valve disease. Eur J Cardiothorac Surg 2024; 65:ezae087. [PMID: 38515198 DOI: 10.1093/ejcts/ezae087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Revised: 02/08/2024] [Accepted: 03/06/2024] [Indexed: 03/23/2024] Open
Abstract
Treatment decisions in healthcare often carry lifelong consequences that can be challenging to foresee. As such, tools that visualize and estimate outcome after different lifetime treatment strategies are lacking and urgently needed to support clinical decision-making in the setting of rapidly evolving healthcare systems, with increasingly numerous potential treatments. In this regard, microsimulation models may prove to be valuable additions to current risk-prediction models. Notable advantages of microsimulation encompass input from multiple data sources, the ability to move beyond time-to-first-event analysis, accounting for multiple types of events and generating projections of lifelong outcomes. This review aims to clarify the concept of microsimulation, also known as individualized state-transition models, and help clinicians better understand its potential in clinical decision-making. A practical example of a patient with heart valve disease is used to illustrate key components of microsimulation models, such as health states, transition probabilities, input parameters (e.g. evidence-based risks of events) and various aspects of mortality. Finally, this review focuses on future efforts needed in microsimulation to allow for increasing patient-tailoring of the models by extending the general structure with patient-specific prediction models and translating them to meaningful, user-friendly tools that may be used by both clinician and patient to support clinical decision-making.
Collapse
Affiliation(s)
- Maximiliaan L Notenboom
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Reda Rhellab
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Jonathan R G Etnel
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| | | | - Jesper Hjortnaes
- Department of Cardiothoracic Surgery, Leiden University Medical Center, Rotterdam, Netherlands
| | - Jolanda Kluin
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Johanna J M Takkenberg
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| | - Kevin M Veen
- Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, Netherlands
| |
Collapse
|
7
|
Alves CL, Toutain TGLDO, Porto JAM, Aguiar PMDC, de Sena EP, Rodrigues FA, Pineda AM, Thielemann C. Analysis of functional connectivity using machine learning and deep learning in different data modalities from individuals with schizophrenia. J Neural Eng 2023; 20:056025. [PMID: 37673060 DOI: 10.1088/1741-2552/acf734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 09/06/2023] [Indexed: 09/08/2023]
Abstract
Objective. Schizophrenia(SCZ) is a severe mental disorder associated with persistent or recurrent psychosis, hallucinations, delusions, and thought disorders that affect approximately 26 million people worldwide, according to the World Health Organization. Several studies encompass machine learning (ML) and deep learning algorithms to automate the diagnosis of this mental disorder. Others study SCZ brain networks to get new insights into the dynamics of information processing in individuals suffering from the condition. In this paper, we offer a rigorous approach with ML and deep learning techniques for evaluating connectivity matrices and measures of complex networks to establish an automated diagnosis and comprehend the topology and dynamics of brain networks in SCZ individuals.Approach.For this purpose, we employed an functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) dataset. In addition, we combined EEG measures, i.e. Hjorth mobility and complexity, with complex network measurements to be analyzed in our model for the first time in the literature.Main results.When comparing the SCZ group to the control group, we found a high positive correlation between the left superior parietal lobe and the left motor cortex and a positive correlation between the left dorsal posterior cingulate cortex and the left primary motor. Regarding complex network measures, the diameter, which corresponds to the longest shortest path length in a network, may be regarded as a biomarker because it is the most crucial measure in different data modalities. Furthermore, the SCZ brain networks exhibit less segregation and a lower distribution of information. As a result, EEG measures outperformed complex networks in capturing the brain alterations associated with SCZ.Significance. Our model achieved an area under receiver operating characteristic curve (AUC) of 100% and an accuracy of 98.5% for the fMRI, an AUC of 95%, and an accuracy of 95.4% for the EEG data set. These are excellent classification results. Furthermore, we investigated the impact of specific brain connections and network measures on these results, which helped us better describe changes in the diseased brain.
Collapse
Affiliation(s)
- Caroline L Alves
- University of São Paulo (USP), Institute of Mathematical and Computer Sciences (ICMC), São Paulo, Brazil
- BioMEMS Lab, Aschaffenburg University of Applied Sciences, Aschaffenburg, Germany
| | | | | | - Patrícia Maria de Carvalho Aguiar
- Hospital Israelita Albert Einstein, São Paulo, Brazil
- Federal University of São Paulo, Department of Neurology and Neurosurgery, São Paulo, Brazil
| | | | - Francisco A Rodrigues
- University of São Paulo (USP), Institute of Mathematical and Computer Sciences (ICMC), São Paulo, Brazil
| | - Aruane M Pineda
- University of São Paulo (USP), Institute of Mathematical and Computer Sciences (ICMC), São Paulo, Brazil
| | | |
Collapse
|
8
|
Loftus TJ, Altieri MS, Balch JA, Abbott KL, Choi J, Marwaha JS, Hashimoto DA, Brat GA, Raftopoulos Y, Evans HL, Jackson GP, Walsh DS, Tignanelli CJ. Artificial Intelligence-enabled Decision Support in Surgery: State-of-the-art and Future Directions. Ann Surg 2023; 278:51-58. [PMID: 36942574 DOI: 10.1097/sla.0000000000005853] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/23/2023]
Abstract
OBJECTIVE To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting. BACKGROUND To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown. METHODS Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. RESULTS Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2000; 7 of these 8 articles (87.5%) had below-average (<0.83) area under the receiver operating characteristic or accuracy. Overall, 29 articles (80.6%) performed internal validation only, 5 (13.8%) performed external validation, and 2 (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy. CONCLUSIONS Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.
Collapse
Affiliation(s)
- Tyler J Loftus
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Maria S Altieri
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Pennsylvania, Philadelphia, PA
| | - Jeremy A Balch
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Kenneth L Abbott
- Department of Surgery, University of Florida Health, Gainesville, FL
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
| | - Jeff Choi
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Stanford University, Stanford, CA
| | - Jayson S Marwaha
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Beth Israel Deaconess Medical Center
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA
| | - Daniel A Hashimoto
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Pennsylvania Perelman School of Medicine
- General Robotics, Automation, Sensing, and Perception Laboratory, University of Pennsylvania School of Engineering and Applied Science, Philadelphia, PA
| | - Gabriel A Brat
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Beth Israel Deaconess Medical Center
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA
| | - Yannis Raftopoulos
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Weight Management Program, Holyoke Medical Center, Holyoke, MA
| | - Heather L Evans
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, Medical University of South Carolina, Charleston, SC
| | - Gretchen P Jackson
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Digital, Intuitive Surgical, Sunnyvale, CA; Departments of Pediatric Surgery, Pediatrics, and Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Danielle S Walsh
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery, University of Kentucky, Lexington, KY
| | - Christopher J Tignanelli
- American College of Surgeons Health Information Technology Committee and Artificial Intelligence Subcommittee, Chicago, IL
- Department of Surgery
- Institute for Health Informatics
- Program for Clinical Artificial Intelligence, Center for Learning Health Systems Science, University of Minnesota, Minneapolis, MN
| |
Collapse
|
9
|
Alves CL, Toutain TGLDO, de Carvalho Aguiar P, Pineda AM, Roster K, Thielemann C, Porto JAM, Rodrigues FA. Diagnosis of autism spectrum disorder based on functional brain networks and machine learning. Sci Rep 2023; 13:8072. [PMID: 37202411 DOI: 10.1038/s41598-023-34650-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Accepted: 05/04/2023] [Indexed: 05/20/2023] Open
Abstract
Autism is a multifaceted neurodevelopmental condition whose accurate diagnosis may be challenging because the associated symptoms and severity vary considerably. The wrong diagnosis can affect families and the educational system, raising the risk of depression, eating disorders, and self-harm. Recently, many works have proposed new methods for the diagnosis of autism based on machine learning and brain data. However, these works focus on only one pairwise statistical metric, ignoring the brain network organization. In this paper, we propose a method for the automatic diagnosis of autism based on functional brain imaging data recorded from 500 subjects, where 242 present autism spectrum disorder considering the regions of interest throughout Bootstrap Analysis of Stable Cluster map. Our method can distinguish the control group from autism spectrum disorder patients with high accuracy. Indeed the best performance provides an AUC near 1.0, which is higher than that found in the literature. We verify that the left ventral posterior cingulate cortex region is less connected to an area in the cerebellum of patients with this neurodevelopment disorder, which agrees with previous studies. The functional brain networks of autism spectrum disorder patients show more segregation, less distribution of information across the network, and less connectivity compared to the control cases. Our workflow provides medical interpretability and can be used on other fMRI and EEG data, including small data sets.
Collapse
Affiliation(s)
- Caroline L Alves
- Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), São Paulo, Brazil.
- BioMEMS Lab, Aschaffenburg University of Applied Sciences, Aschaffenburg, Germany.
| | | | - Patricia de Carvalho Aguiar
- Hospital Israelita Albert Einstein, São Paulo, Brazil
- Department of Neurology and Neurosurgery, Federal University of São Paulo, São Paulo, Brazil
| | - Aruane M Pineda
- Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), São Paulo, Brazil
| | - Kirstin Roster
- Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), São Paulo, Brazil
| | | | | | - Francisco A Rodrigues
- Institute of Mathematical and Computer Sciences (ICMC), University of São Paulo (USP), São Paulo, Brazil
| |
Collapse
|
10
|
Behnoush AH, Khalaji A, Rezaee M, Momtahen S, Mansourian S, Bagheri J, Masoudkabir F, Hosseini K. Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery. Clin Cardiol 2023; 46:269-278. [PMID: 36588391 PMCID: PMC10018097 DOI: 10.1002/clc.23963] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 12/12/2022] [Accepted: 12/19/2022] [Indexed: 01/03/2023] Open
Abstract
BACKGROUND Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG. HYOTHESIS ML algorithms can significantly improve mortality prediction after CABG. METHODS Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models. RESULTS Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients. CONCLUSIONS All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).
Collapse
Affiliation(s)
- Amir Hossein Behnoush
- Tehran Heart Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- School of MedicineTehran University of Medical SciencesTehranIran
- Non‐Communicable Diseases Research Center, Endocrinology and Metabolism Population Sciences InstituteTehran University of Medical SciencesTehranIran
| | - Amirmohammad Khalaji
- Tehran Heart Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- School of MedicineTehran University of Medical SciencesTehranIran
- Non‐Communicable Diseases Research Center, Endocrinology and Metabolism Population Sciences InstituteTehran University of Medical SciencesTehranIran
| | - Malihe Rezaee
- Tehran Heart Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Non‐Communicable Diseases Research Center, Endocrinology and Metabolism Population Sciences InstituteTehran University of Medical SciencesTehranIran
- School of MedicineShahid Beheshti University of Medical SciencesTehranIran
| | - Shahram Momtahen
- Department of Surgery, Tehran Heart CenterTehran University of Medical SciencesTehranIran
| | - Soheil Mansourian
- Department of Surgery, Tehran Heart CenterTehran University of Medical SciencesTehranIran
| | - Jamshid Bagheri
- Department of Surgery, Tehran Heart CenterTehran University of Medical SciencesTehranIran
| | - Farzad Masoudkabir
- Tehran Heart Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
| | - Kaveh Hosseini
- Tehran Heart Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
- Cardiac Primary Prevention Research Center, Cardiovascular Diseases Research InstituteTehran University of Medical SciencesTehranIran
| |
Collapse
|
11
|
Morris MX, Song EY, Rajesh A, Asaad M, Phillips BT. Ethical, Legal, and Financial Considerations of Artificial Intelligence in Surgery. Am Surg 2023; 89:55-60. [PMID: 35978473 DOI: 10.1177/00031348221117042] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Machine learning systems have become integrated into some of the most vital decision-making aspects of humanity, including hiring decisions, loan applications, and automobile safety, to name just a few. As applications increase in both gravity and complexity, the data quality and algorithmic interpretability of the systems must rise to meet those challenges. This is especially vital for navigating the nuances of health care, particularly among the high stakes of surgical operations. In addition to inherent ethical challenges of enabling a "black box" system to influence decision-making in patient care, the creation of biased datasets leads to biased algorithms with the power to perpetuate discrimination and reinforce disparities. Transparency and responsibility are paramount to the implementation of artificial intelligence in surgical decision-making and autonomous robotic surgery. Machine learning has been permeating health care across diverse clinical and surgical contexts but continues to face sizable obstacles, including apprehension from patients and providers alike. To integrate the technology fully while upholding standard of care and patient-provider trust, one must acknowledge and address the ethical, financial, and legal implications of using artificial intelligence for patient care.
Collapse
Affiliation(s)
- Miranda X Morris
- 12277Duke University School of Medicine, Durham, NC, USA.,22957Duke Pratt School of Engineering, Durham, NC, USA
| | - Ethan Y Song
- Division of Plastic, Maxillofacial, and Oral Surgery, Department of Surgery, 14742Duke University Hospital, Durham, NC, USA
| | - Aashish Rajesh
- Department of Surgery, 571198University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
| | - Malke Asaad
- Department of Plastic Surgery, 22957University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Brett T Phillips
- Division of Plastic, Maxillofacial, and Oral Surgery, Department of Surgery, 14742Duke University Hospital, Durham, NC, USA
| |
Collapse
|
12
|
Alves CL, Cury RG, Roster K, Pineda AM, Rodrigues FA, Thielemann C, Ciba M. Application of machine learning and complex network measures to an EEG dataset from ayahuasca experiments. PLoS One 2022; 17:e0277257. [PMID: 36525422 PMCID: PMC9757568 DOI: 10.1371/journal.pone.0277257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 10/23/2022] [Indexed: 12/23/2022] Open
Abstract
Ayahuasca is a blend of Amazonian plants that has been used for traditional medicine by the inhabitants of this region for hundreds of years. Furthermore, this plant has been demonstrated to be a viable therapy for a variety of neurological and mental diseases. EEG experiments have found specific brain regions that changed significantly due to ayahuasca. Here, we used an EEG dataset to investigate the ability to automatically detect changes in brain activity using machine learning and complex networks. Machine learning was applied at three different levels of data abstraction: (A) the raw EEG time series, (B) the correlation of the EEG time series, and (C) the complex network measures calculated from (B). Further, at the abstraction level of (C), we developed new measures of complex networks relating to community detection. As a result, the machine learning method was able to automatically detect changes in brain activity, with case (B) showing the highest accuracy (92%), followed by (A) (88%) and (C) (83%), indicating that connectivity changes between brain regions are more important for the detection of ayahuasca. The most activated areas were the frontal and temporal lobe, which is consistent with the literature. F3 and PO4 were the most important brain connections, a significant new discovery for psychedelic literature. This connection may point to a cognitive process akin to face recognition in individuals during ayahuasca-mediated visual hallucinations. Furthermore, closeness centrality and assortativity were the most important complex network measures. These two measures are also associated with diseases such as Alzheimer's disease, indicating a possible therapeutic mechanism. Moreover, the new measures were crucial to the predictive model and suggested larger brain communities associated with the use of ayahuasca. This suggests that the dissemination of information in functional brain networks is slower when this drug is present. Overall, our methodology was able to automatically detect changes in brain activity during ayahuasca consumption and interpret how these psychedelics alter brain networks, as well as provide insights into their mechanisms of action.
Collapse
Affiliation(s)
- Caroline L. Alves
- BioMEMS Lab, Aschaffenburg University of Applied Sciences (UAS), Aschaffenburg, Germany
- Institute of Mathematical and Computer Sciences, University of São Paulo (USP), São Paulo, Brazil
- * E-mail:
| | - Rubens Gisbert Cury
- Department of Neurology, Movement Disorders Center, University of São Paulo (USP), São Paulo, Brazil
| | - Kirstin Roster
- Institute of Mathematical and Computer Sciences, University of São Paulo (USP), São Paulo, Brazil
| | - Aruane M. Pineda
- Institute of Mathematical and Computer Sciences, University of São Paulo (USP), São Paulo, Brazil
| | - Francisco A. Rodrigues
- Institute of Mathematical and Computer Sciences, University of São Paulo (USP), São Paulo, Brazil
| | - Christiane Thielemann
- BioMEMS Lab, Aschaffenburg University of Applied Sciences (UAS), Aschaffenburg, Germany
| | - Manuel Ciba
- BioMEMS Lab, Aschaffenburg University of Applied Sciences (UAS), Aschaffenburg, Germany
| |
Collapse
|
13
|
Fransvea P, Fransvea G, Liuzzi P, Sganga G, Mannini A, Costa G. Study and validation of an explainable machine learning-based mortality prediction following emergency surgery in the elderly: A prospective observational study. Int J Surg 2022; 107:106954. [PMID: 36229017 DOI: 10.1016/j.ijsu.2022.106954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Revised: 09/07/2022] [Accepted: 10/03/2022] [Indexed: 10/31/2022]
Abstract
INTRODUCTION The heterogeneity of procedures and the variety of comorbidities of the patients undergoing surgery in an emergency setting makes perioperative risk stratification, planning, and risk mitigation crucial. In this optic, Machine Learning has the capability of deriving data-driven predictions based on multivariate interactions of thousands of instances. Our aim was to cross-validate and test interpretable models for the prediction of post-operative mortality after any surgery in an emergency setting on elderly patients. METHODS This study is a secondary analysis derived from the FRAILESEL study, a multi-center (N = 29 emergency care units), nationwide, observational prospective study with data collected between 06-2017 and 06-2018 investigating perioperative outcomes of elderly patients (age≥65 years) undergoing emergency surgery. Demographic and clinical data, medical and surgical history, preoperative risk factors, frailty, biochemical blood examination, vital parameters, and operative details were collected and the primary outcome was set to the 30-day mortality. RESULTS Of the 2570 included patients (50.66% males, median age 77 [IQR = 13] years) 238 (9.26%) were in the non-survivors group. The best performing solution (MultiLayer Perceptron) resulted in a test accuracy of 94.9% (sensitivity = 92.0%, specificity = 95.2%). Model explanations showed how non-chronic cardiac-related comorbidities reduced activities of daily living, low consciousness levels, high creatinine and low saturation increase the risk of death following surgery. CONCLUSIONS In this prospective observational study, a robustly cross-validated model resulted in better predictive performance than existing tools and scores in literature. By using only preoperative features and by deriving patient-specific explanations, the model provides crucial information during shared decision-making processes required for risk mitigation procedures.
Collapse
Affiliation(s)
- Pietro Fransvea
- Emergency Surgery and Trauma, Fondazione Policlinico Universitario A. Gemelli IRCCS, Università Cattolica del Sacro Cuore, Largo A. Gemelli 8, Rome, Italy The BioRobotics Institute, Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio 34, Pontedera, PI, Italy IRCCS Fondazione Don Carlo Gnocchi ONLUS, Via di Scandicci 269, Firenze, FI, Italy Surgery Center, Colorectal Surgery Unit - Fondazione Policlinico Campus Bio-Medico, University Hospital of University Campus Bio-Medico of Rome, Rome, Italy
| | | | | | | | | | | |
Collapse
|
14
|
Montisci A, Palmieri V, Vietri MT, Sala S, Maiello C, Donatelli F, Napoli C. Big Data in cardiac surgery: real world and perspectives. J Cardiothorac Surg 2022; 17:277. [PMID: 36309702 PMCID: PMC9617748 DOI: 10.1186/s13019-022-02025-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Accepted: 10/14/2022] [Indexed: 11/10/2022] Open
Abstract
Big Data, and the derived analysis techniques, such as artificial intelligence and machine learning, have been considered a revolution in the modern practice of medicine. Big Data comes from multiple sources, encompassing electronic health records, clinical studies, imaging data, registries, administrative databases, patient-reported outcomes and OMICS profiles. The main objective of such analyses is to unveil hidden associations and patterns. In cardiac surgery, the main targets for the use of Big Data are the construction of predictive models to recognize patterns or associations better representing the individual risk or prognosis compared to classical surgical risk scores. The results of these studies contributed to kindle the interest for personalized medicine and contributed to recognize the limitations of randomized controlled trials in representing the real world. However, the main sources of evidence for guidelines and recommendations remain RCTs and meta-analysis. The extent of the revolution of Big Data and new analytical models in cardiac surgery is yet to be determined.
Collapse
|
15
|
Garcia-Canadilla P, Isabel-Roquero A, Aurensanz-Clemente E, Valls-Esteve A, Miguel FA, Ormazabal D, Llanos F, Sanchez-de-Toledo J. Machine Learning-Based Systems for the Anticipation of Adverse Events After Pediatric Cardiac Surgery. Front Pediatr 2022; 10:930913. [PMID: 35832588 PMCID: PMC9271800 DOI: 10.3389/fped.2022.930913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 06/06/2022] [Indexed: 11/18/2022] Open
Abstract
Pediatric congenital heart disease (CHD) patients are at higher risk of postoperative complications and clinical deterioration either due to their underlying pathology or due to the cardiac surgery, contributing significantly to mortality, morbidity, hospital and family costs, and poor quality of life. In current clinical practice, clinical deterioration is detected, in most of the cases, when it has already occurred. Several early warning scores (EWS) have been proposed to assess children at risk of clinical deterioration using vital signs and risk indicators, in order to intervene in a timely manner to reduce the impact of deterioration and risk of death among children. However, EWS are based on measurements performed at a single time point without incorporating trends nor providing information about patient's risk trajectory. Moreover, some of these measurements rely on subjective assessment making them susceptible to different interpretations. All these limitations could explain why the implementation of EWS in high-resource settings failed to show a significant decrease in hospital mortality. By means of machine learning (ML) based algorithms we could integrate heterogeneous and complex data to predict patient's risk of deterioration. In this perspective article, we provide a brief overview of the potential of ML technologies to improve the identification of pediatric CHD patients at high-risk for clinical deterioration after cardiac surgery, and present the CORTEX traffic light, a ML-based predictive system that Sant Joan de Déu Barcelona Children's Hospital is implementing, as an illustration of the application of an ML-based risk stratification system in a relevant hospital setting.
Collapse
Affiliation(s)
- Patricia Garcia-Canadilla
- BCNatal—Barcelona Center for Maternal-Fetal and Neonatal Medicine, Hospital Sant Joan de Déu and Hospital Clínic, University of Barcelona, Barcelona, Spain
- Cardiovascular Diseases and Child Development, Institut de Recerca Sant Joan de Déu, Esplugues de Llobregat, Spain
| | - Alba Isabel-Roquero
- Department of Pediatric Cardiology, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
- BCNMedTech, Universitat Pompeu Fabra, Barcelona, Spain
| | - Esther Aurensanz-Clemente
- Cardiovascular Diseases and Child Development, Institut de Recerca Sant Joan de Déu, Esplugues de Llobregat, Spain
- Department of Pediatric Cardiology, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
| | - Arnau Valls-Esteve
- Innovation in Health Technologies, Institut de Recerca Sant Joan de Déu, Esplugues de Llobregat, Spain
| | - Francesca Aina Miguel
- Department of Engineering, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
| | - Daniel Ormazabal
- Department of Informatics, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
| | - Floren Llanos
- Department of Informatics, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
| | - Joan Sanchez-de-Toledo
- Cardiovascular Diseases and Child Development, Institut de Recerca Sant Joan de Déu, Esplugues de Llobregat, Spain
- Department of Pediatric Cardiology, Hospital Sant Joan de Déu Barcelona, Esplugues de Llobregat, Spain
- Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, PA, United States
| |
Collapse
|
16
|
Campagner A, Sternini F, Cabitza F. Decisions are not all equal-Introducing a utility metric based on case-wise raters' perceptions. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 221:106930. [PMID: 35690505 DOI: 10.1016/j.cmpb.2022.106930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 05/13/2022] [Accepted: 05/31/2022] [Indexed: 06/15/2023]
Abstract
Background and Objective Evaluation of AI-based decision support systems (AI-DSS) is of critical importance in practical applications, nonetheless common evaluation metrics fail to properly consider relevant and contextual information. In this article we discuss a novel utility metric, the weighted Utility (wU), for the evaluation of AI-DSS, which is based on the raters' perceptions of their annotation hesitation and of the relevance of the training cases. Methods We discuss the relationship between the proposed metric and other previous proposals; and we describe the application of the proposed metric for both model evaluation and optimization, through three realistic case studies. Results We show that our metric generalizes the well-known Net Benefit, as well as other common error-based and utility-based metrics. Through the empirical studies, we show that our metric can provide a more flexible tool for the evaluation of AI models. We also show that, compared to other optimization metrics, model optimization based on the wU can provide significantly better performance (AUC 0.862 vs 0.895, p-value <0.05), especially on cases judged to be more complex by the human annotators (AUC 0.85 vs 0.92, p-value <0.05). Conclusions We make the point for having utility as a primary concern in the evaluation and optimization of machine learning models in critical domains, like the medical one; and for the importance of a human-centred approach to assess the potential impact of AI models on human decision making also on the basis of further information that can be collected during the ground-truthing process.
Collapse
Affiliation(s)
- Andrea Campagner
- Dipartimento di Informatica, Sistemistica e Comunicazione, Università di Milano-Bicocca, Milano, Italy.
| | - Federico Sternini
- Polito(BIO)Med Lab, Politecnico di Torino, Torino, Italy; USE-ME-D srl, I3P Politecnico di Torino, Torino, Ital
| | - Federico Cabitza
- Dipartimento di Informatica, Sistemistica e Comunicazione, Università di Milano-Bicocca, Milano, Italy; IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| |
Collapse
|
17
|
Wang X, Zhu T, Xia M, Liu Y, Wang Y, Wang X, Zhuang L, Zhong D, Zhu J, He H, Weng S, Zhu J, Lai D. Predicting the Prognosis of Patients in the Coronary Care Unit: A Novel Multi-Category Machine Learning Model Using XGBoost. Front Cardiovasc Med 2022; 9:764629. [PMID: 35647052 PMCID: PMC9133425 DOI: 10.3389/fcvm.2022.764629] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 04/20/2022] [Indexed: 11/13/2022] Open
Abstract
Background Early prediction and classification of prognosis is essential for patients in the coronary care unit (CCU). We applied a machine learning (ML) model using the eXtreme Gradient Boosting (XGBoost) algorithm to prognosticate CCU patients and compared XGBoost with traditional classification models. Methods CCU patients' data were extracted from the MIMIC-III v1.4 clinical database, and divided into four groups based on the time to death: <30 days, 30 days−1 year, 1–5 years, and ≥5 years. Four classification models, including XGBoost, naïve Bayes (NB), logistic regression (LR), and support vector machine (SVM) were constructed using the Python software. These four models were tested and compared for accuracy, F1 score, Matthews correlation coefficient (MCC), and area under the curve (AUC) of the receiver operating characteristic curves. Subsequently, Local Interpretable Model-Agnostic Explanations method was performed to improve XGBoost model interpretability. We also constructed sub-models of each model based on the different categories of death time and compared the differences by decision curve analysis. The optimal model was further analyzed using a clinical impact curve. At last, feature ablation curves of the XGBoost model were conducted to obtain the simplified model. Results Overall, 5360 CCU patients were included. Compared to NB, LR, and SVM, the XGBoost model showed better accuracy (0.663, 0.605, 0.632, and 0.622), micro-AUCs (0.873, 0.811, 0.841, and 0.818), and MCC (0.337, 0.317, 0.250, and 0.182). In subgroup analysis, the XGBoost model had a better predictive performance in acute myocardial infarction subgroup. The decision curve and clinical impact curve analyses verified the clinical utility of the XGBoost model for different categories of patients. Finally, we obtained a simplified model with thirty features. Conclusions For CCU physicians, the ML technique by XGBoost is a potential predictive tool in patients with different conditions, and it may contribute to improvements in prognosis.
Collapse
Affiliation(s)
- Xingchen Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Tianqi Zhu
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China
| | - Minghong Xia
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yu Liu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Yao Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Xizhi Wang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Lenan Zhuang
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Danfeng Zhong
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Jun Zhu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Hong He
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Shaoxiang Weng
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Junhui Zhu
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
- Junhui Zhu
| | - Dongwu Lai
- Key Laboratory of Cardiovascular Intervention and Regenerative Medicine of Zhejiang Province, Department of Cardiology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, China
- *Correspondence: Dongwu Lai
| |
Collapse
|
18
|
Lu Y, Zhang Q, Jiang J. Development and validation of a prediction model for in-hospital mortality of patients with severe thrombocytopenia. Sci Rep 2022; 12:6316. [PMID: 35428822 PMCID: PMC9012749 DOI: 10.1038/s41598-022-10438-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Accepted: 04/05/2022] [Indexed: 11/16/2022] Open
Abstract
Risk stratification and prognosis evaluation of severe thrombocytopenia are essential for clinical treatment and management. Currently, there is currently no reliable predictive model to identify patients at high risk of severe thrombocytopenia. This study aimed to develop and validate a prognostic nomogram model to predict in-hospital mortality in patients with severe thrombocytopenia in the intensive care unit. Patients diagnosed with severe thrombocytopenia (N = 1561) in the Medical Information Mart for Intensive Care IV database were randomly divided into training (70%) and validation (30%) cohorts. In the training cohort, univariate and multivariate logistic regression analyses with positive stepwise selection were performed to screen the candidate variables, and variables with p < 0.05 were included in the nomogram model. The nomogram model was compared with traditional severity assessment tools and included the following 13 variables: age, cerebrovascular disease, malignant cancer, oxygen saturation, heart rate, mean arterial pressure, respiration rate, mechanical ventilation, vasopressor, continuous renal replacement therapy, prothrombin time, partial thromboplastin time, and blood urea nitrogen. The nomogram was well-calibrated. According to the area under the receiver operating characteristics, reclassification improvement, and integrated discrimination improvement, the nomogram model performed better than the traditional sequential organ failure assessment (SOFA) score and simplified acute physiology score II (SAPS II). Additionally, according to decision curve analysis, a threshold probability between 0.1 and 0.75 indicated that our constructed nomogram model showed more net benefits than the SOFA score and SAPS II. The nomogram model we established showed superior predictive performance and can assist in the quantitative assessment of the prognostic risk in patients with severe thrombocytopenia.
Collapse
Affiliation(s)
- Yan Lu
- Clinical Laboratory, DongYang People's Hospital, 60 West Wuning Road, Dongyang, 322100, Zhejiang, China.
| | - Qiaohong Zhang
- Clinical Laboratory, DongYang People's Hospital, 60 West Wuning Road, Dongyang, 322100, Zhejiang, China
| | - Jinwen Jiang
- Clinical Laboratory, DongYang People's Hospital, 60 West Wuning Road, Dongyang, 322100, Zhejiang, China
| |
Collapse
|
19
|
Aortic Risks Prediction Models after Cardiac Surgeries Using Integrated Data. J Pers Med 2022; 12:jpm12040637. [PMID: 35455753 PMCID: PMC9024528 DOI: 10.3390/jpm12040637] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Revised: 04/09/2022] [Accepted: 04/12/2022] [Indexed: 12/14/2022] Open
Abstract
The complications of thoracic aortic disease include aortic dissection and aneurysm. The risks are frequently compounded by many cardiovascular comorbidities, which makes the process of clinical decision making complicated. The purpose of this study is to develop risk predictive models for patients after thoracic aneurysm surgeries, using integrated data from different medical institutions. Seven risk features were formulated for prediction. The CatBoost classifier performed best and provided an ROC AUC of 0.94–0.98 and an F-score of 0.95–0.98. The obtained results are widely in line with the current literature. The obtained findings provide additional support for clinical decision making, guiding a patient care team prior to surgical treatment, and promoting a safe postoperative period.
Collapse
|
20
|
Rellum SR, Schuurmans J, van der Ven WH, Eberl S, Driessen AHG, Vlaar APJ, Veelo DP. Machine learning methods for perioperative anesthetic management in cardiac surgery patients: a scoping review. J Thorac Dis 2022; 13:6976-6993. [PMID: 35070381 PMCID: PMC8743411 DOI: 10.21037/jtd-21-765] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 08/27/2021] [Indexed: 12/27/2022]
Abstract
Background Machine learning (ML) is developing fast with promising prospects within medicine and already has several applications in perioperative care. We conducted a scoping review to examine the extent and potential limitations of ML implementation in perioperative anesthetic care, specifically in cardiac surgery patients. Methods We mapped the current literature by searching three databases: MEDLINE (Ovid), EMBASE (Ovid), and Cochrane Library. Articles were eligible if they reported on perioperative ML use in the field of cardiac surgery with relevance to anesthetic practices. Data on the applicability of ML and comparability to conventional statistical methods were extracted. Results Forty-six articles on ML relevant to the work of the anesthesiologist in cardiac surgery were identified. Three main categories emerged: (I) event and risk prediction, (II) hemodynamic monitoring, and (III) automation of echocardiography. Prediction models based on ML tend to behave similarly to conventional statistical methods. Using dynamic hemodynamic or ultrasound data in ML models, however, shifts the potential to promising results. Conclusions ML in cardiac surgery is increasingly used in perioperative anesthetic management. The majority is used for prediction purposes similar to conventional clinical scores. Remarkable ML model performances are achieved when using real-time dynamic parameters. However, beneficial clinical outcomes of ML integration have yet to be determined. Nonetheless, the first steps introducing ML in perioperative anesthetic care for cardiac surgery have been taken.
Collapse
Affiliation(s)
- Santino R Rellum
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Jaap Schuurmans
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Ward H van der Ven
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Susanne Eberl
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Antoine H G Driessen
- Department of Cardiothoracic Surgery, Heart Center, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Alexander P J Vlaar
- Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Denise P Veelo
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| |
Collapse
|
21
|
Zhang H, Tian W, Sun Y. A novel nomogram for predicting 3-year mortality in critically ill patients after coronary artery bypass grafting. BMC Surg 2021; 21:407. [PMID: 34847905 PMCID: PMC8638264 DOI: 10.1186/s12893-021-01408-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Accepted: 11/19/2021] [Indexed: 11/10/2022] Open
Abstract
Background The long-term outcomes for patients after coronary artery bypass grafting (CABG) have been received more and more concern. The existing prediction models are mostly focused on in-hospital operative mortality after CABG, but there is still little research on long-term mortality prediction model for patients after CABG. Objective To develop and validate a novel nomogram for predicting 3-year mortality in critically ill patients after CABG. Methods Data for developing novel predictive model were extracted from Medical Information Mart for Intensive cart III (MIMIC-III), of which 2929 critically ill patients who underwent CABG at the first admission were enrolled. Results A novel prognostic nomogram for 3-year mortality was constructed with the seven independent prognostic factors, including age, congestive heart failure, white blood cell, creatinine, SpO2, anion gap, and continuous renal replacement treatment derived from the multivariable logistic regression. The nomogram indicated accurate discrimination in primary (AUC: 0.81) and validation cohort (AUC: 0.802), which were better than traditional severity scores. And good consistency between the predictive and observed outcome was showed by the calibration curve for 3-year mortality. The decision curve analysis also showed higher clinical net benefit than traditional severity scores. Conclusion The novel nomogram had well performance to predict 3-year mortality in critically ill patients after CABG. The prediction model provided valuable information for treatment strategy and postdischarge management, which may be helpful in improving the long-term prognosis in critically ill patients after CABG. Supplementary Information The online version contains supplementary material available at 10.1186/s12893-021-01408-8.
Collapse
Affiliation(s)
- HuanRui Zhang
- Department of Geriatric Cardiology, The First Affiliated Hospital of China Medical University, NO.155 Nanjing North Street, Heping Ward, Shenyang, 110001, China
| | - Wen Tian
- Department of Geriatric Cardiology, The First Affiliated Hospital of China Medical University, NO.155 Nanjing North Street, Heping Ward, Shenyang, 110001, China
| | - YuJiao Sun
- Department of Geriatric Cardiology, The First Affiliated Hospital of China Medical University, NO.155 Nanjing North Street, Heping Ward, Shenyang, 110001, China.
| |
Collapse
|
22
|
Langarizadeh M, HosseiniNezhad M, Hosseini S. Mortality prediction of mitral valve replacement surgery by machine learning. Res Cardiovasc Med 2021. [DOI: 10.4103/rcm.rcm_50_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
|