1
|
Alge OP, Pickard J, Zhang W, Cheng S, Derksen H, Omenn GS, Gryak J, VanEpps JS, Najarian K. Continuous sepsis trajectory prediction using tensor-reduced physiological signals. Sci Rep 2024; 14:18155. [PMID: 39103488 PMCID: PMC11300462 DOI: 10.1038/s41598-024-68901-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 07/29/2024] [Indexed: 08/07/2024] Open
Abstract
The quick Sequential Organ Failure Assessment (qSOFA) system identifies an individual's risk to progress to poor sepsis-related outcomes using minimal variables. We used Support Vector Machine, Learning Using Concave and Convex Kernels, and Random Forest to predict an increase in qSOFA score using electronic health record (EHR) data, electrocardiograms (ECG), and arterial line signals. We structured physiological signals data in a tensor format and used Canonical Polyadic/Parallel Factors (CP) decomposition for feature reduction. Random Forests trained on ECG data show improved performance after tensor decomposition for predictions in a 6-h time frame (AUROC 0.67 ± 0.06 compared to 0.57 ± 0.08, p = 0.01 ). Adding arterial line features can also improve performance (AUROC 0.69 ± 0.07, p < 0.01 ), and benefit from tensor decomposition (AUROC 0.71 ± 0.07, p = 0.01 ). Adding EHR data features to a tensor-reduced signal model further improves performance (AUROC 0.77 ± 0.06, p < 0.01 ). Despite reduction in performance going from an EHR data-informed model to a tensor-reduced waveform data model, the signals-informed model offers distinct advantages. The first is that predictions can be made on a continuous basis in real-time, and second is that these predictions are not limited by the availability of EHR data. Additionally, structuring the waveform features as a tensor conserves structural and temporal information that would otherwise be lost if the data were presented as flat vectors.
Collapse
Affiliation(s)
- Olivia P Alge
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
| | - Joshua Pickard
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Winston Zhang
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Shuyang Cheng
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
| | - Harm Derksen
- Department of Mathematics, Northeastern University, Boston, MA, USA
| | - Gilbert S Omenn
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
- Departments of Internal Medicine, Human Genetics, and Environmental Health, Ann Arbor, MI, USA
| | - Jonathan Gryak
- Department of Computer Science, Queens College, CUNY, Queens, NY, USA
| | - J Scott VanEpps
- Michigan Center for Integrative Research in Critical Care, University of Michigan, Ann Arbor, MI, USA
- Department of Emergency Medicine, University of Michigan, Ann Arbor, MI, USA
- Biointerfaces Institute, University of Michigan, Ann Arbor, MI, USA
- Macromolecular Science and Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Kayvan Najarian
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA
- Michigan Center for Integrative Research in Critical Care, University of Michigan, Ann Arbor, MI, USA
- Department of Emergency Medicine, University of Michigan, Ann Arbor, MI, USA
- Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| |
Collapse
|
2
|
Choi H, Kim Y, Kang H, Seo H, Kim M, Han J, Kee G, Park S, Ko S, Jung H, Kim B, Roh JH, Jun TJ, Kim YH. Time series forecasting of weight for diuretic dose adjustment using bidirectional long short-term memory. Sci Rep 2024; 14:17723. [PMID: 39085306 PMCID: PMC11292016 DOI: 10.1038/s41598-024-68663-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2024] [Accepted: 07/26/2024] [Indexed: 08/02/2024] Open
Abstract
Loop diuretics are prevailing drugs to manage fluid overload in heart failure. However, adjusting to loop diuretic doses is strenuous due to the lack of a diuretic guideline. Accordingly, we developed a novel clinician decision support system for adjusting loop diuretics dosage with a Long Short-Term Memory (LSTM) algorithm using time-series EMRs. Weight measurements were used as the target to estimate fluid loss during diuretic therapy. We designed the TSFD-LSTM, a bi-directional LSTM model with an attention mechanism, to forecast weight change 48 h after heart failure patients were injected with loop diuretics. The model utilized 65 variables, including disease conditions, concurrent medications, laboratory results, vital signs, and physical measurements from EMRs. The framework processed four sequences simultaneously as inputs. An ablation study on attention mechanisms and a comparison with the transformer model as a baseline were conducted. The TSFD-LSTM outperformed the other models, achieving 85% predictive accuracy with MAE and MSE values of 0.56 and 1.45, respectively. Thus, the TSFD-LSTM model can aid in personalized loop diuretic treatment and prevent adverse drug events, contributing to improved healthcare efficacy for heart failure patients.
Collapse
Affiliation(s)
- Heejung Choi
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Yunha Kim
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Heejun Kang
- Division of Cardiology, Asan Medical Center, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Hyeram Seo
- Department of Information Medicine, Asan Medical Center, 88, Olympicro 43GilSongpagu, 05505, Seoul, Republic of Korea
| | - Minkyoung Kim
- Department of Information Medicine, Asan Medical Center, 88, Olympicro 43GilSongpagu, 05505, Seoul, Republic of Korea
| | - JiYe Han
- Department of Information Medicine, Asan Medical Center, 88, Olympicro 43GilSongpagu, 05505, Seoul, Republic of Korea
| | - Gaeun Kee
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Seohyun Park
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Soyoung Ko
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - HyoJe Jung
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Byeolhee Kim
- Department of Medical ScienceAsan Medical Institute of Convergence Science and TechnologyAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43Gil, Songpagu, 05505, Seoul, Republic of Korea
| | - Jae-Hyung Roh
- Department of Internal Medicine, Chungnam National University College of Medicine, Chungnam National University Sejong Hospital, 20, Bodeum 7-Ro, Sejong-Si, 30099, Sejong, Republic of Korea
| | - Tae Joon Jun
- Big Data Research Center, Asan Institute for Life Sciences, AsanMedicalCenter, 88, Olympicro 43GilSongpagu, 05505, Seoul, Republic of Korea
| | - Young-Hak Kim
- Division of CardiologyDepartment of Information MedicineAsan Medical Center, University of Ulsan College of Medicine, 88, Olympicro 43GilSongpagu, 05505, Seoul, Republic of Korea.
| |
Collapse
|
3
|
Zilker S, Weinzierl S, Kraus M, Zschech P, Matzner M. A machine learning framework for interpretable predictions in patient pathways: The case of predicting ICU admission for patients with symptoms of sepsis. Health Care Manag Sci 2024; 27:136-167. [PMID: 38771522 PMCID: PMC11258202 DOI: 10.1007/s10729-024-09673-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 04/13/2024] [Indexed: 05/22/2024]
Abstract
Proactive analysis of patient pathways helps healthcare providers anticipate treatment-related risks, identify outcomes, and allocate resources. Machine learning (ML) can leverage a patient's complete health history to make informed decisions about future events. However, previous work has mostly relied on so-called black-box models, which are unintelligible to humans, making it difficult for clinicians to apply such models. Our work introduces PatWay-Net, an ML framework designed for interpretable predictions of admission to the intensive care unit (ICU) for patients with symptoms of sepsis. We propose a novel type of recurrent neural network and combine it with multi-layer perceptrons to process the patient pathways and produce predictive yet interpretable results. We demonstrate its utility through a comprehensive dashboard that visualizes patient health trajectories, predictive outcomes, and associated risks. Our evaluation includes both predictive performance - where PatWay-Net outperforms standard models such as decision trees, random forests, and gradient-boosted decision trees - and clinical utility, validated through structured interviews with clinicians. By providing improved predictive accuracy along with interpretable and actionable insights, PatWay-Net serves as a valuable tool for healthcare decision support in the critical case of patients with symptoms of sepsis.
Collapse
Affiliation(s)
- Sandra Zilker
- Technische Hochschule Nürnberg Georg Simon Ohm, Professorship for Business Analytics, Hohfederstraße 40, 90489, Nuremberg, Germany.
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany.
| | - Sven Weinzierl
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany
| | - Mathias Kraus
- University of Regensburg, Chair for Explainable AI in Business Value Creation, Bajuwarenstraße 4, 93053, Regensburg, Germany
| | - Patrick Zschech
- Leipzig University, Professorship for Intelligent Information Systems and Processes, Grimmaische Straße 12, 04109, Leipzig, Germany
| | - Martin Matzner
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Chair of Digital Industrial Service Systems, Fürther Straße 248, 90429, Nuremberg, Germany
| |
Collapse
|
4
|
Zhang S, Yu J, Xu X, Yin C, Lu Y, Yao B, Tory M, Padilla LM, Caterino J, Zhang P, Wang D. Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis. PROCEEDINGS OF THE SIGCHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS. CHI CONFERENCE 2024; 2024:445. [PMID: 38835626 PMCID: PMC11149368 DOI: 10.1145/3613904.3642343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2024]
Abstract
Today's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that SepsisLab enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.
Collapse
Affiliation(s)
- Shao Zhang
- Northeastern University, Boston, Massachusetts, United States
| | - Jianing Yu
- Northeastern University, Boston, Massachusetts, United States
| | - Xuhai Xu
- Massachusetts Institute of Technology, Cambridge, Massachusetts,
United States
| | - Changchang Yin
- The Ohio State University, Columbus, Ohio, United States
| | - Yuxuan Lu
- Northeastern University, Boston, Massachusetts, United States
| | - Bingsheng Yao
- Rensselaer Polytechnic Institute, Troy, New York, United
States
| | - Melanie Tory
- Northeastern University, Portland, Maine, United States
| | - Lace M. Padilla
- Northeastern University, Boston, Massachusetts, United States
| | - Jeffrey Caterino
- The Ohio State University Wexner Medical Center, Columbus, Ohio,
United States
| | - Ping Zhang
- The Ohio State University, Columbus, Ohio, United States
| | - Dakuo Wang
- Northeastern University, Boston, Massachusetts, United
States
| |
Collapse
|
5
|
Chae S, Street WN, Ramaraju N, Gilbertson-White S. Prediction of Cancer Symptom Trajectory Using Longitudinal Electronic Health Record Data and Long Short-Term Memory Neural Network. JCO Clin Cancer Inform 2024; 8:e2300039. [PMID: 38471054 PMCID: PMC10948138 DOI: 10.1200/cci.23.00039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 12/08/2023] [Accepted: 01/26/2024] [Indexed: 03/14/2024] Open
Abstract
PURPOSE Ability to predict symptom severity and progression across treatment trajectories would allow clinicians to provide timely intervention and treatment planning. However, such predictions are difficult because of sparse and inconsistent assessment, and simplistic measures such as the last observed symptom severity are often used. The purpose of this study is to develop a model for predicting future cancer symptom experiences on the basis of past symptom experiences. PATIENTS AND METHODS We performed a retrospective, longitudinal analysis using records of patients with cancer (n = 208) hospitalized between 2008 and 2014. A long short-term memory (LSTM)-based recurrent neural network, a linear regression, and random forest models were trained on previous symptoms experienced and used to predict future symptom trajectories. RESULTS We found that at least one of three tested models (LSTM, linear regression, and random forest) outperform predictions based solely on the previous clinical observation. LSTM models significantly outperformed linear regression and random forest models in predicting nausea (P < .1) and psychosocial status (P < .01). Linear regression outperformed all models when predicting oral health (P < .01), while random forest outperformed all models when predicting mobility (P < .01) and nutrition (P < .01). CONCLUSION We can successfully predict patients' symptom trajectories with a prediction model, built with sparse assessment data, using routinely collected nursing documentation. The results of this project can be applied to better individualize symptom management to support cancer patients' quality of life.
Collapse
Affiliation(s)
- Sena Chae
- The University of Iowa College of Nursing, Iowa City, IA
| | - W. Nick Street
- The University of Iowa Tippie College of Business, Iowa City, IA
| | - Naveenkumar Ramaraju
- University of Illinois Urbana-Champaign, Gies College of Business, Champaign, IL
| | | |
Collapse
|
6
|
Piliuk K, Tomforde S. Artificial intelligence in emergency medicine. A systematic literature review. Int J Med Inform 2023; 180:105274. [PMID: 37944275 DOI: 10.1016/j.ijmedinf.2023.105274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 10/21/2023] [Accepted: 10/26/2023] [Indexed: 11/12/2023]
Abstract
Motivation and objective: Emergency medicine is becoming a popular application area for artificial intelligence methods but remains less investigated than other healthcare branches. The need for time-sensitive decision-making on the basis of high data volumes makes the use of quantitative technologies inevitable. However, the specifics of healthcare regulations impose strict requirements for such applications. Published contributions cover separate parts of emergency medicine and use disparate data and algorithms. This study aims to systematize the relevant contributions, investigate the main obstacles to artificial intelligence applications in emergency medicine, and propose directions for further studies. METHODS The contributions selection process was conducted with systematic electronic databases querying and filtering with respect to established exclusion criteria. Among the 380 papers gathered from IEEE Xplore, ACM Digital Library, Springer Library, ScienceDirect, and Nature databases 116 were considered to be a part of the survey. The main features of the selected papers are the focus on emergency medicine and the use of machine learning or deep learning algorithms. FINDINGS AND DISCUSSION The selected papers were classified into two branches: diagnostics-specific and triage-specific. The former ones are focused on either diagnosis prediction or decision support. The latter covers such applications as mortality, outcome, admission prediction, condition severity estimation, and urgent care prediction. The observed contributions are highly specialized within a single disease or medical operation and often use privately collected retrospective data, making them incomparable. These and other issues can be addressed by creating an end-to-end solution based on human-machine interaction. CONCLUSION Artificial intelligence applications are finding their place in emergency medicine, while most of the corresponding studies remain isolated and lack higher generalization and more sophisticated methodology, which can be a matter of forthcoming improvements.
Collapse
Affiliation(s)
| | - Sven Tomforde
- Christian-Albrechts-Universität zu Kiel, 24118 Kiel, Germany
| |
Collapse
|
7
|
Sheikhalishahi S, Bhattacharyya A, Celi LA, Osmani V. An interpretable deep learning model for time-series electronic health records: Case study of delirium prediction in critical care. Artif Intell Med 2023; 144:102659. [PMID: 37783541 DOI: 10.1016/j.artmed.2023.102659] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 07/19/2023] [Accepted: 09/04/2023] [Indexed: 10/04/2023]
Abstract
Deep Learning (DL) models have received increasing attention in the clinical setting, particularly in intensive care units (ICU). In this context, the interpretability of the outcomes estimated by the DL models is an essential step towards increasing adoption of DL models in clinical practice. To address this challenge, we propose an ante-hoc, interpretable neural network model. Our proposed model, named double self-attention architecture (DSA), uses two attention-based mechanisms, including self-attention and effective attention. It can capture the importance of input variables in general, as well as changes in importance along the time dimension for the outcome of interest. We evaluated our model using two real-world clinical datasets covering 22840 patients in predicting onset of delirium 12 h and 48 h in advance. Additionally, we compare the descriptive performance of our model with three post-hoc interpretable algorithms as well as with the opinion of clinicians based on the published literature and clinical experience. We find that our model covers the majority of the top-10 variables ranked by the other three post-hoc interpretable algorithms as well as the clinical opinion, with the advantage of taking into account both, the dependencies among variables as well as dependencies between varying time-steps. Finally, our results show that our model can improve descriptive performance without sacrificing predictive performance.
Collapse
Affiliation(s)
| | | | - Leo Anthony Celi
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA; Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA; Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Venet Osmani
- Fondazione Bruno Kessler Research Institute, Trento, Italy; Information School, University of Sheffield, UK.
| |
Collapse
|
8
|
Lee JM, Hauskrecht M. Personalized event prediction for Electronic Health Records. Artif Intell Med 2023; 143:102620. [PMID: 37673563 PMCID: PMC10503594 DOI: 10.1016/j.artmed.2023.102620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 03/01/2023] [Accepted: 04/24/2023] [Indexed: 09/08/2023]
Abstract
Clinical event sequences consist of hundreds of clinical events that represent records of patient care in time. Developing accurate predictive models of such sequences is of a great importance for supporting a variety of models for interpreting/classifying the current patient condition, or predicting adverse clinical events and outcomes, all aimed to improve patient care. One important challenge of learning predictive models of clinical sequences is their patient-specific variability. Based on underlying clinical conditions, each patient's sequence may consist of different sets of clinical events (observations, lab results, medications, procedures). Hence, simple population-wide models learned from event sequences for many different patients may not accurately predict patient-specific dynamics of event sequences and their differences. To address the problem, we propose and investigate multiple new event sequence prediction models and methods that let us better adjust the prediction for individual patients and their specific conditions. The methods developed in this work pursue refinement of population-wide models to subpopulations, self-adaptation, and a meta-level model switching that is able to adaptively select the model with the best chance to support the immediate prediction. We analyze and test the performance of these models on clinical event sequences of patients in MIMIC-III database.
Collapse
Affiliation(s)
- Jeong Min Lee
- Department of Computer Science, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Milos Hauskrecht
- Department of Computer Science, University of Pittsburgh, Pittsburgh, PA, USA.
| |
Collapse
|
9
|
Pungitore S, Subbian V. Assessment of Prediction Tasks and Time Window Selection in Temporal Modeling of Electronic Health Record Data: a Systematic Review. JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2023; 7:313-331. [PMID: 37637723 PMCID: PMC10449760 DOI: 10.1007/s41666-023-00143-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 04/12/2023] [Accepted: 07/28/2023] [Indexed: 08/29/2023]
Abstract
Temporal electronic health record (EHR) data are often preferred for clinical prediction tasks because they offer more complete representations of a patient's pathophysiology than static data. A challenge when working with temporal EHR data is problem formulation, which includes defining the time windows of interest and the prediction task. Our objective was to conduct a systematic review that assessed the definition and reporting of concepts relevant to temporal clinical prediction tasks. We searched PubMed® and IEEE Xplore® databases for studies from January 1, 2010 applying machine learning models to EHR data for patient outcome prediction. Publications applying time-series methods were selected for further review. We identified 92 studies and summarized them by clinical context and definition and reporting of the prediction problem. For the time windows of interest, 12 studies did not discuss window lengths, 57 used a single set of window lengths, and 23 evaluated the relationship between window length and model performance. We also found that 72 studies had appropriate reporting of the prediction task. However, evaluation of prediction problem formulation for temporal EHR data was complicated by heterogeneity in assessing and reporting of these concepts. Even among studies modeling similar clinical outcomes, there were variations in terminology used to describe the prediction problem, rationale for window lengths, and determination of the outcome of interest. As temporal modeling using EHR data expands, minimal reporting standards should include time-series specific concerns to promote rigor and reproducibility in future studies and facilitate model implementation in clinical settings. Supplementary Information The online version contains supplementary material available at 10.1007/s41666-023-00143-4.
Collapse
Affiliation(s)
- Sarah Pungitore
- Program in Applied Mathematics, Department of Mathematics, 617 N Santa Rita Ave, Tucson, AZ 85721 USA
| | - Vignesh Subbian
- Department of Biomedical Engineering, The University of Arizona, Tucson, AZ 85721-0020 USA
- Department of Systems and Industrial Engineering, The University of Arizona, Tucson, AZ 85721-0020 USA
| |
Collapse
|
10
|
Islam KR, Prithula J, Kumar J, Tan TL, Reaz MBI, Sumon MSI, Chowdhury MEH. Machine Learning-Based Early Prediction of Sepsis Using Electronic Health Records: A Systematic Review. J Clin Med 2023; 12:5658. [PMID: 37685724 PMCID: PMC10488449 DOI: 10.3390/jcm12175658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/13/2023] [Accepted: 08/23/2023] [Indexed: 09/10/2023] Open
Abstract
BACKGROUND Sepsis, a life-threatening infection-induced inflammatory condition, has significant global health impacts. Timely detection is crucial for improving patient outcomes as sepsis can rapidly progress to severe forms. The application of machine learning (ML) and deep learning (DL) to predict sepsis using electronic health records (EHRs) has gained considerable attention for timely intervention. METHODS PubMed, IEEE Xplore, Google Scholar, and Scopus were searched for relevant studies. All studies that used ML/DL to detect or early-predict the onset of sepsis in the adult population using EHRs were considered. Data were extracted and analyzed from all studies that met the criteria and were also evaluated for their quality. RESULTS This systematic review examined 1942 articles, selecting 42 studies while adhering to strict criteria. The chosen studies were predominantly retrospective (n = 38) and spanned diverse geographic settings, with a focus on the United States. Different datasets, sepsis definitions, and prevalence rates were employed, necessitating data augmentation. Heterogeneous parameter utilization, diverse model distribution, and varying quality assessments were observed. Longitudinal data enabled early sepsis prediction, and quality criteria fulfillment varied, with inconsistent funding-article quality correlation. CONCLUSIONS This systematic review underscores the significance of ML/DL methods for sepsis detection and early prediction through EHR data.
Collapse
Affiliation(s)
- Khandaker Reajul Islam
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Johayra Prithula
- Department of Electrical and Electronics Engineering, University of Dhaka, Dhaka 1000, Bangladesh
| | - Jaya Kumar
- Department of Physiology, Faculty of Medicine, University Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Toh Leong Tan
- Department of Emergency Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, Kuala Lumpur 56000, Malaysia
| | - Mamun Bin Ibne Reaz
- Department of Electrical and Electronic Engineering, Independent University, Bangladesh Bashundhara, Dhaka 1229, Bangladesh
| | - Md. Shaheenur Islam Sumon
- Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh
| | | |
Collapse
|
11
|
Kaya U, Yılmaz A, Aşar S. Sepsis Prediction by Using a Hybrid Metaheuristic Algorithm: A Novel Approach for Optimizing Deep Neural Networks. Diagnostics (Basel) 2023; 13:2023. [PMID: 37370918 DOI: 10.3390/diagnostics13122023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 05/11/2023] [Accepted: 05/21/2023] [Indexed: 06/29/2023] Open
Abstract
The early diagnosis of sepsis reduces the risk of the patient's death. Gradient-based algorithms are applied to the neural network models used in the estimation of sepsis in the literature. However, these algorithms become stuck at the local minimum in solution space. In recent years, swarm intelligence and an evolutionary approach have shown proper results. In this study, a novel hybrid metaheuristic algorithm was proposed for optimization with regard to the weights of the deep neural network and applied for the early diagnosis of sepsis. The proposed algorithm aims to reach the global minimum with a local search strategy capable of exploring and exploiting particles in Particle Swarm Optimization (PSO) and using the mental search operator of the Human Mental Search algorithm (HMS). The benchmark functions utilized to compare the performance of HMS, PSO, and HMS-PSO revealed that the proposed approach is more reliable, durable, and adjustable than other applied algorithms. HMS-PSO is integrated with a deep neural network (HMS-PSO-DNN). The study focused on predicting sepsis with HMS-PSO-DNN, utilizing a dataset of 640 patients aged 18 to 60. The HMS-PSO-DNN model gave a better mean squared error (MSE) result than other algorithms in terms of accuracy, robustness, and performance. We obtained the MSE value of 0.22 with 30 independent runs.
Collapse
Affiliation(s)
- Umut Kaya
- Faculty of Engineering and Architecture, Department of Software Engineering, İstanbul Beykent University, Istanbul 34398, Turkey
| | - Atınç Yılmaz
- Faculty of Engineering and Architecture, Department of Computer Engineering, İstanbul Beykent University, Istanbul 34398, Turkey
| | - Sinan Aşar
- Intensive Care Unit, Bakirkoy Dr. Sadi Konuk Training and Research Hospital, Istanbul 34147, Turkey
| |
Collapse
|
12
|
Prediction of Acid-Base and Potassium Imbalances in Intensive Care Patients Using Machine Learning Techniques. Diagnostics (Basel) 2023; 13:diagnostics13061171. [PMID: 36980479 PMCID: PMC10047445 DOI: 10.3390/diagnostics13061171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 03/16/2023] [Accepted: 03/17/2023] [Indexed: 03/22/2023] Open
Abstract
Acid–base disorders occur when the body’s normal pH is out of balance. They can be caused by problems with kidney or respiratory function or by an excess of acids or bases that the body cannot properly eliminate. Acid–base and potassium imbalances are mechanistically linked because acid–base imbalances can alter the transport of potassium. Both acid–base and potassium imbalances are common in critically ill patients. This study investigated machine learning models for predicting the occurrence of acid–base and potassium imbalances in intensive care patients. We used an institutional dataset of 1089 patients with 87 variables, including vital signs, general appearance, and laboratory results. Gradient boosting (GB) was able to predict nine clinical conditions related to acid–base and potassium imbalances: mortality (AUROC = 0.9822), hypocapnia (AUROC = 0.7524), hypercapnia (AUROC = 0.8228), hypokalemia (AUROC = 0.9191), hyperkalemia (AUROC = 0.9565), respiratory acidosis (AUROC = 0.8125), respiratory alkalosis (AUROC = 0.7685), metabolic acidosis (AUROC = 0.8682), and metabolic alkalosis (AUROC = 0.8284). Some predictions remained relatively robust even when the prediction window was increased. Additionally, the decision-making process was made more interpretable and transparent through the use of SHAP analysis. Overall, the results suggest that machine learning could be a useful tool to gain insight into the condition of intensive care patients and assist in the management of acid–base and potassium imbalances.
Collapse
|
13
|
Fischer A, Rietveld A, Teunissen P, Bakker P, Hoogendoorn M. End-to-end learning with interpretation on electrohysterography data to predict preterm birth. Comput Biol Med 2023; 158:106846. [PMID: 37019011 DOI: 10.1016/j.compbiomed.2023.106846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Revised: 03/03/2023] [Accepted: 03/30/2023] [Indexed: 04/03/2023]
Abstract
Prediction of preterm birth is a difficult task for clinicians. By examining an electrohysterogram, electrical activity of the uterus that can lead to preterm birth can be detected. Since signals associated with uterine activity are difficult to interpret for clinicians without a background in signal processing, machine learning may be a viable solution. We are the first to employ Deep Learning models, a long-short term memory and temporal convolutional network model, on electrohysterography data using the Term-Preterm Electrohysterogram database. We show that end-to-end learning achieves an AUC score of 0.58, which is comparable to machine learning models that use handcrafted features. Moreover, we evaluate the effect of adding clinical data to the model and conclude that adding the available clinical data to electrohysterography data does not result in a gain in performance. Also, we propose an interpretability framework for time series classification that is well-suited to use in case of limited data, as opposed to existing methods that require large amounts of data. Clinicians with extensive work experience as gynaecologist used our framework to provide insights on how to link our results to clinical practice and stress that in order to decrease the number of false positives, a dataset with patients at high risk of preterm birth should be collected. All code is made publicly available.
Collapse
|
14
|
Duan Y, Huo J, Chen M, Hou F, Yan G, Li S, Wang H. Early prediction of sepsis using double fusion of deep features and handcrafted features. APPL INTELL 2023; 53:1-17. [PMID: 36685641 PMCID: PMC9843111 DOI: 10.1007/s10489-022-04425-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/21/2022] [Indexed: 01/19/2023]
Abstract
Sepsis is a life-threatening medical condition that is characterized by the dysregulated immune system response to infections, having both high morbidity and mortality rates. Early prediction of sepsis is critical to the decrease of mortality. This paper presents a novel early warning model called Double Fusion Sepsis Predictor (DFSP) for sepsis onset. DFSP is a double fusion framework that combines the benefits of early and late fusion strategies. First, a hybrid deep learning model that combines both the convolutional and recurrent neural networks to extract deep features is proposed. Second, deep features and handcrafted features, such as clinical scores, are concatenated to build the joint feature representation (early fusion). Third, several tree-based models based on joint feature representation are developed to generate the risk scores of sepsis onset that are combined with an End-to-End neural network for final sepsis detection (late fusion). To evaluate DFSP, a retrospective study was conducted, which included patients admitted to the ICUs of a hospital in Shanghai China. The results demonstrate that the DFSP outperforms state-of-the-art approaches in early sepsis prediction.
Collapse
Affiliation(s)
- Yongrui Duan
- School of Economics & Management, Tongji University, Shanghai, China
| | - Jiazhen Huo
- School of Economics & Management, Tongji University, Shanghai, China
| | - Mingzhou Chen
- School of Economics & Management, Tongji University, Shanghai, China
| | - Fenggang Hou
- Department of Oncology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Guoliang Yan
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| | - Shufang Li
- Emergency Department, Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Haihui Wang
- Department of Geriatrics, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai, China
| |
Collapse
|
15
|
Chen Q, Li R, Lin C, Lai C, Chen D, Qu H, Huang Y, Lu W, Tang Y, Li L. Transferability and interpretability of the sepsis prediction models in the intensive care unit. BMC Med Inform Decis Mak 2022; 22:343. [PMID: 36581881 PMCID: PMC9798724 DOI: 10.1186/s12911-022-02090-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Accepted: 12/16/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND We aimed to develop an early warning system for real-time sepsis prediction in the ICU by machine learning methods, with tools for interpretative analysis of the predictions. In particular, we focus on the deployment of the system in a target medical center with small historical samples. METHODS Light Gradient Boosting Machine (LightGBM) and multilayer perceptron (MLP) were trained on Medical Information Mart for Intensive Care (MIMIC-III) dataset and then finetuned on the private Historical Database of local Ruijin Hospital (HDRJH) using transfer learning technique. The Shapley Additive Explanations (SHAP) analysis was employed to characterize the feature importance in the prediction inference. Ultimately, the performance of the sepsis prediction system was further evaluated in the real-world study in the ICU of the target Ruijin Hospital. RESULTS The datasets comprised 6891 patients from MIMIC-III, 453 from HDRJH, and 67 from Ruijin real-world data. The area under the receiver operating characteristic curves (AUCs) for LightGBM and MLP models derived from MIMIC-III were 0.98 - 0.98 and 0.95 - 0.96 respectively on MIMIC-III dataset, and, in comparison, 0.82 - 0.86 and 0.84 - 0.87 respectively on HDRJH, from 1 to 5 h preceding. After transfer learning and ensemble learning, the AUCs of the final ensemble model were enhanced to 0.94 - 0.94 on HDRJH and to 0.86 - 0.9 in the real-world study in the ICU of the target Ruijin Hospital. In addition, the SHAP analysis illustrated the importance of age, antibiotics, net balance, and ventilation for sepsis prediction, making the model interpretable. CONCLUSIONS Our machine learning model allows accurate real-time prediction of sepsis within 5-h preceding. Transfer learning can effectively improve the feasibility to deploy the prediction model in the target cohort, and ameliorate the model performance for external validation. SHAP analysis indicates that the role of antibiotic usage and fluid management needs further investigation. We argue that our system and methodology have the potential to improve ICU management by helping medical practitioners identify at-sepsis-risk patients and prepare for timely diagnosis and intervention. TRIAL REGISTRATION NCT05088850 (retrospectively registered).
Collapse
Affiliation(s)
- Qiyu Chen
- grid.8547.e0000 0001 0125 2443Department of Applied Mathematics, School of Mathematical Sciences, Fudan University, Shanghai, 200433 China
| | - Ranran Li
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - ChihChe Lin
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Chiming Lai
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Dechang Chen
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Hongping Qu
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Yaling Huang
- grid.495525.a0000 0004 0552 4356Shanghai Electric Group Co., Ltd., Central Academe, Shanghai, China
| | - Wenlian Lu
- grid.8547.e0000 0001 0125 2443Department of Applied Mathematics, School of Mathematical Sciences, Fudan University, Shanghai, 200433 China
| | - Yaoqing Tang
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| | - Lei Li
- grid.16821.3c0000 0004 0368 8293Department of Critical Care Medicine, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025 China
| |
Collapse
|
16
|
Singh YV, Singh P, Khan S, Singh RS. A Machine Learning Model for Early Prediction and Detection of Sepsis in Intensive Care Unit Patients. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:9263391. [PMID: 35378945 PMCID: PMC8976655 DOI: 10.1155/2022/9263391] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 03/01/2022] [Accepted: 03/09/2022] [Indexed: 12/17/2022]
Abstract
In today's scenario, sepsis is impacting millions of patients in the intensive care unit due to the fact that the mortality rate is increased exponentially and has become a major challenge in the field of healthcare. Such peoples require determinant care which increases the cost of the treatment by using a large number of resources because of the nonavailability of the resources. The treatment of sepsis is available in the early state, but treatment is not started at the right time, and then it converts to the advanced level of sepsis and increases the fatalities. Thus, an intensive analysis is required to detect and identify sepsis at the early stage. There are some models available that work based on the manual score and based on only the biomark features, but these are not fully automated. Some machine learning-based models are also available, which can reduce the mortality rate, but accuracy is not up to date. This paper proposes a machine learning model for early detecting and predicting sepsis in intensive care unit patients. Various models, random forest (RF), linear regression (LR), support vector machine (SVM), naive Bayes (NB), ensemble (of SVM, RF, NB, and LR), XGBoost, and proposed ensemble (of SVM, RF, NB, LR, and XGBoost), are simulated by using the collected data from intensive care unit patient's database that is based on the clinical laboratory values and vital signs. The performance of the models is evaluated by considering the same datasets. The balanced accuracy of RF, LR, SVM, NB, ensemble (of SVM, RF, NB, and LR), XGBoost, and proposed ensemble (of SVM, RF, NB, LR, and XGBoost) is 0.90, 0.73, 0.93, 0.74, 0.94, 0.95, and 0.96, respectively. It is also evident from the experimental results that the proposed ensemble model performs well as compared to the other models.
Collapse
Affiliation(s)
- Yash Veer Singh
- Department of Information Technology, ABES Engineering College, Ghaziabad (UP) 201009, India
| | - Pushpendra Singh
- Department of Information Technology, Raj Kumar Goel Institute of Technology, Ghaziabad (UP) 101003, India
| | - Shadab Khan
- Department of Computer Science & Engineering, Sunder Deep Engineering College, Ghaziabad (UP) 201002, India
| | - Ram Sewak Singh
- Department of Electronics and Communication,School of Electrical Engineering and Computing, Adama Science and Technology University, Adama, Ethiopia
| |
Collapse
|
17
|
Sepsis prediction in intensive care unit based on genetic feature optimization and stacked deep ensemble learning. Neural Comput Appl 2022. [DOI: 10.1007/s00521-021-06631-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
18
|
Allam A, Feuerriegel S, Rebhan M, Krauthammer M. Analyzing Patient Trajectories With Artificial Intelligence. J Med Internet Res 2021; 23:e29812. [PMID: 34870606 PMCID: PMC8686456 DOI: 10.2196/29812] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Revised: 07/26/2021] [Accepted: 10/29/2021] [Indexed: 01/16/2023] Open
Abstract
In digital medicine, patient data typically record health events over time (eg, through electronic health records, wearables, or other sensing technologies) and thus form unique patient trajectories. Patient trajectories are highly predictive of the future course of diseases and therefore facilitate effective care. However, digital medicine often uses only limited patient data, consisting of health events from only a single or small number of time points while ignoring additional information encoded in patient trajectories. To analyze such rich longitudinal data, new artificial intelligence (AI) solutions are needed. In this paper, we provide an overview of the recent efforts to develop trajectory-aware AI solutions and provide suggestions for future directions. Specifically, we examine the implications for developing disease models from patient trajectories along the typical workflow in AI: problem definition, data processing, modeling, evaluation, and interpretation. We conclude with a discussion of how such AI solutions will allow the field to build robust models for personalized risk scoring, subtyping, and disease pathway discovery.
Collapse
Affiliation(s)
- Ahmed Allam
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
- Biomedical Informatics, University Hospital of Zurich, Zurich, Switzerland
| | - Stefan Feuerriegel
- Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland
- ETH Artificial Intelligence Center, ETH Zurich, Zurich, Switzerland
- Ludwig Maximilian University of Munich, Munich, Germany
| | - Michael Rebhan
- Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland
| | - Michael Krauthammer
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
- Biomedical Informatics, University Hospital of Zurich, Zurich, Switzerland
- Yale Center for Medical Informatics, Yale University School of Medicine, New Haven, CT, United States
| |
Collapse
|
19
|
Comoretto RI, Azzolina D, Amigoni A, Stoppa G, Todino F, Wolfler A, Gregori D. Predicting Hemodynamic Failure Development in PICU Using Machine Learning Techniques. Diagnostics (Basel) 2021; 11:diagnostics11071299. [PMID: 34359385 PMCID: PMC8303657 DOI: 10.3390/diagnostics11071299] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Revised: 07/12/2021] [Accepted: 07/16/2021] [Indexed: 11/16/2022] Open
Abstract
The present work aims to identify the predictors of hemodynamic failure (HF) developed during pediatric intensive care unit (PICU) stay testing a set of machine learning techniques (MLTs), comparing their ability to predict the outcome of interest. The study involved patients admitted to PICUs between 2010 and 2020. Data were extracted from the Italian Network of Pediatric Intensive Care Units (TIPNet) registry. The algorithms considered were generalized linear model (GLM), recursive partition tree (RPART), random forest (RF), neural networks models, and extreme gradient boosting (XGB). Since the outcome is rare, upsampling and downsampling algorithms have been applied for imbalance control. For each approach, the main performance measures were reported. Among an overall sample of 29,494 subjects, only 399 developed HF during the PICU stay. The median age was about two years, and the male gender was the most prevalent. The XGB algorithm outperformed other MLTs in predicting HF development, with a median ROC measure of 0.780 (IQR 0.770-0.793). PIM 3, age, and base excess were found to be the strongest predictors of outcome. The present work provides insights for the prediction of HF development during PICU stay using machine-learning algorithms.
Collapse
Affiliation(s)
- Rosanna I. Comoretto
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, 35131 Padova, Italy; (R.I.C.); (D.A.); (G.S.); (F.T.)
| | - Danila Azzolina
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, 35131 Padova, Italy; (R.I.C.); (D.A.); (G.S.); (F.T.)
- Department of Medical Sciences, University of Ferrara, 44100 Ferrara, Italy
| | - Angela Amigoni
- Pediatric Intensive Care Unit, Department of Women’s and Children’s Health, University Hospital of Padua, Via Giustiniani 2, 35128 Padova, Italy;
| | - Giorgia Stoppa
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, 35131 Padova, Italy; (R.I.C.); (D.A.); (G.S.); (F.T.)
| | - Federica Todino
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, 35131 Padova, Italy; (R.I.C.); (D.A.); (G.S.); (F.T.)
| | - Andrea Wolfler
- Department of Anaesthesia, Gaslini Hospital, 16147 Genova, Italy;
| | - Dario Gregori
- Unit of Biostatistics, Epidemiology and Public Health, Department of Cardiac, Thoracic, Vascular Sciences and Public Health, University of Padova, 35131 Padova, Italy; (R.I.C.); (D.A.); (G.S.); (F.T.)
- Correspondence: ; Tel.: +39-049-8275-384; Fax: +39-02-700-445-089
| | | |
Collapse
|