1
|
Nainamalai V, Qair HA, Pelanis E, Jenssen HB, Fretland ÅA, Edwin B, Elle OJ, Balasingham I. Automated algorithm for medical data structuring, and segmentation using artificial intelligence within secured environment for dataset creation. Eur J Radiol Open 2024; 13:100582. [PMID: 39041057 PMCID: PMC11260947 DOI: 10.1016/j.ejro.2024.100582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 06/02/2024] [Accepted: 06/17/2024] [Indexed: 07/24/2024] Open
Abstract
Objective Routinely collected electronic health records using artificial intelligence (AI)-based systems bring out enormous benefits for patients, healthcare centers, and its industries. Artificial intelligence models can be used to structure a wide variety of unstructured data. Methods We present a semi-automatic workflow for medical dataset management, including data structuring, research extraction, AI-ground truth creation, and updates. The algorithm creates directories based on keywords in new file names. Results Our work focuses on organizing computed tomography (CT), magnetic resonance (MR) images, patient clinical data, and segmented annotations. In addition, an AI model is used to generate different initial labels that can be edited manually to create ground truth labels. The manually verified ground truth labels are later included in the structured dataset using an automated algorithm for future research. Conclusion This is a workflow with an AI model trained on local hospital medical data with output based/adapted to the users and their preferences. The automated algorithms and AI model could be implemented inside a secondary secure environment in the hospital to produce inferences.
Collapse
Affiliation(s)
| | - Hemin Ali Qair
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
| | - Egidijus Pelanis
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
- Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Håvard Bjørke Jenssen
- Department of Radiology and Nuclear Medicine, Oslo University Hospital, Oslo, Norway
| | - Åsmund Avdem Fretland
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
- Department of Hepato-Pancreatic-Biliary surgery, Oslo University Hospital, Oslo, Norway
| | - Bjørn Edwin
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
- Department of Hepato-Pancreatic-Biliary surgery, Oslo University Hospital, Oslo, Norway
| | - Ole Jakob Elle
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
- Department of Informatics, University of Oslo, Oslo, Norway
| | - Ilangko Balasingham
- The Intervention Centre, Rikshospitalet, Oslo University Hospital, Oslo, Norway
- Department of electronic systems (IES), Norwegian University of Science and Technology, Trondheim, Norway
| |
Collapse
|
2
|
Bandyopadhyay A, Albashayreh A, Zeinali N, Fan W, Gilbertson-White S. Using real-world electronic health record data to predict the development of 12 cancer-related symptoms in the context of multimorbidity. JAMIA Open 2024; 7:ooae082. [PMID: 39282082 PMCID: PMC11397936 DOI: 10.1093/jamiaopen/ooae082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2024] [Revised: 08/09/2024] [Accepted: 09/05/2024] [Indexed: 09/18/2024] Open
Abstract
Objective This study uses electronic health record (EHR) data to predict 12 common cancer symptoms, assessing the efficacy of machine learning (ML) models in identifying symptom influencers. Materials and Methods We analyzed EHR data of 8156 adults diagnosed with cancer who underwent cancer treatment from 2017 to 2020. Structured and unstructured EHR data were sourced from the Enterprise Data Warehouse for Research at the University of Iowa Hospital and Clinics. Several predictive models, including logistic regression, random forest (RF), and XGBoost, were employed to forecast symptom development. The performances of the models were evaluated by F1-score and area under the curve (AUC) on the testing set. The SHapley Additive exPlanations framework was used to interpret these models and identify the predictive risk factors associated with fatigue as an exemplar. Results The RF model exhibited superior performance with a macro average AUC of 0.755 and an F1-score of 0.729 in predicting a range of cancer-related symptoms. For instance, the RF model achieved an AUC of 0.954 and an F1-score of 0.914 for pain prediction. Key predictive factors identified included clinical history, cancer characteristics, treatment modalities, and patient demographics depending on the symptom. For example, the odds ratio (OR) for fatigue was significantly influenced by allergy (OR = 2.3, 95% CI: 1.8-2.9) and colitis (OR = 1.9, 95% CI: 1.5-2.4). Discussion Our research emphasizes the critical integration of multimorbidity and patient characteristics in modeling cancer symptoms, revealing the considerable influence of chronic conditions beyond cancer itself. Conclusion We highlight the potential of ML for predicting cancer symptoms, suggesting a pathway for integrating such models into clinical systems to enhance personalized care and symptom management.
Collapse
Affiliation(s)
- Anindita Bandyopadhyay
- Department of Business Analytics, University of Iowa, Iowa City, IA 52242, United States
| | - Alaa Albashayreh
- College of Nursing, University of Iowa, Iowa City, IA 52242, United States
| | - Nahid Zeinali
- Department of Informatics, University of Iowa, Iowa City, IA 52242, United States
| | - Weiguo Fan
- Department of Business Analytics, University of Iowa, Iowa City, IA 52242, United States
| | | |
Collapse
|
3
|
Li Y, Liang Z, Li Y, Cao Y, Zhang H, Dong B. Machine learning value in the diagnosis of vertebral fractures: A systematic review and meta-analysis. Eur J Radiol 2024; 181:111714. [PMID: 39241305 DOI: 10.1016/j.ejrad.2024.111714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2024] [Revised: 07/28/2024] [Accepted: 08/30/2024] [Indexed: 09/09/2024]
Abstract
PURPOSE To evaluate the diagnostic accuracy of machine learning (ML) in detecting vertebral fractures, considering varying fracture classifications, patient populations, and imaging approaches. METHOD A systematic review and meta-analysis were conducted by searching PubMed, Embase, Cochrane Library, and Web of Science up to December 31, 2023, for studies using ML for vertebral fracture diagnosis. Bias risk was assessed using QUADAS-2. A bivariate mixed-effects model was used for the meta-analysis. Meta-analyses were performed according to five task types (vertebral fractures, osteoporotic vertebral fractures, differentiation of benign and malignant vertebral fractures, differentiation of acute and chronic vertebral fractures, and prediction of vertebral fractures). Subgroup analyses were conducted by different ML models (including ML and DL) and modeling methods (including CT, X-ray, MRI, and clinical features). RESULTS Eighty-one studies were included. ML demonstrated a diagnostic sensitivity of 0.91 and specificity of 0.95 for vertebral fractures. Subgroup analysis showed that DL (SROC 0.98) and CT (SROC 0.98) performed best overall. For osteoporotic fractures, ML showed a sensitivity of 0.93 and specificity of 0.96, with DL (SROC 0.99) and X-ray (SROC 0.99) performing better. For differentiating benign from malignant fractures, ML achieved a sensitivity of 0.92 and specificity of 0.93, with DL (SROC 0.96) and MRI (SROC 0.97) performing best. For differentiating acute from chronic vertebral fractures, ML showed a sensitivity of 0.92 and specificity of 0.93, with ML (SROC 0.96) and CT (SROC 0.97) performing best. For predicting vertebral fractures, ML had a sensitivity of 0.76 and specificity of 0.87, with ML (SROC 0.80) and clinical features (SROC 0.86) performing better. CONCLUSIONS ML, especially DL models applied to CT, MRI, and X-ray, shows high diagnostic accuracy for vertebral fractures. ML also effectively predicts osteoporotic vertebral fractures, aiding in tailored prevention strategies. Further research and validation are required to confirm ML's clinical efficacy.
Collapse
Affiliation(s)
- Yue Li
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China
| | - Zhuang Liang
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China
| | - Yingchun Li
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China
| | - Yang Cao
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China
| | - Hui Zhang
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China
| | - Bo Dong
- Pain Ward of Rehabilitation Department, Honghui Hospital, Xi'an Jiaotong University, Xi'an, Shaanxi Province 710054, PR China.
| |
Collapse
|
4
|
Molaei S, Bousejin NG, Ghosheh GO, Thakur A, Chauhan VK, Zhu T, Clifton DA. CliqueFluxNet: Unveiling EHR Insights with Stochastic Edge Fluxing and Maximal Clique Utilisation Using Graph Neural Networks. JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2024; 8:555-575. [PMID: 39131103 PMCID: PMC11310186 DOI: 10.1007/s41666-024-00169-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 05/16/2024] [Accepted: 06/27/2024] [Indexed: 08/13/2024]
Abstract
Electronic Health Records (EHRs) play a crucial role in shaping predictive are models, yet they encounter challenges such as significant data gaps and class imbalances. Traditional Graph Neural Network (GNN) approaches have limitations in fully leveraging neighbourhood data or demanding intensive computational requirements for regularisation. To address this challenge, we introduce CliqueFluxNet, a novel framework that innovatively constructs a patient similarity graph to maximise cliques, thereby highlighting strong inter-patient connections. At the heart of CliqueFluxNet lies its stochastic edge fluxing strategy - a dynamic process involving random edge addition and removal during training. This strategy aims to enhance the model's generalisability and mitigate overfitting. Our empirical analysis, conducted on MIMIC-III and eICU datasets, focuses on the tasks of mortality and readmission prediction. It demonstrates significant progress in representation learning, particularly in scenarios with limited data availability. Qualitative assessments further underscore CliqueFluxNet's effectiveness in extracting meaningful EHR representations, solidifying its potential for advancing GNN applications in healthcare analytics.
Collapse
Affiliation(s)
- Soheila Molaei
- Department of Engineering Science, University of Oxford, Oxford, OX1 3AZ UK
| | | | - Ghadeer O. Ghosheh
- Department of Engineering Science, University of Oxford, Oxford, OX1 3AZ UK
| | - Anshul Thakur
- Department of Engineering Science, University of Oxford, Oxford, OX1 3AZ UK
| | | | - Tingting Zhu
- Department of Engineering Science, University of Oxford, Oxford, OX1 3AZ UK
| | - David A. Clifton
- Department of Engineering Science, University of Oxford, Oxford, OX1 3AZ UK
- Oxford-Suzhou Centre for Advanced Research (OSCAR), Suzhou, 215123 China
| |
Collapse
|
5
|
Moharrami M, Azimian Zavareh P, Watson E, Singhal S, Johnson AEW, Hosni A, Quinonez C, Glogauer M. Prognosing post-treatment outcomes of head and neck cancer using structured data and machine learning: A systematic review. PLoS One 2024; 19:e0307531. [PMID: 39046953 PMCID: PMC11268644 DOI: 10.1371/journal.pone.0307531] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Accepted: 07/07/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND This systematic review aimed to evaluate the performance of machine learning (ML) models in predicting post-treatment survival and disease progression outcomes, including recurrence and metastasis, in head and neck cancer (HNC) using clinicopathological structured data. METHODS A systematic search was conducted across the Medline, Scopus, Embase, Web of Science, and Google Scholar databases. The methodological characteristics and performance metrics of studies that developed and validated ML models were assessed. The risk of bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). RESULTS Out of 5,560 unique records, 34 articles were included. For survival outcome, the ML model outperformed the Cox proportional hazards model in time-to-event analyses for HNC, with a concordance index of 0.70-0.79 vs. 0.66-0.76, and for all sub-sites including oral cavity (0.73-0.89 vs. 0.69-0.77) and larynx (0.71-0.85 vs. 0.57-0.74). In binary classification analysis, the area under the receiver operating characteristics (AUROC) of ML models ranged from 0.75-0.97, with an F1-score of 0.65-0.89 for HNC; AUROC of 0.61-0.91 and F1-score of 0.58-0.86 for the oral cavity; and AUROC of 0.76-0.97 and F1-score of 0.63-0.92 for the larynx. Disease-specific survival outcomes showed higher performance than overall survival outcomes, but the performance of ML models did not differ between three- and five-year follow-up durations. For disease progression outcomes, no time-to-event metrics were reported for ML models. For binary classification of the oral cavity, the only evaluated subsite, the AUROC ranged from 0.67 to 0.97, with F1-scores between 0.53 and 0.89. CONCLUSIONS ML models have demonstrated considerable potential in predicting post-treatment survival and disease progression, consistently outperforming traditional linear models and their derived nomograms. Future research should incorporate more comprehensive treatment features, emphasize disease progression outcomes, and establish model generalizability through external validations and the use of multicenter datasets.
Collapse
Affiliation(s)
- Mohammad Moharrami
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Geneva, Switzerland
| | - Parnia Azimian Zavareh
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Geneva, Switzerland
| | - Erin Watson
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
| | - Sonica Singhal
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Chronic Disease and Injury Prevention Department, Health Promotion, Public Health Ontario, Toronto, Canada
| | - Alistair E. W. Johnson
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Ali Hosni
- Radiation Oncology, Princess Margaret Cancer Center, University of Toronto, Toronto, Canada
| | - Carlos Quinonez
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Schulich School of Medicine & Dentistry, Western University, London, Canada
| | - Michael Glogauer
- Faculty of Dentistry, University of Toronto, Toronto, Canada
- Department of Dental Oncology, Princess Margaret Cancer Centre, Toronto, Canada
- Department of Dentistry, Centre for Advanced Dental Research and Care, Mount Sinai Hospital, Toronto, Canada
| |
Collapse
|
6
|
Garrido NJ, González-Martínez F, Losada S, Plaza A, del Olmo E, Mateo J. Innovation through Artificial Intelligence in Triage Systems for Resource Optimization in Future Pandemics. Biomimetics (Basel) 2024; 9:440. [PMID: 39056881 PMCID: PMC11274710 DOI: 10.3390/biomimetics9070440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Revised: 07/12/2024] [Accepted: 07/16/2024] [Indexed: 07/28/2024] Open
Abstract
Artificial intelligence (AI) systems are already being used in various healthcare areas. Similarly, they can offer many advantages in hospital emergency services. The objective of this work is to demonstrate that through the novel use of AI, a trained system can be developed to detect patients at potential risk of infection in a new pandemic more quickly than standardized triage systems. This identification would occur in the emergency department, thus allowing for the early implementation of organizational preventive measures to block the chain of transmission. MATERIALS AND METHODS In this study, we propose the use of a machine learning system in emergency department triage during pandemics to detect patients at the highest risk of death and infection using the COVID-19 era as an example, where rapid decision making and comprehensive support have becoming increasingly crucial. All patients who consecutively presented to the emergency department were included, and more than 89 variables were automatically analyzed using the extreme gradient boosting (XGB) algorithm. RESULTS The XGB system demonstrated the highest balanced accuracy at 91.61%. Additionally, it obtained results more quickly than traditional triage systems. The variables that most influenced mortality prediction were procalcitonin level, age, and oxygen saturation, followed by lactate dehydrogenase (LDH) level, C-reactive protein, the presence of interstitial infiltrates on chest X-ray, and D-dimer. Our system also identified the importance of oxygen therapy in these patients. CONCLUSIONS These results highlight that XGB is a useful and novel tool in triage systems for guiding the care pathway in future pandemics, thus following the example set by the well-known COVID-19 pandemic.
Collapse
Affiliation(s)
- Nicolás J. Garrido
- Internal Medicine, Virgen de la Luz Hospital, 16002 Cuenca, Spain
- Expert Medical Analysis Group, Institute of Technology, University of Castilla-La Mancha, 16071 Cuenca, Spain
| | - Félix González-Martínez
- Expert Medical Analysis Group, Institute of Technology, University of Castilla-La Mancha, 16071 Cuenca, Spain
- Department of Emergency Medicine, Virgen de la Luz Hospital, 16002 Cuenca, Spain
- Expert Medical Analysis Group, Instituto de Investigación Sanitaria de Castilla-La Mancha (IDISCAM), 45071 Toledo, Spain
| | - Susana Losada
- Department of Emergency Medicine, Virgen de la Luz Hospital, 16002 Cuenca, Spain
| | - Adrián Plaza
- Department of Emergency Medicine, Virgen de la Luz Hospital, 16002 Cuenca, Spain
| | - Eneida del Olmo
- Department of Emergency Medicine, Virgen de la Luz Hospital, 16002 Cuenca, Spain
| | - Jorge Mateo
- Expert Medical Analysis Group, Institute of Technology, University of Castilla-La Mancha, 16071 Cuenca, Spain
- Expert Medical Analysis Group, Instituto de Investigación Sanitaria de Castilla-La Mancha (IDISCAM), 45071 Toledo, Spain
| |
Collapse
|
7
|
Calcote MJ, Mann JR, Adcock KG, Duckworth S, Donald MC. Big Data in Health Care: An Interprofessional Course. Nurse Educ 2024; 49:E187-E191. [PMID: 37994454 DOI: 10.1097/nne.0000000000001571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2023]
Abstract
BACKGROUND The widespread adoption of the electronic health record (EHR) has resulted in vast repositories of EHR big data that are being used to identify patterns and correlations that translate into data-informed health care decision making. PROBLEM Health care professionals need the skills necessary to navigate a digitized, data-rich health care environment as big data plays an increasingly integral role in health care. APPROACH Faculty incorporated the concept of big data in an asynchronous online course allowing an interprofessional mix of students to analyze EHR big data on over a million patients. OUTCOMES Students conducted a descriptive analysis of cohorts of patients with selected diagnoses and presented their findings. CONCLUSIONS Students collaborated with an interprofessional team to analyze EHR big data on selected variables. The teams used data visualization tools to describe an assigned diagnosis patient population.
Collapse
Affiliation(s)
- Margaret J Calcote
- Author Affiliations: Assistant Professor (Dr Calcote), The University of Mississippi Medical Center School of Nursing, Jackson; Professor and Chair (Dr Mann), Department of Preventive Medicine, The University of Mississippi Medical Center School of Medicine, Jackson; Professor (Dr Adcock), Pharmacy Division, The University of Mississippi Medical Center School of Pharmacy, Jackson; Professor (Dr Duckworth), The University of Mississippi Medical Center Division of Internal Medicine, Jackson; and Medical Student M3 (Mr Donald), The University of Mississippi Medical Center School of Medicine, Jackson
| | | | | | | | | |
Collapse
|
8
|
Belge Bilgin G, Bilgin C, Burkett BJ, Orme JJ, Childs DS, Thorpe MP, Halfdanarson TR, Johnson GB, Kendi AT, Sartor O. Theranostics and artificial intelligence: new frontiers in personalized medicine. Theranostics 2024; 14:2367-2378. [PMID: 38646652 PMCID: PMC11024845 DOI: 10.7150/thno.94788] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Accepted: 03/17/2024] [Indexed: 04/23/2024] Open
Abstract
The field of theranostics is rapidly advancing, driven by the goals of enhancing patient care. Recent breakthroughs in artificial intelligence (AI) and its innovative theranostic applications have marked a critical step forward in nuclear medicine, leading to a significant paradigm shift in precision oncology. For instance, AI-assisted tumor characterization, including automated image interpretation, tumor segmentation, feature identification, and prediction of high-risk lesions, improves diagnostic processes, offering a precise and detailed evaluation. With a comprehensive assessment tailored to an individual's unique clinical profile, AI algorithms promise to enhance patient risk classification, thereby benefiting the alignment of patient needs with the most appropriate treatment plans. By uncovering potential factors unseeable to the human eye, such as intrinsic variations in tumor radiosensitivity or molecular profile, AI software has the potential to revolutionize the prediction of response heterogeneity. For accurate and efficient dosimetry calculations, AI technology offers significant advantages by providing customized phantoms and streamlining complex mathematical algorithms, making personalized dosimetry feasible and accessible in busy clinical settings. AI tools have the potential to be leveraged to predict and mitigate treatment-related adverse events, allowing early interventions. Additionally, generative AI can be utilized to find new targets for developing novel radiopharmaceuticals and facilitate drug discovery. However, while there is immense potential and notable interest in the role of AI in theranostics, these technologies do not lack limitations and challenges. There remains still much to be explored and understood. In this study, we investigate the current applications of AI in theranostics and seek to broaden the horizons for future research and innovation.
Collapse
Affiliation(s)
| | - Cem Bilgin
- Department of Radiology, Mayo Clinic Rochester, MN, USA
| | | | - Jacob J. Orme
- Department of Oncology, Mayo Clinic Rochester, MN, USA
| | | | | | | | - Geoffrey B Johnson
- Department of Radiology, Mayo Clinic Rochester, MN, USA
- Department of Immunology, Mayo Clinic Rochester, MN, USA
| | | | - Oliver Sartor
- Department of Radiology, Mayo Clinic Rochester, MN, USA
- Department of Oncology, Mayo Clinic Rochester, MN, USA
- Department of Urology, Mayo Clinic Rochester, MN, USA
| |
Collapse
|
9
|
Wang Y, Yin C, Zhang P. Multimodal risk prediction with physiological signals, medical images and clinical notes. Heliyon 2024; 10:e26772. [PMID: 38455585 PMCID: PMC10918115 DOI: 10.1016/j.heliyon.2024.e26772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 02/17/2024] [Accepted: 02/20/2024] [Indexed: 03/09/2024] Open
Abstract
The broad adoption of electronic health record (EHR) systems brings us a tremendous amount of clinical data and thus provides opportunities to conduct data-based healthcare research to solve various clinical problems in the medical domain. Machine learning and deep learning methods are widely used in the medical informatics and healthcare domain due to their power to mine insights from raw data. When adapting deep learning models for EHR data, it is essential to consider its heterogeneous nature: EHR contains patient records from various sources including medical tests (e.g. blood test, microbiology test), medical imaging, diagnosis, medications, procedures, clinical notes, etc. Those modalities together provide a holistic view of patient health status and complement each other. Therefore, combining data from multiple modalities that are intrinsically different is challenging but intuitively promising in deep learning for EHR. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in EHR for enhanced performance in clinical risk prediction. Early, joint, and late fusion strategies are employed to combine data from various modalities effectively. We test the model with three predictive tasks: in-hospital mortality, long length of stay, and 30-day readmission. Experimental results show that multimodal models outperform uni-modal models in the tasks involved. Additionally, by training models with different input modality combinations, we calculate the Shapley value for each modality to quantify their contribution to multimodal performance. It is shown that temporal variables tend to be more helpful than CXR images and clinical notes in the three explored predictive tasks.
Collapse
Affiliation(s)
- Yuanlong Wang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA
| | - Changchang Yin
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, OH 43210, USA
| | - Ping Zhang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, OH 43210, USA
| |
Collapse
|
10
|
Lin WC, Jordan BK, Scottoline B, Ostmo SR, Coyner AS, Singh P, Kalpathy-Cramer J, Erdogmus D, Chan RP, Chiang MF, Campbell JP. Oxygenation Fluctuations Associated with Severe Retinopathy of Prematurity: Insights from a Multimodal Deep Learning Approach. OPHTHALMOLOGY SCIENCE 2024; 4:100417. [PMID: 38059124 PMCID: PMC10696464 DOI: 10.1016/j.xops.2023.100417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/27/2023] [Accepted: 10/18/2023] [Indexed: 12/08/2023]
Abstract
Purpose Retinopathy of prematurity (ROP) is one of the leading causes of blindness in children. Although the role of oxygen in the pathophysiology of ROP is well established, a precise understanding of the dynamic relationship between oxygen exposure ROP incidence and severity is lacking. The purpose of this study was to evaluate the correlation between time-dependent oxygen variables and the onset of ROP. Design Retrospective cohort study. Participants Two hundred thirty infants who were born at a single academic center and met the inclusion criteria were included. Infants are mainly born between January 2011 and October 2022. Methods Patient data were extracted from electronic health records (EHRs), with sufficient time-dependent oxygen data. Clinical outcomes for ROP were recorded as none/mild or moderate/severe (defined as type II or worse). Mixed-effects linear models were used to compare the 2 groups in terms of dynamic oxygen variables, such as daily average and the coefficient of variation (COV) fraction of inspired oxygen (FiO2). Support vector machine (SVM) and long-short-term memory (LSTM)-based multimodal models were trained with fivefold cross-validation to predict which infants would develop moderate/severe ROP. Gestational age (GA), birth weight, and time-dependent oxygen variables were used to develop predictive models. Main Outcome Measures Model cross-validation performance was evaluated by computing the mean area under the receiver operating characteristic (AUROC) curve, precision, recall, and F1 score. Results We found that both daily average and COV of FiO2 were associated with more severe ROP (adjusted P < 0.001). With fivefold cross-validation, the multimodal LSTM models had higher performance than the best static models (SVM using GA and 3 average FiO2 features) and SVM models trained on GA alone (mean AUROC = 0.89 ± 0.04 vs. 0.86 ± 0.05 vs. 0.83 ± 0.04). Conclusions The development of severe ROP might not only be influenced by oxygen exposure but also by its fluctuation, which provides direction for future study of pathophysiological factors associated with severe ROP development. Additionally, we demonstrated that multimodal neural networks can be a method to extract useful information from time-series data, which may be a valuable methodology for the investigation of other diseases using EHR data. Financial Disclosures Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Wei-Chun Lin
- Department of Ophthalmology, Oregon Health & Science University, Portland, Oregon
| | - Brian K. Jordan
- Department of Neonatology, Oregon Health and Science University, Portland, Oregon
| | - Brian Scottoline
- Department of Neonatology, Oregon Health and Science University, Portland, Oregon
| | - Susan R. Ostmo
- Department of Ophthalmology, Oregon Health & Science University, Portland, Oregon
| | - Aaron S. Coyner
- Department of Ophthalmology, Oregon Health & Science University, Portland, Oregon
| | - Praveer Singh
- Department of Ophthalmology, University of Colorado (CU) School of Medicine, Denver, Colorado
| | | | - Deniz Erdogmus
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - R.V. Paul Chan
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, Illinois
| | - Michael F. Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland
- National Library of Medicine, National Institutes of Health, Bethesda, Maryland
| | - J. Peter Campbell
- Department of Ophthalmology, Oregon Health & Science University, Portland, Oregon
| |
Collapse
|
11
|
Taha MA, Morren JA. The role of artificial intelligence in electrodiagnostic and neuromuscular medicine: Current state and future directions. Muscle Nerve 2024; 69:260-272. [PMID: 38151482 DOI: 10.1002/mus.28023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2023] [Revised: 12/04/2023] [Accepted: 12/09/2023] [Indexed: 12/29/2023]
Abstract
The rapid advancements in artificial intelligence (AI), including machine learning (ML), and deep learning (DL) have ushered in a new era of technological breakthroughs in healthcare. These technologies are revolutionizing the way we utilize medical data, enabling improved disease classification, more precise diagnoses, better treatment selection, therapeutic monitoring, and highly accurate prognostication. Different ML and DL models have been used to distinguish between electromyography signals in normal individuals and those with amyotrophic lateral sclerosis and myopathy, with accuracy ranging from 67% to 99.5%. DL models have also been successfully applied in neuromuscular ultrasound, with the use of segmentation techniques achieving diagnostic accuracy of at least 90% for nerve entrapment disorders, and 87% for inflammatory myopathies. Other successful AI applications include prediction of treatment response, and prognostication including prediction of intensive care unit admissions for patients with myasthenia gravis. Despite these remarkable strides, significant knowledge, attitude, and practice gaps persist, including within the field of electrodiagnostic and neuromuscular medicine. In this narrative review, we highlight the fundamental principles of AI and draw parallels with the intricacies of human brain networks. Specifically, we explore the immense potential that AI holds for applications in electrodiagnostic studies, neuromuscular ultrasound, and other aspects of neuromuscular medicine. While there are exciting possibilities for the future, it is essential to acknowledge and understand the limitations of AI and take proactive steps to mitigate these challenges. This collective endeavor holds immense potential for the advancement of healthcare through the strategic and responsible integration of AI technologies.
Collapse
Affiliation(s)
- Mohamed A Taha
- Neuromuscular Center, Neurological Institute, Cleveland Clinic, Cleveland, Ohio, USA
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - John A Morren
- Neuromuscular Center, Neurological Institute, Cleveland Clinic, Cleveland, Ohio, USA
| |
Collapse
|
12
|
Li F, Rasmy L, Xiang Y, Feng J, Abdelhameed A, Hu X, Sun Z, Aguilar D, Dhoble A, Du J, Wang Q, Niu S, Dang Y, Zhang X, Xie Z, Nian Y, He J, Zhou Y, Li J, Prosperi M, Bian J, Zhi D, Tao C. Dynamic Prognosis Prediction for Patients on DAPT After Drug-Eluting Stent Implantation: Model Development and Validation. J Am Heart Assoc 2024; 13:e029900. [PMID: 38293921 PMCID: PMC11056175 DOI: 10.1161/jaha.123.029900] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 12/01/2023] [Indexed: 02/01/2024]
Abstract
BACKGROUND The rapid evolution of artificial intelligence (AI) in conjunction with recent updates in dual antiplatelet therapy (DAPT) management guidelines emphasizes the necessity for innovative models to predict ischemic or bleeding events after drug-eluting stent implantation. Leveraging AI for dynamic prediction has the potential to revolutionize risk stratification and provide personalized decision support for DAPT management. METHODS AND RESULTS We developed and validated a new AI-based pipeline using retrospective data of drug-eluting stent-treated patients, sourced from the Cerner Health Facts data set (n=98 236) and Optum's de-identified Clinformatics Data Mart Database (n=9978). The 36 months following drug-eluting stent implantation were designated as our primary forecasting interval, further segmented into 6 sequential prediction windows. We evaluated 5 distinct AI algorithms for their precision in predicting ischemic and bleeding risks. Model discriminative accuracy was assessed using the area under the receiver operating characteristic curve, among other metrics. The weighted light gradient boosting machine stood out as the preeminent model, thus earning its place as our AI-DAPT model. The AI-DAPT demonstrated peak accuracy in the 30 to 36 months window, charting an area under the receiver operating characteristic curve of 90% [95% CI, 88%-92%] for ischemia and 84% [95% CI, 82%-87%] for bleeding predictions. CONCLUSIONS Our AI-DAPT excels in formulating iterative, refined dynamic predictions by assimilating ongoing updates from patients' clinical profiles, holding value as a novel smart clinical tool to facilitate optimal DAPT duration management with high accuracy and adaptability.
Collapse
Affiliation(s)
- Fang Li
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| | - Laila Rasmy
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Yang Xiang
- Peng Cheng LaboratoryShenzhenGuangdongChina
| | - Jingna Feng
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| | - Ahmed Abdelhameed
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| | - Xinyue Hu
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| | - Zenan Sun
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - David Aguilar
- Department of Internal Medicine, McGovern Medical SchoolUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- LSU School of Medicine, LSU Health New OrleansNew OrleansLAUSA
| | - Abhijeet Dhoble
- Department of Internal Medicine, McGovern Medical SchoolUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Jingcheng Du
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Qing Wang
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Shuteng Niu
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Yifang Dang
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Xinyuan Zhang
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Ziqian Xie
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Yi Nian
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - JianPing He
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Yujia Zhou
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Jianfu Li
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| | - Mattia Prosperi
- Data Intelligence Systems Lab, Department of Epidemiology, College of Public Health and Health Professions & College of MedicineUniversity of FloridaGainesvilleFLUSA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of MedicineUniversity of FloridaGainesvilleFLUSA
| | - Degui Zhi
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
| | - Cui Tao
- McWilliams School of Biomedical InformaticsUniversity of Texas Health Science Center at HoustonHoustonTXUSA
- Department of Artificial Intelligence and InformaticsMayo ClinicJacksonvilleFLUSA
| |
Collapse
|
13
|
Lin WC, Chen A, Song X, Weiskopf NG, Chiang MF, Hribar MR. Prediction of multiclass surgical outcomes in glaucoma using multimodal deep learning based on free-text operative notes and structured EHR data. J Am Med Inform Assoc 2024; 31:456-464. [PMID: 37964658 PMCID: PMC10797280 DOI: 10.1093/jamia/ocad213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 10/16/2023] [Accepted: 10/25/2023] [Indexed: 11/16/2023] Open
Abstract
OBJECTIVE Surgical outcome prediction is challenging but necessary for postoperative management. Current machine learning models utilize pre- and post-op data, excluding intraoperative information in surgical notes. Current models also usually predict binary outcomes even when surgeries have multiple outcomes that require different postoperative management. This study addresses these gaps by incorporating intraoperative information into multimodal models for multiclass glaucoma surgery outcome prediction. MATERIALS AND METHODS We developed and evaluated multimodal deep learning models for multiclass glaucoma trabeculectomy surgery outcomes using both structured EHR data and free-text operative notes. We compare those to baseline models that use structured EHR data exclusively, or neural network models that leverage only operative notes. RESULTS The multimodal neural network had the highest performance with a macro AUROC of 0.750 and F1 score of 0.583. It outperformed the baseline machine learning model with structured EHR data alone (macro AUROC of 0.712 and F1 score of 0.486). Additionally, the multimodal model achieved the highest recall (0.692) for hypotony surgical failure, while the surgical success group had the highest precision (0.884) and F1 score (0.775). DISCUSSION This study shows that operative notes are an important source of predictive information. The multimodal predictive model combining perioperative notes and structured pre- and post-op EHR data outperformed other models. Multiclass surgical outcome prediction can provide valuable insights for clinical decision-making. CONCLUSIONS Our results show the potential of deep learning models to enhance clinical decision-making for postoperative management. They can be applied to other specialties to improve surgical outcome predictions.
Collapse
Affiliation(s)
- Wei-Chun Lin
- Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd, Portland, OR, 97239, United States
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, 545 SW Campus Dr, Portland, OR, 97239, United States
| | - Aiyin Chen
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, 545 SW Campus Dr, Portland, OR, 97239, United States
| | - Xubo Song
- Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd, Portland, OR, 97239, United States
| | - Nicole G Weiskopf
- Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd, Portland, OR, 97239, United States
| | - Michael F Chiang
- National Eye Institute, National Institutes of Health, 31 Center Dr MSC 2510, Bethesda, MD, 20892, United States
- National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, United States
| | - Michelle R Hribar
- Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd, Portland, OR, 97239, United States
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, 545 SW Campus Dr, Portland, OR, 97239, United States
- National Eye Institute, National Institutes of Health, 31 Center Dr MSC 2510, Bethesda, MD, 20892, United States
| |
Collapse
|
14
|
Ma M, Wang M, Gao B, Li Y, Huang J, Chen H. Research on Multimodal Fusion of Temporal Electronic Medical Records. Bioengineering (Basel) 2024; 11:94. [PMID: 38247971 PMCID: PMC10813197 DOI: 10.3390/bioengineering11010094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 12/16/2023] [Accepted: 12/18/2023] [Indexed: 01/23/2024] Open
Abstract
The surge in deep learning-driven EMR research has centered on harnessing diverse data forms. Yet, the amalgamation of diverse modalities within time series data remains an underexplored realm. This study probes a multimodal fusion approach, merging temporal and non-temporal clinical notes along with tabular data. We leveraged data from 1271 myocardial infarction and 6450 stroke inpatients at a Beijing tertiary hospital. Our dataset encompassed static, and time series note data, coupled with static and time series table data. The temporal data underwent a preprocessing phase, padding to a 30-day interval, and segmenting into 3-day sub-sequences. These were fed into a long short-term memory (LSTM) network for sub-sequence representation. Multimodal attention gates were implemented for both static and temporal subsequence representations, culminating in fused representations. An attention-backtracking module was introduced for the latter, adept at capturing enduring dependencies in temporal fused representations. The concatenated results were channeled into an LSTM to yield the ultimate fused representation. Initially, two note modalities were designated as primary modes, and subsequently, the proposed fusion model was compared with comparative models including recent models such as Crossformer. The proposed model consistently exhibited superior predictive prowess in both tasks. Removing the attention-backtracking module led to performance decline. The proposed model consistently shows excellent predictive capabilities in both tasks. The proposed method not only effectively integrates data from the four modalities, but also has a good understanding of how to handle irregular time series data and lengthy clinical texts. An effective method is provided, which is expected to be more widely used in multimodal medical data representation.
Collapse
Affiliation(s)
- Moxuan Ma
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| | - Muyu Wang
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| | - Binyu Gao
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| | - Yichen Li
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| | - Jun Huang
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| | - Hui Chen
- School of Biomedical Engineering, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China; (M.M.)
- Beijing Key Laboratory of Fundamental Research on Biomechanics in Clinical Application, Capital Medical University, No. 10, Xitoutiao, You An Men, Fengtai District, Beijing 100069, China
| |
Collapse
|
15
|
Ostropolets A, Hripcsak G, Husain SA, Richter LR, Spotnitz M, Elhussein A, Ryan PB. Scalable and interpretable alternative to chart review for phenotype evaluation using standardized structured data from electronic health records. J Am Med Inform Assoc 2023; 31:119-129. [PMID: 37847668 PMCID: PMC10746303 DOI: 10.1093/jamia/ocad202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 09/23/2023] [Accepted: 10/02/2023] [Indexed: 10/19/2023] Open
Abstract
OBJECTIVES Chart review as the current gold standard for phenotype evaluation cannot support observational research on electronic health records and claims data sources at scale. We aimed to evaluate the ability of structured data to support efficient and interpretable phenotype evaluation as an alternative to chart review. MATERIALS AND METHODS We developed Knowledge-Enhanced Electronic Profile Review (KEEPER) as a phenotype evaluation tool that extracts patient's structured data elements relevant to a phenotype and presents them in a standardized fashion following clinical reasoning principles. We evaluated its performance (interrater agreement, intermethod agreement, accuracy, and review time) compared to manual chart review for 4 conditions using randomized 2-period, 2-sequence crossover design. RESULTS Case ascertainment with KEEPER was twice as fast compared to manual chart review. 88.1% of the patients were classified concordantly using charts and KEEPER, but agreement varied depending on the condition. Missing data and differences in interpretation accounted for most of the discrepancies. Pairs of clinicians agreed in case ascertainment in 91.2% of the cases when using KEEPER compared to 76.3% when using charts. Patient classification aligned with the gold standard in 88.1% and 86.9% of the cases respectively. CONCLUSION Structured data can be used for efficient and interpretable phenotype evaluation if they are limited to relevant subset and organized according to the clinical reasoning principles. A system that implements these principles can achieve noninferior performance compared to chart review at a fraction of time.
Collapse
Affiliation(s)
- Anna Ostropolets
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
- Medical Informatics Services, New York-Presbyterian Hospital, New York, NY 10032, United States
| | - Syed A Husain
- Division of Nephrology, Department of Medicine, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY 10032, United States
| | - Lauren R Richter
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
| | - Matthew Spotnitz
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
| | - Ahmed Elhussein
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
| | - Patrick B Ryan
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY 10032, United States
- Observational Health Data Analytics, Janssen Research and Development, Titusville, NJ 08560, United States
| |
Collapse
|
16
|
Galimzhanov A, Matetic A, Tenekecioglu E, Mamas MA. Prediction of clinical outcomes after percutaneous coronary intervention: Machine-learning analysis of the National Inpatient Sample. Int J Cardiol 2023; 392:131339. [PMID: 37678434 DOI: 10.1016/j.ijcard.2023.131339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 08/08/2023] [Accepted: 09/03/2023] [Indexed: 09/09/2023]
Abstract
BACKGROUND This study aimed to develop a multiclass machine-learning (ML) model to predict all-cause mortality, ischemic and hemorrhagic events in unselected hospitalized patients undergoing percutaneous coronary intervention (PCI). METHODS This retrospective study included 1,815,595 unselected weighted hospitalizations undergoing PCI from the National Inpatient Sample (2016-2019). Five most common ML algorithms (logistic regression, support vector machine (SVM), naive Bayes, random forest (RF), and extreme gradient boosting (XGBoost)) were trained and tested with 101 input features. The study endpoints were different combinations of all-cause mortality, ischemic cerebrovascular events (CVE) and major bleeding. An area under the curve (AUC) with 95% confidence interval (95% CI) was selected as a performance metric. RESULTS The study population was split to a training cohort of 1,186,880 PCI discharges, validation cohort (for calibration) of 296,725 hospitalizations and a test cohort of 331,990 PCI discharges. A total of 98,180 (5.4%) hospital entries included study outcomes. Logistic regression, SVM, naive Bayes, and RF model demonstrated AUCs of 0.83 (95% CI 0.82-0.84), 0.84 (95% CI 0.83-0.86), 0.81 (95% CI 0.80-0.82), and 0.83 (95% CI 0.81-0.84), retrospectively. The XGBoost classifier performed the best with an AUC of 0.86 (95% CI 0.85-0.87) with excellent calibration. We then built a web-based application that provides predictions based on the XGBoost model. CONCLUSION We derived the multi-task XGBoost classifier based on 101 features to predict different combinations of all-cause death, ischemic CVE and major bleeding. Such models may be useful in benchmarking and risk prediction using routinely collected administrative data.
Collapse
Affiliation(s)
- Akhmetzhan Galimzhanov
- Department of Propedeutics of Internal Disease, Semey Medical University, Semey, Kazakhstan; Keele Cardiovascular Research Group, Keele University, Keele, UK.
| | - Andrija Matetic
- Keele Cardiovascular Research Group, Keele University, Keele, UK; Department of Cardiology, University Hospital of Split, Split 21000, Croatia
| | - Erhan Tenekecioglu
- Department of Cardiology, Bursa Education and Research Hospital, Health Sciences University, Bursa,Turkey; Department of Cardiology, Thoraxcenter, Erasmus MC, Erasmus University, Rotterdam, the Netherlands
| | - Mamas A Mamas
- Keele Cardiovascular Research Group, Keele University, Keele, UK
| |
Collapse
|
17
|
Zou B, Ding Y, Li J, Yu B, Kui X. TGRA-P: Task-driven model predicts 90-day mortality from ICU clinical notes on mechanical ventilation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107783. [PMID: 37716220 DOI: 10.1016/j.cmpb.2023.107783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Revised: 08/14/2023] [Accepted: 08/28/2023] [Indexed: 09/18/2023]
Abstract
BACKGROUND With the outbreak and spread of COVID-19 worldwide, limited ventilators fail to meet the surging demand for mechanical ventilation in the ICU. Clinical models based on structured data that have been proposed to rationalize ventilator allocation often suffer from poor ductility due to fixed fields and laborious normalization processes. The advent of pre-trained models and downstream fine-tuning methods allows for learning large amounts of unstructured clinical text for different tasks. But the hardware requirements of large-scale pre-trained models and purposeless networks downstream have led to a lack of promotion in the clinical domain. OBJECTIVE In this study, an innovative architecture of a task-driven predictive model is proposed and a Task-driven Gated Recurrent Attention Pool model (TGRA-P) is developed based on the architecture. TGRA-P predicts early mortality risk from patients' clinical notes on mechanical ventilation in the ICU, which is used to assist clinicians in diagnosis and decision-making. METHODS Specifically, a Task-Specific Embedding Module is proposed to fine-tune the embedding with task labels and save it as static files for downstream calls. It serves the task better and prevents GPU overload. The Gated Recurrent Attention Unit (GRA) is proposed to further enhance the dependency of the information preceding and following the text sequence with fewer parameters. In addition, we propose a Residual Max Pool (RMP) to avoid ignoring words in common text classification tasks by incorporating all word-level features of the notes for prediction. Finally, we use a fully connected decoding network as a classifier to predict the mortality risk. RESULT The proposed model shows very promising results with an AUROC of 0.8245±0.0096, an AUPRC of 0.7532±0.0115, an accuracy of 0.7422±0.0028 and F1-score of 0.6612±0.0059 for 90-day mortality prediction using clinical notes of ICU mechanically ventilated patients on the MIMIC-III dataset, all of which are better than previous studies. Moreover, the superiority of the proposed model in comparison with other baseline models is also statistically validated through the calculated Cohen's d effect sizes. CONCLUSION The experimental results show that TGRA-P based on the innovative task-driven prognostic architecture obtains state-of-the-art performance. In future work, we will build upon the provided code and investigate its applicability to different datasets. The model balances performance and efficiency, not only reducing the cost of early mortality risk prediction but also assisting physicians in making timely clinical interventions and decisions. By incorporating textual records that are challenging for clinicians to utilize, the model serves as a valuable complement to physicians' judgment, enhancing their decision-making process.
Collapse
Affiliation(s)
- Beiji Zou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China.
| | - Yuting Ding
- School of Computer Science and Engineering, Central South University, Changsha 410083, China.
| | - Jinxiu Li
- The Second Xiangya Hospital, Central South University, Changsha 410011, China.
| | - Bo Yu
- The Second Xiangya Hospital, Central South University, Changsha 410011, China.
| | - Xiaoyan Kui
- School of Computer Science and Engineering, Central South University, Changsha 410083, China.
| |
Collapse
|
18
|
Jaotombo F, Adorni L, Ghattas B, Boyer L. Finding the best trade-off between performance and interpretability in predicting hospital length of stay using structured and unstructured data. PLoS One 2023; 18:e0289795. [PMID: 38032876 PMCID: PMC10688642 DOI: 10.1371/journal.pone.0289795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Accepted: 07/25/2023] [Indexed: 12/02/2023] Open
Abstract
OBJECTIVE This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data. METHODS The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines. RESULTS The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery. CONCLUSIONS Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.
Collapse
Affiliation(s)
- Franck Jaotombo
- EMLYON Business School, Ecully, France
- Research Centre on Health Services and Quality of Life, Aix Marseille University, Marseille, France
| | - Luca Adorni
- Becker Friedman Institute, Chicago, IL, United States of America
| | - Badih Ghattas
- Aix Marseille University, CNRS, AMSE, Marseille, France
| | - Laurent Boyer
- Research Centre on Health Services and Quality of Life, Aix Marseille University, Marseille, France
- Department of Public Health, Assistance Publique–Hopitaux de Marseille, Marseille, France
| |
Collapse
|
19
|
MacDougall C. A Cloudy Crystal Ball: Critically Assessing and Rethinking the Antibiogram. Clin Infect Dis 2023; 77:1501-1503. [PMID: 37658904 DOI: 10.1093/cid/ciad468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 08/08/2023] [Indexed: 09/05/2023] Open
Affiliation(s)
- Conan MacDougall
- Department of Clinical Pharmacy, School of Pharmacy, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
20
|
Garriga R, Buda TS, Guerreiro J, Omaña Iglesias J, Estella Aguerri I, Matić A. Combining clinical notes with structured electronic health records enhances the prediction of mental health crises. Cell Rep Med 2023; 4:101260. [PMID: 37913776 PMCID: PMC10694623 DOI: 10.1016/j.xcrm.2023.101260] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 07/12/2023] [Accepted: 10/05/2023] [Indexed: 11/03/2023]
Abstract
An automatic prediction of mental health crises can improve caseload prioritization and enable preventative interventions, improving patient outcomes and reducing costs. We combine structured electronic health records (EHRs) with clinical notes from 59,750 de-identified patients to predict the risk of mental health crisis relapse within the next 28 days. The results suggest that an ensemble machine learning model that relies on structured EHRs and clinical notes when available, and relying solely on structured data when the notes are unavailable, offers superior performance over models trained with either of the two data streams alone. Furthermore, the study provides key takeaways related to the required amount of clinical notes to add value in predictive analytics. This study sheds light on the untapped potential of clinical notes in the prediction of mental health crises and highlights the importance of choosing an appropriate machine learning method to combine structured and unstructured EHRs.
Collapse
Affiliation(s)
- Roger Garriga
- Koa Health, 08019 Barcelona, Spain; Department of Information and Communication Technologies, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | | | | | | | | | | |
Collapse
|
21
|
Stam WT, Ingwersen EW, Ali M, Spijkerman JT, Kazemier G, Bruns ERJ, Daams F. Machine learning models in clinical practice for the prediction of postoperative complications after major abdominal surgery. Surg Today 2023; 53:1209-1215. [PMID: 36840764 PMCID: PMC10520164 DOI: 10.1007/s00595-023-02662-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 02/07/2023] [Indexed: 02/26/2023]
Abstract
Complications after surgery have a major impact on short- and long-term outcomes, and decades of technological advancement have not yet led to the eradication of their risk. The accurate prediction of complications, recently enhanced by the development of machine learning algorithms, has the potential to completely reshape surgical patient management. In this paper, we reflect on multiple issues facing the implementation of machine learning, from the development to the actual implementation of machine learning models in daily clinical practice, providing suggestions on the use of machine learning models for predicting postoperative complications after major abdominal surgery.
Collapse
Affiliation(s)
- Wessel T Stam
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
- AGEM Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam, The Netherlands
| | - Erik W Ingwersen
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
- AGEM Amsterdam Gastroenterology, Endocrinology and Metabolism, Amsterdam, The Netherlands
| | - Mahsoem Ali
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Jorik T Spijkerman
- Independent Consultant in Computational Intelligence, Amsterdam, The Netherlands
| | - Geert Kazemier
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Emma R J Bruns
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Freek Daams
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands.
- Cancer Center Amsterdam, Cancer Treatment and Quality of Life, Amsterdam, The Netherlands.
| |
Collapse
|
22
|
Athaya T, Ripan RC, Li X, Hu H. Multimodal deep learning approaches for single-cell multi-omics data integration. Brief Bioinform 2023; 24:bbad313. [PMID: 37651607 PMCID: PMC10516349 DOI: 10.1093/bib/bbad313] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 06/23/2023] [Accepted: 07/18/2023] [Indexed: 09/02/2023] Open
Abstract
Integrating single-cell multi-omics data is a challenging task that has led to new insights into complex cellular systems. Various computational methods have been proposed to effectively integrate these rapidly accumulating datasets, including deep learning. However, despite the proven success of deep learning in integrating multi-omics data and its better performance over classical computational methods, there has been no systematic study of its application to single-cell multi-omics data integration. To fill this gap, we conducted a literature review to explore the use of multimodal deep learning techniques in single-cell multi-omics data integration, taking into account recent studies from multiple perspectives. Specifically, we first summarized different modalities found in single-cell multi-omics data. We then reviewed current deep learning techniques for processing multimodal data and categorized deep learning-based integration methods for single-cell multi-omics data according to data modality, deep learning architecture, fusion strategy, key tasks and downstream analysis. Finally, we provided insights into using these deep learning models to integrate multi-omics data and better understand single-cell biological mechanisms.
Collapse
Affiliation(s)
- Tasbiraha Athaya
- Department of Computer Science, University of Central Florida, Orlando, Florida, United States of America
| | - Rony Chowdhury Ripan
- Department of Computer Science, University of Central Florida, Orlando, Florida, United States of America
| | - Xiaoman Li
- Burnett School of Biomedical Science, College of Medicine, University of Central Florida, Orlando, Florida, United States of America
| | - Haiyan Hu
- Department of Computer Science, University of Central Florida, Orlando, Florida, United States of America
| |
Collapse
|
23
|
Dhingra LS, Shen M, Mangla A, Khera R. Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record. Am J Cardiol 2023; 203:136-148. [PMID: 37499593 PMCID: PMC10865722 DOI: 10.1016/j.amjcard.2023.06.104] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 05/24/2023] [Accepted: 06/29/2023] [Indexed: 07/29/2023]
Abstract
The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.
Collapse
Affiliation(s)
| | - Miles Shen
- Section of Cardiovascular Medicine, Department of Internal Medicine; Department of Internal Medicine
| | - Anjali Mangla
- Section of Cardiovascular Medicine, Department of Internal Medicine; Department of Neuroscience, Yale School of Medicine, New Haven, Connecticut
| | - Rohan Khera
- Section of Cardiovascular Medicine, Department of Internal Medicine; Center for Outcomes Research and Evaluation (CORE), Yale New Haven Hospital, New Haven, Connecticut; Section of Health Informatics, Department of Biostatistics, Yale School of Public Health, New Haven, Connecticut.; Section of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, Connecticut.
| |
Collapse
|
24
|
Wang Y, Jin X, Castro C. Accelerating the characterization of dynamic DNA origami devices with deep neural networks. Sci Rep 2023; 13:15196. [PMID: 37709771 PMCID: PMC10502017 DOI: 10.1038/s41598-023-41459-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Accepted: 08/27/2023] [Indexed: 09/16/2023] Open
Abstract
Mechanical characterization of dynamic DNA nanodevices is essential to facilitate their use in applications like molecular diagnostics, force sensing, and nanorobotics that rely on device reconfiguration and interactions with other materials. A common approach to evaluate the mechanical properties of dynamic DNA nanodevices is by quantifying conformational distributions, where the magnitude of fluctuations correlates to the stiffness. This is generally carried out through manual measurement from experimental images, which is a tedious process and a critical bottleneck in the characterization pipeline. While many tools support the analysis of static molecular structures, there is a need for tools to facilitate the rapid characterization of dynamic DNA devices that undergo large conformational fluctuations. Here, we develop a data processing pipeline based on Deep Neural Networks (DNNs) to address this problem. The YOLOv5 and Resnet50 network architecture were used for the two key subtasks: particle detection and pose (i.e. conformation) estimation. We demonstrate effective network performance (F1 score 0.85 in particle detection) and good agreement with experimental distributions with limited user input and small training sets (~ 5 to 10 images). We also demonstrate this pipeline can be applied to multiple nanodevices, providing a robust approach for the rapid characterization of dynamic DNA devices.
Collapse
Affiliation(s)
- Yuchen Wang
- Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, 43210, USA.
| | - Xin Jin
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210, USA
| | - Carlos Castro
- Department of Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, 43210, USA.
| |
Collapse
|
25
|
Liu J, Capurro D, Nguyen A, Verspoor K. Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities. J Biomed Inform 2023; 145:104466. [PMID: 37549722 DOI: 10.1016/j.jbi.2023.104466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Revised: 06/09/2023] [Accepted: 08/01/2023] [Indexed: 08/09/2023]
Abstract
OBJECTIVE With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities. METHODS We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities. RESULTS Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results. CONCLUSION We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.
Collapse
Affiliation(s)
- Jinghui Liu
- Australian e-Health Research Centre, CSIRO, Queensland, Australia; School of Computing and Information Systems, The University of Melbourne, Victoria, Australia
| | - Daniel Capurro
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, The University of Melbourne, Victoria, Australia
| | - Anthony Nguyen
- Australian e-Health Research Centre, CSIRO, Queensland, Australia
| | - Karin Verspoor
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; School of Computing Technologies, RMIT University, Victoria, Australia.
| |
Collapse
|
26
|
Ingwersen EW, Stam WT, Meijs BJV, Roor J, Besselink MG, Groot Koerkamp B, de Hingh IHJT, van Santvoort HC, Stommel MWJ, Daams F. Machine learning versus logistic regression for the prediction of complications after pancreatoduodenectomy. Surgery 2023; 174:435-440. [PMID: 37150712 DOI: 10.1016/j.surg.2023.03.012] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 03/02/2023] [Accepted: 03/20/2023] [Indexed: 05/09/2023]
Abstract
BACKGROUND Machine learning is increasingly advocated to develop prediction models for postoperative complications. It is, however, unclear if machine learning is superior to logistic regression when using structured clinical data. Postoperative pancreatic fistula and delayed gastric emptying are the two most common complications with the biggest impact on patient condition and length of hospital stay after pancreatoduodenectomy. This study aimed to compare the performance of machine learning and logistic regression in predicting pancreatic fistula and delayed gastric emptying after pancreatoduodenectomy. METHODS This retrospective observational study used nationwide data from 16 centers in the Dutch Pancreatic Cancer Audit between January 2014 and January 2021. The area under the curve of a machine learning and logistic regression model for clinically relevant postoperative pancreatic fistula and delayed gastric emptying were compared. RESULTS Overall, 799 (16.3%) patients developed a postoperative pancreatic fistula, and 943 developed (19.2%) delayed gastric emptying. For postoperative pancreatic fistula, the area under the curve of the machine learning model was 0.74, and the area under the curve of the logistic regression model was 0.73. For delayed gastric emptying, the area under the curve of the machine learning model and logistic regression was 0.59. CONCLUSION Machine learning did not outperform logistic regression modeling in predicting postoperative complications after pancreatoduodenectomy.
Collapse
Affiliation(s)
- Erik W Ingwersen
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Wessel T Stam
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Bono J V Meijs
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands
| | - Joran Roor
- SAS institute B.V., Huizen, the Netherlands
| | - Marc G Besselink
- Cancer Center Amsterdam, the Netherlands; Amsterdam Gastroenterology Endocrinology and Metabolism, the Netherlands; Department of Surgery, Amsterdam UMC, University of Amsterdam, the Netherlands
| | | | | | | | | | - Freek Daams
- Amsterdam UMC, location Vrije Universiteit Amsterdam, Department of Surgery, Amsterdam, the Netherlands; Cancer Center Amsterdam, the Netherlands.
| |
Collapse
|
27
|
Chen J, Engelhard M, Henao R, Berchuck S, Eichner B, Perrin EM, Sapiro G, Dawson G. Enhancing early autism prediction based on electronic records using clinical narratives. J Biomed Inform 2023; 144:104390. [PMID: 37182592 PMCID: PMC10526711 DOI: 10.1016/j.jbi.2023.104390] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 04/14/2023] [Accepted: 05/09/2023] [Indexed: 05/16/2023]
Abstract
Recent work has shown that predictive models can be applied to structured electronic health record (EHR) data to stratify autism likelihood from an early age (<1 year). Integrating clinical narratives (or notes) with structured data has been shown to improve prediction performance in other clinical applications, but the added predictive value of this information in early autism prediction has not yet been explored. In this study, we aimed to enhance the performance of early autism prediction by using both structured EHR data and clinical narratives. We built models based on structured data and clinical narratives separately, and then an ensemble model that integrated both sources of data. We assessed the predictive value of these models from Duke University Health System over a 14-year span to evaluate ensemble models predicting later autism diagnosis (by age 4 years) from data collected from ages 30 to 360 days. Our sample included 11,750 children above by age 3 years (385 meeting autism diagnostic criteria). The ensemble model for autism prediction showed superior performance and at age 30 days achieved 46.8% sensitivity (95% confidence interval, CI: 22.0%, 52.9%), 28.0% positive predictive value (PPV) at high (90%) specificity (CI: 2.0%, 33.1%), and AUC4 (with at least 4-year follow-up for controls) reaching 0.769 (CI: 0.715, 0.811). Prediction by 360 days achieved 44.5% sensitivity (CI: 23.6%, 62.9%), and 13.7% PPV at high (90%) specificity (CI: 9.6%, 18.9%), and AUC4 reaching 0.797 (CI: 0.746, 0.840). Results show that incorporating clinical narratives in early autism prediction achieved promising accuracy by age 30 days, outperforming models based on structured data only. Furthermore, findings suggest that additional features learned from clinician narratives might be hypothesis generating for understanding early development in autism.
Collapse
Affiliation(s)
- Junya Chen
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States.
| | - Matthew Engelhard
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Ricardo Henao
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Samuel Berchuck
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Brian Eichner
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Eliana M Perrin
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Guillermo Sapiro
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| | - Geraldine Dawson
- Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27705, United States
| |
Collapse
|
28
|
Sax DR, Warton EM, Sofrygin O, Mark DG, Ballard DW, Kene MV, Vinson DR, Reed ME. Automated analysis of unstructured clinical assessments improves emergency department triage performance: A retrospective deep learning analysis. J Am Coll Emerg Physicians Open 2023; 4:e13003. [PMID: 37448487 PMCID: PMC10337523 DOI: 10.1002/emp2.13003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 05/11/2023] [Accepted: 06/20/2023] [Indexed: 07/15/2023] Open
Abstract
Objectives Efficient and accurate emergency department (ED) triage is critical to prioritize the sickest patients and manage department flow. We explored the use of electronic health record data and advanced predictive analytics to improve triage performance. Methods Using a data set of over 5 million ED encounters of patients 18 years and older across 21 EDs from 2016 to 2020, we derived triage models using deep learning to predict 2 outcomes: hospitalization (primary outcome) and fast-track eligibility (exploratory outcome), defined as ED discharge with <2 resource types used (eg, laboratory or imaging studies) and no critical events (eg, resuscitative medications use or intensive care unit [ICU] admission). We report area under the receiver operator characteristic curve (AUC) and 95% confidence intervals (CI) for models using (1) triage variables alone (demographics and vital signs), (2) triage nurse clinical assessment alone (unstructured notes), and (3) triage variables plus clinical assessment for each prediction target. Results We found 12.7% of patients were hospitalized (n = 673,659) and 37.0% were fast-track eligible (n = 1,966,615). The AUC was lowest for models using triage variables alone: AUC 0.77 (95% CI 0.77-0.78) and 0.70 (95% CI 0.70-0.71) for hospitalization and fast-track eligibility, respectively, and highest for models incorporating clinical assessment with triage variables for both hospitalization and fast-track eligibility: AUC 0.87 (95% CI 0.87-0.87) for both prediction targets. Conclusion Our findings highlight the potential to use advanced predictive analytics to accurately predict key ED triage outcomes. Predictive accuracy was optimized when clinical assessments were added to models using simple structured variables alone.
Collapse
Affiliation(s)
- Dana R. Sax
- Department of Emergency MedicineKaiser East Bay and Kaiser Permanente NorthernCalifornia Division of ResearchOaklandCaliforniaUSA
| | - E. Margaret Warton
- Kaiser Permanente Northern California Division of ResearchOaklandCaliforniaUSA
| | | | - Dustin G. Mark
- Department of Emergency MedicineKaiser East Bay and Kaiser Permanente NorthernCalifornia Division of ResearchOaklandCaliforniaUSA
| | - Dustin W. Ballard
- Department of Emergency MedicineKaiser San Rafael and Kaiser Permanente Northern California Division of ResearchOaklandCaliforniaUSA
| | - Mamata V. Kene
- Department of Emergency MedicineKaiser San Rafael and Kaiser Permanente Northern California Division of ResearchOaklandCaliforniaUSA
| | - David R. Vinson
- Department of Emergency MedicineRoseville, and Kaiser Permanente Northern California Division of ResearchOaklandCaliforniaUSA
| | - Mary E. Reed
- Kaiser Permanente Northern California Division of ResearchOaklandCaliforniaUSA
| |
Collapse
|
29
|
Lybarger K, Dobbins NJ, Long R, Singh A, Wedgeworth P, Uzuner Ö, Yetisgen M. Leveraging natural language processing to augment structured social determinants of health data in the electronic health record. J Am Med Inform Assoc 2023; 30:1389-1397. [PMID: 37130345 PMCID: PMC10354760 DOI: 10.1093/jamia/ocad073] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2022] [Revised: 04/06/2023] [Accepted: 04/12/2023] [Indexed: 05/04/2023] Open
Abstract
OBJECTIVE Social determinants of health (SDOH) impact health outcomes and are documented in the electronic health record (EHR) through structured data and unstructured clinical notes. However, clinical notes often contain more comprehensive SDOH information, detailing aspects such as status, severity, and temporality. This work has two primary objectives: (1) develop a natural language processing information extraction model to capture detailed SDOH information and (2) evaluate the information gain achieved by applying the SDOH extractor to clinical narratives and combining the extracted representations with existing structured data. MATERIALS AND METHODS We developed a novel SDOH extractor using a deep learning entity and relation extraction architecture to characterize SDOH across various dimensions. In an EHR case study, we applied the SDOH extractor to a large clinical data set with 225 089 patients and 430 406 notes with social history sections and compared the extracted SDOH information with existing structured data. RESULTS The SDOH extractor achieved 0.86 F1 on a withheld test set. In the EHR case study, we found extracted SDOH information complements existing structured data with 32% of homeless patients, 19% of current tobacco users, and 10% of drug users only having these health risk factors documented in the clinical narrative. CONCLUSIONS Utilizing EHR data to identify SDOH health risk factors and social needs may improve patient care and outcomes. Semantic representations of text-encoded SDOH information can augment existing structured data, and this more comprehensive SDOH representation can assist health systems in identifying and addressing these social needs.
Collapse
Affiliation(s)
- Kevin Lybarger
- Department of Information Sciences and Technology, George Mason University, Fairfax, Virginia, USA
| | - Nicholas J Dobbins
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, Washington, USA
- Department of Research IT, UW Medicine, University of Washington, Seattle, Washington, USA
| | - Ritche Long
- Department of Research IT, UW Medicine, University of Washington, Seattle, Washington, USA
| | - Angad Singh
- Department of Medicine, University of Washington, Seattle, Washington, USA
| | - Patrick Wedgeworth
- Department of Medicine, University of Washington, Seattle, Washington, USA
| | - Özlem Uzuner
- Department of Information Sciences and Technology, George Mason University, Fairfax, Virginia, USA
| | - Meliha Yetisgen
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, Washington, USA
| |
Collapse
|
30
|
Wehkamp K, Krawczak M, Schreiber S. The Quality and Utility of Artificial Intelligence in Patient Care. DEUTSCHES ARZTEBLATT INTERNATIONAL 2023; 120:463-469. [PMID: 37218054 PMCID: PMC10487679 DOI: 10.3238/arztebl.m2023.0124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Revised: 11/30/2022] [Accepted: 05/08/2023] [Indexed: 05/24/2023]
Abstract
BACKGROUND Artificial intelligence (AI) is increasingly being used in patient care. In the future, physicians will need to understand not only the basic functioning of AI applications, but also their quality, utility, and risks. METHODS This article is based on a selective review of the literature on the principles, quality, limitations, and benefits AI applications in patient care, along with examples of individual applications. RESULTS The number of AI applications in patient care is rising, with more than 500 approvals in the United States to date. Their quality and utility are based on a number of interdependent factors, including the real-life setting, the type and amount of data collected, the choice of variables used by the application, the algorithms used, and the goal and implementation of each application. Bias (which may be hidden) and errors can arise at all these levels. Any evaluation of the quality and utility of an AI application must, therefore, be conducted according to the scientific principles of evidence-based medicine-a requirement that is often hampered by a lack of transparency. CONCLUSION AI has the potential to improve patient care while meeting the challenge of dealing with an ever-increasing surfeit of information and data in medicine with limited human resources. The limitations and risks of AI applications require critical and responsible consideration. This can best be achieved through a combination of scientific.
Collapse
Affiliation(s)
- Kai Wehkamp
- Department of Internal Medicine I, University Medical Center Schleswig-Holstein, Campus Lübeck, Kiel, Germany
- Department for Medical Management, MSH Medical School Hamburg, Hamburg, Germany
| | - Michael Krawczak
- Institute of Medical Informatics and Statistics, Christian-Albrechts-University of Kiel, University Medical Center Schleswig-Holstein Campus Kiel, Germany
| | - Stefan Schreiber
- Department of Internal Medicine I, University Medical Center Schleswig-Holstein, Campus Lübeck, Kiel, Germany
- Institute of Clinical Molecular Biology, Christian-Albrechts-University of Kiel, University Medical Center Schleswig-Holstein Campus Kiel, Germany
| |
Collapse
|
31
|
Allen KS, Hood DR, Cummins J, Kasturi S, Mendonca EA, Vest JR. Natural language processing-driven state machines to extract social factors from unstructured clinical documentation. JAMIA Open 2023; 6:ooad024. [PMID: 37081945 PMCID: PMC10112959 DOI: 10.1093/jamiaopen/ooad024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 03/08/2023] [Accepted: 03/28/2023] [Indexed: 04/22/2023] Open
Abstract
Objective This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability. Materials and Methods Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity. Results PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Discussion We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics. Conclusion The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.
Collapse
Affiliation(s)
- Katie S Allen
- Corresponding Author: Katie S. Allen, BS, Center for Biomedical Informatics, Regenstrief Institute, Inc., 1101 W. 10th Street, Indianapolis, IN 46202, USA;
| | - Dan R Hood
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Jonathan Cummins
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Suranga Kasturi
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
| | - Eneida A Mendonca
- Division of Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio, USA
- Department of Pediatrics, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Joshua R Vest
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, USA
- Department of Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, USA
| |
Collapse
|
32
|
Wang Y, Yin C, Zhang P. Multimodal Risk Prediction with Physiological Signals, Medical Images and Clinical Notes. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.05.18.23290207. [PMID: 37293005 PMCID: PMC10246140 DOI: 10.1101/2023.05.18.23290207] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The broad adoption of electronic health records (EHRs) provides great opportunities to conduct healthcare research and solve various clinical problems in medicine. With recent advances and success, methods based on machine learning and deep learning have become increasingly popular in medical informatics. Combining data from multiple modalities may help in predictive tasks. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in Electronic Health Record (EHR) for enhanced performance in downstream predictive tasks. Early, joint, and late fusion strategies were employed to effectively combine data from various modalities. Model performance and contribution scores show that multimodal models outperform uni-modal models in various tasks. Additionally, temporal signs contain more information than CXR images and clinical notes in three explored predictive tasks. Therefore, models integrating different data modalities can work better in predictive tasks.
Collapse
Affiliation(s)
- Yuanlong Wang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio 43210, USA
| | - Changchang Yin
- Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio 43210, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, Ohio 43210 USA
| | - Ping Zhang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio 43210, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, Ohio 43210 USA
| |
Collapse
|
33
|
Gan Z, Zhou D, Rush E, Panickan VA, Ho YL, Ostrouchov G, Xu Z, Shen S, Xiong X, Greco KF, Hong C, Bonzel CL, Wen J, Costa L, Cai T, Begoli E, Xia Z, Gaziano JM, Liao KP, Cho K, Cai T, Lu J. ARCH: Large-scale Knowledge Graph via Aggregated Narrative Codified Health Records Analysis. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.05.14.23289955. [PMID: 37293026 PMCID: PMC10246054 DOI: 10.1101/2023.05.14.23289955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Objective Electronic health record (EHR) systems contain a wealth of clinical data stored as both codified data and free-text narrative notes, covering hundreds of thousands of clinical concepts available for research and clinical care. The complex, massive, heterogeneous, and noisy nature of EHR data imposes significant challenges for feature representation, information extraction, and uncertainty quantification. To address these challenges, we proposed an efficient Aggregated naRrative Codified Health (ARCH) records analysis to generate a large-scale knowledge graph (KG) for a comprehensive set of EHR codified and narrative features. Methods The ARCH algorithm first derives embedding vectors from a co-occurrence matrix of all EHR concepts and then generates cosine similarities along with associated p -values to measure the strength of relatedness between clinical features with statistical certainty quantification. In the final step, ARCH performs a sparse embedding regression to remove indirect linkage between entity pairs. We validated the clinical utility of the ARCH knowledge graph, generated from 12.5 million patients in the Veterans Affairs (VA) healthcare system, through downstream tasks including detecting known relationships between entity pairs, predicting drug side effects, disease phenotyping, as well as sub-typing Alzheimer's disease patients. Results ARCH produces high-quality clinical embeddings and KG for over 60,000 EHR concepts, as visualized in the R-shiny powered web-API (https://celehs.hms.harvard.edu/ARCH/). The ARCH embeddings attained an average area under the ROC curve (AUC) of 0.926 and 0.861 for detecting pairs of similar EHR concepts when the concepts are mapped to codified data and to NLP data; and 0.810 (codified) and 0.843 (NLP) for detecting related pairs. Based on the p -values computed by ARCH, the sensitivity of detecting similar and related entity pairs are 0.906 and 0.888 under false discovery rate (FDR) control of 5%. For detecting drug side effects, the cosine similarity based on the ARCH semantic representations achieved an AUC of 0.723 while the AUC improved to 0.826 after few-shot training via minimizing the loss function on the training data set. Incorporating NLP data substantially improved the ability to detect side effects in the EHR. For example, based on unsupervised ARCH embeddings, the power of detecting drug-side effects pairs when using codified data only was 0.15, much lower than the power of 0.51 when using both codified and NLP concepts. Compared to existing large-scale representation learning methods including PubmedBERT, BioBERT and SAPBERT, ARCH attains the most robust performance and substantially higher accuracy in detecting these relationships. Incorporating ARCH selected features in weakly supervised phenotyping algorithms can improve the robustness of algorithm performance, especially for diseases that benefit from NLP features as supporting evidence. For example, the phenotyping algorithm for depression attained an AUC of 0.927 when using ARCH selected features but only 0.857 when using codified features selected via the KESER network[1]. In addition, embeddings and knowledge graphs generated from the ARCH network were able to cluster AD patients into two subgroups, where the fast progression subgroup had a much higher mortality rate. Conclusions The proposed ARCH algorithm generates large-scale high-quality semantic representations and knowledge graph for both codified and NLP EHR features, useful for a wide range of predictive modeling tasks.
Collapse
Affiliation(s)
| | - Doudou Zhou
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Everett Rush
- Oak Ridge National Laboratory, Oak Ridge, TN USA
| | - Vidul A Panickan
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Yuk-Lam Ho
- VA Boston Healthcare System, Boston, MA, USA
| | | | - Zhiwei Xu
- University of Michigan, Ann Arbor, MI, USA
| | - Shuting Shen
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Xin Xiong
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | | | | | - Clara-Lea Bonzel
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Jun Wen
- Harvard Medical School, Boston, MA, USA
| | | | - Tianrun Cai
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Edmon Begoli
- Oak Ridge National Laboratory, Oak Ridge, TN USA
| | - Zongqi Xia
- University of Pittsburgh, Pittsburgh, USA
| | - J Michael Gaziano
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Katherine P Liao
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Kelly Cho
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Tianxi Cai
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Junwei Lu
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| |
Collapse
|
34
|
González-Castro L, Chávez M, Duflot P, Bleret V, Martin AG, Zobel M, Nateqi J, Lin S, Pazos-Arias JJ, Del Fiol G, López-Nores M. Machine Learning Algorithms to Predict Breast Cancer Recurrence Using Structured and Unstructured Sources from Electronic Health Records. Cancers (Basel) 2023; 15:2741. [PMID: 37345078 DOI: 10.3390/cancers15102741] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Revised: 04/26/2023] [Accepted: 05/06/2023] [Indexed: 06/23/2023] Open
Abstract
Recurrence is a critical aspect of breast cancer (BC) that is inexorably tied to mortality. Reuse of healthcare data through Machine Learning (ML) algorithms offers great opportunities to improve the stratification of patients at risk of cancer recurrence. We hypothesized that combining features from structured and unstructured sources would provide better prediction results for 5-year cancer recurrence than either source alone. We collected and preprocessed clinical data from a cohort of BC patients, resulting in 823 valid subjects for analysis. We derived three sets of features: structured information, features from free text, and a combination of both. We evaluated the performance of five ML algorithms to predict 5-year cancer recurrence and selected the best-performing to test our hypothesis. The XGB (eXtreme Gradient Boosting) model yielded the best performance among the five evaluated algorithms, with precision = 0.900, recall = 0.907, F1-score = 0.897, and area under the receiver operating characteristic AUROC = 0.807. The best prediction results were achieved with the structured dataset, followed by the unstructured dataset, while the combined dataset achieved the poorest performance. ML algorithms for BC recurrence prediction are valuable tools to improve patient risk stratification, help with post-cancer monitoring, and plan more effective follow-up. Structured data provides the best results when fed to ML algorithms. However, an approach based on natural language processing offers comparable results while potentially requiring less mapping effort.
Collapse
Affiliation(s)
| | - Marcela Chávez
- Department of Information System Management, Centre Hospitalier Universitaire de Liège, 4000 Liège, Belgium
| | - Patrick Duflot
- Department of Information System Management, Centre Hospitalier Universitaire de Liège, 4000 Liège, Belgium
| | - Valérie Bleret
- Senology Department, Centre Hospitalier Universitaire de Liège, 4000 Liège, Belgium
| | | | - Marc Zobel
- Science Department, Symptoma GmbH, 1030 Vienna, Austria
| | - Jama Nateqi
- Science Department, Symptoma GmbH, 1030 Vienna, Austria
- Department of Internal Medicine, Paracelsus Medical University, 5020 Salzburg, Austria
| | - Simon Lin
- Science Department, Symptoma GmbH, 1030 Vienna, Austria
- Department of Internal Medicine, Paracelsus Medical University, 5020 Salzburg, Austria
| | - José J Pazos-Arias
- atlanTTic Research Center, Department of Telematics Engineering, University of Vigo, 36310 Vigo, Spain
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah School of Medicine, Salt Lake City, UT 84108, USA
| | - Martín López-Nores
- atlanTTic Research Center, Department of Telematics Engineering, University of Vigo, 36310 Vigo, Spain
| |
Collapse
|
35
|
Uddin Y, Nair A, Shariq S, Hannan SH. Transforming primary healthcare through natural language processing and big data analytics. BMJ 2023; 381:948. [PMID: 37137492 DOI: 10.1136/bmj.p948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
|
36
|
Pham TH, Yin C, Mehta L, Zhang X, Zhang P. A fair and interpretable network for clinical risk prediction: a regularized multi-view multi-task learning approach. Knowl Inf Syst 2023; 65:1487-1521. [PMID: 36998311 PMCID: PMC10046420 DOI: 10.1007/s10115-022-01813-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 12/06/2022] [Accepted: 12/12/2022] [Indexed: 12/24/2022]
Abstract
In healthcare domain, complication risk profiling which can be seen as multiple clinical risk prediction tasks is challenging due to the complex interaction between heterogeneous clinical entities. With the availability of real-world data, many deep learning methods are proposed for complication risk profiling. However, the existing methods face three open challenges. First, they leverage clinical data from a single view and then lead to suboptimal models. Second, most existing methods lack an effective mechanism to interpret predictions. Third, models learned from clinical data may have inherent pre-existing biases and exhibit discrimination against certain social groups. We then propose a multi-view multi-task network (MuViTaNet) to tackle these issues. MuViTaNet complements patient representation by using a multi-view encoder to exploit more information. Moreover, it uses a multi-task learning to generate more generalized representations using both labeled and unlabeled datasets. Last, a fairness variant (F-MuViTaNet) is proposed to mitigate the unfairness issues and promote healthcare equity. The experiments show that MuViTaNet outperforms existing methods for cardiac complication profiling. Its architecture also provides an effective mechanism for interpreting the predictions, which helps clinicians discover the underlying mechanism triggering the complication onsets. F-MuViTaNet can also effectively mitigate the unfairness with only negligible impact on accuracy.
Collapse
Affiliation(s)
- Thai-Hoang Pham
- Department of Computer Science and Engineering, The Ohio State University, Columbus, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, USA
| | - Changchang Yin
- Department of Computer Science and Engineering, The Ohio State University, Columbus, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, USA
| | - Laxmi Mehta
- Division of Cardiology, Department of Medicine, The Ohio State University, Columbus, USA
| | - Xueru Zhang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, USA
| | - Ping Zhang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, USA
- Department of Biomedical Informatics, The Ohio State University, Columbus, USA
| |
Collapse
|
37
|
Chiu CC, Wu CM, Chien TN, Kao LJ, Li C, Chu CM. Integrating Structured and Unstructured EHR Data for Predicting Mortality by Machine Learning and Latent Dirichlet Allocation Method. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:4340. [PMID: 36901354 PMCID: PMC10001457 DOI: 10.3390/ijerph20054340] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Revised: 02/22/2023] [Accepted: 02/24/2023] [Indexed: 06/18/2023]
Abstract
An ICU is a critical care unit that provides advanced medical support and continuous monitoring for patients with severe illnesses or injuries. Predicting the mortality rate of ICU patients can not only improve patient outcomes, but also optimize resource allocation. Many studies have attempted to create scoring systems and models that predict the mortality of ICU patients using large amounts of structured clinical data. However, unstructured clinical data recorded during patient admission, such as notes made by physicians, is often overlooked. This study used the MIMIC-III database to predict mortality in ICU patients. In the first part of the study, only eight structured variables were used, including the six basic vital signs, the GCS, and the patient's age at admission. In the second part, unstructured predictor variables were extracted from the initial diagnosis made by physicians when the patients were admitted to the hospital and analyzed using Latent Dirichlet Allocation techniques. The structured and unstructured data were combined using machine learning methods to create a mortality risk prediction model for ICU patients. The results showed that combining structured and unstructured data improved the accuracy of the prediction of clinical outcomes in ICU patients over time. The model achieved an AUROC of 0.88, indicating accurate prediction of patient vital status. Additionally, the model was able to predict patient clinical outcomes over time, successfully identifying important variables. This study demonstrated that a small number of easily collectible structured variables, combined with unstructured data and analyzed using LDA topic modeling, can significantly improve the predictive performance of a mortality risk prediction model for ICU patients. These results suggest that initial clinical observations and diagnoses of ICU patients contain valuable information that can aid ICU medical and nursing staff in making important clinical decisions.
Collapse
Affiliation(s)
- Chih-Chou Chiu
- Department of Business Management, National Taipei University of Technology, Taipei 106, Taiwan
| | - Chung-Min Wu
- Department of Business Management, National Taipei University of Technology, Taipei 106, Taiwan
| | - Te-Nien Chien
- College of Management, National Taipei University of Technology, Taipei 106, Taiwan
| | - Ling-Jing Kao
- Department of Business Management, National Taipei University of Technology, Taipei 106, Taiwan
| | - Chengcheng Li
- College of Management, National Taipei University of Technology, Taipei 106, Taiwan
| | - Chuan-Mei Chu
- College of Management, National Taipei University of Technology, Taipei 106, Taiwan
| |
Collapse
|
38
|
Long J, Wang M, Li W, Cheng J, Yuan M, Zhong M, Zhang Z, Zhang C. The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis. Intensive Crit Care Nurs 2023; 76:103378. [PMID: 36805167 DOI: 10.1016/j.iccn.2022.103378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2022] [Revised: 12/07/2022] [Accepted: 12/13/2022] [Indexed: 02/17/2023]
Abstract
OBJECTIVE To review and evaluate existing risk assessment tools for intensive care unitreadmission. METHODS Nine electronic databases (Medline, CINAHL, Web of Science, Cochrane Library, Embase, Sino Med, CNKI, VIP, and Wan fang) were systematically searched from their inception to September 2022. Two authors independently extracted data from the literature included. Meta-analysis was performed under the bivariate modeling and summary receiver operating characteristic curve method. RESULTS A total of 29 studies were included in this review, among which 11 were quantitatively Meta-analyzed. The results showed Stability and Workload Index for Transfer: Sensitivity = 0.55, Specificity = 0.65, Area under curve = 0.63. And Early warning score: Sensitivity = 0.78, Specificity = 0.83, Area under curve = 0.88. The remaining tools included scores, nomograms, machine learning models, and deep learning models. These studies, with varying reports on thresholds, case selection, data preprocessing, and model performance, have a high risk of bias. CONCLUSION We cannot identify a tool that can be used directly in intensive care unit readmission risk assessment. Scores based on early warning score are moderately accurate in predicting readmission, but there is heterogeneity and publication bias that requires model adjustment for local factors such as resources, demographics, and case mix. Machine learning models present a promising modeling technique but have a high methodological bias and require further validation. IMPLICATIONS FOR CLINICAL PRACTICE Using reliable risk assessment tools is essential for the early identification of unplanned intensive care unit readmission risk in critically ill patients. A reliable risk assessment tool must be developed, which is the focus of further research.
Collapse
Affiliation(s)
- Jianying Long
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Min Wang
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Wenrui Li
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Jie Cheng
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Mengyuan Yuan
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Mingming Zhong
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China
| | - Zhigang Zhang
- Department of Critical Care Medicine, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China; School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China.
| | - Caiyun Zhang
- School of Nursing, Lanzhou University, Lanzhou, Gansu 730000, PR China; Outpatient Department, The First Hospital of Lanzhou University, Lanzhou, Gansu 730000, PR China.
| |
Collapse
|
39
|
Tang S, Tariq A, Dunnmon JA, Sharma U, Elugunti P, Rubin DL, Patel BN, Banerjee I. Predicting 30-day all-cause hospital readmission using multimodal spatiotemporal graph neural networks. IEEE J Biomed Health Inform 2023; PP:10.1109/JBHI.2023.3236888. [PMID: 37018684 PMCID: PMC11073780 DOI: 10.1109/jbhi.2023.3236888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
Reduction in 30-day readmission rate is an important quality factor for hospitals as it can reduce the overall cost of care and improve patient post-discharge outcomes. While deep-learning-based studies have shown promising empirical results, several limitations exist in prior models for hospital readmission prediction, such as: (a) only patients with certain conditions are considered, (b) do not leverage data temporality, (c) individual admissions are assumed independent of each other, which ignores patient similarity, (d) limited to single modality or single center data. In this study, we propose a multimodal, spatiotemporal graph neural network (MM-STGNN) for prediction of 30-day all-cause hospital readmission, which fuses in-patient multimodal, longitudinal data and models patient similarity using a graph. Using longitudinal chest radiographs and electronic health records from two independent centers, we show that MM-STGNN achieved an area under the receiver operating characteristic curve (AUROC) of 0.79 on both datasets. Furthermore, MM-STGNN significantly outperformed the current clinical reference standard, LACE+ (AUROC=0.61), on the internal dataset. For subset populations of patients with heart disease, our model significantly outperformed baselines, such as gradient-boosting and Long Short-Term Memory models (e.g., AUROC improved by 3.7 points in patients with heart disease). Qualitative interpretability analysis indicated that while patients' primary diagnoses were not explicitly used to train the model, features crucial for model prediction may reflect patients' diagnoses. Our model could be utilized as an additional clinical decision aid during discharge disposition and triaging high-risk patients for closer post-discharge follow-up for potential preventive measures.
Collapse
|
40
|
Natural Language Processing Applications for Computer-Aided Diagnosis in Oncology. Diagnostics (Basel) 2023; 13:diagnostics13020286. [PMID: 36673096 PMCID: PMC9857980 DOI: 10.3390/diagnostics13020286] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2022] [Revised: 12/24/2022] [Accepted: 01/05/2023] [Indexed: 01/15/2023] Open
Abstract
In the era of big data, text-based medical data, such as electronic health records (EHR) and electronic medical records (EMR), are growing rapidly. EHR and EMR are collected from patients to record their basic information, lab tests, vital signs, clinical notes, and reports. EHR and EMR contain the helpful information to assist oncologists in computer-aided diagnosis and decision making. However, it is time consuming for doctors to extract the valuable information they need and analyze the information from the EHR and EMR data. Recently, more and more research works have applied natural language processing (NLP) techniques, i.e., rule-based, machine learning-based, and deep learning-based techniques, on the EHR and EMR data for computer-aided diagnosis in oncology. The objective of this review is to narratively review the recent progress in the area of NLP applications for computer-aided diagnosis in oncology. Moreover, we intend to reduce the research gap between artificial intelligence (AI) experts and clinical specialists to design better NLP applications. We originally identified 295 articles from the three electronic databases: PubMed, Google Scholar, and ACL Anthology; then, we removed the duplicated papers and manually screened the irrelevant papers based on the content of the abstract; finally, we included a total of 23 articles after the screening process of the literature review. Furthermore, we provided an in-depth analysis and categorized these studies into seven cancer types: breast cancer, lung cancer, liver cancer, prostate cancer, pancreatic cancer, colorectal cancer, and brain tumors. Additionally, we identified the current limitations of NLP applications on supporting the clinical practices and we suggest some promising future research directions in this paper.
Collapse
|
41
|
Jujjavarapu C, Suri P, Pejaver V, Friedly J, Gold LS, Meier E, Cohen T, Mooney SD, Heagerty PJ, Jarvik JG. Predicting decompression surgery by applying multimodal deep learning to patients' structured and unstructured health data. BMC Med Inform Decis Mak 2023; 23:2. [PMID: 36609379 PMCID: PMC9824905 DOI: 10.1186/s12911-022-02096-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 12/29/2022] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND Low back pain (LBP) is a common condition made up of a variety of anatomic and clinical subtypes. Lumbar disc herniation (LDH) and lumbar spinal stenosis (LSS) are two subtypes highly associated with LBP. Patients with LDH/LSS are often started with non-surgical treatments and if those are not effective then go on to have decompression surgery. However, recommendation of surgery is complicated as the outcome may depend on the patient's health characteristics. We developed a deep learning (DL) model to predict decompression surgery for patients with LDH/LSS. MATERIALS AND METHOD We used datasets of 8387 and 8620 patients from a prospective study that collected data from four healthcare systems to predict early (within 2 months) and late surgery (within 12 months after a 2 month gap), respectively. We developed a DL model to use patients' demographics, diagnosis and procedure codes, drug names, and diagnostic imaging reports to predict surgery. For each prediction task, we evaluated the model's performance using classical and generalizability evaluation. For classical evaluation, we split the data into training (80%) and testing (20%). For generalizability evaluation, we split the data based on the healthcare system. We used the area under the curve (AUC) to assess performance for each evaluation. We compared results to a benchmark model (i.e. LASSO logistic regression). RESULTS For classical performance, the DL model outperformed the benchmark model for early surgery with an AUC of 0.725 compared to 0.597. For late surgery, the DL model outperformed the benchmark model with an AUC of 0.655 compared to 0.635. For generalizability performance, the DL model outperformed the benchmark model for early surgery. For late surgery, the benchmark model outperformed the DL model. CONCLUSIONS For early surgery, the DL model was preferred for classical and generalizability evaluation. However, for late surgery, the benchmark and DL model had comparable performance. Depending on the prediction task, the balance of performance may shift between DL and a conventional ML method. As a result, thorough assessment is needed to quantify the value of DL, a relatively computationally expensive, time-consuming and less interpretable method.
Collapse
Affiliation(s)
- Chethan Jujjavarapu
- Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, Box 358047, Seattle, WA, 98195, USA
| | - Pradeep Suri
- Clinical Learning, Evidence and Research Center, University of Washington, 4333 Brooklyn Ave NE, Seattle, WA, 98105, USA
- Department of Rehabilitation Medicine, University of Washington, 1959 NE Pacific St, Seattle, WA, 98195, USA
| | - Vikas Pejaver
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, 10029, USA
| | - Janna Friedly
- Clinical Learning, Evidence and Research Center, University of Washington, 4333 Brooklyn Ave NE, Seattle, WA, 98105, USA
- Department of Rehabilitation Medicine, University of Washington, 1959 NE Pacific St, Seattle, WA, 98195, USA
| | - Laura S Gold
- Clinical Learning, Evidence and Research Center, University of Washington, 4333 Brooklyn Ave NE, Seattle, WA, 98105, USA
- Department of Radiology, University of Washington, 1959 NE Pacific Street, Seattle, WA, 98195, USA
| | - Eric Meier
- Clinical Learning, Evidence and Research Center, University of Washington, 4333 Brooklyn Ave NE, Seattle, WA, 98105, USA
- Department of Biostatistics, University of Washington, Box 357232, Seattle, WA, 98195-7232, USA
- Center for Biomedical Statistics, University of Washington, Seattle, WA, USA
| | - Trevor Cohen
- Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, Box 358047, Seattle, WA, 98195, USA
| | - Sean D Mooney
- Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington, Box 358047, Seattle, WA, 98195, USA
| | - Patrick J Heagerty
- Department of Biostatistics, University of Washington, Box 357232, Seattle, WA, 98195-7232, USA
- Center for Biomedical Statistics, University of Washington, Seattle, WA, USA
| | - Jeffrey G Jarvik
- Clinical Learning, Evidence and Research Center, University of Washington, 4333 Brooklyn Ave NE, Seattle, WA, 98105, USA.
- Department of Radiology, University of Washington, 1959 NE Pacific Street, Seattle, WA, 98195, USA.
- Department of Neurological Surgery, University of Washington, 1959 NE Pacific Street, Seattle, WA, 98195, USA.
- Department of Health Services, University of Washington, Box 357660, Seattle, WA, 98195-7660, USA.
| |
Collapse
|
42
|
Khairuddin MZF, Hasikin K, Razak NAA, Mohshim SA, Ibrahim SS. Harnessing the Multimodal Data Integration and Deep Learning for Occupational Injury Severity Prediction. IEEE ACCESS 2023; 11:85284-85302. [DOI: 10.1109/access.2023.3304328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Affiliation(s)
| | - Khairunnisa Hasikin
- Department of Biomedical Engineering, Faculty of Engineering, University Malaya, Kuala Lumpur, Malaysia
| | - Nasrul Anuar Abd Razak
- Department of Biomedical Engineering, Faculty of Engineering, University Malaya, Kuala Lumpur, Malaysia
| | - Siti Afifah Mohshim
- Medical Engineering Technology Section, British Malaysian Institute, Universiti Kuala Lumpur, Kuala Lumpur, Selangor, Malaysia
| | - Siti Salwa Ibrahim
- Negeri Sembilan State Health Department, Ministry of Health, Seremban, Negeri Sembilan, Malaysia
| |
Collapse
|
43
|
Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, Dong H, Poon MTC, Fitzpatrick N, Levine AP, Slater LT, Handy A, Karwath A, Gkoutos GV, Chelala C, Shah AD, Stewart R, Collier N, Alex B, Whiteley W, Sudlow C, Roberts A, Dobson RJB. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med 2022; 5:186. [PMID: 36544046 PMCID: PMC9770568 DOI: 10.1038/s41746-022-00730-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.
Collapse
Affiliation(s)
- Honghan Wu
- Institute of Health Informatics, University College London, London, UK.
| | - Minhong Wang
- Institute of Health Informatics, University College London, London, UK
| | - Jinge Wu
- Institute of Health Informatics, University College London, London, UK
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Farah Francis
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Yun-Hsuan Chang
- Institute of Health Informatics, University College London, London, UK
| | - Alex Shavick
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Hang Dong
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Department of Computer Science, University of Oxford, Oxford, UK
| | | | | | - Adam P Levine
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Luke T Slater
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Alex Handy
- Institute of Health Informatics, University College London, London, UK
- University College London Hospitals NHS Trust, London, UK
| | - Andreas Karwath
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Georgios V Gkoutos
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Claude Chelala
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Anoop Dinesh Shah
- Institute of Health Informatics, University College London, London, UK
| | - Robert Stewart
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK
- South London and Maudsley NHS Foundation Trust, London, UK
| | - Nigel Collier
- Theoretical and Applied Linguistics, Faculty of Modern & Medieval Languages & Linguistics, University of Cambridge, Cambridge, UK
| | - Beatrice Alex
- Edinburgh Futures Institute, University of Edinburgh, Edinburgh, UK
| | | | - Cathie Sudlow
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Angus Roberts
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London, UK
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| |
Collapse
|
44
|
Zhang X, Gavaldà R, Baixeries J. Interpretable prediction of mortality in liver transplant recipients based on machine learning. Comput Biol Med 2022; 151:106188. [PMID: 36306583 DOI: 10.1016/j.compbiomed.2022.106188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2022] [Revised: 09/24/2022] [Accepted: 10/08/2022] [Indexed: 12/27/2022]
Abstract
BACKGROUND Accurate prediction of the mortality of post-liver transplantation is an important but challenging task. It relates to optimizing organ allocation and estimating the risk of possible dysfunction. Existing risk scoring models, such as the Balance of Risk (BAR) score and the Survival Outcomes Following Liver Transplantation (SOFT) score, do not predict the mortality of post-liver transplantation with sufficient accuracy. In this study, we evaluate the performance of machine learning models and establish an explainable machine learning model for predicting mortality in liver transplant recipients. METHOD The optimal feature set for the prediction of the mortality was selected by a wrapper method based on binary particle swarm optimization (BPSO). With the selected optimal feature set, seven machine learning models were applied to predict mortality over different time windows. The best-performing model was used to predict mortality through a comprehensive comparison and evaluation. An interpretable approach based on machine learning and SHapley Additive exPlanations (SHAP) is used to explicitly explain the model's decision and make new discoveries. RESULTS With regard to predictive power, our results demonstrated that the feature set selected by BPSO outperformed both the feature set in the existing risk score model (BAR score, SOFT score) and the feature set processed by principal component analysis (PCA). The best-performing model, extreme gradient boosting (XGBoost), was found to improve the Area Under a Curve (AUC) values for mortality prediction by 6.7%, 11.6%, and 17.4% at 3 months, 3 years, and 10 years, respectively, compared to the SOFT score. The main predictors of mortality and their impact were discussed for different age groups and different follow-up periods. CONCLUSIONS Our analysis demonstrates that XGBoost can be an ideal method to assess the mortality risk in liver transplantation. In combination with the SHAP approach, the proposed framework provides a more intuitive and comprehensive interpretation of the predictive model, thereby allowing the clinician to better understand the decision-making process of the model and the impact of factors associated with mortality risk in liver transplantation.
Collapse
Affiliation(s)
- Xiao Zhang
- Department of Computer Science, Universitat Politècnica de Catalunya, Barcelona, 08034, Spain.
| | | | - Jaume Baixeries
- Department of Computer Science, Universitat Politècnica de Catalunya, Barcelona, 08034, Spain
| |
Collapse
|
45
|
Kline A, Wang H, Li Y, Dennis S, Hutch M, Xu Z, Wang F, Cheng F, Luo Y. Multimodal machine learning in precision health: A scoping review. NPJ Digit Med 2022; 5:171. [PMID: 36344814 PMCID: PMC9640667 DOI: 10.1038/s41746-022-00712-8] [Citation(s) in RCA: 74] [Impact Index Per Article: 37.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 10/14/2022] [Indexed: 11/09/2022] Open
Abstract
Machine learning is frequently being leveraged to tackle problems in the health sector including utilization for clinical decision-support. Its use has historically been focused on single modal data. Attempts to improve prediction and mimic the multimodal nature of clinical expert decision-making has been met in the biomedical field of machine learning by fusing disparate data. This review was conducted to summarize the current studies in this field and identify topics ripe for future research. We conducted this review in accordance with the PRISMA extension for Scoping Reviews to characterize multi-modal data fusion in health. Search strings were established and used in databases: PubMed, Google Scholar, and IEEEXplore from 2011 to 2021. A final set of 128 articles were included in the analysis. The most common health areas utilizing multi-modal methods were neurology and oncology. Early fusion was the most common data merging strategy. Notably, there was an improvement in predictive performance when using data fusion. Lacking from the papers were clear clinical deployment strategies, FDA-approval, and analysis of how using multimodal approaches from diverse sub-populations may improve biases and healthcare disparities. These findings provide a summary on multimodal data fusion as applied to health diagnosis/prognosis problems. Few papers compared the outputs of a multimodal approach with a unimodal prediction. However, those that did achieved an average increase of 6.4% in predictive accuracy. Multi-modal machine learning, while more robust in its estimations over unimodal methods, has drawbacks in its scalability and the time-consuming nature of information concatenation.
Collapse
Affiliation(s)
- Adrienne Kline
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Hanyin Wang
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Yikuan Li
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Saya Dennis
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Meghan Hutch
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA
| | - Zhenxing Xu
- Department of Population Health Sciences, Cornell University, New York, 10065, NY, USA
| | - Fei Wang
- Department of Population Health Sciences, Cornell University, New York, 10065, NY, USA
| | - Feixiong Cheng
- Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, 44195, OH, USA
| | - Yuan Luo
- Department of Preventive Medicine, Northwestern University, Chicago, 60201, IL, USA.
| |
Collapse
|
46
|
Rabii KB, Javaid W, Nabeel I. Development and implementation of centralised, cloud-based, employee health contact tracing database and predictive modelling framework in the COVID-19 pandemic. Lancet Digit Health 2022; 4:e770-e772. [PMID: 36307190 PMCID: PMC9597572 DOI: 10.1016/s2589-7500(22)00171-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 07/13/2022] [Accepted: 08/21/2022] [Indexed: 12/05/2022]
Affiliation(s)
- Kristine Bahareh Rabii
- Department of Infection Prevention and Control, Mount Sinai Beth Israel, New York, NY 10003, USA.
| | - Waleed Javaid
- Department of Infection Prevention and Control, Mount Sinai Beth Israel, New York, NY 10003, USA; Division of Infectious Diseases, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Ismail Nabeel
- Department of Environmental Medicine and Public Health, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
47
|
Xie J, Wang Z, Yu Z, Guo B. Enabling Timely Medical Intervention by Exploring Health-Related Multivariate Time Series with a Hybrid Attentive Model. SENSORS (BASEL, SWITZERLAND) 2022; 22:6104. [PMID: 36015865 PMCID: PMC9414519 DOI: 10.3390/s22166104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 08/03/2022] [Accepted: 08/10/2022] [Indexed: 06/15/2023]
Abstract
Modern healthcare practice, especially in intensive care units, produces a vast amount of multivariate time series of health-related data, e.g., multi-lead electrocardiogram (ECG), pulse waveform, blood pressure waveform and so on. As a result, timely and accurate prediction of medical intervention (e.g., intravenous injection) becomes possible, by exploring such semantic-rich time series. Existing works mainly focused on onset prediction at the granularity of hours that was not suitable for medication intervention in emergency medicine. This research proposes a Multi-Variable Hybrid Attentive Model (MVHA) to predict the impending need of medical intervention, by jointly mining multiple time series. Specifically, a two-level attention mechanism is designed to capture the pattern of fluctuations and trends of different time series. This work applied MVHA to the prediction of the impending intravenous injection need of critical patients at the intensive care units. Experiments on the MIMIC Waveform Database demonstrated that the proposed model achieves a prediction accuracy of 0.8475 and an ROC-AUC of 0.8318, which significantly outperforms baseline models.
Collapse
Affiliation(s)
| | - Zhu Wang
- School of Computer Science, Northwestern Polytechnical University, 1 Dongxiang Road, Chang’an District, Xi’an 710129, China
| | | | | |
Collapse
|
48
|
Liu J, Capurro D, Nguyen A, Verspoor K. "Note Bloat" impacts deep learning-based NLP models for clinical prediction tasks. J Biomed Inform 2022; 133:104149. [PMID: 35878821 DOI: 10.1016/j.jbi.2022.104149] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/28/2022] [Accepted: 07/19/2022] [Indexed: 10/17/2022]
Abstract
One unintended consequence of the Electronic Health Records (EHR) implementation is the overuse of content-importing technology, such as copy-and-paste, that creates "bloated" notes containing large amounts of textual redundancy. Despite the rising interest in applying machine learning models to learn from real-patient data, it is unclear how the phenomenon of note bloat might affect the Natural Language Processing (NLP) models derived from these notes. Therefore, in this work we examine the impact of redundancy on deep learning-based NLP models, considering four clinical prediction tasks using a publicly available EHR database. We applied two deduplication methods to the hospital notes, identifying large quantities of redundancy, and found that removing the redundancy usually has little negative impact on downstream performances, and can in certain circumstances assist models to achieve significantly better results. We also showed it is possible to attack model predictions by simply adding note duplicates, causing changes of correct predictions made by trained models into wrong predictions. In conclusion, we demonstrated that EHR text redundancy substantively affects NLP models for clinical prediction tasks, showing that the awareness of clinical contexts and robust modeling methods are important to create effective and reliable NLP systems in healthcare contexts.
Collapse
Affiliation(s)
- Jinghui Liu
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Australian e-Health Research Centre, CSIRO, Brisbane, Australia.
| | - Daniel Capurro
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, Melbourne Medical School, The University of Melbourne, Victoria, Australia.
| | - Anthony Nguyen
- Australian e-Health Research Centre, CSIRO, Brisbane, Australia.
| | - Karin Verspoor
- School of Computing and Information Systems, The University of Melbourne, Victoria, Australia; Centre for Digital Transformation of Health, Melbourne Medical School, The University of Melbourne, Victoria, Australia; School of Computing Technologies, RMIT University, Victoria, Australia.
| |
Collapse
|
49
|
Hernandez M, Epelde G, Alberdi A, Cilla R, Rankin D. Synthetic data generation for tabular health records: A systematic review. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.04.053] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
50
|
Chen PF, Chen L, Lin YK, Li GH, Lai F, Lu CW, Yang CY, Chen KC, Lin TY. Predicting Postoperative Mortality With Deep Neural Networks and Natural Language Processing: Model Development and Validation. JMIR Med Inform 2022; 10:e38241. [PMID: 35536634 PMCID: PMC9131148 DOI: 10.2196/38241] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 04/18/2022] [Accepted: 04/26/2022] [Indexed: 11/23/2022] Open
Abstract
Background Machine learning (ML) achieves better predictions of postoperative mortality than previous prediction tools. Free-text descriptions of the preoperative diagnosis and the planned procedure are available preoperatively. Because reading these descriptions helps anesthesiologists evaluate the risk of the surgery, we hypothesized that deep learning (DL) models with unstructured text could improve postoperative mortality prediction. However, it is challenging to extract meaningful concept embeddings from this unstructured clinical text. Objective This study aims to develop a fusion DL model containing structured and unstructured features to predict the in-hospital 30-day postoperative mortality before surgery. ML models for predicting postoperative mortality using preoperative data with or without free clinical text were assessed. Methods We retrospectively collected preoperative anesthesia assessments, surgical information, and discharge summaries of patients undergoing general and neuraxial anesthesia from electronic health records (EHRs) from 2016 to 2020. We first compared the deep neural network (DNN) with other models using the same input features to demonstrate effectiveness. Then, we combined the DNN model with bidirectional encoder representations from transformers (BERT) to extract information from clinical texts. The effects of adding text information on the model performance were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Statistical significance was evaluated using P<.05. Results The final cohort contained 121,313 patients who underwent surgeries. A total of 1562 (1.29%) patients died within 30 days of surgery. Our BERT-DNN model achieved the highest AUROC (0.964, 95% CI 0.961-0.967) and AUPRC (0.336, 95% CI 0.276-0.402). The AUROC of the BERT-DNN was significantly higher compared to logistic regression (AUROC=0.952, 95% CI 0.949-0.955) and the American Society of Anesthesiologist Physical Status (ASAPS AUROC=0.892, 95% CI 0.887-0.896) but not significantly higher compared to the DNN (AUROC=0.959, 95% CI 0.956-0.962) and the random forest (AUROC=0.961, 95% CI 0.958-0.964). The AUPRC of the BERT-DNN was significantly higher compared to the DNN (AUPRC=0.319, 95% CI 0.260-0.384), the random forest (AUPRC=0.296, 95% CI 0.239-0.360), logistic regression (AUPRC=0.276, 95% CI 0.220-0.339), and the ASAPS (AUPRC=0.149, 95% CI 0.107-0.203). Conclusions Our BERT-DNN model has an AUPRC significantly higher compared to previously proposed models using no text and an AUROC significantly higher compared to logistic regression and the ASAPS. This technique helps identify patients with higher risk from the surgical description text in EHRs.
Collapse
Affiliation(s)
- Pei-Fu Chen
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan.,Department of Anesthesiology, Far Eastern Memorial Hospital, New Taipei City, Taiwan
| | - Lichin Chen
- Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan
| | - Yow-Kuan Lin
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan.,Department of Computer Science, Columbia University, New York, NY, United States
| | - Guo-Hung Li
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan
| | - Feipei Lai
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan.,Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan.,Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan
| | - Cheng-Wei Lu
- Department of Anesthesiology, Far Eastern Memorial Hospital, New Taipei City, Taiwan.,Department of Mechanical Engineering, Yuan Ze University, Taoyuan, Taiwan
| | - Chi-Yu Yang
- Department of Information Technology, Far Eastern Memorial Hospital, New Taipei City, Taiwan.,Section of Cardiovascular Medicine, Cardiovascular Center, Far Eastern Memorial Hospital, New Taipei City, Taiwan
| | - Kuan-Chih Chen
- Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Taipei, Taiwan.,Department of Internal Medicine, Far Eastern Memorial Hospital, New Taipei City, Taiwan
| | - Tzu-Yu Lin
- Department of Anesthesiology, Far Eastern Memorial Hospital, New Taipei City, Taiwan.,Department of Mechanical Engineering, Yuan Ze University, Taoyuan, Taiwan
| |
Collapse
|