1
|
Jeon K, Park WY, Kahn CE, Nagy P, You SC, Yoon SH. Advancing Medical Imaging Research Through Standardization: The Path to Rapid Development, Rigorous Validation, and Robust Reproducibility. Invest Radiol 2024:00004424-990000000-00232. [PMID: 38985896 DOI: 10.1097/rli.0000000000001106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/12/2024]
Abstract
ABSTRACT Artificial intelligence (AI) has made significant advances in radiology. Nonetheless, challenges in AI development, validation, and reproducibility persist, primarily due to the lack of high-quality, large-scale, standardized data across the world. Addressing these challenges requires comprehensive standardization of medical imaging data and seamless integration with structured medical data.Developed by the Observational Health Data Sciences and Informatics community, the OMOP Common Data Model enables large-scale international collaborations with structured medical data. It ensures syntactic and semantic interoperability, while supporting the privacy-protected distribution of research across borders. The recently proposed Medical Imaging Common Data Model is designed to encompass all DICOM-formatted medical imaging data and integrate imaging-derived features with clinical data, ensuring their provenance.The harmonization of medical imaging data and its seamless integration with structured clinical data at a global scale will pave the way for advanced AI research in radiology. This standardization will enable federated learning, ensuring privacy-preserving collaboration across institutions and promoting equitable AI through the inclusion of diverse patient populations. Moreover, it will facilitate the development of foundation models trained on large-scale, multimodal datasets, serving as powerful starting points for specialized AI applications. Objective and transparent algorithm validation on a standardized data infrastructure will enhance reproducibility and interoperability of AI systems, driving innovation and reliability in clinical applications.
Collapse
Affiliation(s)
- Kyulee Jeon
- From the Department of Biomedical Systems Informatics, Yonsei University, Seoul, South Korea (K.J., S.C.Y.); Institution for Innovation in Digital Healthcare, Yonsei University, Seoul, South Korea (K.J., S.C.Y.); Biomedical Informatics and Data Science, Johns Hopkins University, Baltimore, MD (W.Y.P., P.N.); Department of Radiology, University of Pennsylvania, Philadelphia, PA (C.E.K.); and Department of Radiology, Seoul National University College of Medicine, Seoul National University Hospital, Seoul, South Korea (S.H.Y.)
| | | | | | | | | | | |
Collapse
|
2
|
Keloth VK, Selek S, Chen Q, Gilman C, Fu S, Dang Y, Chen X, Hu X, Zhou Y, He H, Fan JW, Wang K, Brandt C, Tao C, Liu H, Xu H. Large Language Models for Social Determinants of Health Information Extraction from Clinical Notes - A Generalizable Approach across Institutions. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.21.24307726. [PMID: 38826441 PMCID: PMC11142292 DOI: 10.1101/2024.05.21.24307726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
The consistent and persuasive evidence illustrating the influence of social determinants on health has prompted a growing realization throughout the health care sector that enhancing health and health equity will likely depend, at least to some extent, on addressing detrimental social determinants. However, detailed social determinants of health (SDoH) information is often buried within clinical narrative text in electronic health records (EHRs), necessitating natural language processing (NLP) methods to automatically extract these details. Most current NLP efforts for SDoH extraction have been limited, investigating on limited types of SDoH elements, deriving data from a single institution, focusing on specific patient cohorts or note types, with reduced focus on generalizability. This study aims to address these issues by creating cross-institutional corpora spanning different note types and healthcare systems, and developing and evaluating the generalizability of classification models, including novel large language models (LLMs), for detecting SDoH factors from diverse types of notes from four institutions: Harris County Psychiatric Center, University of Texas Physician Practice, Beth Israel Deaconess Medical Center, and Mayo Clinic. Four corpora of deidentified clinical notes were annotated with 21 SDoH factors at two levels: level 1 with SDoH factor types only and level 2 with SDoH factors along with associated values. Three traditional classification algorithms (XGBoost, TextCNN, Sentence BERT) and an instruction tuned LLM-based approach (LLaMA) were developed to identify multiple SDoH factors. Substantial variation was noted in SDoH documentation practices and label distributions based on patient cohorts, note types, and hospitals. The LLM achieved top performance with micro-averaged F1 scores over 0.9 on level 1 annotated corpora and an F1 over 0.84 on level 2 annotated corpora. While models performed well when trained and tested on individual datasets, cross-dataset generalization highlighted remaining obstacles. To foster collaboration, access to partial annotated corpora and models trained by merging all annotated datasets will be made available on the PhysioNet repository.
Collapse
Affiliation(s)
- Vipina K. Keloth
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Salih Selek
- Department of Psychiatry and Behavioral Sciences, UTHealth McGovern Medical School, Houston, TX, USA
| | - Qingyu Chen
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Christopher Gilman
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Sunyang Fu
- McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Yifang Dang
- McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Xinghan Chen
- School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Xinyue Hu
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, USA
| | - Yujia Zhou
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Huan He
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Jungwei W. Fan
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Karen Wang
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
- Equity Research and Innovation Center, Yale School of Medicine, New Haven, CT, USA
| | - Cynthia Brandt
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| | - Cui Tao
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL, USA
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Hongfang Liu
- McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Hua Xu
- Department of Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| |
Collapse
|
3
|
Park WY, Jeon K, Schmidt TS, Kondylakis H, Alkasab T, Dewey BE, You SC, Nagy P. Development of Medical Imaging Data Standardization for Imaging-Based Observational Research: OMOP Common Data Model Extension. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:899-908. [PMID: 38315345 PMCID: PMC11031512 DOI: 10.1007/s10278-024-00982-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/10/2023] [Accepted: 11/14/2023] [Indexed: 02/07/2024]
Abstract
The rapid growth of artificial intelligence (AI) and deep learning techniques require access to large inter-institutional cohorts of data to enable the development of robust models, e.g., targeting the identification of disease biomarkers and quantifying disease progression and treatment efficacy. The Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) has been designed to accommodate a harmonized representation of observational healthcare data. This study proposes the Medical Imaging CDM (MI-CDM) extension, adding two new tables and two vocabularies to the OMOP CDM to address the structural and semantic requirements to support imaging research. The tables provide the capabilities of linking DICOM data sources as well as tracking the provenance of imaging features derived from those images. The implementation of the extension enables phenotype definitions using imaging features and expanding standardized computable imaging biomarkers. This proposal offers a comprehensive and unified approach for conducting imaging research and outcome studies utilizing imaging features.
Collapse
Affiliation(s)
- Woo Yeon Park
- Biomedical Informatics and Data Science, Johns Hopkins University, 855 N Wolfe St, Rangos 616, Baltimore, MD, USA.
| | - Kyulee Jeon
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Korea
| | - Teri Sippel Schmidt
- Biomedical Informatics and Data Science, Johns Hopkins University, 855 N Wolfe St, Rangos 616, Baltimore, MD, USA
| | - Haridimos Kondylakis
- Institute of Computer Science, Foundation of Research & Technology-Hellas (FORTH), Heraklion, Greece
| | - Tarik Alkasab
- Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Blake E Dewey
- Department of Neurology, Johns Hopkins University, Baltimore, MD, USA
| | - Seng Chan You
- Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea
- Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Korea
| | - Paul Nagy
- Biomedical Informatics and Data Science, Johns Hopkins University, 855 N Wolfe St, Rangos 616, Baltimore, MD, USA
| |
Collapse
|
4
|
Esteban S, Szmulewicz A. Making causal inferences from transactional data: A narrative review of opportunities and challenges when implementing the target trial framework. J Int Med Res 2024; 52:3000605241241920. [PMID: 38548473 PMCID: PMC10981242 DOI: 10.1177/03000605241241920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 03/10/2024] [Indexed: 04/01/2024] Open
Abstract
The target trial framework has emerged as a powerful tool for addressing causal questions in clinical practice and in public health. In the healthcare sector, where decision-making is increasingly data-driven, transactional databases, such as electronic health records (EHR) and insurance claims, present an untapped potential for answering complex causal questions. This narrative review explores the potential of the integration of the target trial framework with real-world data to enhance healthcare decision-making processes. We outline essential elements of the target trial framework, and identify pertinent challenges in data quality, privacy concerns, and methodological limitations, proposing solutions to overcome these obstacles and optimize the framework's application.
Collapse
Affiliation(s)
- Santiago Esteban
- Instituto de Efectividad Clínica y Sanitaria, Centro de Implementación e Innovación en Políticas de Salud, Buenos Aires, Argentina
- Hospital Italiano de Buenos Aires, Family and Community Medicine Division Buenos Aires, Buenos Aires, Argentina
| | | |
Collapse
|