1
|
Fu YV, Ramachandran GK, Halwani A, McInnes BT, Xia F, Lybarger K, Yetisgen M, Uzuner Ö. CACER: Clinical concept Annotations for Cancer Events and Relations. J Am Med Inform Assoc 2024:ocae231. [PMID: 39225779 DOI: 10.1093/jamia/ocae231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2024] [Revised: 08/08/2024] [Accepted: 08/12/2024] [Indexed: 09/04/2024] Open
Abstract
OBJECTIVE Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. MATERIALS AND METHODS We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48 000 medical problems and drug events and 10 000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). RESULTS In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. DISCUSSION The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. CONCLUSIONS We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.
Collapse
Affiliation(s)
- Yujuan Velvin Fu
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States
| | | | - Ahmad Halwani
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, United States
| | - Bridget T McInnes
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, United States
| | - Fei Xia
- Department of Linguistics, University of Washington, Seattle, WA 98195, United States
| | - Kevin Lybarger
- Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States
| | - Meliha Yetisgen
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States
| | - Özlem Uzuner
- Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States
| |
Collapse
|
2
|
Xu W, Wang L, Zhang M, Zhu J, Yan J, Wu Q. A joint entity Relation Extraction method for document level Traditional Chinese Medicine texts. Artif Intell Med 2024; 154:102915. [PMID: 38936309 DOI: 10.1016/j.artmed.2024.102915] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 05/03/2024] [Accepted: 06/17/2024] [Indexed: 06/29/2024]
Abstract
Chinese medicine is a unique and complex medical system with complete and rich scientific theories. The textual data of Traditional Chinese Medicine (TCM) contains a large amount of relevant knowledge in the field of TCM, which can serve as guidance for accurate disease diagnosis as well as efficient disease prevention and treatment. Existing TCM texts are disorganized and lack a uniform standard. For this reason, this paper proposes a joint extraction framework by using graph convolutional networks to extract joint entity relations on document-level TCM texts to achieve TCM entity relation mining. More specifically, we first finetune the pre-trained language model by using the TCM domain knowledge to obtain the task-specific model. Taking the integrity of TCM into account, we extract the complete entities as well as the relations corresponding to diagnosis and treatment from the document-level medical cases by using multiple features such as word fusion coding, TCM lexicon information, and multi-relational graph convolutional networks. The experimental results show that the proposed method outperforms the state-of-the-art methods. It has an F1-score of 90.7% for Name Entity Recognization and 76.14% for Relation Extraction on the TCM dataset, which significantly improves the ability to extract entity relations from TCM texts. Code is available at https://github.com/xxxxwx/TCMERE.
Collapse
Affiliation(s)
- Wenxuan Xu
- School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, China.
| | - Lin Wang
- School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, China.
| | - Mingchuan Zhang
- School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, China.
| | - Junlong Zhu
- School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, China.
| | - Junqiang Yan
- The First Affiliated Hospital, Henan University of Science and Technology, Luoyang, 471003, China.
| | - Qingtao Wu
- School of Information Engineering, Henan University of Science and Technology, Luoyang, 471023, China.
| |
Collapse
|
3
|
Shi Y, He M, Chen J, Han F, Cai Y. SubGE-DDI: A new prediction model for drug-drug interaction established through biomedical texts and drug-pairs knowledge subgraph enhancement. PLoS Comput Biol 2024; 20:e1011989. [PMID: 38626249 PMCID: PMC11051621 DOI: 10.1371/journal.pcbi.1011989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 04/26/2024] [Accepted: 03/11/2024] [Indexed: 04/18/2024] Open
Abstract
Biomedical texts provide important data for investigating drug-drug interactions (DDIs) in the field of pharmacovigilance. Although researchers have attempted to investigate DDIs from biomedical texts and predict unknown DDIs, the lack of accurate manual annotations significantly hinders the performance of machine learning algorithms. In this study, a new DDI prediction framework, Subgraph Enhance model, was developed for DDI (SubGE-DDI) to improve the performance of machine learning algorithms. This model uses drug pairs knowledge subgraph information to achieve large-scale plain text prediction without many annotations. This model treats DDI prediction as a multi-class classification problem and predicts the specific DDI type for each drug pair (e.g. Mechanism, Effect, Advise, Interact and Negative). The drug pairs knowledge subgraph was derived from a huge drug knowledge graph containing various public datasets, such as DrugBank, TwoSIDES, OffSIDES, DrugCentral, EntrezeGene, SMPDB (The Small Molecule Pathway Database), CTD (The Comparative Toxicogenomics Database) and SIDER. The SubGE-DDI was evaluated from the public dataset (SemEval-2013 Task 9 dataset) and then compared with other state-of-the-art baselines. SubGE-DDI achieves 83.91% micro F1 score and 84.75% macro F1 score in the test dataset, outperforming the other state-of-the-art baselines. These findings show that the proposed drug pairs knowledge subgraph-assisted model can effectively improve the prediction performance of DDIs from biomedical texts.
Collapse
Affiliation(s)
- Yiyang Shi
- School of Medical Information and Engineering, Guangdong Pharmaceutical University, Guangzhou, China
| | - Mingxiu He
- School of Medical Information and Engineering, Guangdong Pharmaceutical University, Guangzhou, China
| | - Junheng Chen
- School of Medical Information and Engineering, Guangdong Pharmaceutical University, Guangzhou, China
| | - Fangfang Han
- School of Medical Information and Engineering, Guangdong Pharmaceutical University, Guangzhou, China
- NMPA Key Laboratory for Technology Research and Evaluation of Pharmacovigilance, Guangzhou, China
| | - Yongming Cai
- School of Medical Information and Engineering, Guangdong Pharmaceutical University, Guangzhou, China
- NMPA Key Laboratory for Technology Research and Evaluation of Pharmacovigilance, Guangzhou, China
- Guangdong Provincial Traditional Chinese Medicine Precision Medicine Big Data Engineering Technology Research Center, Guangzhou, China
| |
Collapse
|
4
|
Modi S, Kasmiran KA, Mohd Sharef N, Sharum MY. Extracting adverse drug events from clinical Notes: A systematic review of approaches used. J Biomed Inform 2024; 151:104603. [PMID: 38331081 DOI: 10.1016/j.jbi.2024.104603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/10/2024]
Abstract
BACKGROUND An adverse drug event (ADE) is any unfavorable effect that occurs due to the use of a drug. Extracting ADEs from unstructured clinical notes is essential to biomedical text extraction research because it helps with pharmacovigilance and patient medication studies. OBJECTIVE From the considerable amount of clinical narrative text, natural language processing (NLP) researchers have developed methods for extracting ADEs and their related attributes. This work presents a systematic review of current methods. METHODOLOGY Two biomedical databases have been searched from June 2022 until December 2023 for relevant publications regarding this review, namely the databases PubMed and Medline. Similarly, we searched the multi-disciplinary databases IEEE Xplore, Scopus, ScienceDirect, and the ACL Anthology. We adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement guidelines and recommendations for reporting systematic reviews in conducting this review. Initially, we obtained 5,537 articles from the search results from the various databases between 2015 and 2023. Based on predefined inclusion and exclusion criteria for article selection, 100 publications have undergone full-text review, of which we consider 82 for our analysis. RESULTS We determined the general pattern for extracting ADEs from clinical notes, with named entity recognition (NER) and relation extraction (RE) being the dual tasks considered. Researchers that tackled both NER and RE simultaneously have approached ADE extraction as a "pipeline extraction" problem (n = 22), as a "joint task extraction" problem (n = 7), and as a "multi-task learning" problem (n = 6), while others have tackled only NER (n = 27) or RE (n = 20). We further grouped the reviews based on the approaches for data extraction, namely rule-based (n = 8), machine learning (n = 11), deep learning (n = 32), comparison of two or more approaches (n = 11), hybrid (n = 12) and large language models (n = 8). The most used datasets are MADE 1.0, TAC 2017 and n2c2 2018. CONCLUSION Extracting ADEs is crucial, especially for pharmacovigilance studies and patient medications. This survey showcases advances in ADE extraction research, approaches, datasets, and state-of-the-art performance in them. Challenges and future research directions are highlighted. We hope this review will guide researchers in gaining background knowledge and developing more innovative ways to address the challenges.
Collapse
Affiliation(s)
- Salisu Modi
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia; Department of Computer Science, Sokoto State University, Sokoto, Nigeria.
| | - Khairul Azhar Kasmiran
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Nurfadhlina Mohd Sharef
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Mohd Yunus Sharum
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| |
Collapse
|
5
|
Deimazar G, Sheikhtaheri A. Machine learning models to detect and predict patient safety events using electronic health records: A systematic review. Int J Med Inform 2023; 180:105246. [PMID: 37837710 DOI: 10.1016/j.ijmedinf.2023.105246] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 10/02/2023] [Accepted: 10/08/2023] [Indexed: 10/16/2023]
Abstract
INTRODUCTION Identifying patient safety events using electronic health records (EHRs) and automated machine learning-based detection methods can help improve the efficiency and quality of healthcare service provision. OBJECTIVE This study aimed to systematically review machine learning-based methods and techniques, as well as their results for patient safety event management using EHRs. METHODS We reviewed the studies that focused on machine learning techniques, including automatic prediction and detection of patient safety events and medical errors through EHR analysis to manage patient safety events. The data were collected by searching Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE Xplore databases. RESULTS After screening, 41 papers were reviewed. Support vector machine (SVM), random forest, conditional random field (CRF), and bidirectional long short-term memory with conditional random field (BiLSTM-CRF) algorithms were mostly applied to predict, identify, and classify patient safety events using EHRs; however, they had different performances. BiLSTM-CRF was employed in most of the studies to extract and identify concepts, e.g., adverse drug events (ADEs) and adverse drug reactions (ADRs), as well as relationships between drug and severity, drug and ADEs, drug and ADRs. Recurrent neural networks (RNN) and BiLSTM-CRF had the best results in detecting ADEs compared to other patient safety events. Linear classifiers and Naive Bayes (NB) had the highest performance for ADR detection. Logistic regression had the best results in detecting surgical site infections. According to the findings, the quality of articles has non-significantly improved in recent years, but they had low average scores. CONCLUSIONS Machine learning can be useful in automatic detection and prediction of patient safety events. However, most of these algorithms have not yet been externally validated or prospectively tested. Therefore, further studies are required to improve the performance of these automated systems.
Collapse
Affiliation(s)
- Ghasem Deimazar
- Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
| | - Abbas Sheikhtaheri
- Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
6
|
Feng J, Liang Y, Yu T. MM-GANN-DDI: Multimodal Graph-Agnostic Neural Networks for Predicting Drug-Drug Interaction Events. Comput Biol Med 2023; 166:107492. [PMID: 37820558 DOI: 10.1016/j.compbiomed.2023.107492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Revised: 08/15/2023] [Accepted: 09/15/2023] [Indexed: 10/13/2023]
Abstract
Personalized treatment of complex diseases relies on combined medication. However, the occurrence of unexpected drug-drug interactions (DDIs) in these combinations can lead to adverse effects or even fatalities. Although recent computational methods exhibit promising performance in DDI screening, their practical implementation faces two significant challenges: (i) the availability of comprehensive datasets to support clinical application, and (ii) the ability to infer DDI types for new drugs beyond the existing dataset coverage. To mitigate these challenges, we propose MM-GANN-DDI: a Multimodal Graph-Agnostic Neural Network for Predicting Drug-Drug Interaction Events. We first mine six drug modalities and incorporate a graph attention (GAT) mechanism to fuse these modalities with the topological features of the DDI graph. We further propose a novel graph neural network training mechanism called graph-agnostic meta-training (GAMT), which effectively leverages topological information from the DDI graph and efficiently predicts DDI types for new drugs beyond the available dataset. Specifically, GAMT samples meta-graphs from the original DDI graph, splitting them into support and query sets to simulate seen and unseen drugs. Two-level optimizations are applied to enhance the model's generalization capability. We evaluate our model on two datasets (DB-v1 and DB-v2) across three tasks. Our MM-GANN-DDI demonstrates competitive performance on all three tasks. Notably, in Task 2, which focuses on predicting DDI types for drugs outside the dataset, our proposed model outperforms other methods, exhibiting an improvement of 4.6 percentage points in AUPR on DB-v1 and 5.9 percentage points on DB-v2. Additionally, our model surpasses state-of-the-art methods and classic approaches in terms of accuracy, F1 score, precision, and recall. Ablation experiments provide further validation of the effectiveness of the proposed model design. Importantly, our model exhibits the potential to discover unobserved DDIs, demonstrating its practical application in clinical medication.
Collapse
Affiliation(s)
- Junning Feng
- Faculty of Innovation Engineering, Macau University of Science and Technology, 999078, Macao Special Administrative Region of China; School of Data Science, The Chinese University of Hong Kong-Shenzhen, Shenzhen, 518055, China
| | - Yong Liang
- Peng Cheng Laboratory, Shenzhen, 518055, China.
| | - Tianwei Yu
- School of Data Science, The Chinese University of Hong Kong-Shenzhen, Shenzhen, 518055, China
| |
Collapse
|
7
|
Durango MC, Torres-Silva EA, Orozco-Duque A. Named Entity Recognition in Electronic Health Records: A Methodological Review. Healthc Inform Res 2023; 29:286-300. [PMID: 37964451 PMCID: PMC10651400 DOI: 10.4258/hir.2023.29.4.286] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 07/29/2023] [Accepted: 09/03/2023] [Indexed: 11/16/2023] Open
Abstract
OBJECTIVES A substantial portion of the data contained in Electronic Health Records (EHR) is unstructured, often appearing as free text. This format restricts its potential utility in clinical decision-making. Named entity recognition (NER) methods address the challenge of extracting pertinent information from unstructured text. The aim of this study was to outline the current NER methods and trace their evolution from 2011 to 2022. METHODS We conducted a methodological literature review of NER methods, with a focus on distinguishing the classification models, the types of tagging systems, and the languages employed in various corpora. RESULTS Several methods have been documented for automatically extracting relevant information from EHRs using natural language processing techniques such as NER and relation extraction (RE). These methods can automatically extract concepts, events, attributes, and other data, as well as the relationships between them. Most NER studies conducted thus far have utilized corpora in English or Chinese. Additionally, the bidirectional encoder representation from transformers using the BIO tagging system architecture is the most frequently reported classification scheme. We discovered a limited number of papers on the implementation of NER or RE tasks in EHRs within a specific clinical domain. CONCLUSIONS EHRs play a pivotal role in gathering clinical information and could serve as the primary source for automated clinical decision support systems. However, the creation of new corpora from EHRs in specific clinical domains is essential to facilitate the swift development of NER and RE models applied to EHRs for use in clinical practice.
Collapse
Affiliation(s)
- María C. Durango
- Grupo de Investigación e Innovación Biomédica, Instituto Tecnológico Metropolitano, Antioquia,
Colombia
| | - Ever A. Torres-Silva
- Grupo de Investigación e Innovación Biomédica, Instituto Tecnológico Metropolitano, Antioquia,
Colombia
| | - Andrés Orozco-Duque
- Grupo de Investigación e Innovación Biomédica, Instituto Tecnológico Metropolitano, Antioquia,
Colombia
- Facultad de Ingenierías, Universidad de Medellín, Antioquia,
Colombia
| |
Collapse
|
8
|
Peng C, Yang X, Yu Z, Bian J, Hogan WR, Wu Y. Clinical concept and relation extraction using prompt-based machine reading comprehension. J Am Med Inform Assoc 2023; 30:1486-1493. [PMID: 37316988 PMCID: PMC10436141 DOI: 10.1093/jamia/ocad107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Revised: 05/08/2023] [Accepted: 06/05/2023] [Indexed: 06/16/2023] Open
Abstract
OBJECTIVE To develop a natural language processing system that solves both clinical concept extraction and relation extraction in a unified prompt-based machine reading comprehension (MRC) architecture with good generalizability for cross-institution applications. METHODS We formulate both clinical concept extraction and relation extraction using a unified prompt-based MRC architecture and explore state-of-the-art transformer models. We compare our MRC models with existing deep learning models for concept extraction and end-to-end relation extraction using 2 benchmark datasets developed by the 2018 National NLP Clinical Challenges (n2c2) challenge (medications and adverse drug events) and the 2022 n2c2 challenge (relations of social determinants of health [SDoH]). We also evaluate the transfer learning ability of the proposed MRC models in a cross-institution setting. We perform error analyses and examine how different prompting strategies affect the performance of MRC models. RESULTS AND CONCLUSION The proposed MRC models achieve state-of-the-art performance for clinical concept and relation extraction on the 2 benchmark datasets, outperforming previous non-MRC transformer models. GatorTron-MRC achieves the best strict and lenient F1-scores for concept extraction, outperforming previous deep learning models on the 2 datasets by 1%-3% and 0.7%-1.3%, respectively. For end-to-end relation extraction, GatorTron-MRC and BERT-MIMIC-MRC achieve the best F1-scores, outperforming previous deep learning models by 0.9%-2.4% and 10%-11%, respectively. For cross-institution evaluation, GatorTron-MRC outperforms traditional GatorTron by 6.4% and 16% for the 2 datasets, respectively. The proposed method is better at handling nested/overlapped concepts, extracting relations, and has good portability for cross-institute applications. Our clinical MRC package is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerMRC.
Collapse
Affiliation(s)
- Cheng Peng
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Xi Yang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, Florida, USA
| | - Zehao Yu
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, Florida, USA
| | - William R Hogan
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Yonghui Wu
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, Florida, USA
| |
Collapse
|
9
|
Botsis T, Kreimeyer K. Improving drug safety with adverse event detection using natural language processing. Expert Opin Drug Saf 2023; 22:659-668. [PMID: 37339273 DOI: 10.1080/14740338.2023.2228197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 06/19/2023] [Indexed: 06/22/2023]
Abstract
INTRODUCTION Pharmacovigilance (PV) involves monitoring and aggregating adverse event information from a variety of data sources, including health records, biomedical literature, spontaneous adverse event reports, product labels, and patient-generated content like social media posts, but the most pertinent details in these sources are typically available in narrative free-text formats. Natural language processing (NLP) techniques can be used to extract clinically relevant information from PV texts to inform decision-making. AREAS COVERED We conducted a non-systematic literature review by querying the PubMed database to examine the uses of NLP in drug safety and distilled the findings to present our expert opinion on the topic. EXPERT OPINION New NLP techniques and approaches continue to be applied for drug safety use cases; however, systems that are fully deployed and in use in a clinical environment remain vanishingly rare. To see high-performing NLP techniques implemented in the real setting will require long-term engagement with end users and other stakeholders and revised workflows in fully formulated business plans for the targeted use cases. Additionally, we found little to no evidence of extracted information placed into standardized data models, which should be a way to make implementations more portable and adaptable.
Collapse
Affiliation(s)
- Taxiarchis Botsis
- Department of Oncology, the Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Kory Kreimeyer
- Department of Oncology, the Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
10
|
Yang L, Huang X, Wang J, Yang X, Ding L, Li Z, Li J. Identifying stroke-related quantified evidence from electronic health records in real-world studies. Artif Intell Med 2023; 140:102552. [PMID: 37210153 DOI: 10.1016/j.artmed.2023.102552] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Revised: 02/28/2023] [Accepted: 04/11/2023] [Indexed: 05/22/2023]
Abstract
BACKGROUND Stroke is one of the leading causes of death and disability worldwide. The National Institutes of Health Stroke Scale (NIHSS) scores in electronic health records (EHRs), which quantitatively describe patients' neurological deficits in evidence-based treatment, are crucial in stroke-related clinical investigations. However, the free-text format and lack of standardization inhibit their effective use. Automatically extracting the scale scores from the clinical free text so that its potential value in real-world studies is realized has become an important goal. OBJECTIVE This study aims to develop an automated method to extract scale scores from the free text of EHRs. METHODS We propose a two-step pipeline method to identify NIHSS items and numerical scores and validate its feasibility using a freely accessible critical care database: MIMIC-III (Medical Information Mart for Intensive Care III). First, we utilize MIMIC-III to create an annotated corpus. Then, we investigate possible machine learning methods for two subtasks, NIHSS item and score recognition and item-score relation extraction. In the evaluation, we conduct both task-specific and end-to-end evaluations and compare our method with the rule-based method using precision, recall and F1 scores as evaluation metrics. RESULTS We use all available discharge summaries of stroke cases in MIMIC-III. The annotated NIHSS corpus contains 312 cases, 2929 scale items, 2774 scores and 2733 relations. The results show that the best F1-score of our method was 0.9006, which was attained by combining BERT-BiLSTM-CRF and Random Forest, and it outperformed the rule-based method (F1-score = 0.8098). In the end-to-end task, our method could successfully recognize the item "1b level of consciousness questions", the score "1" and their relation "('1b level of consciousness questions', '1', 'has value')" from the sentence "1b level of consciousness questions: said name = 1", while the rule-based method could not. CONCLUSIONS The two-step pipeline method we propose is an effective approach to identify NIHSS items, scores and their relations. With its help, clinical investigators can easily retrieve and access structured scale data, thereby supporting stroke-related real-world studies.
Collapse
Affiliation(s)
- Lin Yang
- Institute of Medical Information and Library, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing 100020, China; Key Laboratory of Medical Information Intelligent Technology, Chinese Academy of Medical Sciences, Beijing 100020, China
| | - Xiaoshuo Huang
- Institute of Medical Information and Library, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing 100020, China; School of Health Care Technology, Dalian Neusoft University of Information, Dalian 116023, China
| | - Jiayang Wang
- Institute of Medical Information and Library, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing 100020, China
| | - Xin Yang
- China National Clinical Research Center for Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China; National Center for Healthcare Quality Management in Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Lingling Ding
- China National Clinical Research Center for Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Zixiao Li
- China National Clinical Research Center for Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China; National Center for Healthcare Quality Management in Neurological Diseases, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China; Department of Neurology, Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
| | - Jiao Li
- Institute of Medical Information and Library, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing 100020, China; Key Laboratory of Medical Information Intelligent Technology, Chinese Academy of Medical Sciences, Beijing 100020, China.
| |
Collapse
|
11
|
Gan Z, Zhou D, Rush E, Panickan VA, Ho YL, Ostrouchov G, Xu Z, Shen S, Xiong X, Greco KF, Hong C, Bonzel CL, Wen J, Costa L, Cai T, Begoli E, Xia Z, Gaziano JM, Liao KP, Cho K, Cai T, Lu J. ARCH: Large-scale Knowledge Graph via Aggregated Narrative Codified Health Records Analysis. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.05.14.23289955. [PMID: 37293026 PMCID: PMC10246054 DOI: 10.1101/2023.05.14.23289955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Objective Electronic health record (EHR) systems contain a wealth of clinical data stored as both codified data and free-text narrative notes, covering hundreds of thousands of clinical concepts available for research and clinical care. The complex, massive, heterogeneous, and noisy nature of EHR data imposes significant challenges for feature representation, information extraction, and uncertainty quantification. To address these challenges, we proposed an efficient Aggregated naRrative Codified Health (ARCH) records analysis to generate a large-scale knowledge graph (KG) for a comprehensive set of EHR codified and narrative features. Methods The ARCH algorithm first derives embedding vectors from a co-occurrence matrix of all EHR concepts and then generates cosine similarities along with associated p -values to measure the strength of relatedness between clinical features with statistical certainty quantification. In the final step, ARCH performs a sparse embedding regression to remove indirect linkage between entity pairs. We validated the clinical utility of the ARCH knowledge graph, generated from 12.5 million patients in the Veterans Affairs (VA) healthcare system, through downstream tasks including detecting known relationships between entity pairs, predicting drug side effects, disease phenotyping, as well as sub-typing Alzheimer's disease patients. Results ARCH produces high-quality clinical embeddings and KG for over 60,000 EHR concepts, as visualized in the R-shiny powered web-API (https://celehs.hms.harvard.edu/ARCH/). The ARCH embeddings attained an average area under the ROC curve (AUC) of 0.926 and 0.861 for detecting pairs of similar EHR concepts when the concepts are mapped to codified data and to NLP data; and 0.810 (codified) and 0.843 (NLP) for detecting related pairs. Based on the p -values computed by ARCH, the sensitivity of detecting similar and related entity pairs are 0.906 and 0.888 under false discovery rate (FDR) control of 5%. For detecting drug side effects, the cosine similarity based on the ARCH semantic representations achieved an AUC of 0.723 while the AUC improved to 0.826 after few-shot training via minimizing the loss function on the training data set. Incorporating NLP data substantially improved the ability to detect side effects in the EHR. For example, based on unsupervised ARCH embeddings, the power of detecting drug-side effects pairs when using codified data only was 0.15, much lower than the power of 0.51 when using both codified and NLP concepts. Compared to existing large-scale representation learning methods including PubmedBERT, BioBERT and SAPBERT, ARCH attains the most robust performance and substantially higher accuracy in detecting these relationships. Incorporating ARCH selected features in weakly supervised phenotyping algorithms can improve the robustness of algorithm performance, especially for diseases that benefit from NLP features as supporting evidence. For example, the phenotyping algorithm for depression attained an AUC of 0.927 when using ARCH selected features but only 0.857 when using codified features selected via the KESER network[1]. In addition, embeddings and knowledge graphs generated from the ARCH network were able to cluster AD patients into two subgroups, where the fast progression subgroup had a much higher mortality rate. Conclusions The proposed ARCH algorithm generates large-scale high-quality semantic representations and knowledge graph for both codified and NLP EHR features, useful for a wide range of predictive modeling tasks.
Collapse
Affiliation(s)
| | - Doudou Zhou
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Everett Rush
- Oak Ridge National Laboratory, Oak Ridge, TN USA
| | - Vidul A Panickan
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Yuk-Lam Ho
- VA Boston Healthcare System, Boston, MA, USA
| | | | - Zhiwei Xu
- University of Michigan, Ann Arbor, MI, USA
| | - Shuting Shen
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Xin Xiong
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | | | | | - Clara-Lea Bonzel
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Jun Wen
- Harvard Medical School, Boston, MA, USA
| | | | - Tianrun Cai
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Edmon Begoli
- Oak Ridge National Laboratory, Oak Ridge, TN USA
| | - Zongqi Xia
- University of Pittsburgh, Pittsburgh, USA
| | - J Michael Gaziano
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Katherine P Liao
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Kelly Cho
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
- Brigham and Women's Hospital, Boston, MA, USA
| | - Tianxi Cai
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| | - Junwei Lu
- Harvard T.H. Chan School of Public Health, Boston, MA, USA
- VA Boston Healthcare System, Boston, MA, USA
| |
Collapse
|
12
|
Yang Y, Lu Y, Yan W. A comprehensive review on knowledge graphs for complex diseases. Brief Bioinform 2023; 24:6931722. [PMID: 36528805 DOI: 10.1093/bib/bbac543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 11/02/2022] [Accepted: 11/10/2022] [Indexed: 12/23/2022] Open
Abstract
In recent years, knowledge graphs (KGs) have gained a great deal of popularity as a tool for storing relationships between entities and for performing higher level reasoning. KGs in biomedicine and clinical practice aim to provide an elegant solution for diagnosing and treating complex diseases more efficiently and flexibly. Here, we provide a systematic review to characterize the state-of-the-art of KGs in the area of complex disease research. We cover the following topics: (1) knowledge sources, (2) entity extraction methods, (3) relation extraction methods and (4) the application of KGs in complex diseases. As a result, we offer a complete picture of the domain. Finally, we discuss the challenges in the field by identifying gaps and opportunities for further research and propose potential research directions of KGs for complex disease diagnosis and treatment.
Collapse
Affiliation(s)
- Yang Yang
- School of Computer Science & Technology, Soochow University, Suzhou 215000, China.,Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 210000, China
| | - Yuwei Lu
- School of Computer Science & Technology, Soochow University, Suzhou 215000, China.,Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 210000, China
| | - Wenying Yan
- Department of Bioinformatics, School of Biology and Basic Medical Sciences, Medical College of Soochow University, and Center for Systems Biology, Soochow University, Suzhou 215123, China
| |
Collapse
|
13
|
Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, Dong H, Poon MTC, Fitzpatrick N, Levine AP, Slater LT, Handy A, Karwath A, Gkoutos GV, Chelala C, Shah AD, Stewart R, Collier N, Alex B, Whiteley W, Sudlow C, Roberts A, Dobson RJB. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med 2022; 5:186. [PMID: 36544046 PMCID: PMC9770568 DOI: 10.1038/s41746-022-00730-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.
Collapse
Affiliation(s)
- Honghan Wu
- Institute of Health Informatics, University College London, London, UK.
| | - Minhong Wang
- Institute of Health Informatics, University College London, London, UK
| | - Jinge Wu
- Institute of Health Informatics, University College London, London, UK
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Farah Francis
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Yun-Hsuan Chang
- Institute of Health Informatics, University College London, London, UK
| | - Alex Shavick
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Hang Dong
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Department of Computer Science, University of Oxford, Oxford, UK
| | | | | | - Adam P Levine
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Luke T Slater
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Alex Handy
- Institute of Health Informatics, University College London, London, UK
- University College London Hospitals NHS Trust, London, UK
| | - Andreas Karwath
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Georgios V Gkoutos
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Claude Chelala
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Anoop Dinesh Shah
- Institute of Health Informatics, University College London, London, UK
| | - Robert Stewart
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK
- South London and Maudsley NHS Foundation Trust, London, UK
| | - Nigel Collier
- Theoretical and Applied Linguistics, Faculty of Modern & Medieval Languages & Linguistics, University of Cambridge, Cambridge, UK
| | - Beatrice Alex
- Edinburgh Futures Institute, University of Edinburgh, Edinburgh, UK
| | | | - Cathie Sudlow
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Angus Roberts
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London, UK
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| |
Collapse
|
14
|
Timilsina M, Tandan M, Nováček V. Machine learning approaches for predicting the onset time of the adverse drug events in oncology. MACHINE LEARNING WITH APPLICATIONS 2022. [DOI: 10.1016/j.mlwa.2022.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
|
15
|
Kopperud BT, Lidgard S, Liow LH. Enhancing georeferenced biodiversity inventories: automated information extraction from literature records reveal the gaps. PeerJ 2022; 10:e13921. [PMID: 35999848 PMCID: PMC9393005 DOI: 10.7717/peerj.13921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 07/29/2022] [Indexed: 01/19/2023] Open
Abstract
We use natural language processing (NLP) to retrieve location data for cheilostome bryozoan species (text-mined occurrences (TMO)) in an automated procedure. We compare these results with data combined from two major public databases (DB): the Ocean Biodiversity Information System (OBIS), and the Global Biodiversity Information Facility (GBIF). Using DB and TMO data separately and in combination, we present latitudinal species richness curves using standard estimators (Chao2 and the Jackknife) and range-through approaches. Our combined DB and TMO species richness curves quantitatively document a bimodal global latitudinal diversity gradient for extant cheilostomes for the first time, with peaks in the temperate zones. A total of 79% of the georeferenced species we retrieved from TMO (N = 1,408) and DB (N = 4,549) are non-overlapping. Despite clear indications that global location data compiled for cheilostomes should be improved with concerted effort, our study supports the view that many marine latitudinal species richness patterns deviate from the canonical latitudinal diversity gradient (LDG). Moreover, combining online biodiversity databases with automated information retrieval from the published literature is a promising avenue for expanding taxon-location datasets.
Collapse
Affiliation(s)
- Bjørn Tore Kopperud
- Natural History Museum, University of Oslo, Oslo, Norway,GeoBio-Center, Ludwig-Maximilians-Universität München, München, Germany,Department of Earth and Environmental Sciences, Ludwig-Maximilians-Universität München, München, Germany
| | - Scott Lidgard
- Negaunee Integrative Research Center, Field Museum of Natural History, Chicago, Illinois, U.S.A.
| | - Lee Hsiang Liow
- Natural History Museum, University of Oslo, Oslo, Norway,Centre for Ecological and Evolutionary Synthesis, University of Oslo, Oslo, Norway
| |
Collapse
|
16
|
Guo S, Yang W, Han L, Song X, Wang G. A multi-layer soft lattice based model for Chinese clinical named entity recognition. BMC Med Inform Decis Mak 2022; 22:201. [PMID: 35908055 PMCID: PMC9338545 DOI: 10.1186/s12911-022-01924-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Objective Named entity recognition (NER) is a key and fundamental part of many medical and clinical tasks, including the establishment of a medical knowledge graph, decision-making support, and question answering systems. When extracting entities from electronic health records (EHRs), NER models mostly apply long short-term memory (LSTM) and have surprising performance in clinical NER. However, increasing the depth of the network is often required by these LSTM-based models to capture long-distance dependencies. Therefore, these LSTM-based models that have achieved high accuracy generally require long training times and extensive training data, which has obstructed the adoption of LSTM-based models in clinical scenarios with limited training time. Method Inspired by Transformer, we combine Transformer with Soft Term Position Lattice to form soft lattice structure Transformer, which models long-distance dependencies similarly to LSTM. Our model consists of four components: the WordPiece module, the BERT module, the soft lattice structure Transformer module, and the CRF module. Result Our experiments demonstrated that this approach increased the F1 by 1–5% in the CCKS NER task compared to other models based on LSTM with CRF and consumed less training time. Additional evaluations showed that lattice structure transformer shows good performance for recognizing long medical terms, abbreviations, and numbers. The proposed model achieve 91.6% f-measure in recognizing long medical terms and 90.36% f-measure in abbreviations, and numbers. Conclusions By using soft lattice structure Transformer, the method proposed in this paper captured Chinese words to lattice information, making our model suitable for Chinese clinical medical records. Transformers with Mutilayer soft lattice Chinese word construction can capture potential interactions between Chinese characters and words.
Collapse
Affiliation(s)
- Shuli Guo
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Wentao Yang
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Lina Han
- Department of Cardiology, The Second Medical Center, National Clinical Research Center for Geriatric Diseases, Chinese PLA General Hospital, Beijing, China.
| | - Xiaowei Song
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| | - Guowei Wang
- State Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
17
|
Kaas-Hansen BS, Placido D, Rodríguez CL, Thorsen-Meyer HC, Gentile S, Nielsen AP, Brunak S, Jürgens G, Andersen SE. Language-agnostic pharmacovigilant text mining to elicit side effects from clinical notes and hospital medication records. Basic Clin Pharmacol Toxicol 2022; 131:282-293. [PMID: 35834334 PMCID: PMC9541191 DOI: 10.1111/bcpt.13773] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 06/10/2022] [Accepted: 07/09/2022] [Indexed: 11/26/2022]
Abstract
We sought to craft a drug safety signalling pipeline associating latent information in clinical free text with exposures to single drugs and drug pairs. Data arose from 12 secondary and tertiary public hospitals in two Danish regions, comprising approximately half the Danish population. Notes were operationalised with a fastText embedding, based on which we trained 10,720 neural-network models (one for each distinct single-drug/drug-pair exposure) predicting the risk of exposure given an embedding vector. We included 2,905,251 admissions between May 2008 and June 2016, with 13,740,564 distinct drug prescriptions; the median number of prescriptions was 5 (IQR: 3-9) and in 1,184,340 (41%) admissions patients used ≥5 drugs concomitantly. 10,788,259 clinical notes were included, with 179,441,739 tokens retained after pruning. Of 345 single-drug signals reviewed, 28 (8.1%) represented possibly undescribed relationships; 186 (54%) signals were clinically meaningful. 16 (14%) of the 115 drug-pair signals were possible interactions and 2 (1.7%) were known. In conclusion, we built a language-agnostic pipeline for mining associations between free-text information and medication exposure without manual curation, predicting not the likely outcome of a range of exposures, but the likely exposures for outcomes of interest. Our approach may help overcome limitations of text mining methods relying on curated data in English and can help leverage non-English free text for pharmacovigilance.
Collapse
Affiliation(s)
- Benjamin Skov Kaas-Hansen
- Clinical Pharmacology Unit, Zealand University Hospital, Denmark.,NNF Center for Protein Research, University of Copenhagen, Denmark.,Section of Biostatistics, Department of Public Health, University of Copenhagen, Denmark
| | - Davide Placido
- NNF Center for Protein Research, University of Copenhagen, Denmark
| | | | | | | | | | - Søren Brunak
- NNF Center for Protein Research, University of Copenhagen, Denmark
| | - Gesche Jürgens
- Clinical Pharmacology Unit, Zealand University Hospital, Denmark
| | | |
Collapse
|
18
|
Combining Machine Learning with a Rule-Based Algorithm to Detect and Identify Related Entities of Documented Adverse Drug Reactions on Hospital Discharge Summaries. Drug Saf 2022; 45:853-862. [PMID: 35794349 DOI: 10.1007/s40264-022-01196-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/29/2022] [Indexed: 12/28/2022]
Abstract
INTRODUCTION Discharge summaries contain valuable information about adverse drug reactions, but their unstructured nature makes them challenging to analyse and use as a signal source for pharmacovigilance. Machine learning has shown promise in identifying discharge summaries that contain related drug-adverse event pairs but has fared relatively poorer in entity extraction. METHODS A hybrid model is developed combining rule-based and machine learning algorithms using discharge summaries with the aim of maximising capture of related drug-adverse event pairs. The rule first identifies segments containing adverse event entities within a 100-character distance from a drug term; machine learning subsequently estimates the relatedness of the drug and adverse event entities contained. The approach is validated on four independent datasets that are temporally and geographically separated from model development data. The impact of restricted drug-adverse event pair detection on recall is evaluated by using two of the four validation datasets that do not impose rule-based restrictions to annotations. RESULTS The hybrid model achieves a recall of 0.80 (fivefold cross validation), 0.80 (temporal) and 0.76 (geographical) on validation using datasets containing only pre-identified target text segments that fulfil the rule-based algorithm criteria. When tested on datasets that additionally contained drug-adverse event pairs not restricted by the rule-based criteria, recall of the model declines to 0.68 and 0.62 on temporally and geographically separated datasets, respectively. CONCLUSIONS The proposed hybrid model demonstrates reasonable generalisability on external validation. Rule-based restriction of the detection space results in an approximately 12-14% reduction in recall but improves identification of the related drug and adverse event terms.
Collapse
|
19
|
He L, Liu H, Yang Y, Wang B. A Multi-attention Collaborative Deep Learning Approach for Blood Pressure Prediction. ACM TRANSACTIONS ON MANAGEMENT INFORMATION SYSTEMS 2022. [DOI: 10.1145/3471571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
We develop a deep learning model based on Long Short-term Memory (LSTM) to predict blood pressure based on a unique data set collected from physical examination centers capturing comprehensive multi-year physical examination and lab results. In the Multi-attention Collaborative Deep Learning model (MAC-LSTM) we developed for this type of data, we incorporate three types of attention to generate more explainable and accurate results. In addition, we leverage information from similar users to enhance the predictive power of the model due to the challenges with short examination history. Our model significantly reduces predictive errors compared to several state-of-the-art baseline models. Experimental results not only demonstrate our model’s superiority but also provide us with new insights about factors influencing blood pressure. Our data is collected in a natural setting instead of a setting designed specifically to study blood pressure, and the physical examination items used to predict blood pressure are common items included in regular physical examinations for all the users. Therefore, our blood pressure prediction results can be easily used in an alert system for patients and doctors to plan prevention or intervention. The same approach can be used to predict other health-related indexes such as BMI.
Collapse
Affiliation(s)
- Luo He
- Department of Management Science and Engineering, Schoolof Economics and Management, Tsinghua University, Beijing, China
| | - Hongyan Liu
- Department of Management Science and Engineering, Schoolof Economics and Management, Tsinghua University, Beijing, China
| | - Yinghui Yang
- Graduate School of Management, University of California, California, USA
| | - Bei Wang
- School of Information, Renmin University of China, Beijing, China
| |
Collapse
|
20
|
Yang C, Xiao D, Luo Y, Li B, Zhao X, Zhang H. A hybrid method based on semi-supervised learning for relation extraction in Chinese EMRs. BMC Med Inform Decis Mak 2022; 22:169. [PMID: 35761319 PMCID: PMC9235238 DOI: 10.1186/s12911-022-01908-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Accepted: 06/16/2022] [Indexed: 11/14/2022] Open
Abstract
Background Building a large-scale medical knowledge graphs needs to automatically extract the relations between entities from electronic medical records (EMRs) . The main challenges are the scarcity of available labeled corpus and the identification of complexity semantic relations in text of Chinese EMRs. A hybrid method based on semi-supervised learning is proposed to extract the medical entity relations from small-scale complex Chinese EMRs. Methods The semantic features of sentences are extracted by a residual network and the long dependent information is captured by bidirectional gated recurrent unit. Then the attention mechanism is used to assign weights for the extracted features respectively, and the output of two attention mechanisms is integrated for relation prediction. We adjusted the training process with manually annotated small-scale relational corpus and bootstrapping semi-supervised learning algorithm, and continuously expanded the datasets during the training process. Results We constructed a small corpus of Chinese EMRs relation extraction based on the EMR datasets released at the China Conference on Knowledge Graph and Semantic Computing. The experimental results show that the best F1-score of the proposed method on the overall relation categories reaches 89.78%, which is 13.07% higher than the baseline CNN.
Collapse
Affiliation(s)
- Chunming Yang
- School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China. .,Sichuan Big Data and Intelligent Systems Engineering Technology Research Center, Mianyang, 621010, Sichuan, China.
| | - Dan Xiao
- School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China
| | - Yuanyuan Luo
- School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China
| | - Bo Li
- School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China
| | - Xujian Zhao
- School of Computer Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China
| | - Hui Zhang
- School of Science, Southwest University of Science and Technology, Mianyang, 621010, Sichuan, China
| |
Collapse
|
21
|
Chanda AK, Bai T, Yang Z, Vucetic S. Improving medical term embeddings using UMLS Metathesaurus. BMC Med Inform Decis Mak 2022; 22:114. [PMID: 35488252 PMCID: PMC9052653 DOI: 10.1186/s12911-022-01850-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2020] [Accepted: 03/29/2022] [Indexed: 11/25/2022] Open
Abstract
BACKGROUND Health providers create Electronic Health Records (EHRs) to describe the conditions and procedures used to treat their patients. Medical notes entered by medical staff in the form of free text are a particularly insightful component of EHRs. There is a great interest in applying machine learning tools on medical notes in numerous medical informatics applications. Learning vector representations, or embeddings, of terms in the notes, is an important pre-processing step in such applications. However, learning good embeddings is challenging because medical notes are rich in specialized terminology, and the number of available EHRs in practical applications is often very small. METHODS In this paper, we propose a novel algorithm to learn embeddings of medical terms from a limited set of medical notes. The algorithm, called definition2vec, exploits external information in the form of medical term definitions. It is an extension of a skip-gram algorithm that incorporates textual definitions of medical terms provided by the Unified Medical Language System (UMLS) Metathesaurus. RESULTS To evaluate the proposed approach, we used a publicly available Medical Information Mart for Intensive Care (MIMIC-III) EHR data set. We performed quantitative and qualitative experiments to measure the usefulness of the learned embeddings. The experimental results show that definition2vec keeps the semantically similar medical terms together in the embedding vector space even when they are rare or unobserved in the corpus. We also demonstrate that learned vector embeddings are helpful in downstream medical informatics applications. CONCLUSION This paper shows that medical term definitions can be helpful when learning embeddings of rare or previously unseen medical terms from a small corpus of specialized documents such as medical notes.
Collapse
Affiliation(s)
- Ashis Kumar Chanda
- Department of Computer and Information Sciences, Temple University, Philadelphia, PA, USA
| | - Tian Bai
- Department of Computer and Information Sciences, Temple University, Philadelphia, PA, USA
| | - Ziyu Yang
- Department of Computer and Information Sciences, Temple University, Philadelphia, PA, USA
| | - Slobodan Vucetic
- Department of Computer and Information Sciences, Temple University, Philadelphia, PA, USA.
| |
Collapse
|
22
|
Sato H. Development of Clinical Pharmaceutical Services <i>via</i> Artificial Intelligence Adaptation. YAKUGAKU ZASSHI 2022; 142:337-340. [DOI: 10.1248/yakushi.21-00178-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
Affiliation(s)
- Hiroyasu Sato
- Department of Pharmacy, Obihiro Kosei General Hospital
| |
Collapse
|
23
|
Iwata H, Kojima R, Okuno Y. AIM in Pharmacology and Drug Discovery. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
24
|
An attentive joint model with transformer-based weighted graph convolutional network for extracting adverse drug event relation. J Biomed Inform 2021; 125:103968. [PMID: 34871807 DOI: 10.1016/j.jbi.2021.103968] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 11/25/2021] [Accepted: 11/27/2021] [Indexed: 11/21/2022]
Abstract
Adverse drug event (ADE) relation extraction is a crucial task for drug safety surveillance which aims to discover potential relations between ADE mentions from unstructured medical texts. To date, the graph convolutional networks (GCN) have been the state-of-the-art solutions for improving the ability of relation extraction task. However, there are many challenging issues that should be addressed. Among these, the syntactic information is not fully exploited by GCN-based methods, especially the diversified dependency edges. Still, these methods fail to effectively extract complex relations that include nested, discontinuous and overlapping mentions. Besides, the task is primarily regarded as a classification problem where each candidate relation is treated independently which neglects the interaction between other relations. To deal with these issues, in this paper, we propose an attentive joint model with transformer-based weighted GCN for extracting ADE Relations, called ADERel. Firstly, the ADERel system formulates the ADE relation extraction task as an N-level sequence labelling so as to model the complex relations in different levels and capture greater interaction between relations. Then, it exploits our neural joint model to process the N-level sequences jointly. The joint model leverages the contextual and structural information by adopting a shared representation that combines a bidirectional encoder representation from transformers (BERT) and our proposed weighted GCN (WGCN). The latter assigns a score to each dependency edge within a sentence so as to capture rich syntactic features and determine the most influential edges for extracting ADE relations. Finally, the system employs a multi-head attention to exchange boundary knowledge across levels. We evaluate ADERel on two benchmark datasets from TAC 2017 and n2c2 2018 shared tasks. The experimental results show that ADERel is superior in performance compared with several state-of-the-art methods. The results also demonstrate that incorporating a transformer model with WGCN makes the proposed system more effective for extracting various types of ADE relations. The evaluations further highlight that ADERel takes advantage of joint learning, showing its effectiveness in recognizing complex relations.
Collapse
|
25
|
Martin JA, Crane-Droesch A, Lapite FC, Puhl JC, Kmiec TE, Silvestri JA, Ungar LH, Kinosian BP, Himes BE, Hubbard RA, Diamond JM, Ahya V, Sims MW, Halpern SD, Weissman GE. Development and validation of a prediction model for actionable aspects of frailty in the text of clinicians' encounter notes. J Am Med Inform Assoc 2021; 29:109-119. [PMID: 34791302 DOI: 10.1093/jamia/ocab248] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 10/16/2021] [Accepted: 10/28/2021] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE Frailty is a prevalent risk factor for adverse outcomes among patients with chronic lung disease. However, identifying frail patients who may benefit from interventions is challenging using standard data sources. We therefore sought to identify phrases in clinical notes in the electronic health record (EHR) that describe actionable frailty syndromes. MATERIALS AND METHODS We used an active learning strategy to select notes from the EHR and annotated each sentence for 4 actionable aspects of frailty: respiratory impairment, musculoskeletal problems, fall risk, and nutritional deficiencies. We compared the performance of regression, tree-based, and neural network models to predict the labels for each sentence. We evaluated performance with the scaled Brier score (SBS), where 1 is perfect and 0 is uninformative, and the positive predictive value (PPV). RESULTS We manually annotated 155 952 sentences from 326 patients. Elastic net regression had the best performance across all 4 frailty aspects (SBS 0.52, 95% confidence interval [CI] 0.49-0.54) followed by random forests (SBS 0.49, 95% CI 0.47-0.51), and multi-task neural networks (SBS 0.39, 95% CI 0.37-0.42). For the elastic net model, the PPV for identifying the presence of respiratory impairment was 54.8% (95% CI 53.3%-56.6%) at a sensitivity of 80%. DISCUSSION Classification models using EHR notes can effectively identify actionable aspects of frailty among patients living with chronic lung disease. Regression performed better than random forest and neural network models. CONCLUSIONS NLP-based models offer promising support to population health management programs that seek to identify and refer community-dwelling patients with frailty for evidence-based interventions.
Collapse
Affiliation(s)
- Jacob A Martin
- Division of Cardiology, Department of Medicine, New York University Grossman School of Medicine, New York, New York, USA.,Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Andrew Crane-Droesch
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | | | - Joseph C Puhl
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Tyler E Kmiec
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Jasmine A Silvestri
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Lyle H Ungar
- Department of Computer and Information Science, University of Pennsylvania School of Engineering and Applied Science, Philadelphia, Pennsylvania, USA
| | - Bruce P Kinosian
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA.,Division of Geriatrics, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.,Geriatrics and Extended Care Data Analysis Center, Corporal Michael J Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| | - Blanca E Himes
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Rebecca A Hubbard
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Joshua M Diamond
- Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Vivek Ahya
- Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Michael W Sims
- Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Scott D Halpern
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA.,Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.,Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| | - Gary E Weissman
- Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA.,Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
| |
Collapse
|
26
|
Lee S, Shin J, Kim HS, Lee MJ, Yoon JM, Lee S, Kim Y, Kim JY, Lee S. Hybrid Method Incorporating a Rule-Based Approach and Deep Learning for Prescription Error Prediction. Drug Saf 2021; 45:27-35. [PMID: 34766251 DOI: 10.1007/s40264-021-01123-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/21/2021] [Indexed: 11/26/2022]
Abstract
INTRODUCTION Recently, automated detection has been a new approach to address the risks posed by prescribing errors. This study focused on prescription errors and utilized real medical data to supplement the Drug Utilization Review (DUR)-based rules, the current prescription error detection method. We developed a new hybrid method through artificial intelligence for prescription error prediction by utilizing actual detection accuracy improvement to reduce 'warning fatigue' for doctors and improve medical care quality. OBJECT This study was conducted in the Department of Pediatrics, targeting children sensitive to drugs to develop a prescription error detection system. Based on the DUR prescription history, 15,281 patient-level observations of children from Konyang University Hospital (KYUH)'s common data model (CDM) and DUR were collected and analyzed retrospectively. METHOD Among the CDM data, inspection information was interlocked with DUR and reflected as standard information for model development; this included outpatient prescriptions from January 1 to December 31, 2018. Through consultation with pediatric clinicians, rule definitions and model development were conducted for 35 drugs, with 137,802 normal and 1609 prescription errors. RESULTS We developed a novel hybrid method of error detection in the form of an advanced rule-based deep neural network (ARDNN), which showed the expected performance (precision: 72.86, recall: 81.01, F1 score: 76.72) and reduced alarm pop-up alert fatigue to below 10%. We also created an ARDNN-based comprehensive dashboard that allows doctors to monitor prescription errors with alarm pop-ups when prescribing medications. CONCLUSION These results can advance the existing rule-based model by developing a prescription error detection model using deep learning. This method can improve overall medical efficiency and service quality by reducing doctors' fatigue.
Collapse
Affiliation(s)
- Seunghee Lee
- Health Care Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea
| | | | | | | | - Jung Min Yoon
- Department of Pediatrics, Konyang University Hospital, Daejeon, Republic of Korea
| | - Sohee Lee
- Department of Biomedical Engineering, Konyang University, Daejeon, Republic of Korea
| | - Yongsuk Kim
- Department of Medical Artificial Intelligence, Konyang University, Daejeon, Republic of Korea
| | - Jong-Yeup Kim
- Health Care Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea.
- Departments of Biomedical Informatics, College of Medicine, Konyang University, 158 Gwanjeodong-ro, Seo-gu, Daejeon, 35365, Republic of Korea.
| | - Suehyun Lee
- Health Care Data Science Center, Konyang University Hospital, Daejeon, Republic of Korea.
- Departments of Biomedical Informatics, College of Medicine, Konyang University, 158 Gwanjeodong-ro, Seo-gu, Daejeon, 35365, Republic of Korea.
| |
Collapse
|
27
|
A Survey on Recent Named Entity Recognition and Relationship Extraction Techniques on Clinical Texts. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11188319] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Significant growth in Electronic Health Records (EHR) over the last decade has provided an abundance of clinical text that is mostly unstructured and untapped. This huge amount of clinical text data has motivated the development of new information extraction and text mining techniques. Named Entity Recognition (NER) and Relationship Extraction (RE) are key components of information extraction tasks in the clinical domain. In this paper, we highlight the present status of clinical NER and RE techniques in detail by discussing the existing proposed NLP models for the two tasks and their performances and discuss the current challenges. Our comprehensive survey on clinical NER and RE encompass current challenges, state-of-the-art practices, and future directions in information extraction from clinical text. This is the first attempt to discuss both of these interrelated topics together in the clinical context. We identified many research articles published based on different approaches and looked at applications of these tasks. We also discuss the evaluation metrics that are used in the literature to measure the effectiveness of the two these NLP methods and future research directions.
Collapse
|
28
|
Yu C, Xue H, Wang M, An L. Towards an entity relation extraction framework in the cross-lingual context. ELECTRONIC LIBRARY 2021. [DOI: 10.1108/el-10-2020-0304] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose
Owing to the uneven distribution of annotated corpus among different languages, it is necessary to bridge the gap between low resource languages and high resource languages. From the perspective of entity relation extraction, this paper aims to extend the knowledge acquisition task from a single language context to a cross-lingual context, and to improve the relation extraction performance for low resource languages.
Design/methodology/approach
This paper proposes a cross-lingual adversarial relation extraction (CLARE) framework, which decomposes cross-lingual relation extraction into parallel corpus acquisition and adversarial adaptation relation extraction. Based on the proposed framework, this paper conducts extensive experiments in two tasks, i.e. the English-to-Chinese and the English-to-Arabic cross-lingual entity relation extraction.
Findings
The Macro-F1 values of the optimal models in the two tasks are 0.880 1 and 0.789 9, respectively, indicating that the proposed CLARE framework for CLARE can significantly improve the effect of low resource language entity relation extraction. The experimental results suggest that the proposed framework can effectively transfer the corpus as well as the annotated tags from English to Chinese and Arabic. This study reveals that the proposed approach is less human labour intensive and more effective in the cross-lingual entity relation extraction than the manual method. It shows that this approach has high generalizability among different languages.
Originality/value
The research results are of great significance for improving the performance of the cross-lingual knowledge acquisition. The cross-lingual transfer may greatly reduce the time and cost of the manual construction of the multi-lingual corpus. It sheds light on the knowledge acquisition and organization from the unstructured text in the era of big data.
Collapse
|
29
|
Mitra A, Rawat BPS, McManus DD, Yu H. Relation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study. JMIR Med Inform 2021; 9:e27527. [PMID: 34255697 PMCID: PMC8285744 DOI: 10.2196/27527] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 03/19/2021] [Accepted: 05/30/2021] [Indexed: 01/05/2023] Open
Abstract
BACKGROUND Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. OBJECTIVE In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event-related relation classification. METHODS We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). RESULTS Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. CONCLUSIONS In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation.
Collapse
Affiliation(s)
- Avijit Mitra
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States
| | - Bhanu Pratap Singh Rawat
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States
| | - David D McManus
- Department of Medicine, University of Massachusetts Medical School, Worcester, MA, United States
| | - Hong Yu
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States.,Department of Medicine, University of Massachusetts Medical School, Worcester, MA, United States.,Department of Computer Science, University of Massachusetts Lowell, Lowell, MA, United States.,Center for Healthcare Organization and Implementation Research, Bedford Veterans Affairs Medical Center, Bedford, MA, United States
| |
Collapse
|
30
|
Zhang T, Lin H, Ren Y, Yang Z, Wang J, Zhang S, Xu B, Duan X. Adversarial transfer network with bilinear attention for the detection of adverse drug reactions from social media. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
31
|
RMAN: Relational multi-head attention neural network for joint extraction of entities and relations. APPL INTELL 2021. [DOI: 10.1007/s10489-021-02600-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
32
|
Kaushik M, Chandra Joshi R, Kushwah AS, Gupta MK, Banerjee M, Burget R, Dutta MK. Cytokine gene variants and socio-demographic characteristics as predictors of cervical cancer: A machine learning approach. Comput Biol Med 2021; 134:104559. [PMID: 34147008 DOI: 10.1016/j.compbiomed.2021.104559] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Revised: 05/30/2021] [Accepted: 06/04/2021] [Indexed: 01/03/2023]
Abstract
Cervical cancer is still one of the most prevalent cancers in women and a significant cause of mortality. Cytokine gene variants and socio-demographic characteristics have been reported as biomarkers for determining the cervical cancer risk in the Indian population. This study was designed to apply a machine learning-based model using these risk factors for better prognosis and prediction of cervical cancer. This study includes the dataset of cytokine gene variants, clinical and socio-demographic characteristics of normal healthy control subjects, and cervical cancer cases. Different risk factors, including demographic details and cytokine gene variants, were analysed using different machine learning approaches. Various statistical parameters were used for evaluating the proposed method. After multi-step data processing and random splitting of the dataset, machine learning methods were applied and evaluated with 5-fold cross-validation and also tested on the unseen data records of a collected dataset for proper evaluation and analysis. The proposed approaches were verified after analysing various performance metrics. The logistic regression technique achieved the highest average accuracy of 82.25% and the highest average F1-score of 82.58% among all the methods. Ridge classifiers and the Gaussian Naïve Bayes classifier achieved the highest sensitivity-85%. The ridge classifier surpasses most of the machine learning classifiers with 84.78% accuracy and 97.83% sensitivity. The risk factors analysed in this study can be taken as biomarkers in developing a cervical cancer diagnosis system. The outcomes demonstrate that the machine learning assisted analysis of cytokine gene variants and socio-demographic characteristics can be utilised effectively for predicting the risk of developing cervical cancer.
Collapse
Affiliation(s)
- Manoj Kaushik
- Centre for Advanced Studies, Dr. A. P. J. Abdul Kalam Technical University, Lucknow, Uttar Pradesh, India
| | - Rakesh Chandra Joshi
- Centre for Advanced Studies, Dr. A. P. J. Abdul Kalam Technical University, Lucknow, Uttar Pradesh, India
| | - Atar Singh Kushwah
- Molecular & Human Genetics Laboratory, Department of Zoology, University of Lucknow, Lucknow, Uttar Pradesh, India; Department of Zoology, Institute of Science, Banaras Hindu University, Varanasi, Uttar Pradesh, India
| | - Maneesh Kumar Gupta
- Molecular & Human Genetics Laboratory, Department of Zoology, University of Lucknow, Lucknow, Uttar Pradesh, India
| | - Monisha Banerjee
- Molecular & Human Genetics Laboratory, Department of Zoology, University of Lucknow, Lucknow, Uttar Pradesh, India
| | - Radim Burget
- Brno University of Technology, Faculty of Electrical Engineering, Brno, Czech Republic
| | - Malay Kishore Dutta
- Centre for Advanced Studies, Dr. A. P. J. Abdul Kalam Technical University, Lucknow, Uttar Pradesh, India.
| |
Collapse
|
33
|
Mahendran D, McInnes BT. Extracting Adverse Drug Events from Clinical Notes. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2021; 2021:420-429. [PMID: 34457157 PMCID: PMC8378605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Adverse drug events (ADEs) are unexpected incidents caused by the administration of a drug or medication. To identify and extract these events, we require information about not just the drug itself but attributes describing the drug (e.g., strength, dosage), the reason why the drug was initially prescribed, and any adverse reaction to the drug. This paper explores the relationship between a drug and its associated attributes using relation extraction techniques. We explore three approaches: a rule-based approach, a deep learning-based approach, and a contextualized language model-based approach. We evaluate our system on the n2c2-2018 ADE extraction dataset. Our experimental results demonstrate that the contextualized language model-based approach outperformed other models overall and obtain the state-of-the-art performance in ADE extraction with a Precision of 0.93, Recall of 0.96, and an F1 score of 0.94; however, for certain relation types, the rule-based approach obtained a higher Precision and Recall than either learning approach.
Collapse
Affiliation(s)
- Darshini Mahendran
- Computer Science Department, Virginia Commonwealth University, Richmond, VA, USA
| | - Bridget T McInnes
- Computer Science Department, Virginia Commonwealth University, Richmond, VA, USA
| |
Collapse
|
34
|
Alfattni G, Belousov M, Peek N, Nenadic G. Extracting Drug Names and Associated Attributes From Discharge Summaries: Text Mining Study. JMIR Med Inform 2021; 9:e24678. [PMID: 33949962 PMCID: PMC8135022 DOI: 10.2196/24678] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 02/15/2021] [Accepted: 02/20/2021] [Indexed: 11/13/2022] Open
Abstract
Background Drug prescriptions are often recorded in free-text clinical narratives; making this information available in a structured form is important to support many health-related tasks. Although several natural language processing (NLP) methods have been proposed to extract such information, many challenges remain. Objective This study evaluates the feasibility of using NLP and deep learning approaches for extracting and linking drug names and associated attributes identified in clinical free-text notes and presents an extensive error analysis of different methods. This study initiated with the participation in the 2018 National NLP Clinical Challenges (n2c2) shared task on adverse drug events and medication extraction. Methods The proposed system (DrugEx) consists of a named entity recognizer (NER) to identify drugs and associated attributes and a relation extraction (RE) method to identify the relations between them. For NER, we explored deep learning-based approaches (ie, bidirectional long-short term memory with conditional random fields [BiLSTM-CRFs]) with various embeddings (ie, word embedding, character embedding [CE], and semantic-feature embedding) to investigate how different embeddings influence the performance. A rule-based method was implemented for RE and compared with a context-aware long-short term memory (LSTM) model. The methods were trained and evaluated using the 2018 n2c2 shared task data. Results The experiments showed that the best model (BiLSTM-CRFs with pretrained word embeddings [PWE] and CE) achieved lenient micro F-scores of 0.921 for NER, 0.927 for RE, and 0.855 for the end-to-end system. NER, which relies on the pretrained word and semantic embeddings, performed better on most individual entity types, but NER with PWE and CE had the highest classification efficiency among the proposed approaches. Extracting relations using the rule-based method achieved higher accuracy than the context-aware LSTM for most relations. Interestingly, the LSTM model performed notably better in the reason-drug relations, the most challenging relation type. Conclusions The proposed end-to-end system achieved encouraging results and demonstrated the feasibility of using deep learning methods to extract medication information from free-text data.
Collapse
Affiliation(s)
- Ghada Alfattni
- Department of Computer Science, University of Manchester, Manchester, United Kingdom.,Department of Computer Science, Jamoum University College, Umm Al-Qura University, Makkah, Saudi Arabia
| | - Maksim Belousov
- Department of Computer Science, University of Manchester, Manchester, United Kingdom
| | - Niels Peek
- Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, University of Manchester, Manchester, United Kingdom.,National Institute of Health Research Manchester Biomedical Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom.,The Alan Turing Institute, Manchester, United Kingdom
| | - Goran Nenadic
- Department of Computer Science, University of Manchester, Manchester, United Kingdom.,The Alan Turing Institute, Manchester, United Kingdom
| |
Collapse
|
35
|
Lybarger K, Ostendorf M, Thompson M, Yetisgen M. Extracting COVID-19 diagnoses and symptoms from clinical text: A new annotated corpus and neural event extraction framework. J Biomed Inform 2021; 117:103761. [PMID: 33781918 PMCID: PMC7997694 DOI: 10.1016/j.jbi.2021.103761] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 03/02/2021] [Accepted: 03/20/2021] [Indexed: 12/29/2022]
Abstract
Coronavirus disease 2019 (COVID-19) is a global pandemic. Although much has been learned about the novel coronavirus since its emergence, there are many open questions related to tracking its spread, describing symptomology, predicting the severity of infection, and forecasting healthcare utilization. Free-text clinical notes contain critical information for resolving these questions. Data-driven, automatic information extraction models are needed to use this text-encoded information in large-scale studies. This work presents a new clinical corpus, referred to as the COVID-19 Annotated Clinical Text (CACT) Corpus, which comprises 1,472 notes with detailed annotations characterizing COVID-19 diagnoses, testing, and clinical presentation. We introduce a span-based event extraction model that jointly extracts all annotated phenomena, achieving high performance in identifying COVID-19 and symptom events with associated assertion values (0.83-0.97 F1 for events and 0.73-0.79 F1 for assertions). Our span-based event extraction model outperforms an extractor built on MetaMapLite for the identification of symptoms with assertion values. In a secondary use application, we predicted COVID-19 test results using structured patient data (e.g. vital signs and laboratory results) and automatically extracted symptom information, to explore the clinical presentation of COVID-19. Automatically extracted symptoms improve COVID-19 prediction performance, beyond structured data alone.
Collapse
Affiliation(s)
- Kevin Lybarger
- Biomedical & Health Informatics, University of Washington, Box 358047, Seattle, WA 98109, USA.
| | - Mari Ostendorf
- Department of Electrical & Computer Engineering, University of Washington, Campus Box 352500 185, Seattle, WA 98195-2500, USA
| | - Matthew Thompson
- Department of Family Medicine, University of Washington, Box 354696, Seattle, WA 98195-2500, USA
| | - Meliha Yetisgen
- Biomedical & Health Informatics, University of Washington, Box 358047, Seattle, WA 98109, USA
| |
Collapse
|
36
|
El-allaly ED, Sarrouti M, En-Nahnahi N, Ouatik El Alaoui S. MTTLADE: A multi-task transfer learning-based method for adverse drug events extraction. Inf Process Manag 2021. [DOI: 10.1016/j.ipm.2020.102473] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
|
37
|
Peterson KS, Lewis J, Patterson OV, Chapman AB, Denhalter DW, Lye PA, Stevens VW, Gamage SD, Roselle GA, Wallace KS, Jones M. Automated Travel History Extraction From Clinical Notes for Informing the Detection of Emergent Infectious Disease Events: Algorithm Development and Validation. JMIR Public Health Surveill 2021; 7:e26719. [PMID: 33759790 PMCID: PMC7993087 DOI: 10.2196/26719] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Revised: 02/05/2021] [Accepted: 02/12/2021] [Indexed: 02/02/2023] Open
Abstract
Background Patient travel history can be crucial in evaluating evolving infectious disease events. Such information can be challenging to acquire in electronic health records, as it is often available only in unstructured text. Objective This study aims to assess the feasibility of annotating and automatically extracting travel history mentions from unstructured clinical documents in the Department of Veterans Affairs across disparate health care facilities and among millions of patients. Information about travel exposure augments existing surveillance applications for increased preparedness in responding quickly to public health threats. Methods Clinical documents related to arboviral disease were annotated following selection using a semiautomated bootstrapping process. Using annotated instances as training data, models were developed to extract from unstructured clinical text any mention of affirmed travel locations outside of the continental United States. Automated text processing models were evaluated, involving machine learning and neural language models for extraction accuracy. Results Among 4584 annotated instances, 2659 (58%) contained an affirmed mention of travel history, while 347 (7.6%) were negated. Interannotator agreement resulted in a document-level Cohen kappa of 0.776. Automated text processing accuracy (F1 85.6, 95% CI 82.5-87.9) and computational burden were acceptable such that the system can provide a rapid screen for public health events. Conclusions Automated extraction of patient travel history from clinical documents is feasible for enhanced passive surveillance public health systems. Without such a system, it would usually be necessary to manually review charts to identify recent travel or lack of travel, use an electronic health record that enforces travel history documentation, or ignore this potential source of information altogether. The development of this tool was initially motivated by emergent arboviral diseases. More recently, this system was used in the early phases of response to COVID-19 in the United States, although its utility was limited to a relatively brief window due to the rapid domestic spread of the virus. Such systems may aid future efforts to prevent and contain the spread of infectious diseases.
Collapse
Affiliation(s)
- Kelly S Peterson
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| | - Julia Lewis
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| | - Olga V Patterson
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| | - Alec B Chapman
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| | - Daniel W Denhalter
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Department of Rocky Mountain Cancer Data Systems, University of Utah, Salt Lake City, UT, United States
| | - Patricia A Lye
- National Infectious Diseases Service, Specialty Care Services, US Department of Veterans Affairs, Cincinnati, OH, United States
| | - Vanessa W Stevens
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| | - Shantini D Gamage
- National Infectious Diseases Service, Specialty Care Services, US Department of Veterans Affairs, Cincinnati, OH, United States.,Division of Infectious Diseases, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Gary A Roselle
- National Infectious Diseases Service, Specialty Care Services, US Department of Veterans Affairs, Cincinnati, OH, United States.,Division of Infectious Diseases, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States.,Cincinnati VA Medical Center, US Department of Veterans Affairs, Cincinnati, OH, United States
| | - Katherine S Wallace
- Office of Biosurveillance, Veterans Affairs Central Office, US Department of Veterans Affairs, Washington, DC, United States.,National Biosurveillance Integration Center, Countering Weapons of Mass Destruction, Department of Homeland Security, Washington, DC, United States
| | - Makoto Jones
- VA Salt Lake City Health Care System, US Department of Veterans Affairs, Salt Lake City, UT, United States.,Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
38
|
Jouffroy J, Feldman SF, Lerner I, Rance B, Burgun A, Neuraz A. Hybrid Deep Learning for Medication-Related Information Extraction From Clinical Texts in French: MedExt Algorithm Development Study. JMIR Med Inform 2021; 9:e17934. [PMID: 33724196 PMCID: PMC8077811 DOI: 10.2196/17934] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Revised: 12/29/2020] [Accepted: 01/20/2021] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Information related to patient medication is crucial for health care; however, up to 80% of the information resides solely in unstructured text. Manual extraction is difficult and time-consuming, and there is not a lot of research on natural language processing extracting medical information from unstructured text from French corpora. OBJECTIVE We aimed to develop a system to extract medication-related information from clinical text written in French. METHODS We developed a hybrid system combining an expert rule-based system, contextual word embedding (embedding for language model) trained on clinical notes, and a deep recurrent neural network (bidirectional long short term memory-conditional random field). The task consisted of extracting drug mentions and their related information (eg, dosage, frequency, duration, route, condition). We manually annotated 320 clinical notes from a French clinical data warehouse to train and evaluate the model. We compared the performance of our approach to those of standard approaches: rule-based or machine learning only and classic word embeddings. We evaluated the models using token-level recall, precision, and F-measure. RESULTS The overall F-measure was 89.9% (precision 90.8; recall: 89.2) when combining expert rules and contextualized embeddings, compared to 88.1% (precision 89.5; recall 87.2) without expert rules or contextualized embeddings. The F-measures for each category were 95.3% for medication name, 64.4% for drug class mentions, 95.3% for dosage, 92.2% for frequency, 78.8% for duration, and 62.2% for condition of the intake. CONCLUSIONS Associating expert rules, deep contextualized embedding, and deep neural networks improved medication information extraction. Our results revealed a synergy when associating expert knowledge and latent knowledge.
Collapse
Affiliation(s)
- Jordan Jouffroy
- Department of Biomedical Informatics, Necker-Enfants malades Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
| | - Sarah F Feldman
- Department of Biomedical Informatics, Necker-Enfants malades Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
| | - Ivan Lerner
- Department of Biomedical Informatics, Necker-Enfants malades Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
| | - Bastien Rance
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
- Department of Biomedical Informatics, Georges Pompidou European Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Anita Burgun
- Department of Biomedical Informatics, Necker-Enfants malades Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
| | - Antoine Neuraz
- Department of Biomedical Informatics, Necker-Enfants malades Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
- UMRS 1138 team 22, Institut National de la Santé et de la Recherche Médicale, Université de Paris, Paris, France
| |
Collapse
|
39
|
Ávila-Tomás JF, Mayer-Pujadas MA, Quesada-Varela VJ. [Artificial intelligence and its applications in medicine II: Current importance and practical applications]. Aten Primaria 2021; 53:81-88. [PMID: 32571595 PMCID: PMC7752970 DOI: 10.1016/j.aprim.2020.04.014] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Accepted: 04/22/2020] [Indexed: 12/16/2022] Open
Abstract
Technology and medicine follow a parallel path during the last decades. Technological advances are changing the concept of health and health needs are influencing the development of technology. Artificial intelligence (AI) is made up of a series of sufficiently trained logical algorithms from which machines are capable of making decisions for specific cases based on general rules. This technology has applications in the diagnosis and follow-up of patients with an individualized prognostic evaluation of them. Furthermore, if we combine this technology with robotics, we can create intelligent machines that make more efficient diagnostic proposals in their work. Therefore, AI is going to be a technology present in our daily work through machines or computer programs, which in a more or less transparent way for the user, will become a daily reality in health processes. Health professionals have to know this technology, its advantages and disadvantages, because it will be an integral part of our work. In these two articles we intend to give a basic vision of this technology adapted to doctors with a review of its history and evolution, its real applications at the present time and a vision of a future in which AI and Big Data will shape the personalized medicine that will characterize the 21st century.
Collapse
Affiliation(s)
- Jose Francisco Ávila-Tomás
- Medicina de Familia y Comunitaria, Centro de Salud Santa Isabel, Madrid, España; Medicina Preventiva y Salud Pública, Universidad Rey Juan Carlos, Móstoles, Madrid, España; Estrutura Organizativa de Xestión Integrada (EOXI), Vigo, Pontevedra, España.
| | - Miguel Angel Mayer-Pujadas
- Medicina de Familia y Comunitaria, Research Programme on Biomedical Informatics (GRIB), Instituto Hospital del Mar de Investigaciones Médicas, Barcelona, España; Universitat Pompeu Fabra, Barcelona, España; Miembro del Grupo de Trabajo de Innovación Tecnológica y Sistemas de Información de la semFYC
| | - Victor Julio Quesada-Varela
- Medicina de Familia y Comunitaria, Centro de Salud de A Guarda, A Guarda, Pontevedra, España; Estrutura Organizativa de Xestión Integrada (EOXI), Vigo, Pontevedra, España; Miembro del Grupo de Trabajo de Innovación Tecnológica y Sistemas de Información de la semFYC
| |
Collapse
|
40
|
AIM in Pharmacology and Drug Discovery. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_145-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
41
|
Yang X, Zhang H, He X, Bian J, Wu Y. Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models. JMIR Med Inform 2020; 8:e22982. [PMID: 33320104 PMCID: PMC7772072 DOI: 10.2196/22982] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 10/05/2020] [Accepted: 11/20/2020] [Indexed: 12/16/2022] Open
Abstract
Background Patients’ family history (FH) is a critical risk factor associated with numerous diseases. However, FH information is not well captured in the structured database but often documented in clinical narratives. Natural language processing (NLP) is the key technology to extract patients’ FH from clinical narratives. In 2019, the National NLP Clinical Challenge (n2c2) organized shared tasks to solicit NLP methods for FH information extraction. Objective This study presents our end-to-end FH extraction system developed during the 2019 n2c2 open shared task as well as the new transformer-based models that we developed after the challenge. We seek to develop a machine learning–based solution for FH information extraction without task-specific rules created by hand. Methods We developed deep learning–based systems for FH concept extraction and relation identification. We explored deep learning models including long short-term memory-conditional random fields and bidirectional encoder representations from transformers (BERT) as well as developed ensemble models using a majority voting strategy. To further optimize performance, we systematically compared 3 different strategies to use BERT output representations for relation identification. Results Our system was among the top-ranked systems (3 out of 21) in the challenge. Our best system achieved micro-averaged F1 scores of 0.7944 and 0.6544 for concept extraction and relation identification, respectively. After challenge, we further explored new transformer-based models and improved the performances of both subtasks to 0.8249 and 0.6775, respectively. For relation identification, our system achieved a performance comparable to the best system (0.6810) reported in the challenge. Conclusions This study demonstrated the feasibility of utilizing deep learning methods to extract FH information from clinical narratives.
Collapse
Affiliation(s)
- Xi Yang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States.,Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, United States
| | - Hansi Zhang
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States
| | - Xing He
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States.,Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, United States
| | - Yonghui Wu
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, United States.,Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL, United States
| |
Collapse
|
42
|
Abstract
OBJECTIVE To determine how the representation of women's health has changed in clinical studies over the course of 70 years. DESIGN Observational study of 71 866 research articles published between 1948 and 2018 in The BMJ. MAIN OUTCOME MEASURES The incidence of women-specific health topics over time. General linear, additive and segmented regression models were used to estimate trends. RESULTS Over 70 years, the overall odds that a word in a BMJ research article was 'woman' or 'women' increased by an annual factor of 1.023, but this rate of increase varied by clinical specialty with some showing little or no change. The odds that an article was about some aspect of women-specific health increased much more slowly, by an annual factor of 1.004. The incidence of articles about particular areas of women-specific medicine such as pregnancy did not show a general increase, but rather fluctuated over time. The incidence of articles making any mention of women, gender or sex declined between 1948 and 2005, after which it rose steeply so that by 2018 few papers made no mention of them at all. CONCLUSIONS Over time women have become ever more prominent in BMJ research articles. However, the importance of women-specific health topics has waxed and waned as researchers responded ephemerally to medical advances, public health programmes, and sociolegal changes. The appointment of a woman editor-inchief in 2005 may have had a dramatic effect on whether women were mentioned in research articles.
Collapse
Affiliation(s)
- Eva N Hamulyák
- Department of Vascular Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
| | - Austin J Brockmeier
- Department of Electrical & Computer Engineering, University of Delaware, Newark, Delaware, USA
| | - Johanna D Killas
- Health Studies Programme, University of Toronto, Toronto, Ontario, Canada
| | - Sophia Ananiadou
- Department of Computer Science, The University of Manchester National Centre for Text Mining, Manchester, UK
- The Alan Turing Institute, London, UK
| | - Saskia Middeldorp
- Department of Vascular Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
| | - Armand M Leroi
- Department of Life Sciences, Imperial College London, South Kensington Campus, London, UK
- Data Science Institute, Imperial College London, South Kensington Campus, London, UK
| |
Collapse
|
43
|
Abstract
OBJECTIVES We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes-diseases and drugs (or medications)-and relations between them. METHODS For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. RESULTS In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. CONCLUSIONS The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.
Collapse
Affiliation(s)
- Udo Hahn
- Jena University Language & Information Engineering (JULIE) Lab, Friedrich-Schiller-Universität Jena, Jena, Germany
| | - Michel Oleynik
- Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria
| |
Collapse
|
44
|
Dandala B, Joopudi V, Tsou CH, Liang JJ, Suryanarayanan P. Extraction of Information Related to Drug Safety Surveillance From Electronic Health Record Notes: Joint Modeling of Entities and Relations Using Knowledge-Aware Neural Attentive Models. JMIR Med Inform 2020; 8:e18417. [PMID: 32459650 PMCID: PMC7382020 DOI: 10.2196/18417] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 05/12/2020] [Accepted: 05/13/2020] [Indexed: 12/25/2022] Open
Abstract
BACKGROUND An adverse drug event (ADE) is commonly defined as "an injury resulting from medical intervention related to a drug." Providing information related to ADEs and alerting caregivers at the point of care can reduce the risk of prescription and diagnostic errors and improve health outcomes. ADEs captured in structured data in electronic health records (EHRs) as either coded problems or allergies are often incomplete, leading to underreporting. Therefore, it is important to develop capabilities to process unstructured EHR data in the form of clinical notes, which contain a richer documentation of a patient's ADE. Several natural language processing (NLP) systems have been proposed to automatically extract information related to ADEs. However, the results from these systems showed that significant improvement is still required for the automatic extraction of ADEs from clinical notes. OBJECTIVE This study aims to improve the automatic extraction of ADEs and related information such as drugs, their attributes, and reason for administration from the clinical notes of patients. METHODS This research was conducted using discharge summaries from the Medical Information Mart for Intensive Care III (MIMIC-III) database obtained through the 2018 National NLP Clinical Challenges (n2c2) annotated with drugs, drug attributes (ie, strength, form, frequency, route, dosage, duration), ADEs, reasons, and relations between drugs and other entities. We developed a deep learning-based system for extracting these drug-centric concepts and relations simultaneously using a joint method enhanced with contextualized embeddings, a position-attention mechanism, and knowledge representations. The joint method generated different sentence representations for each drug, which were then used to extract related concepts and relations simultaneously. Contextualized representations trained on the MIMIC-III database were used to capture context-sensitive meanings of words. The position-attention mechanism amplified the benefits of the joint method by generating sentence representations that capture long-distance relations. Knowledge representations were obtained from graph embeddings created using the US Food and Drug Administration Adverse Event Reporting System database to improve relation extraction, especially when contextual clues were insufficient. RESULTS Our system achieved new state-of-the-art results on the n2c2 data set, with significant improvements in recognizing crucial drug-reason (F1=0.650 versus F1=0.579) and drug-ADE (F1=0.490 versus F1=0.476) relations. CONCLUSIONS This study presents a system for extracting drug-centric concepts and relations that outperformed current state-of-the-art results and shows that contextualized embeddings, position-attention mechanisms, and knowledge graph embeddings effectively improve deep learning-based concepts and relation extraction. This study demonstrates the potential for deep learning-based methods to help extract real-world evidence from unstructured patient data for drug safety surveillance.
Collapse
|
45
|
Harris DR, Henderson DW, Corbeau A. sig2db: a Workflow for Processing Natural Language from Prescription Instructions for Clinical Data Warehouses. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2020; 2020:221-230. [PMID: 32477641 PMCID: PMC7233058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
We present sig2db as an open-source solution for clinical data warehouses desiring to process natural language from prescription instructions, often referred to as "sigs". In electronic prescribing, the sig is typically an unstructured text field intended to capture all requirements for medication administration. The sig captures certain fields that the structured data may lack such as days supply, time of day, or meal-time considerations. Our open-source software package facilitates the workflow needed to process sigs into a structured format usable by clinical data warehouses. Our solution focuses on extracting concepts from prescriptions in order to understand the intended semantics by leveraging known natural language processing tools. We demonstrate the utility of concept extraction from sigs and present our findings in processing 1023 unique sigs from 5.7 million unique prescriptions.
Collapse
Affiliation(s)
- Daniel R Harris
- Institute for Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Kentucky, Lexington, Kentucky 40506
- Center for Clinical and Translational Sciences, University of Kentucky, Lexington, KY 40506
| | - Darren W Henderson
- Center for Clinical and Translational Sciences, University of Kentucky, Lexington, KY 40506
| | - Alexandria Corbeau
- Center for Clinical and Translational Sciences, University of Kentucky, Lexington, KY 40506
| |
Collapse
|
46
|
Henry S, Buchan K, Filannino M, Stubbs A, Uzuner O. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. J Am Med Inform Assoc 2020; 27:3-12. [PMID: 31584655 PMCID: PMC7489085 DOI: 10.1093/jamia/ocz166] [Citation(s) in RCA: 94] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2019] [Revised: 08/19/2019] [Accepted: 08/23/2019] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE This article summarizes the preparation, organization, evaluation, and results of Track 2 of the 2018 National NLP Clinical Challenges shared task. Track 2 focused on extraction of adverse drug events (ADEs) from clinical records and evaluated 3 tasks: concept extraction, relation classification, and end-to-end systems. We perform an analysis of the results to identify the state of the art in these tasks, learn from it, and build on it. MATERIALS AND METHODS For all tasks, teams were given raw text of narrative discharge summaries, and in all the tasks, participants proposed deep learning-based methods with hand-designed features. In the concept extraction task, participants used sequence labelling models (bidirectional long short-term memory being the most popular), whereas in the relation classification task, they also experimented with instance-based classifiers (namely support vector machines and rules). Ensemble methods were also popular. RESULTS A total of 28 teams participated in task 1, with 21 teams in tasks 2 and 3. The best performing systems set a high performance bar with F1 scores of 0.9418 for concept extraction, 0.9630 for relation classification, and 0.8905 for end-to-end. However, the results were much lower for concepts and relations of Reasons and ADEs. These were often missed because local context is insufficient to identify them. CONCLUSIONS This challenge shows that clinical concept extraction and relation classification systems have a high performance for many concept types, but significant improvement is still required for ADEs and Reasons. Incorporating the larger context or outside knowledge will likely improve the performance of future systems.
Collapse
Affiliation(s)
- Sam Henry
- Department is Information Sciences and Technology, George Mason University, Fairfax, Virginia, USA
| | - Kevin Buchan
- Department of Information Science, University at Albany – State University of New York, Albany, New York, USA
| | - Michele Filannino
- Department is Information Sciences and Technology, George Mason University, Fairfax, Virginia, USA
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
| | - Amber Stubbs
- Department is Mathematics and Computer Science, Simmons University, Boston, Massachusetts, USA
| | - Ozlem Uzuner
- Department is Information Sciences and Technology, George Mason University, Fairfax, Virginia, USA
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|