1
|
Remy F, Demuynck K, Demeester T. BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights. J Am Med Inform Assoc 2024; 31:1844-1855. [PMID: 38412333 PMCID: PMC11339519 DOI: 10.1093/jamia/ocae029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 01/16/2024] [Accepted: 02/02/2024] [Indexed: 02/29/2024] Open
Abstract
OBJECTIVE In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in the training of semantic models for the biomedical and clinical domains. MATERIALS AND METHODS Drawing on the wealth of the Unified Medical Language System knowledge graph and harnessing cutting-edge LLMs, we propose a new state-of-the-art approach for obtaining high-fidelity representations of biomedical concepts and sentences, consisting of 3 steps: an improved contrastive learning phase, a novel self-distillation phase, and a weight averaging phase. RESULTS Through rigorous evaluations of diverse downstream tasks, we demonstrate consistent and substantial improvements over the previous state of the art for semantic textual similarity (STS), biomedical concept representation (BCR), and clinically named entity linking, across 15+ datasets. Besides our new state-of-the-art biomedical model for English, we also distill and release a multilingual model compatible with 50+ languages and finetuned on 7 European languages. DISCUSSION Many clinical pipelines can benefit from our latest models. Our new multilingual model enables a range of languages to benefit from our advancements in biomedical semantic representation learning, opening a new avenue for bioinformatics researchers around the world. As a result, we hope to see BioLORD-2023 becoming a precious tool for future biomedical applications. CONCLUSION In this article, we introduced BioLORD-2023, a state-of-the-art model for STS and BCR designed for the clinical domain.
Collapse
Affiliation(s)
- François Remy
- Internet and Data Science Lab, imec, Ghent University, Ghent, Belgium
| | - Kris Demuynck
- Internet and Data Science Lab, imec, Ghent University, Ghent, Belgium
| | - Thomas Demeester
- Internet and Data Science Lab, imec, Ghent University, Ghent, Belgium
| |
Collapse
|
2
|
Singh A, Krishnamoorthy S, Ortega JE. NeighBERT: Medical Entity Linking Using Relation-Induced Dense Retrieval. JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2024; 8:353-369. [PMID: 38681752 PMCID: PMC11052986 DOI: 10.1007/s41666-023-00136-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 05/08/2023] [Accepted: 07/03/2023] [Indexed: 05/01/2024]
Abstract
One of the common tasks in clinical natural language processing is medical entity linking (MEL) which involves mention detection followed by linking the mention to an entity in a knowledge base. One reason that MEL has not been solved is due to a problem that occurs in language where ambiguous texts can be resolved to several named entities. This problem is exacerbated when processing the text found in electronic health records. Recent work has shown that deep learning models based on transformers outperform previous methods on linking at higher rates of performance. We introduce NeighBERT, a custom pre-training technique which extends BERT (Devlin et al [1]) by encoding how entities are related within a knowledge graph. This technique adds relational context that has been traditionally missing in original BERT, helping resolve the ambiguity found in clinical text. In our experiments, NeighBERT improves the precision, recall, and F1-score of the state of the art by 1-3 points for named entity recognition and 10-15 points for MEL on two widely known clinical datasets. Supplementary Information The online version contains supplementary material available at 10.1007/s41666-023-00136-3.
Collapse
Affiliation(s)
- Ayush Singh
- inQbator AI, Evernorth Health Services, Saint Louis, MO USA
| | | | - John E. Ortega
- inQbator AI, Evernorth Health Services, Saint Louis, MO USA
| |
Collapse
|
3
|
Graham SS, Shifflet S, Amjad M, Claborn K. An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records. PLoS One 2024; 19:e0292170. [PMID: 38289927 PMCID: PMC10826931 DOI: 10.1371/journal.pone.0292170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Accepted: 09/14/2023] [Indexed: 02/01/2024] Open
Abstract
The goal of this study is to develop and validate a lightweight, interpretable machine learning (ML) classifier to identify opioid overdoses in emergency medical services (EMS) records. We conducted a comparative assessment of three feature engineering approaches designed for use with unstructured narrative data. Opioid overdose annotations were provided by two harm reduction paramedics and two supporting annotators trained to reliably match expert annotations. Candidate feature engineering techniques included term frequency-inverse document frequency (TF-IDF), a highly performant approach to concept vectorization, and a custom approach based on the count of empirically-identified keywords. Each feature set was trained using four model architectures: generalized linear model (GLM), Naïve Bayes, neural network, and Extreme Gradient Boost (XGBoost). Ensembles of trained models were also evaluated. The custom feature models were also assessed for variable importance to aid interpretation. Models trained using TF-IDF feature engineering ranged from AUROC = 0.59 (95% CI: 0.53-0.66) for the Naïve Bayes to AUROC = 0.76 (95% CI: 0.71-0.81) for the neural network. Models trained using concept vectorization features ranged from AUROC = 0.83 (95% 0.78-0.88)for the Naïve Bayes to AUROC = 0.89 (95% CI: 0.85-0.94) for the ensemble. Models trained using custom features were the most performant, with benchmarks ranging from AUROC = 0.92 (95% CI: 0.88-0.95) with the GLM to 0.93 (95% CI: 0.90-0.96) for the ensemble. The custom features model achieved positive predictive values (PPV) ranging for 80 to 100%, which represent substantial improvements over previously published EMS encounter opioid overdose classifiers. The application of this approach to county EMS data can productively inform local and targeted harm reduction initiatives.
Collapse
Affiliation(s)
- S. Scott Graham
- Department of Rhetoric & Writing, Center for Health Communication, University of Texas at Austin, Austin, TX, United States of Amedrica
- Addiction Research Institute, University of Texas at Austin, Austin, TX, United States of Amedrica
| | - Savannah Shifflet
- Addiction Research Institute, University of Texas at Austin, Austin, TX, United States of Amedrica
| | - Maaz Amjad
- Addiction Research Institute, University of Texas at Austin, Austin, TX, United States of Amedrica
| | - Kasey Claborn
- Addiction Research Institute, University of Texas at Austin, Austin, TX, United States of Amedrica
- Steve Hicks School of Social Work, University of Texas at Austin, Austin, TX, United States of Amedrica
| |
Collapse
|
4
|
Hänsel K, Dudgeon SN, Cheung KH, Durant TJS, Schulz WL. From Data to Wisdom: Biomedical Knowledge Graphs for Real-World Data Insights. J Med Syst 2023; 47:65. [PMID: 37195430 PMCID: PMC10191934 DOI: 10.1007/s10916-023-01951-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 04/15/2023] [Indexed: 05/18/2023]
Abstract
Graph data models are an emerging approach to structure clinical and biomedical information. These models offer intriguing opportunities for novel approaches in healthcare, such as disease phenotyping, risk prediction, and personalized precision care. The combination of data and information in a graph model to create knowledge graphs has rapidly expanded in biomedical research, but the integration of real-world data from the electronic health record has been limited. To broadly apply knowledge graphs to EHR and other real-world data, a deeper understanding of how to represent these data in a standardized graph model is needed. We provide an overview of the state-of-the-art research for clinical and biomedical data integration and summarize the potential to accelerate healthcare and precision medicine research through insight generation from integrated knowledge graphs.
Collapse
Affiliation(s)
- Katrin Hänsel
- Department of Laboratory Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Sarah N Dudgeon
- Department of Laboratory Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Kei-Hoi Cheung
- Section of Biomedical Informatics, Department of Emergency Medicine, Yale School of Medicine, 55 Park Street, PS 210, New Haven, CT, 06510, USA
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
| | - Thomas J S Durant
- Department of Laboratory Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Wade L Schulz
- Department of Laboratory Medicine, Yale School of Medicine, New Haven, CT, USA.
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA.
| |
Collapse
|
5
|
Li G, Siddharth L, Luo J. Embedding knowledge graph of patent metadata to measure knowledge proximity. J Assoc Inf Sci Technol 2023. [DOI: 10.1002/asi.24736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Affiliation(s)
- Guangtong Li
- Data‐Driven Innovation Lab, Engineering Product Development Pillar Singapore University of Technology and Design Singapore Singapore
| | - L. Siddharth
- Data‐Driven Innovation Lab, Engineering Product Development Pillar Singapore University of Technology and Design Singapore Singapore
| | - Jianxi Luo
- Data‐Driven Innovation Lab, Engineering Product Development Pillar Singapore University of Technology and Design Singapore Singapore
| |
Collapse
|
6
|
Lyu X, Costas R. Studying the cognitive relatedness between topics in the global science landscape: The case of Big Data research. J Inf Sci 2022. [DOI: 10.1177/01655515221121970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Taking Big Data research as a case study, this article intends to investigate the cognitive relatedness of research topics across the global science landscape to a focal topic. Several levels of cognitive relatedness are established depending on the citation distance between the citing publications and a core set of publications. The concept of citation generation is adopted for identifying and classifying other publications with different levels of relatedness to the core set. The micro publication-level classification system of Centre for Science and Technology Studies (CWTS) is applied for determining clusters of publication sets at the topic level. The overall cognitive relatedness of micro clusters to Big Data core publications are measured based on the mean citation generation of all the publications in corresponding clusters. In addition to the given clusters, this study also explores the ‘topics’ relatedness from a semantic point of view, by extracting high-frequency title terms of publications in each generation. Results show that data analysis methods and technologies are the topics with the strongest cognitive relatedness to Big Data research, while topics on physics and astronomy studies present the weakest relatedness. This approach allows assessment of relatedness between research topics by considering the citations distribution across multiple citation generations, and can provide useful insights to study and characterise topics with fuzzy boundaries or are difficult to delineate, thus representing a novel toolset relevant in the context of studying interdisciplinary research.
Collapse
Affiliation(s)
- Xiaozan Lyu
- Department of Administrative Management, School of Law, Zhejiang University City College, China
| | - Rodrigo Costas
- Centre for Science and Technology Studies (CWTS), Leiden University, The Netherlands; Centre for Research on Evaluation, Science and Technology (CREST), Stellenbosch University, South Africa
| |
Collapse
|
7
|
Empirical Analysis of Early Childhood Enlightenment Education Using Neural Network. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:3601941. [PMID: 36072750 PMCID: PMC9444361 DOI: 10.1155/2022/3601941] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 07/06/2022] [Accepted: 07/11/2022] [Indexed: 12/02/2022]
Abstract
This exploration aims to study the value orientation and essence of early childhood enlightenment education based on the deep neural network (DNN). Based on the acquisition and feature learning of cross-media education big data, the DNN correlation learning of cross-media education big data, and the intelligent search of cross-media education big data based on the DNN, the intelligent search system of cross-media children's enlightenment education big data based on DNN is designed and implemented. The system includes three functional modules: the feature learning module of cross-media infant enlightenment education big data, the deep semantic correlation learning module of cross-media childhood enlightenment education big data, and the intelligent search module of cross-media childhood enlightenment education big data based on DNN. This exploration realizes the acquisition and feature learning of big data of cross-media early childhood enlightenment education, DNN learning of cross-media education big data of early childhood enlightenment, and intelligent computing of cross-media education big data based on DNN. The experimental results show that the proposed system's mean average precision (MAP) performance is improved by 15.6% on the public dataset of early childhood enlightenment education published by the Ministry of Education. Moreover, the system has also significantly improved the MAP performance of the constructed dataset in the field of early childhood enlightenment education; that is, the MAP performance has been improved by 20.6% on the dataset of Taylor's University in Malaysia (NUS-WIDE). This exploration has certain theoretical significance and empirical value for early childhood enlightenment education research.
Collapse
|
8
|
Saeed N, Naveed H. Medical terminology-based computing system: a lightweight post-processing solution for out-of-vocabulary multi-word terms. Front Mol Biosci 2022; 9:928530. [PMID: 36032678 PMCID: PMC9411640 DOI: 10.3389/fmolb.2022.928530] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Accepted: 07/04/2022] [Indexed: 11/24/2022] Open
Abstract
The linguistic rules of medical terminology assist in gaining acquaintance with rare/complex clinical and biomedical terms. The medical language follows a Greek and Latin-inspired nomenclature. This nomenclature aids the stakeholders in simplifying the medical terms and gaining semantic familiarity. However, natural language processing models misrepresent rare and complex biomedical words. In this study, we present MedTCS—a lightweight, post-processing module—to simplify hybridized or compound terms into regular words using medical nomenclature. MedTCS enabled the word-based embedding models to achieve 100% coverage and enabled the BiowordVec model to achieve high correlation scores (0.641 and 0.603 in UMNSRS similarity and relatedness datasets, respectively) that significantly surpass the n-gram and sub-word approaches of FastText and BERT. In the downstream task of named entity recognition (NER), MedTCS enabled the latest clinical embedding model of FastText-OA-All-300d to improve the F1-score from 0.45 to 0.80 on the BC5CDR corpus and from 0.59 to 0.81 on the NCBI-Disease corpus, respectively. Similarly, in the drug indication classification task, our model was able to increase the coverage by 9% and the F1-score by 1%. Our results indicate that incorporating a medical terminology-based module provides distinctive contextual clues to enhance vocabulary as a post-processing step on pre-trained embeddings. We demonstrate that the proposed module enables the word embedding models to generate vectors of out-of-vocabulary words effectively. We expect that our study can be a stepping stone for the use of biomedical knowledge-driven resources in NLP.
Collapse
|
9
|
Chang E. A vector-based semantic relatedness measure using multiple relations within SNOMED CT and UMLS. J Biomed Inform 2022; 131:104118. [PMID: 35690349 DOI: 10.1016/j.jbi.2022.104118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2022] [Revised: 05/26/2022] [Accepted: 06/05/2022] [Indexed: 11/27/2022]
Abstract
OBJECTIVE To propose a new vector-based relatedness metric that derives word vectors from the intrinsic structure of biomedical ontologies, without consulting external resources such as large-scale biomedical corpora. MATERIALS AND METHODS SNOMED CT on the mapping layer of UMLS was used as a testbed ontology. Vectors were created for every concept at the end of all semantic relations-attribute-value relations and descendants as well as is_a relation-of the defining concept. The cosine similarity between the averages of those vectors with respect to each defining concept was computed to produce a final semantic relatedness. RESULTS Two benchmark sets that include a total of 62 biomedical term pairs were used for evaluation. Spearman's rank coefficient of the current method was 0.655, 0.744, and 0.742 with the relatedness rated by physicians, coders, and medical experts, respectively. The proposed method was comparable to a word-embedding method and outperformed path-based, information content-based, and another multiple relation-based relatedness metrics. DISCUSSION The current study demonstrated that the addition of attribute relations to the is_a hierarchy of SNOMED CT better conforms to the human sense of relatedness than models based on taxonomic relations. The current approach also showed that it is robust to the design inconsistency of ontologies. CONCLUSION Unlike the previous vector-based approach, the current study exploited the intrinsic semantic structure of an ontology, precluding the need for external textual resources to obtain context information of defining terms. Future research is recommended to prove the validity of the current method with other biomedical ontologies.
Collapse
Affiliation(s)
- Eunsuk Chang
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
| |
Collapse
|
10
|
Zhu Z, Li J, Huang J, Li Z, Zhang H, Chen S, Zhong Q, Xie Y, Hu S, Wang Y, Wang D, Yu G. An intelligent prediagnosis system for disease prediction and examination recommendation based on electronic medical record and a medical-semantic-aware convolution neural network (MSCNN) for pediatric chronic cough. Transl Pediatr 2022; 11:1216-1233. [PMID: 35958012 PMCID: PMC9360821 DOI: 10.21037/tp-22-275] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 07/10/2022] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Due to the phenotypic similarities among different pediatric respiratory diseases with chronic cough, primary doctors often misdiagnose and the misuse of examinations is prevalent. In the pre-diagnosis stage, the patients' chief complaints and other information in the electronic medical record (EMR) provide a powerful reference for respiratory experts to make preliminary disease judgment and examination plan. In this paper, we proposed an intelligent prediagnosis system to predict disease diagnosis and recommend examinations based on EMR text. METHODS We examined the clinical notes of 178,293 children with chronic cough symptoms from retrospective EMR data. The dataset is split into 7:3 for training and testing. From the testing set, we also extract 5% of samples for validation. We proposed a medical-semantic-aware convolution neural network (MSCNN) framework that can accomplish two downstream tasks from the same medical language model through transfer learning. First, a medical language model based on the word2vec algorithm was built to generate embeddings for the text data. Then, text convolutional neural network (TextCNN) was used to build models for disease prediction and examination recommendation. RESULTS We implemented 5 algorithms for disease prediction. In the disease prediction task, our algorithm outperformed the baseline methods on all metrics, with a top-1 accuracy (AC) of 0.68 and a top-3 AC of 0.923 on the testing set. By adding data enhancement, the top-3 AC reached 0.926. In the examination recommendation task, the overall AC on the testing set was 0.93 and the macro average (MA) F1-score was 0.88. The average area under the curve (AUC) on the training set was 0.97 while on the testing set it was 0.86. CONCLUSIONS We constructed an intelligent prediagnosis system with an MSCNN framework that can predict diseases and make examination recommendations based on EMR data. Our approach achieved good results on a retrospective clinical dataset and thus has great potential for the application of automated diagnosis assist in clinical practice during pre-diagnosis stage, which will provide help for primary level doctors or doctors in basic-level hospitals. Due to the generality of the proposed framework, it can be straight forwardly extended to prediagnosis for other diseases.
Collapse
Affiliation(s)
- Zhu Zhu
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China
| | - Jing Li
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China
| | - Jian Huang
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China
| | - Zheming Li
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China
| | | | - Siyu Chen
- Avain (Hangzhou) Technology Co., Ltd., Hangzhou, China
| | - Qianhui Zhong
- Avain (Hangzhou) Technology Co., Ltd., Hangzhou, China
| | - Yulan Xie
- Avain (Hangzhou) Technology Co., Ltd., Hangzhou, China
| | - Shasha Hu
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China
| | - Yinshuo Wang
- National Clinical Research Center for Child Health, Hangzhou, China.,Department of Pulmonology, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China
| | - Dejian Wang
- Department of R&D, Hangzhou Healink Technology, Hangzhou, China
| | - Gang Yu
- Department of Data and Information, The Children's Hospital, Zhejiang University School of Medicine, Hangzhou, China.,National Clinical Research Center for Child Health, Hangzhou, China.,Polytechnic Institute, Zhejiang University, Hangzhou, China
| |
Collapse
|
11
|
Kaur M, Costello J, Willis E, Kelm K, Reformat MZ, Bolduc FV. Deciphering Diversity of Mental Models in Neurodevelopmental Disorders: Knowledge Graph Representation of Public Data Using Natural Language Processing (Preprint). J Med Internet Res 2022; 24:e39888. [PMID: 35930346 PMCID: PMC9391978 DOI: 10.2196/39888] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 07/05/2022] [Accepted: 07/08/2022] [Indexed: 11/26/2022] Open
Abstract
Background Understanding how individuals think about a topic, known as the mental model, can significantly improve communication, especially in the medical domain where emotions and implications are high. Neurodevelopmental disorders (NDDs) represent a group of diagnoses, affecting up to 18% of the global population, involving differences in the development of cognitive or social functions. In this study, we focus on 2 NDDs, attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD), which involve multiple symptoms and interventions requiring interactions between 2 important stakeholders: parents and health professionals. There is a gap in our understanding of differences between mental models for each stakeholder, making communication between stakeholders more difficult than it could be. Objective We aim to build knowledge graphs (KGs) from web-based information relevant to each stakeholder as proxies of mental models. These KGs will accelerate the identification of shared and divergent concerns between stakeholders. The developed KGs can help improve knowledge mobilization, communication, and care for individuals with ADHD and ASD. Methods We created 2 data sets by collecting the posts from web-based forums and PubMed abstracts related to ADHD and ASD. We utilized the Unified Medical Language System (UMLS) to detect biomedical concepts and applied Positive Pointwise Mutual Information followed by truncated Singular Value Decomposition to obtain corpus-based concept embeddings for each data set. Each data set is represented as a KG using a property graph model. Semantic relatedness between concepts is calculated to rank the relation strength of concepts and stored in the KG as relation weights. UMLS disorder-relevant semantic types are used to provide additional categorical information about each concept’s domain. Results The developed KGs contain concepts from both data sets, with node sizes representing the co-occurrence frequency of concepts and edge sizes representing relevance between concepts. ADHD- and ASD-related concepts from different semantic types shows diverse areas of concerns and complex needs of the conditions. KG identifies converging and diverging concepts between health professionals literature (PubMed) and parental concerns (web-based forums), which may correspond to the differences between mental models for each stakeholder. Conclusions We show for the first time that generating KGs from web-based data can capture the complex needs of families dealing with ADHD or ASD. Moreover, we showed points of convergence between families and health professionals’ KGs. Natural language processing–based KG provides access to a large sample size, which is often a limiting factor for traditional in-person mental model mapping. Our work offers a high throughput access to mental model maps, which could be used for further in-person validation, knowledge mobilization projects, and basis for communication about potential blind spots from stakeholders in interactions about NDDs. Future research will be needed to identify how concepts could interact together differently for each stakeholder.
Collapse
Affiliation(s)
- Manpreet Kaur
- Department of Pediatrics, University of Alberta, Edmonton, AB, Canada
| | - Jeremy Costello
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
| | - Elyse Willis
- Department of Pediatrics, University of Alberta, Edmonton, AB, Canada
| | - Karen Kelm
- Department of Pediatrics, University of Alberta, Edmonton, AB, Canada
| | - Marek Z Reformat
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
- Information Technology Institute, University of Social Sciences, Łódź, Poland
| | - Francois V Bolduc
- Department of Pediatrics, University of Alberta, Edmonton, AB, Canada
- Department of Medical Genetics, University of Alberta, Edmonton, AB, Canada
- Women and Children Health Research Institute, University of Alberta, Edmonton, AB, Canada
- Neuroscience and Mental Health Research Institute, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
12
|
Nguyen V, Yip HY, Bajaj G, Wijesiriwardene T, Javangula V, Parthasarathy S, Sheth A, Bodenreider O. Context-Enriched Learning Models for Aligning Biomedical Vocabularies at Scale in the UMLS Metathesaurus. PROCEEDINGS OF THE ... INTERNATIONAL WORLD-WIDE WEB CONFERENCE. INTERNATIONAL WWW CONFERENCE 2022; 2022:1037-1046. [PMID: 36108322 PMCID: PMC9455675 DOI: 10.1145/3485447.3511946] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The Unified Medical Language System (UMLS) Metathesaurus construction process mainly relies on lexical algorithms and manual expert curation for integrating over 200 biomedical vocabularies. A lexical-based learning model (LexLM) was developed to predict synonymy among Metathesaurus terms and largely outperforms a rule-based approach (RBA) that approximates the current construction process. However, the LexLM has the potential for being improved further because it only uses lexical information from the source vocabularies, while the RBA also takes advantage of contextual information. We investigate the role of multiple types of contextual information available to the UMLS editors, namely source synonymy (SS), source semantic group (SG), and source hierarchical relations (HR), for the UMLS vocabulary alignment (UVA) problem. In this paper, we develop multiple variants of context-enriched learning models (ConLMs) by adding to the LexLM the types of contextual information listed above. We represent these context types in context-enriched knowledge graphs (ConKGs) with four variants ConSS, ConSG, ConHR, and ConAll. We train these ConKG embeddings using seven KG embedding techniques. We create the ConLMs by concatenating the ConKG embedding vectors with the word embedding vectors from the LexLM. We evaluate the performance of the ConLMs using the UVA generalization test datasets with hundreds of millions of pairs. Our extensive experiments show a significant performance improvement from the ConLMs over the LexLM, namely +5.0% in precision (93.75%), +0.69% in recall (93.23%), +2.88% in F1 (93.49%) for the best ConLM. Our experiments also show that the ConAll variant including the three context types takes more time, but does not always perform better than other variants with a single context type. Finally, our experiments show that the pairs of terms with high lexical similarity benefit most from adding contextual information, namely +6.56% in precision (94.97%), +2.13% in recall (93.23%), +4.35% in F1 (94.09%) for the best ConLM. The pairs with lower degrees of lexical similarity also show performance improvement with +0.85% in F1 (96%) for low similarity and +1.31% in F1 (96.34%) for no similarity. These results demonstrate the importance of using contextual information in the UVA problem.
Collapse
Affiliation(s)
- Vinh Nguyen
- National Library of Medicine, Bethesda, Maryland, USA
| | - Hong Yung Yip
- University of South Carolina, Columbia, South Carolina, USA
| | | | | | | | | | - Amit Sheth
- University of South Carolina, Columbia, South Carolina, USA
| | | |
Collapse
|
13
|
Kalyan KS, Rajasekharan A, Sangeetha S. AMMU: A survey of transformer-based biomedical pretrained language models. J Biomed Inform 2021; 126:103982. [PMID: 34974190 DOI: 10.1016/j.jbi.2021.103982] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 12/12/2021] [Accepted: 12/20/2021] [Indexed: 01/04/2023]
Abstract
Transformer-based pretrained language models (PLMs) have started a new era in modern natural language processing (NLP). These models combine the power of transformers, transfer learning, and self-supervised learning (SSL). Following the success of these models in the general domain, the biomedical research community has developed various in-domain PLMs starting from BioBERT to the latest BioELECTRA and BioALBERT models. We strongly believe there is a need for a survey paper that can provide a comprehensive survey of various transformer-based biomedical pretrained language models (BPLMs). In this survey, we start with a brief overview of foundational concepts like self-supervised learning, embedding layer and transformer encoder layers. We discuss core concepts of transformer-based PLMs like pretraining methods, pretraining tasks, fine-tuning methods, and various embedding types specific to biomedical domain. We introduce a taxonomy for transformer-based BPLMs and then discuss all the models. We discuss various challenges and present possible solutions. We conclude by highlighting some of the open issues which will drive the research community to further improve transformer-based BPLMs. The list of all the publicly available transformer-based BPLMs along with their links is provided at https://mr-nlp.github.io/posts/2021/05/transformer-based-biomedical-pretrained-language-models-list/.
Collapse
|
14
|
Mitra A, Rawat BPS, McManus DD, Yu H. Relation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study. JMIR Med Inform 2021; 9:e27527. [PMID: 34255697 PMCID: PMC8285744 DOI: 10.2196/27527] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 03/19/2021] [Accepted: 05/30/2021] [Indexed: 01/05/2023] Open
Abstract
BACKGROUND Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. OBJECTIVE In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event-related relation classification. METHODS We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). RESULTS Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. CONCLUSIONS In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation.
Collapse
Affiliation(s)
- Avijit Mitra
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States
| | - Bhanu Pratap Singh Rawat
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States
| | - David D McManus
- Department of Medicine, University of Massachusetts Medical School, Worcester, MA, United States
| | - Hong Yu
- College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA, United States.,Department of Medicine, University of Massachusetts Medical School, Worcester, MA, United States.,Department of Computer Science, University of Massachusetts Lowell, Lowell, MA, United States.,Center for Healthcare Organization and Implementation Research, Bedford Veterans Affairs Medical Center, Bedford, MA, United States
| |
Collapse
|
15
|
Nguyen V, Yip HY, Bodenreider O. Biomedical Vocabulary Alignment at Scale in the UMLS Metathesaurus. PROCEEDINGS OF THE ... INTERNATIONAL WORLD-WIDE WEB CONFERENCE. INTERNATIONAL WWW CONFERENCE 2021; 2021:2672-2683. [PMID: 34514472 PMCID: PMC8434895 DOI: 10.1145/3442381.3450128] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/04/2022]
Abstract
With 214 source vocabularies, the construction and maintenance process of the UMLS (Unified Medical Language System) Metathesaurus terminology integration system is costly, time-consuming, and error-prone as it primarily relies on (1) lexical and semantic processing for suggesting groupings of synonymous terms, and (2) the expertise of UMLS editors for curating these synonymy predictions. This paper aims to improve the UMLS Metathesaurus construction process by developing a novel supervised learning approach for improving the task of suggesting synonymous pairs that can scale to the size and diversity of the UMLS source vocabularies. We evaluate this deep learning (DL) approach against a rule-based approach (RBA) that approximates the current UMLS Metathesaurus construction process. The key to the generalizability of our approach is the use of various degrees of lexical similarity in negative pairs during the training process. Our initial experiments demonstrate the strong performance across multiple datasets of our DL approach in terms of recall (91-92%), precision (88-99%), and F1 score (89-95%). Our DL approach largely outperforms the RBA method in recall (+23%), precision (+2.4%), and F1 score (+14.1%). This novel approach has great potential for improving the UMLS Metathesaurus construction process by providing better synonymy suggestions to the UMLS editors.
Collapse
Affiliation(s)
- Vinh Nguyen
- National Library of Medicine, Bethesda, Maryland, USA
| | - Hong Yung Yip
- University of South Carolina, Columbia, South Carolina, USA
| | | |
Collapse
|
16
|
Humphreys BL, Del Fiol G, Xu H. The UMLS knowledge sources at 30: indispensable to current research and applications in biomedical informatics. J Am Med Inform Assoc 2020; 27:1499-1501. [PMID: 33059366 DOI: 10.1093/jamia/ocaa208] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Indexed: 01/22/2023] Open
Affiliation(s)
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
| | - Hua Xu
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| |
Collapse
|