1
|
Zhang Z, Jiang A. Interactive dual-stream contrastive learning for radiology report generation. J Biomed Inform 2024; 157:104718. [PMID: 39209086 DOI: 10.1016/j.jbi.2024.104718] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2024] [Revised: 08/08/2024] [Accepted: 08/25/2024] [Indexed: 09/04/2024]
Abstract
Radiology report generation automates diagnostic narrative synthesis from medical imaging data. Current report generation methods primarily employ knowledge graphs for image enhancement, neglecting the interpretability and guiding function of the knowledge graphs themselves. Additionally, few approaches leverage the stable modal alignment information from multimodal pre-trained models to facilitate the generation of radiology reports. We propose the Terms-Guided Radiology Report Generation (TGR), a simple and practical model for generating reports guided primarily by anatomical terms. Specifically, we utilize a dual-stream visual feature extraction module comprised of detail extraction module and a frozen multimodal pre-trained model to separately extract visual detail features and semantic features. Furthermore, a Visual Enhancement Module (VEM) is proposed to further enrich the visual features, thereby facilitating the generation of a list of anatomical terms. We integrate anatomical terms with image features and proceed to engage contrastive learning with frozen text embeddings, utilizing the stable feature space from these embeddings to boost modal alignment capabilities further. Our model incorporates the capability for manual input, enabling it to generate a list of organs for specifically focused abnormal areas or to produce more accurate single-sentence descriptions based on selected anatomical terms. Comprehensive experiments demonstrate the effectiveness of our method in report generation tasks, our TGR-S model reduces training parameters by 38.9% while performing comparably to current state-of-the-art models, and our TGR-B model exceeds the best baseline models across multiple metrics.
Collapse
Affiliation(s)
- Ziqi Zhang
- College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan 030600, China
| | - Ailian Jiang
- College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan 030600, China.
| |
Collapse
|
2
|
Jani M, Alfattni G, Belousov M, Laidlaw L, Zhang Y, Cheng M, Webb K, Hamilton R, Kanter AS, Dixon WG, Nenadic G. Development and evaluation of a text analytics algorithm for automated application of national COVID-19 shielding criteria in rheumatology patients. Ann Rheum Dis 2024; 83:1082-1091. [PMID: 38575324 PMCID: PMC11287580 DOI: 10.1136/ard-2024-225544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Accepted: 03/26/2024] [Indexed: 04/06/2024]
Abstract
INTRODUCTION At the beginning of the COVID-19 pandemic, the UK's Scientific Committee issued extreme social distancing measures, termed 'shielding', aimed at a subpopulation deemed extremely clinically vulnerable to infection. National guidance for risk stratification was based on patients' age, comorbidities and immunosuppressive therapies, including biologics that are not captured in primary care records. This process required considerable clinician time to manually review outpatient letters. Our aim was to develop and evaluate an automated shielding algorithm by text-mining outpatient letter diagnoses and medications, reducing the need for future manual review. METHODS Rheumatology outpatient letters from a large UK foundation trust were retrieved. Free-text diagnoses were processed using Intelligent Medical Objects software (Concept Tagger), which used interface terminology for each condition mapped to Systematized Medical Nomenclature for Medicine-Clinical Terminology (SNOMED-CT) codes. We developed the Medication Concept Recognition tool (Named Entity Recognition) to retrieve medications' type, dose, duration and status (active/past) at the time of the letter. Age, diagnosis and medication variables were then combined to calculate a shielding score based on the most recent letter. The algorithm's performance was evaluated using clinical review as the gold standard. The time taken to deploy the developed algorithm on a larger patient subset was measured. RESULTS In total, 5942 free-text diagnoses were extracted and mapped to SNOMED-CT, with 13 665 free-text medications (n=803 patients). The automated algorithm demonstrated a sensitivity of 80% (95% CI: 75%, 85%) and specificity of 92% (95% CI: 90%, 94%). Positive likelihood ratio was 10 (95% CI: 8, 14), negative likelihood ratio was 0.21 (95% CI: 0.16, 0.28) and F1 score was 0.81. Evaluation of mismatches revealed that the algorithm performed correctly against the gold standard in most cases. The developed algorithm was then deployed on records from an additional 15 865 patients, which took 18 hours for data extraction and 1 hour to deploy. DISCUSSION An automated algorithm for risk stratification has several advantages including reducing clinician time for manual review to allow more time for direct care, improving efficiency and increasing transparency in individual patient communication. It has the potential to be adapted for future public health initiatives that require prompt automated review of hospital outpatient letters.
Collapse
Affiliation(s)
- Meghna Jani
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, The University of Manchester, Manchester, UK
- Department of Rheumatology, Northern Care Alliance NHS Foundation Trust Salford Care Organisation, Salford, UK
- NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK
| | - Ghada Alfattni
- Department of Computer Science, The University of Manchester, Manchester, UK
- Department of Computer Science, Jamoum University College, Umm Al-Qura University, Makkah, Saudi Arabia
| | - Maksim Belousov
- Department of Computer Science, The University of Manchester, Manchester, UK
| | - Lynn Laidlaw
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, The University of Manchester, Manchester, UK
| | - Yuanyuan Zhang
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, The University of Manchester, Manchester, UK
| | - Michael Cheng
- Department of Business Intelligence, Northern Care Alliance NHS Foundation Trust, Salford Care Organisation, Salford, UK
| | - Karim Webb
- Department of Business Intelligence, Northern Care Alliance NHS Foundation Trust, Salford Care Organisation, Salford, UK
| | - Robyn Hamilton
- Department of Business Intelligence, Northern Care Alliance NHS Foundation Trust, Salford Care Organisation, Salford, UK
| | - Andrew S Kanter
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - William G Dixon
- Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, The University of Manchester, Manchester, UK
- Department of Rheumatology, Northern Care Alliance NHS Foundation Trust Salford Care Organisation, Salford, UK
- NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK
| | - Goran Nenadic
- Department of Computer Science, The University of Manchester, Manchester, UK
| |
Collapse
|
3
|
Patel MA, Daley M, Van Nynatten LR, Slessarev M, Cepinskas G, Fraser DD. A reduced proteomic signature in critically ill Covid-19 patients determined with plasma antibody micro-array and machine learning. Clin Proteomics 2024; 21:33. [PMID: 38760690 PMCID: PMC11100131 DOI: 10.1186/s12014-024-09488-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 05/06/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND COVID-19 is a complex, multi-system disease with varying severity and symptoms. Identifying changes in critically ill COVID-19 patients' proteomes enables a better understanding of markers associated with susceptibility, symptoms, and treatment. We performed plasma antibody microarray and machine learning analyses to identify novel proteins of COVID-19. METHODS A case-control study comparing the concentration of 2000 plasma proteins in age- and sex-matched COVID-19 inpatients, non-COVID-19 sepsis controls, and healthy control subjects. Machine learning was used to identify a unique proteome signature in COVID-19 patients. Protein expression was correlated with clinically relevant variables and analyzed for temporal changes over hospitalization days 1, 3, 7, and 10. Expert-curated protein expression information was analyzed with Natural language processing (NLP) to determine organ- and cell-specific expression. RESULTS Machine learning identified a 28-protein model that accurately differentiated COVID-19 patients from ICU non-COVID-19 patients (accuracy = 0.89, AUC = 1.00, F1 = 0.89) and healthy controls (accuracy = 0.89, AUC = 1.00, F1 = 0.88). An optimal nine-protein model (PF4V1, NUCB1, CrkL, SerpinD1, Fen1, GATA-4, ProSAAS, PARK7, and NET1) maintained high classification ability. Specific proteins correlated with hemoglobin, coagulation factors, hypertension, and high-flow nasal cannula intervention (P < 0.01). Time-course analysis of the 28 leading proteins demonstrated no significant temporal changes within the COVID-19 cohort. NLP analysis identified multi-system expression of the key proteins, with the digestive and nervous systems being the leading systems. CONCLUSIONS The plasma proteome of critically ill COVID-19 patients was distinguishable from that of non-COVID-19 sepsis controls and healthy control subjects. The leading 28 proteins and their subset of 9 proteins yielded accurate classification models and are expressed in multiple organ systems. The identified COVID-19 proteomic signature helps elucidate COVID-19 pathophysiology and may guide future COVID-19 treatment development.
Collapse
Affiliation(s)
- Maitray A Patel
- Epidemiology and Biostatistics, Western University, London, ON, N6A 3K7, Canada
| | - Mark Daley
- Epidemiology and Biostatistics, Western University, London, ON, N6A 3K7, Canada
- Computer Science, Western University, London, ON, N6A 3K7, Canada
| | | | - Marat Slessarev
- Medicine, Western University, London, ON, N6A 3K7, Canada
- Lawson Health Research Institute, London, ON, N6C 2R5, Canada
| | - Gediminas Cepinskas
- Lawson Health Research Institute, London, ON, N6C 2R5, Canada
- Medical Biophysics, Western University, London, ON, N6A 3K7, Canada
| | - Douglas D Fraser
- Lawson Health Research Institute, London, ON, N6C 2R5, Canada.
- Children's Health Research Institute, London, ON, N6C 4V3, Canada.
- Pediatrics, Western University, London, ON, N6A 3K7, Canada.
- Clinical Neurological Sciences, Western University, London, ON, N6A 3K7, Canada.
- Physiology & Pharmacology, Western University, London, ON, N6A 3K7, Canada.
- London Health Sciences Centre, 800 Commissioners Road East, London, ON, N6A 5W9, Canada.
| |
Collapse
|
4
|
Böhringer D, Angelova P, Fuhrmann L, Zimmermann J, Schargus M, Eter N, Reinhard T. Automatic inference of ICD-10 codes from German ophthalmologic physicians' letters using natural language processing. Sci Rep 2024; 14:9035. [PMID: 38641674 PMCID: PMC11031573 DOI: 10.1038/s41598-024-59926-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 04/16/2024] [Indexed: 04/21/2024] Open
Abstract
Physicians' letters are the optimal source of diagnoses for registries. However, most registries demand for diagnosis codes such as ICD-10. We herein describe an algorithm that infers ICD-10 codes from German ophthalmologic physicians' letters. We assess the method in three German eye hospitals. Our algorithm is based on the nearest-neighbor method as well as on a large thesaurus for ICD-10 codes. This thesaurus was embedded into a Word2Vec space created from anonymized physicians' reports of the first hospital. For evaluation, each of the three hospitals sent all diagnoses taken from 100 letters. The inferred ICD-10 codes were evaluated for correctness by the senders. A total of 3332 natural language terms had been sent in (812 hospital one, 1473 hospital two, 1047 hospital three). A total of 526 non-diagnoses were excluded upfront. 2806 ICD-10 codes were inferred (771 hospital one, 1226 hospital two, 809 hospital three). In the first hospital, 98% were fully correct and 99% correct at the level of the superordinate disease concept. The percentages in hospital two were 69% and 86%. The respective numbers for hospital three were 69% and 91%. Our simple method is capable of inferring ICD-10 codes for German natural language diagnoses, especially when the embedding space has been built with physicians' letters from the same hospital. The method may yield sufficient accuracy for many tasks in the multi-centric setting and can easily be adapted to other languages/specialities.
Collapse
Affiliation(s)
- D Böhringer
- Eye Center of the University Hospital Freiburg, Medical Faculty of the Albert-Ludwigs-University Freiburg, Freiburg, Germany.
| | - P Angelova
- Eye Center of the University Hospital Freiburg, Medical Faculty of the Albert-Ludwigs-University Freiburg, Freiburg, Germany
| | - L Fuhrmann
- Department of Ophthalmology, Asklepios Hospital Nord-Heidberg, Hamburg, Germany
| | - J Zimmermann
- Department of Ophthalmology, Medical Center, University of Münster, Münster, Germany
| | - M Schargus
- Department of Ophthalmology, Asklepios Hospital Nord-Heidberg, Hamburg, Germany
| | - N Eter
- Department of Ophthalmology, Medical Center, University of Münster, Münster, Germany
| | - T Reinhard
- Eye Center of the University Hospital Freiburg, Medical Faculty of the Albert-Ludwigs-University Freiburg, Freiburg, Germany
| |
Collapse
|
5
|
Patel MA, Fraser DD, Daley M, Cepinskas G, Veraldi N, Grazioli S. The plasma proteome differentiates the multisystem inflammatory syndrome in children (MIS-C) from children with SARS-CoV-2 negative sepsis. Mol Med 2024; 30:51. [PMID: 38632526 PMCID: PMC11022403 DOI: 10.1186/s10020-024-00806-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 03/09/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND The Multi-System Inflammatory Syndrome in Children (MIS-C) can develop several weeks after SARS-CoV-2 infection and requires a distinct treatment protocol. Distinguishing MIS-C from SARS-CoV-2 negative sepsis (SCNS) patients is important to quickly institute the correct therapies. We performed targeted proteomics and machine learning analysis to identify novel plasma proteins of MIS-C for early disease recognition. METHODS A case-control study comparing the expression of 2,870 unique blood proteins in MIS-C versus SCNS patients, measured using proximity extension assays. The 2,870 proteins were reduced in number with either feature selection alone or with a prior COMBAT-Seq batch effect adjustment. The leading proteins were correlated with demographic and clinical variables. Organ system and cell type expression patterns were analyzed with Natural Language Processing (NLP). RESULTS The cohorts were well-balanced for age and sex. Of the 2,870 unique blood proteins, 58 proteins were identified with feature selection (FDR-adjusted P < 0.005, P < 0.0001; accuracy = 0.96, AUC = 1.00, F1 = 0.95), and 15 proteins were identified with a COMBAT-Seq batch effect adjusted feature selection (FDR-adjusted P < 0.05, P < 0.0001; accuracy = 0.92, AUC = 1.00, F1 = 0.89). All of the latter 15 proteins were present in the former 58-protein model. Several proteins were correlated with illness severity scores, length of stay, and interventions (LTA4H, PTN, PPBP, and EGF; P < 0.001). NLP analysis highlighted the multi-system nature of MIS-C, with the 58-protein set expressed in all organ systems; the highest levels of expression were found in the digestive system. The cell types most involved included leukocytes not yet determined, lymphocytes, macrophages, and platelets. CONCLUSIONS The plasma proteome of MIS-C patients was distinct from that of SCNS. The key proteins demonstrated expression in all organ systems and most cell types. The unique proteomic signature identified in MIS-C patients could aid future diagnostic and therapeutic advancements, as well as predict hospital length of stays, interventions, and mortality risks.
Collapse
Affiliation(s)
- Maitray A Patel
- Epidemiology and Biostatistics, Western University, N6A 3K7, London, ON, Canada
| | - Douglas D Fraser
- Lawson Health Research Institute, N6C 2R5, London, ON, Canada.
- Children's Health Research Institute, N6C 4V3, London, ON, Canada.
- Pediatrics, Western University, N6A 3K7, London, ON, Canada.
- Clinical Neurological Sciences, Western University, N6A 3K7, London, ON, Canada.
- Physiology & Pharmacology, Western University, N6A 3K7, London, ON, Canada.
- London Health Sciences Centre, Room C2-C82, 800 Commissioners Road East, N6A 5W9, London, ON, Canada.
| | - Mark Daley
- Epidemiology and Biostatistics, Western University, N6A 3K7, London, ON, Canada
- Computer Science, Western University, N6A 3K7, London, ON, Canada
| | - Gediminas Cepinskas
- Lawson Health Research Institute, N6C 2R5, London, ON, Canada
- Medical Biophysics, Western University, N6A 3K7, London, ON, Canada
| | - Noemi Veraldi
- Department of Pediatrics, Gynaecology and Obstetrics, Faculty of Medicine, University of Geneva, Geneva, Switzerland
| | - Serge Grazioli
- Department of Pediatrics, Gynaecology and Obstetrics, Faculty of Medicine, University of Geneva, Geneva, Switzerland
- Division of Neonatal and Pediatric Intensive Care, Department of Child, Woman, and Adolescent Medicine, Geneva University Hospitals, Geneva, Switzerland
| |
Collapse
|
6
|
Mateu-Sanz M, Fuenteslópez CV, Uribe-Gomez J, Haugen HJ, Pandit A, Ginebra MP, Hakimi O, Krallinger M, Samara A. Redefining biomaterial biocompatibility: challenges for artificial intelligence and text mining. Trends Biotechnol 2024; 42:402-417. [PMID: 37858386 DOI: 10.1016/j.tibtech.2023.09.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 09/25/2023] [Accepted: 09/26/2023] [Indexed: 10/21/2023]
Abstract
The surge in 'Big data' has significantly influenced biomaterials research and development, with vast data volumes emerging from clinical trials, scientific literature, electronic health records, and other sources. Biocompatibility is essential in developing safe medical devices and biomaterials to perform as intended without provoking adverse reactions. Therefore, establishing an artificial intelligence (AI)-driven biocompatibility definition has become decisive for automating data extraction and profiling safety effectiveness. This definition should both reflect the attributes related to biocompatibility and be compatible with computational data-mining methods. Here, we discuss the need for a comprehensive and contemporary definition of biocompatibility and the challenges in developing one. We also identify the key elements that comprise biocompatibility, and propose an integrated biocompatibility definition that enables data-mining approaches.
Collapse
Affiliation(s)
- Miguel Mateu-Sanz
- Biomaterials, Biomechanics, and Tissue Engineering Group, Department of Materials Science and Engineering, Universitat Politècnica de Catalunya, Barcelona 08019, Spain
| | - Carla V Fuenteslópez
- Institute of Biomedical Engineering, Botnar Research Centre, Nuffield Orthopaedic Centre, University of Oxford, Oxford OX3 7LD, UK
| | - Juan Uribe-Gomez
- CÚRAM, SFI Research Centre for Medical Devices, University of Galway, Galway H92 W2TY, Ireland
| | - Håvard Jostein Haugen
- Department of Biomaterials, Center for Functional Tissue Reconstruction, Faculty of Dentistry, University of Oslo, Oslo 0317, Norway
| | - Abhay Pandit
- CÚRAM, SFI Research Centre for Medical Devices, University of Galway, Galway H92 W2TY, Ireland
| | - Maria-Pau Ginebra
- Biomaterials, Biomechanics, and Tissue Engineering Group, Department of Materials Science and Engineering, Universitat Politècnica de Catalunya, Barcelona 08019, Spain
| | - Osnat Hakimi
- aMoon Ventures, Yerushalaim Rd 34, Ra'anana 4350108, Israel
| | | | - Athina Samara
- Department of Biomaterials, Center for Functional Tissue Reconstruction, Faculty of Dentistry, University of Oslo, Oslo 0317, Norway.
| |
Collapse
|
7
|
Gu S, Lee EW, Zhang W, Simpson RL, Hertzberg VS, Ho JC. Evaluating Natural Language Processing Packages for Predicting Hospital-Acquired Pressure Injuries From Clinical Notes. Comput Inform Nurs 2024; 42:184-192. [PMID: 37607706 PMCID: PMC10884344 DOI: 10.1097/cin.0000000000001053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Incidence of hospital-acquired pressure injury, a key indicator of nursing quality, is directly proportional to adverse outcomes, increased hospital stays, and economic burdens on patients, caregivers, and society. Thus, predicting hospital-acquired pressure injury is important. Prediction models use structured data more often than unstructured notes, although the latter often contain useful patient information. We hypothesize that unstructured notes, such as nursing notes, can predict hospital-acquired pressure injury. We evaluate the impact of using various natural language processing packages to identify salient patient information from unstructured text. We use named entity recognition to identify keywords, which comprise the feature space of our classifier for hospital-acquired pressure injury prediction. We compare scispaCy and Stanza, two different named entity recognition models, using unstructured notes in Medical Information Mart for Intensive Care III, a publicly available ICU data set. To assess the impact of vocabulary size reduction, we compare the use of all clinical notes with only nursing notes. Our results suggest that named entity recognition extraction using nursing notes can yield accurate models. Moreover, the extracted keywords play a significant role in the prediction of hospital-acquired pressure injury.
Collapse
Affiliation(s)
- Siyi Gu
- Author Affiliations: Department of Computer Science, Center for Data Science (Ms Gu, Mr Lee, and Dr Ho), and Nell Hodgson Woodruff School of Nursing (Drs Zhang, Simpson, and Hertzberg), Emory University, Atlanta, GA
| | | | | | | | | | | |
Collapse
|
8
|
Wang R, Jayathunge K, Page R, Li H, Zhang JJ, Yang X. Hybrid architecture based intelligent diagnosis assistant for GP. BMC Med Inform Decis Mak 2024; 24:15. [PMID: 38200559 PMCID: PMC10777579 DOI: 10.1186/s12911-023-02398-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Accepted: 12/07/2023] [Indexed: 01/12/2024] Open
Abstract
As the first point of contact for patients, General Practitioners (GPs) play a crucial role in the National Health Service (NHS). An accurate primary diagnosis from the GP can alleviate the burden on specialists and reduce the time needed to re-confirm the patient's condition, allowing for more efficient further examinations. However, GPs have broad but less specialized knowledge, which limits the accuracy of their diagnosis. Therefore, it is imperative to introduce an intelligent system to assist GPs in making decisions. This paper introduces two data augmentation methods, the Complaint Symptoms Integration Method and Symptom Dot Separating Method, to integrate essential information into the Integration dataset. Additionally, it proposes a hybrid architecture that fuses the features of words from different representation spaces. Experiments demonstrate that, compared to commonly used pre-trained attention-based models, our hybrid architecture delivers the best classification performance for four common neurological diseases on the enhanced Integration dataset. For example, the classification accuracy of the BERT+CNN hybrid architecture is 0.897, which is a 5.1% improvement over both BERT and CNN with 0.846. Finally, this paper develops an AI diagnosis assistant web application that leverages the superior performance of this architecture to help GPs complete primary diagnosis efficiently and accurately.
Collapse
Affiliation(s)
- Ruibin Wang
- National Centre for Computer Animation, Bournemouth University, Bournemouth, UK.
| | - Kavisha Jayathunge
- National Centre for Computer Animation, Bournemouth University, Bournemouth, UK
| | - Rupert Page
- Poole Hospital NHS Foundation Trust, Poole, UK
| | - Hailing Li
- Animation and Digital Art, Communication University of China, Beijing, China
| | - Jian Jun Zhang
- National Centre for Computer Animation, Bournemouth University, Bournemouth, UK
| | - Xiaosong Yang
- National Centre for Computer Animation, Bournemouth University, Bournemouth, UK
| |
Collapse
|
9
|
Liao Y, Liu H, Spasić I. Fine-tuning coreference resolution for different styles of clinical narratives. J Biomed Inform 2024; 149:104578. [PMID: 38122841 DOI: 10.1016/j.jbi.2023.104578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 11/22/2023] [Accepted: 12/12/2023] [Indexed: 12/23/2023]
Abstract
OBJECTIVE Coreference resolution (CR) is a natural language processing (NLP) task that is concerned with finding all expressions within a single document that refer to the same entity. This makes it crucial in supporting downstream NLP tasks such as summarization, question answering and information extraction. Despite great progress in CR, our experiments have highlighted a substandard performance of the existing open-source CR tools in the clinical domain. We set out to explore some practical solutions to fine-tune their performance on clinical data. METHODS We first explored the possibility of automatically producing silver standards following the success of such an approach in other clinical NLP tasks. We designed an ensemble approach that leverages multiple models to automatically annotate co-referring mentions. Subsequently, we looked into other ways of incorporating human feedback to improve the performance of an existing neural network approach. We proposed a semi-automatic annotation process to facilitate the manual annotation process. We also compared the effectiveness of active learning relative to random sampling in an effort to further reduce the cost of manual annotation. RESULTS Our experiments demonstrated that the silver standard approach was ineffective in fine-tuning the CR models. Our results indicated that active learning should also be applied with caution. The semi-automatic annotation approach combined with continued training was found to be well suited for the rapid transfer of CR models under low-resource conditions. The ensemble approach demonstrated a potential to further improve accuracy by leveraging multiple fine-tuned models. CONCLUSION Overall, we have effectively transferred a general CR model to a clinical domain. Our findings based on extensive experimentation have been summarized into practical suggestions for rapid transferring of CR models across different styles of clinical narratives.
Collapse
Affiliation(s)
- Yuxiang Liao
- School of Computer Science and Informatics, Cardiff University, United Kingdom.
| | - Hantao Liu
- School of Computer Science and Informatics, Cardiff University, United Kingdom.
| | - Irena Spasić
- School of Computer Science and Informatics, Cardiff University, United Kingdom.
| |
Collapse
|
10
|
Ouis MY, A Akhloufi M. Deep learning for report generation on chest X-ray images. Comput Med Imaging Graph 2024; 111:102320. [PMID: 38134726 DOI: 10.1016/j.compmedimag.2023.102320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 11/13/2023] [Accepted: 11/29/2023] [Indexed: 12/24/2023]
Abstract
Medical imaging, specifically chest X-ray image analysis, is a crucial component of early disease detection and screening in healthcare. Deep learning techniques, such as convolutional neural networks (CNNs), have emerged as powerful tools for computer-aided diagnosis (CAD) in chest X-ray image analysis. These techniques have shown promising results in automating tasks such as classification, detection, and segmentation of abnormalities in chest X-ray images, with the potential to surpass human radiologists. In this review, we provide an overview of the importance of chest X-ray image analysis, historical developments, impact of deep learning techniques, and availability of labeled databases. We specifically focus on advancements and challenges in radiology report generation using deep learning, highlighting potential future advancements in this area. The use of deep learning for report generation has the potential to reduce the burden on radiologists, improve patient care, and enhance the accuracy and efficiency of chest X-ray image analysis in medical imaging.
Collapse
Affiliation(s)
- Mohammed Yasser Ouis
- Perception, Robotics and Intelligent Machines Lab(PRIME), Department of Computer Science, Université de Moncton, Moncton, NB E1C 3E9, Canada.
| | - Moulay A Akhloufi
- Perception, Robotics and Intelligent Machines Lab(PRIME), Department of Computer Science, Université de Moncton, Moncton, NB E1C 3E9, Canada.
| |
Collapse
|
11
|
Fraile Navarro D, Ijaz K, Rezazadegan D, Rahimi-Ardabili H, Dras M, Coiera E, Berkovsky S. Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review. Int J Med Inform 2023; 177:105122. [PMID: 37295138 DOI: 10.1016/j.ijmedinf.2023.105122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Revised: 04/14/2023] [Accepted: 06/03/2023] [Indexed: 06/12/2023]
Abstract
BACKGROUND Natural Language Processing (NLP) applications have developed over the past years in various fields including its application to clinical free text for named entity recognition and relation extraction. However, there has been rapid developments the last few years that there's currently no overview of it. Moreover, it is unclear how these models and tools have been translated into clinical practice. We aim to synthesize and review these developments. METHODS We reviewed literature from 2010 to date, searching PubMed, Scopus, the Association of Computational Linguistics (ACL), and Association of Computer Machinery (ACM) libraries for studies of NLP systems performing general-purpose (i.e., not disease- or treatment-specific) information extraction and relation extraction tasks in unstructured clinical text (e.g., discharge summaries). RESULTS We included in the review 94 studies with 30 studies published in the last three years. Machine learning methods were used in 68 studies, rule-based in 5 studies, and both in 22 studies. 63 studies focused on Named Entity Recognition, 13 on Relation Extraction and 18 performed both. The most frequently extracted entities were "problem", "test" and "treatment". 72 studies used public datasets and 22 studies used proprietary datasets alone. Only 14 studies defined clearly a clinical or information task to be addressed by the system and just three studies reported its use outside the experimental setting. Only 7 studies shared a pre-trained model and only 8 an available software tool. DISCUSSION Machine learning-based methods have dominated the NLP field on information extraction tasks. More recently, Transformer-based language models are taking the lead and showing the strongest performance. However, these developments are mostly based on a few datasets and generic annotations, with very few real-world use cases. This may raise questions about the generalizability of findings, translation into practice and highlights the need for robust clinical evaluation.
Collapse
Affiliation(s)
- David Fraile Navarro
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia.
| | - Kiran Ijaz
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Dana Rezazadegan
- Department of Computer Science and Software Engineering. School of Software and Electrical Engineering, Swinburne University of Technology, Melbourne, Australia
| | - Hania Rahimi-Ardabili
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Mark Dras
- Department of Computing, Macquarie University, Sydney, Australia
| | - Enrico Coiera
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| | - Shlomo Berkovsky
- Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia
| |
Collapse
|
12
|
Raza S, Schwartz B, Lakamana S, Ge Y, Sarker A. A framework for multi-faceted content analysis of social media chatter regarding non-medical use of prescription medications. BMC DIGITAL HEALTH 2023; 1:29. [PMID: 37680768 PMCID: PMC10483682 DOI: 10.1186/s44247-023-00029-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 07/17/2023] [Indexed: 09/09/2023]
Abstract
Background Substance use, including the non-medical use of prescription medications, is a global health problem resulting in hundreds of thousands of overdose deaths and other health problems. Social media has emerged as a potent source of information for studying substance use-related behaviours and their consequences. Mining large-scale social media data on the topic requires the development of natural language processing (NLP) and machine learning frameworks customized for this problem. Our objective in this research is to develop a framework for conducting a content analysis of Twitter chatter about the non-medical use of a set of prescription medications. Methods We collected Twitter data for four medications-fentanyl and morphine (opioids), alprazolam (benzodiazepine), and Adderall® (stimulant), and identified posts that indicated non-medical use using an automatic machine learning classifier. In our NLP framework, we applied supervised named entity recognition (NER) to identify other substances mentioned, symptoms, and adverse events. We applied unsupervised topic modelling to identify latent topics associated with the chatter for each medication. Results The quantitative analysis demonstrated the performance of the proposed NER approach in identifying substance-related entities from data with a high degree of accuracy compared to the baseline methods. The performance evaluation of the topic modelling was also notable. The qualitative analysis revealed knowledge about the use, non-medical use, and side effects of these medications in individuals and communities. Conclusions NLP-based analyses of Twitter chatter associated with prescription medications belonging to different categories provide multi-faceted insights about their use and consequences. Our developed framework can be applied to chatter about other substances. Further research can validate the predictive value of this information on the prevention, assessment, and management of these disorders.
Collapse
Affiliation(s)
- Shaina Raza
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| | - Brian Schwartz
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Sahithi Lakamana
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Yao Ge
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| |
Collapse
|
13
|
Yoon W, Yi S, Jackson R, Kim H, Kim S, Kang J. Biomedical relation extraction with knowledge base-refined weak supervision. Database (Oxford) 2023; 2023:baad054. [PMID: 37551911 PMCID: PMC10407973 DOI: 10.1093/database/baad054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 05/13/2023] [Accepted: 07/04/2023] [Indexed: 08/09/2023]
Abstract
Biomedical relation extraction (BioRE) is the task of automatically extracting and classifying relations between two biomedical entities in biomedical literature. Recent advances in BioRE research have largely been powered by supervised learning and large language models (LLMs). However, training of LLMs for BioRE with supervised learning requires human-annotated data, and the annotation process often accompanies challenging and expensive work. As a result, the quantity and coverage of annotated data are limiting factors for BioRE systems. In this paper, we present our system for the BioCreative VII challenge-DrugProt track, a BioRE system that leverages a language model structure and weak supervision. Our system is trained on weakly labelled data and then fine-tuned using human-labelled data. To create the weakly labelled dataset, we combined two approaches. First, we trained a model on the original dataset to predict labels on external literature, which will become a model-labelled dataset. Then, we refined the model-labelled dataset using an external knowledge base. Based on our experiment, our approach using refined weak supervision showed significant performance gain over the model trained using standard human-labelled datasets. Our final model showed outstanding performance at the BioCreative VII challenge, achieving 3rd place (this paper focuses on our participating system in the BioCreative VII challenge). Database URL: http://wonjin.info/biore-yoon-et-al-2022.
Collapse
Affiliation(s)
- Wonjin Yoon
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
| | - Sean Yi
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
| | - Richard Jackson
- AstraZeneca UK, 1 Francis Crick Ave, Trumpington, Cambridge CB2 0AA, UK
| | - Hyunjae Kim
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
| | - Sunkyu Kim
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
- AIGEN Sciences Inc., 25 Ttukseom-ro 1-gil, Seongdong-gu, Seoul 04778, South Korea
| | - Jaewoo Kang
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
- AIGEN Sciences Inc., 25 Ttukseom-ro 1-gil, Seongdong-gu, Seoul 04778, South Korea
| |
Collapse
|
14
|
Adams G, Nguyen BH, Smith J, Xia Y, Xie S, Ostropolets A, Deb B, Chen YJ, Naumann T, Elhadad N. What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarization. PROCEEDINGS OF THE CONFERENCE. ASSOCIATION FOR COMPUTATIONAL LINGUISTICS. MEETING 2023; 2023:10520-10542. [PMID: 38689884 PMCID: PMC11059202 DOI: 10.18653/v1/2023.acl-long.587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
Summarization models often generate text that is poorly calibrated to quality metrics because they are trained to maximize the likelihood of a single reference (MLE). To address this, recent work has added a calibration step, which exposes a model to its own ranked outputs to improve relevance or, in a separate line of work, contrasts positive and negative sets to improve faithfulness. While effective, much of this work has focused on how to generate and optimize these sets. Less is known about why one setup is more effective than another. In this work, we uncover the underlying characteristics of effective sets. For each training instance, we form a large, diverse pool of candidates and systematically vary the subsets used for calibration fine-tuning. Each selection strategy targets distinct aspects of the sets, such as lexical diversity or the size of the gap between positive and negatives. On three diverse scientific long-form summarization datasets (spanning biomedical, clinical, and chemical domains), we find, among others, that faithfulness calibration is optimal when the negative sets are extractive and more likely to be generated, whereas for relevance calibration, the metric margin between candidates should be maximized and surprise-the disagreement between model and metric defined candidate rankings-minimized. Code to create, select, and optimize calibration sets is available at https://github.com/griff4692/calibrating-summaries.
Collapse
Affiliation(s)
- Griffin Adams
- Computer Science, Columbia University
- Biomedical Informatics, Columbia University
| | | | | | | | | | | | | | | | | | - Noémie Elhadad
- Computer Science, Columbia University
- Biomedical Informatics, Columbia University
| |
Collapse
|
15
|
Gao Y, Dligach D, Miller T, Churpek MM, Uzuner O, Afshar M. Progress Note Understanding - Assessment and Plan Reasoning: Overview of the 2022 N2C2 Track 3 shared task. J Biomed Inform 2023; 142:104346. [PMID: 37061012 PMCID: PMC11178099 DOI: 10.1016/j.jbi.2023.104346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Revised: 03/16/2023] [Accepted: 03/21/2023] [Indexed: 04/17/2023]
Abstract
Daily progress notes are a common note type in the electronic health record (EHR) where healthcare providers document the patient's daily progress and treatment plans. The EHR is designed to document all the care provided to patients, but it also enables note bloat with extraneous information that distracts from the diagnoses and treatment plans. Applications of natural language processing (NLP) in the EHR is a growing field with the majority of methods in information extraction. Few tasks use NLP methods for downstream diagnostic decision support. We introduced the 2022 National NLP Clinical Challenge (N2C2) Track 3: Progress Note Understanding - Assessment and Plan Reasoning as one step towards a new suite of tasks. The Assessment and Plan Reasoning task focuses on the most critical components of progress notes, Assessment and Plan subsections where health problems and diagnoses are contained. The goal of the task was to develop and evaluate NLP systems that automatically predict causal relations between the overall status of the patient contained in the Assessment section and its relation to each component of the Plan section which contains the diagnoses and treatment plans. The goal of the task was to identify and prioritize diagnoses as the first steps in diagnostic decision support to find the most relevant information in long documents like daily progress notes. We present the results of the 2022 N2C2 Track 3 and provide a description of the data, evaluation, participation and system performance.
Collapse
Affiliation(s)
- Yanjun Gao
- ICU Data Science Lab, Department of Medicine, University of Wisconsin Madison, United States of America.
| | - Dmitriy Dligach
- Department of Computer Science, Loyola University Chicago, United States of America
| | - Timothy Miller
- Boston Children's Hospital, Harvard University, United States of America
| | - Matthew M Churpek
- ICU Data Science Lab, Department of Medicine, University of Wisconsin Madison, United States of America
| | - Ozlem Uzuner
- Department of Information Sciences and Technology, George Mason University, United States of America
| | - Majid Afshar
- ICU Data Science Lab, Department of Medicine, University of Wisconsin Madison, United States of America
| |
Collapse
|
16
|
Alasmari A, Kudryashov L, Yadav S, Lee H, Demner-Fushman D. CHQ- SocioEmo: Identifying Social and Emotional Support Needs in Consumer-Health Questions. Sci Data 2023; 10:329. [PMID: 37244917 DOI: 10.1038/s41597-023-02203-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 05/02/2023] [Indexed: 05/29/2023] Open
Abstract
General public, often called consumers, are increasingly seeking health information online. To be satisfactory, answers to health-related questions often have to go beyond informational needs. Automated approaches to consumer health question answering should be able to recognize the need for social and emotional support. Recently, large scale datasets have addressed the issue of medical question answering and highlighted the challenges associated with question classification from the standpoint of informational needs. However, there is a lack of annotated datasets for the non-informational needs. We introduce a new dataset for non-informational support needs, called CHQ-SocioEmo. The Dataset of Consumer Health Questions was collected from a community question answering forum and annotated with basic emotions and social support needs. This is the first publicly available resource for understanding non-informational support needs in consumer health-related questions online. We benchmark the corpus against multiple state-of-the-art classification models to demonstrate the dataset's effectiveness.
Collapse
Affiliation(s)
| | | | | | - Heera Lee
- University of Maryland, College Park, USA
| | | |
Collapse
|
17
|
Launer-Wachs S, Taub-Tabib H, Tokarev Madem J, Bar-Natan O, Goldberg Y, Shamay Y. From Centralized to Ad-Hoc Knowledge Base Construction for Hypotheses Generation. J Biomed Inform 2023; 142:104383. [PMID: 37196989 DOI: 10.1016/j.jbi.2023.104383] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 04/27/2023] [Accepted: 05/03/2023] [Indexed: 05/19/2023]
Abstract
OBJECTIVE To demonstrate and develop an approach enabling individual researchers or small teams to create their own ad-hoc, lightweight knowledge bases tailored for specialized scientific interests, using text-mining over scientific literature, and demonstrate the effectiveness of these knowledge bases in hypothesis generation and literature-based discovery (LBD). METHODS We propose a lightweight process using an extractive search framework to create ad-hoc knowledge bases, which require minimal training and no background in bio-curation or computer science. These knowledge bases are particularly effective for LBD and hypothesis generation using Swanson's ABC method. The personalized nature of the knowledge bases allows for a somewhat higher level of noise than "public facing" ones, as researchers are expected to have prior domain experience to separate signal from noise. Fact verification is shifted from exhaustive verification of the knowledge base to post-hoc verification of specific entries of interest, allowing researchers to assess the correctness of relevant knowledge base entries by considering the paragraphs in which the facts were introduced. RESULTS We demonstrate the methodology by constructing several knowledge bases of different kinds: three knowledge bases that support lab-internal hypothesis generation: Drug Delivery to Ovarian Tumors (DDOT); Tissue Engineering and Regeneration; Challenges in Cancer Research; and an additional comprehensive, accurate knowledge base designated as a public resource for the wider community on the topic of Cell Specific Drug Delivery (CSDD). In each case, we show the design and construction process, along with relevant visualizations for data exploration, and hypothesis generation. For CSDD and DDOT we also show meta-analysis, human evaluation, and in vitro experimental evaluation. CONCLUSION Our approach enables researchers to create personalized, lightweight knowledge bases for specialized scientific interests, effectively facilitating hypothesis generation and literature-based discovery (LBD). By shifting fact verification efforts to post-hoc verification of specific entries, researchers can focus on exploring and generating hypotheses based on their expertise. The constructed knowledge bases demonstrate the versatility and adaptability of our approach to versatile research interests. The web-based platform, available at https://spike-kbc.apps.allenai.org , provides researchers with a valuable tool for rapid construction of knowledge bases tailored to their needs.
Collapse
Affiliation(s)
- Shaked Launer-Wachs
- Faculty of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | | | - Jennie Tokarev Madem
- Faculty of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Orr Bar-Natan
- Faculty of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Yoav Goldberg
- Allen Institute for AI, Tel Aviv, Israel; Bar-Ilan University, Ramat-Gan, Israel
| | - Yosi Shamay
- Faculty of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel.
| |
Collapse
|
18
|
Luo L, Wei CH, Lai PT, Leaman R, Chen Q, Lu Z. AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning. Bioinformatics 2023; 39:btad310. [PMID: 37171899 PMCID: PMC10212279 DOI: 10.1093/bioinformatics/btad310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 04/12/2023] [Accepted: 05/11/2023] [Indexed: 05/14/2023] Open
Abstract
MOTIVATION Biomedical named entity recognition (BioNER) seeks to automatically recognize biomedical entities in natural language text, serving as a necessary foundation for downstream text mining tasks and applications such as information extraction and question answering. Manually labeling training data for the BioNER task is costly, however, due to the significant domain expertise required for accurate annotation. The resulting data scarcity causes current BioNER approaches to be prone to overfitting, to suffer from limited generalizability, and to address a single entity type at a time (e.g. gene or disease). RESULTS We therefore propose a novel all-in-one (AIO) scheme that uses external data from existing annotated resources to enhance the accuracy and stability of BioNER models. We further present AIONER, a general-purpose BioNER tool based on cutting-edge deep learning and our AIO schema. We evaluate AIONER on 14 BioNER benchmark tasks and show that AIONER is effective, robust, and compares favorably to other state-of-the-art approaches such as multi-task learning. We further demonstrate the practical utility of AIONER in three independent tasks to recognize entity types not previously seen in training data, as well as the advantages of AIONER over existing methods for processing biomedical text at a large scale (e.g. the entire PubMed data). AVAILABILITY AND IMPLEMENTATION The source code, trained models and data for AIONER are freely available at https://github.com/ncbi/AIONER.
Collapse
Affiliation(s)
- Ling Luo
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
- School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China
| | - Chih-Hsuan Wei
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
| | - Po-Ting Lai
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
| | - Robert Leaman
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
| | - Qingyu Chen
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
| | - Zhiyong Lu
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD 20894, United States
| |
Collapse
|
19
|
Tinn R, Cheng H, Gu Y, Usuyama N, Liu X, Naumann T, Gao J, Poon H. Fine-tuning large neural language models for biomedical natural language processing. PATTERNS (NEW YORK, N.Y.) 2023; 4:100729. [PMID: 37123444 PMCID: PMC10140607 DOI: 10.1016/j.patter.2023.100729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 12/12/2022] [Accepted: 03/17/2023] [Indexed: 05/02/2023]
Abstract
Large neural language models have transformed modern natural language processing (NLP) applications. However, fine-tuning such models for specific tasks remains challenging as model size increases, especially with small labeled datasets, which are common in biomedical NLP. We conduct a systematic study on fine-tuning stability in biomedical NLP. We show that fine-tuning performance may be sensitive to pretraining settings and conduct an exploration of techniques for addressing fine-tuning instability. We show that these techniques can substantially improve fine-tuning performance for low-resource biomedical NLP applications. Specifically, freezing lower layers is helpful for standard BERT- B A S E models, while layerwise decay is more effective for BERT- L A R G E and ELECTRA models. For low-resource text similarity tasks, such as BIOSSES, reinitializing the top layers is the optimal strategy. Overall, domain-specific vocabulary and pretraining facilitate robust models for fine-tuning. Based on these findings, we establish a new state of the art on a wide range of biomedical NLP applications.
Collapse
Affiliation(s)
| | - Hao Cheng
- Microsoft Research, Redmond, WA, USA
| | - Yu Gu
- Microsoft Research, Redmond, WA, USA
| | | | | | | | | | - Hoifung Poon
- Microsoft Research, Redmond, WA, USA
- Corresponding author
| |
Collapse
|
20
|
Guo L, Wang W, Wu YJ. What Do MBA Program in Southeast Asia Scholars Propose for Future COVID-19 Research in Academic Publications? A Topic Analysis Based on Autoencoder. SAGE OPEN 2023; 13:21582440231182060. [PMID: 37362769 PMCID: PMC10280124 DOI: 10.1177/21582440231182060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/28/2023]
Abstract
To analyze the directions for future research suggested and to project future research plans, we extract relevant text from these publications with respect to COVID-19-related research based on 54,136 relevant academic journals published from the initial outbreak of COVID-19 in January 2020 until December 2020. First, we extract and preprocess the corpus and then determine that, according to the Elbow method, the optimal number of clusters is 7. Then, we construct a text clustering model based on an autoencoder, with the support of an artificial neural network. Distance measurements, such as correlation, cosine, Braycurtis, and Jaccard are compared, and the clustering results are evaluated with normal mutual information. The results show that cosine similarity has the best effect on clustering of COVID-19-related documents. A topic model analysis shows that the directions of future research can mainly be grouped into the following seven categories: infectivity testing, genome analysis, vaccine testing, diagnosis and infection characteristics, pandemic management, nursing care, and clinical testing. Among them, the topics of pandemic management, diagnosis and infection characteristics, and clinical testing trended upward in proportion to future directions. The topic of vaccine testing remains steady over the observation window, whereas other topics (infectivity testing, genome analysis, and nursing care) slowly trended downward. Among all the topics, medical research comprises 80%, and about 20% of the topics are related to public management, government functions, and economic development. This study enriches our scientific understanding of COVID-19 and helps us to effectively predict future scientific research output on COVID-19.
Collapse
Affiliation(s)
- Lihuan Guo
- Tan Siu Lin Business School, Quanzhou
Normal University, Quanzhou, Fujian, China
- Cloud Computing, IoT, E-commerce
Intelligence Engineering Research Center of Colleges and universities in Fujian
Province, Quanzhou Normal University, Quanzhou, Fujian, China
| | - Wei Wang
- College of Business Administration,
Huaqiao University, Quanzhou, Fujian, China
| | - Yenchun Jim Wu
- MBA Program in Southeast Asia, National
Taipei University of Education, Taipei, Taiwan
- Graduate Institute of Global Business
and Strategy, National Taiwan Normal University, Taipei, Taiwan
| |
Collapse
|
21
|
Leaman R, Islamaj R, Adams V, Alliheedi MA, Almeida JR, Antunes R, Bevan R, Chang YC, Erdengasileng A, Hodgskiss M, Ida R, Kim H, Li K, Mercer RE, Mertová L, Mobasher G, Shin HC, Sung M, Tsujimura T, Yeh WC, Lu Z. Chemical identification and indexing in full-text articles: an overview of the NLM-Chem track at BioCreative VII. Database (Oxford) 2023; 2023:7071696. [PMID: 36882099 PMCID: PMC9991492 DOI: 10.1093/database/baad005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 01/06/2023] [Accepted: 02/15/2023] [Indexed: 03/09/2023]
Abstract
The BioCreative National Library of Medicine (NLM)-Chem track calls for a community effort to fine-tune automated recognition of chemical names in the biomedical literature. Chemicals are one of the most searched biomedical entities in PubMed, and-as highlighted during the coronavirus disease 2019 pandemic-their identification may significantly advance research in multiple biomedical subfields. While previous community challenges focused on identifying chemical names mentioned in titles and abstracts, the full text contains valuable additional detail. We, therefore, organized the BioCreative NLM-Chem track as a community effort to address automated chemical entity recognition in full-text articles. The track consisted of two tasks: (i) chemical identification and (ii) chemical indexing. The chemical identification task required predicting all chemicals mentioned in recently published full-text articles, both span [i.e. named entity recognition (NER)] and normalization (i.e. entity linking), using Medical Subject Headings (MeSH). The chemical indexing task required identifying which chemicals reflect topics for each article and should therefore appear in the listing of MeSH terms for the document in the MEDLINE article indexing. This manuscript summarizes the BioCreative NLM-Chem track and post-challenge experiments. We received a total of 85 submissions from 17 teams worldwide. The highest performance achieved for the chemical identification task was 0.8672 F-score (0.8759 precision and 0.8587 recall) for strict NER performance and 0.8136 F-score (0.8621 precision and 0.7702 recall) for strict normalization performance. The highest performance achieved for the chemical indexing task was 0.6073 F-score (0.7417 precision and 0.5141 recall). This community challenge demonstrated that (i) the current substantial achievements in deep learning technologies can be utilized to improve automated prediction accuracy further and (ii) the chemical indexing task is substantially more challenging. We look forward to further developing biomedical text-mining methods to respond to the rapid growth of biomedical literature. The NLM-Chem track dataset and other challenge materials are publicly available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BC7-NLM-Chem-track/. Database URL https://ftp.ncbi.nlm.nih.gov/pub/lu/BC7-NLM-Chem-track/.
Collapse
Affiliation(s)
| | | | - Virginia Adams
- NVIDIA, 2788 San Tomas Expressway, Santa Clara, CA 95051, USA
| | - Mohammed A Alliheedi
- Department of Computer Science, Al Baha University, 4781 King Fahd Rd, Al Aqiq 65779, Saudi Arabia
| | - João Rafael Almeida
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal
- Department of Information and Communications Technologies, University of A Coruña, Camiño do Lagar de Castro, A Coruña 15008, Spain
| | - Rui Antunes
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, Campus Universitário de Santiago, Aveiro 3810-193, Portugal
| | - Robert Bevan
- Informatics Department, Medicines Discovery Catapult, Alderley Park, Block 35, Mereside, Macclesfield SK10 4ZF, UK
| | - Yung-Chun Chang
- Graduate Institute of Data Science, Taipei Medical University, No. 172-1, Section 2, Keelung Rd, Da’an District, Taipei City , Taipei 106, Taiwan
| | - Arslan Erdengasileng
- Department of Statistics, Florida State University, 117 N. Woodward Ave, Tallahassee, FL 32306, USA
| | - Matthew Hodgskiss
- Informatics Department, Medicines Discovery Catapult, Alderley Park, Block 35, Mereside, Macclesfield SK10 4ZF, UK
| | - Ryuki Ida
- Computational Intelligence Laboratory, Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, Aichi 468-8511, Japan
| | - Hyunjae Kim
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
| | - Keqiao Li
- Department of Statistics, Florida State University, 117 N. Woodward Ave, Tallahassee, FL 32306, USA
| | - Robert E Mercer
- Department of Computer Science, The University of Western Ontario, Room 355, Middlesex College, Ontario , London N6A 5B7, Canada
| | - Lukrécia Mertová
- Scientific Databases and Visualization Group, Heidelberg Institute for Theoretical Studies (HITS gGmbH), Schloss-Wolfsbrunnenweg 35, Heidelberg 69118, Germany
| | - Ghadeer Mobasher
- Scientific Databases and Visualization Group, Heidelberg Institute for Theoretical Studies (HITS gGmbH), Schloss-Wolfsbrunnenweg 35, Heidelberg 69118, Germany
- Institute of Computer Science, Heidelberg University, Im Neuenheimer Feld 205, Heidelberg 69120, Germany
| | - Hoo-Chang Shin
- NVIDIA, 2788 San Tomas Expressway, Santa Clara, CA 95051, USA
| | - Mujeen Sung
- Department of Computer Science and Engineering, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul 02841, South Korea
| | - Tomoki Tsujimura
- Computational Intelligence Laboratory, Toyota Technological Institute, 2-12-1 Hisakata, Tempaku-ku, Nagoya, Aichi 468-8511, Japan
| | - Wen-Chao Yeh
- Institute of Information Systems and Applications, National Tsing Hua University, No. 101, Section 2, Kuang-Fu Road, Hsinchu 30013, Taiwan
| | - Zhiyong Lu
- *Corresponding author: Tel: +1-301-594-7089; Fax: +1-301-480-2290;
| |
Collapse
|
22
|
Zhang Y, Grant BMM, Hope AJ, Hung RJ, Warkentin MT, Lam ACL, Aggawal R, Xu M, Shepherd FA, Tsao MS, Xu W, Pakkal M, Liu G, McInnis MC. Using Recurrent Neural Networks to Extract High-Quality Information From Lung Cancer Screening Computerized Tomography Reports for Inter-Radiologist Audit and Feedback Quality Improvement. JCO Clin Cancer Inform 2023; 7:e2200153. [PMID: 36930839 DOI: 10.1200/cci.22.00153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/19/2023] Open
Abstract
PURPOSE Lung cancer screening programs generate a high volume of low-dose computed tomography (LDCT) reports that contain valuable information, typically in a free-text format. High-performance named-entity recognition (NER) models can extract relevant information from these reports automatically for inter-radiologist quality control. METHODS Using LDCT report data from a longitudinal lung cancer screening program (8,305 reports; 3,124 participants; 2006-2019), we trained a rule-based model and two bidirectional long short-term memory (Bi-LSTM) NER neural network models to detect clinically relevant information from LDCT reports. Model performance was tested using F1 scores and compared with a published open-source radiology NER model (Stanza) in an independent evaluation set of 150 reports. The top performing model was applied to a data set of 6,948 reports for an inter-radiologist quality control assessment. RESULTS The best performing model, a Bi-LSTM NER recurrent neural network model, had an overall F1 score of 0.950, which outperformed Stanza (F1 score = 0.872) and a rule-based NER model (F1 score = 0.809). Recall (sensitivity) for the best Bi-LSTM model ranged from 0.916 to 0.991 for different entity types; precision (positive predictive value) ranged from 0.892 to 0.997. Test performance remained stable across time periods. There was an average of a 2.86-fold difference in the number of identified entities between the most and the least detailed radiologists. CONCLUSION We built an open-source Bi-LSTM NER model that outperformed other open-source or rule-based radiology NER models. This model can efficiently extract clinically relevant information from lung cancer screening computerized tomography reports with high accuracy, enabling efficient audit and feedback to improve quality of patient care.
Collapse
Affiliation(s)
- Yucheng Zhang
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Benjamin M M Grant
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Andrew J Hope
- Radiation Medicine Program, Princess Margaret Cancer Centre, and Department of Radiation Oncology, University of Toronto, Toronto, ON, Canada
| | - Rayjean J Hung
- Prosserman Centre for Population Health Research, Lunenfeld-Tanenbaum Research Institute, Sinai Health Systems, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Matthew T Warkentin
- Prosserman Centre for Population Health Research, Lunenfeld-Tanenbaum Research Institute, Sinai Health Systems, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Andrew C L Lam
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Reenika Aggawal
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Maria Xu
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Frances A Shepherd
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Ming-Sound Tsao
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Laboratory Medicine and Pathology, University Health Network, Toronto, ON, Canada
| | - Wei Xu
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Biostatistics, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Computational Biology and Medicine Program, Princess Margaret Cancer Centre, Toronto, ON, Canada
| | - Mini Pakkal
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Division of Cardiothoracic Imaging, Joint Department of Medical Imaging, Toronto General Hospital, Toronto, ON, Canada
| | - Geoffrey Liu
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Medical Oncology and Hematology, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Biostatistics, Princess Margaret Cancer Centre, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Micheal C McInnis
- Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- Division of Cardiothoracic Imaging, Joint Department of Medical Imaging, Toronto General Hospital, Toronto, ON, Canada
| |
Collapse
|
23
|
Patel MA, Knauer MJ, Nicholson M, Daley M, Van Nynatten LR, Cepinskas G, Fraser DD. Organ and cell-specific biomarkers of Long-COVID identified with targeted proteomics and machine learning. Mol Med 2023; 29:26. [PMID: 36809921 PMCID: PMC9942653 DOI: 10.1186/s10020-023-00610-z] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 01/13/2023] [Indexed: 02/24/2023] Open
Abstract
BACKGROUND Survivors of acute COVID-19 often suffer prolonged, diffuse symptoms post-infection, referred to as "Long-COVID". A lack of Long-COVID biomarkers and pathophysiological mechanisms limits effective diagnosis, treatment and disease surveillance. We performed targeted proteomics and machine learning analyses to identify novel blood biomarkers of Long-COVID. METHODS A case-control study comparing the expression of 2925 unique blood proteins in Long-COVID outpatients versus COVID-19 inpatients and healthy control subjects. Targeted proteomics was accomplished with proximity extension assays, and machine learning was used to identify the most important proteins for identifying Long-COVID patients. Organ system and cell type expression patterns were identified with Natural Language Processing (NLP) of the UniProt Knowledgebase. RESULTS Machine learning analysis identified 119 relevant proteins for differentiating Long-COVID outpatients (Bonferonni corrected P < 0.01). Protein combinations were narrowed down to two optimal models, with nine and five proteins each, and with both having excellent sensitivity and specificity for Long-COVID status (AUC = 1.00, F1 = 1.00). NLP expression analysis highlighted the diffuse organ system involvement in Long-COVID, as well as the involved cell types, including leukocytes and platelets, as key components associated with Long-COVID. CONCLUSIONS Proteomic analysis of plasma from Long-COVID patients identified 119 highly relevant proteins and two optimal models with nine and five proteins, respectively. The identified proteins reflected widespread organ and cell type expression. Optimal protein models, as well as individual proteins, hold the potential for accurate diagnosis of Long-COVID and targeted therapeutics.
Collapse
Affiliation(s)
- Maitray A Patel
- Epidemiology and Biostatistics, Western University, London, ON, N6A 3K7, Canada
| | - Michael J Knauer
- Pathology and Laboratory Medicine, Western University, London, ON, N6A 3K7, Canada
| | | | - Mark Daley
- Epidemiology and Biostatistics, Western University, London, ON, N6A 3K7, Canada.,Computer Science, Western University, London, ON, N6A 3K7, Canada
| | | | - Gediminas Cepinskas
- Lawson Health Research Institute, London, ON, N6C 2R5, Canada.,Medical Biophysics, Western University, London, ON, N6A 3K7, Canada
| | - Douglas D Fraser
- Lawson Health Research Institute, London, ON, N6C 2R5, Canada. .,Children's Health Research Institute, London, ON, N6C 4V3, Canada. .,Pediatrics, Western University, London, ON, N6A 3K7, Canada. .,Clinical Neurological Sciences, Western University, London, ON, N6A 3K7, Canada. .,Physiology and Pharmacology, Western University, London, ON, N6A 3K7, Canada. .,Room C2-C82, London Health Sciences Centre, 800 Commissioners Road East, London, ON, N6A 5W9, Canada.
| |
Collapse
|
24
|
Fan JW, Wang W, Huang M, Liu H, Hooten WM. Retrospective content analysis of consumer product reviews related to chronic pain. Front Digit Health 2023; 5:958338. [PMID: 37168528 PMCID: PMC10165495 DOI: 10.3389/fdgth.2023.958338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 03/09/2023] [Indexed: 05/13/2023] Open
Abstract
Chronic pain (CP) lasts for more than 3 months, causing prolonged physical and mental burdens to patients. According to the US Centers for Disease Control and Prevention, CP contributes to more than 500 billion US dollars yearly in direct medical cost plus the associated productivity loss. CP is complex in etiology and can occur anywhere in the body, making it difficult to treat and manage. There is a pressing need for research to better summarize the common health issues faced by consumers living with CP and their experience in accessing over-the-counter analgesics or therapeutic devices. Modern online shopping platforms offer a broad array of opportunities for the secondary use of consumer-generated data in CP research. In this study, we performed an exploratory data mining study that analyzed CP-related Amazon product reviews. Our descriptive analyses characterized the review language, the reviewed products, the representative topics, and the network of comorbidities mentioned in the reviews. The results indicated that most of the reviews were concise yet rich in terms of representing the various health issues faced by people with CP. Despite the noise in the online reviews, we see potential in leveraging the data to capture certain consumer-reported outcomes or to identify shortcomings of the available products.
Collapse
Affiliation(s)
- Jungwei W. Fan
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
- Correspondence: Jungwei W. Fan
| | - Wanjing Wang
- Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, United States
| | - Ming Huang
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Hongfang Liu
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - W. Michael Hooten
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
25
|
Tanwar A, Zhang J, Ive J, Gupta V, Guo Y. Phenotyping in clinical text with unsupervised numerical reasoning for patient stratification. Exp Biol Med (Maywood) 2022; 247:2038-2052. [PMID: 36217914 PMCID: PMC9791305 DOI: 10.1177/15353702221118092] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
Phenotypic information of patients, as expressed in clinical text, is important in many clinical applications such as identifying patients at risk of hard-to-diagnose conditions. Extracting and inferring some phenotypes from clinical text requires numerical reasoning, for example, a temperature of 102°F suggests the phenotype Fever. However, while current state-of-the-art phenotyping models using natural language processing (NLP) are in general very efficient in extracting phenotypes, they struggle to extract phenotypes that require numerical reasoning. In this article, we propose a novel unsupervised method that leverages external clinical knowledge and contextualized word embeddings by ClinicalBERT for numerical reasoning in different phenotypic contexts. Experiments show that the proposed method achieves significant improvement against unsupervised baseline methods with absolute increase in generalized Recall and F1 scores of up to 79% and 71%, respectively. Also, the proposed method outperforms supervised baseline methods with absolute increase in generalized Recall and F1 scores of up to 70% and 44%, respectively. In addition, we validate the methodology on clinical use cases where the detected phenotypes significantly contribute to patient stratification systems for a set of diseases, namely, HIV and myocardial infarction (heart attack). Moreover, we find that these phenotypes from clinical text can be used to impute the missing values in structured data, which enrich and improve data quality.
Collapse
|
26
|
Kaur N, Mittal A. CheXPrune: sparse chest X-ray report generation model using multi-attention and one-shot global pruning. JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING 2022; 14:7485-7497. [PMID: 36338854 PMCID: PMC9628486 DOI: 10.1007/s12652-022-04454-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 10/05/2022] [Indexed: 05/25/2023]
Abstract
Automatic radiological report generation (ARRG) smoothens the clinical workflow by speeding the report generation task. Recently, various deep neural networks (DNNs) have been used for report generation and have achieved promising results. Despite the impressive results, their deployment remains challenging because of their size and complexity. Researchers have proposed several pruning methods to reduce the size of DNNs. Inspired by the one-shot weight pruning methods, we present CheXPrune, a multi-attention based sparse radiology report generation method. It uses encoder-decoder based architecture equipped with a visual and semantic attention mechanism. The model is 70% pruned during the training to achieve 3.33× compression without sacrificing its accuracy. The empirical results evaluated on the OpenI dataset using BLEU, ROUGE, and CIDEr metrics confirm the accuracy of the sparse model viz-a ` -viz the dense model.
Collapse
Affiliation(s)
- Navdeep Kaur
- UIET, Panjab University, Sector 25, Chandigarh, 160025 India
- Mehr Chand DAV College for Women, Sector 36 A, Chandigarh, 160036 India
| | - Ajay Mittal
- UIET, Panjab University, Sector 25, Chandigarh, 160025 India
| |
Collapse
|
27
|
Kühnel L, Fluck J. We are not ready yet: limitations of state-of-the-art disease named entity recognizers. J Biomed Semantics 2022; 13:26. [PMID: 36303237 DOI: 10.1186/s13326-022-00280-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 10/12/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Intense research has been done in the area of biomedical natural language processing. Since the breakthrough of transfer learning-based methods, BERT models are used in a variety of biomedical and clinical applications. For the available data sets, these models show excellent results - partly exceeding the inter-annotator agreements. However, biomedical named entity recognition applied on COVID-19 preprints shows a performance drop compared to the results on test data. The question arises how well trained models are able to predict on completely new data, i.e. to generalize. RESULTS Based on the example of disease named entity recognition, we investigate the robustness of different machine learning-based methods - thereof transfer learning - and show that current state-of-the-art methods work well for a given training and the corresponding test set but experience a significant lack of generalization when applying to new data. CONCLUSIONS We argue that there is a need for larger annotated data sets for training and testing. Therefore, we foresee the curation of further data sets and, moreover, the investigation of continual learning processes for machine learning-based models.
Collapse
Affiliation(s)
- Lisa Kühnel
- ZB MED - Information Centre for Life Sciences, Gleueler Str. 60, Cologne, Germany. .,Graduate School DILS, Bielefeld Institute for Bioinformatics Infrastructure (BIBI), Faculty of Technology, Bielefeld University, Postfach 10 01 31, 33501, Bielefeld, Germany.
| | - Juliane Fluck
- ZB MED - Information Centre for Life Sciences, Gleueler Str. 60, Cologne, Germany.,Institute of Geodesy and Geoinformation, Agricultural Faculty, University of Bonn, Nussallee 1, 53115, Bonn, Germany
| |
Collapse
|
28
|
Tang A, Deléger L, Bossy R, Zweigenbaum P, Nédellec C. Do syntactic trees enhance Bidirectional Encoder Representations from Transformers (BERT) models for chemical–drug relation extraction? Database (Oxford) 2022; 2022:6675625. [PMID: 36006843 PMCID: PMC9408061 DOI: 10.1093/database/baac070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 07/14/2022] [Accepted: 08/12/2022] [Indexed: 11/14/2022]
Abstract
Collecting relations between chemicals and drugs is crucial in biomedical research. The pre-trained transformer model, e.g. Bidirectional Encoder Representations from Transformers (BERT), is shown to have limitations on biomedical texts; more specifically, the lack of annotated data makes relation extraction (RE) from biomedical texts very challenging. In this paper, we hypothesize that enriching a pre-trained transformer model with syntactic information may help improve its performance on chemical–drug RE tasks. For this purpose, we propose three syntax-enhanced models based on the domain-specific BioBERT model: Chunking-Enhanced-BioBERT and Constituency-Tree-BioBERT in which constituency information is integrated and a Multi-Task-Learning framework Multi-Task-Syntactic (MTS)-BioBERT in which syntactic information is injected implicitly by adding syntax-related tasks as training objectives. Besides, we test an existing model Late-Fusion which is enhanced by syntactic dependency information and build ensemble systems combining syntax-enhanced models and non-syntax-enhanced models. Experiments are conducted on the BioCreative VII DrugProt corpus, a manually annotated corpus for the development and evaluation of RE systems. Our results reveal that syntax-enhanced models in general degrade the performance of BioBERT in the scenario of biomedical RE but improve the performance when the subject–object distance of candidate semantic relation is long. We also explore the impact of quality of dependency parses. [Our code is available at: https://github.com/Maple177/syntax-enhanced-RE/tree/drugprot (for only MTS-BioBERT); https://github.com/Maple177/drugprot-relation-extraction (for the rest of experiments)] Database URLhttps://github.com/Maple177/drugprot-relation-extraction
Collapse
Affiliation(s)
- Anfu Tang
- INRAE, MaIAGE, Université Paris-Saclay , Domaine de Vilvert, Jouy-en-Josas 78352, France
- CNRS, Laboratoire interdisciplinaire des sciences du numérique, Université Paris-Saclay , Campus universitaire bât 507, Rue du Belvedère, Orsay 91405, France
| | - Louise Deléger
- INRAE, MaIAGE, Université Paris-Saclay , Domaine de Vilvert, Jouy-en-Josas 78352, France
| | - Robert Bossy
- INRAE, MaIAGE, Université Paris-Saclay , Domaine de Vilvert, Jouy-en-Josas 78352, France
| | - Pierre Zweigenbaum
- CNRS, Laboratoire interdisciplinaire des sciences du numérique, Université Paris-Saclay , Campus universitaire bât 507, Rue du Belvedère, Orsay 91405, France
| | - Claire Nédellec
- INRAE, MaIAGE, Université Paris-Saclay , Domaine de Vilvert, Jouy-en-Josas 78352, France
| |
Collapse
|
29
|
Frei J, Soto-Rey I, Kramer F. DrNote: An open medical annotation service. PLOS DIGITAL HEALTH 2022; 1:e0000086. [PMID: 36812581 PMCID: PMC9931362 DOI: 10.1371/journal.pdig.0000086] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Accepted: 07/12/2022] [Indexed: 11/19/2022]
Abstract
In the context of clinical trials and medical research medical text mining can provide broader insights for various research scenarios by tapping additional text data sources and extracting relevant information that is often exclusively present in unstructured fashion. Although various works for data like electronic health reports are available for English texts, only limited work on tools for non-English text resources has been published that offers immediate practicality in terms of flexibility and initial setup. We introduce DrNote, an open source text annotation service for medical text processing. Our work provides an entire annotation pipeline with its focus on a fast yet effective and easy to use software implementation. Further, the software allows its users to define a custom annotation scope by filtering only for relevant entities that should be included in its knowledge base. The approach is based on OpenTapioca and combines the publicly available datasets from WikiData and Wikipedia, and thus, performs entity linking tasks. In contrast to other related work our service can easily be built upon any language-specific Wikipedia dataset in order to be trained on a specific target language. We provide a public demo instance of our DrNote annotation service at https://drnote.misit-augsburg.de/.
Collapse
Affiliation(s)
- Johann Frei
- IT-Infrastructure for Translational Medical Research, Faculty of Applied Computer Science, University of Augsburg, Augsburg, Germany
- * E-mail:
| | - Iñaki Soto-Rey
- Medical Data Integration Center, Institute for Digital Medicine, University Hospital Augsburg, Augsburg, Germany
| | - Frank Kramer
- IT-Infrastructure for Translational Medical Research, Faculty of Applied Computer Science, University of Augsburg, Augsburg, Germany
| |
Collapse
|
30
|
Luo L, Lai PT, Wei CH, Lu Z. A sequence labeling framework for extracting drug-protein relations from biomedical literature. Database (Oxford) 2022; 2022:baac058. [PMID: 35856889 PMCID: PMC9297941 DOI: 10.1093/database/baac058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 05/24/2022] [Accepted: 07/14/2022] [Indexed: 06/15/2023]
Abstract
UNLABELLED Automatic extracting interactions between chemical compound/drug and gene/protein are significantly beneficial to drug discovery, drug repurposing, drug design and biomedical knowledge graph construction. To promote the development of the relation extraction between drug and protein, the BioCreative VII challenge organized the DrugProt track. This paper describes the approach we developed for this task. In addition to the conventional text classification framework that has been widely used in relation extraction tasks, we propose a sequence labeling framework to drug-protein relation extraction. We first comprehensively compared the cutting-edge biomedical pre-trained language models for both frameworks. Then, we explored several ensemble methods to further improve the final performance. In the evaluation of the challenge, our best submission (i.e. the ensemble of models in two frameworks via major voting) achieved the F1-score of 0.795 on the official test set. Further, we realized the sequence labeling framework is more efficient and achieves better performance than the text classification framework. Finally, our ensemble of the sequence labeling models with majority voting achieves the best F1-score of 0.800 on the test set. DATABASE URL https://github.com/lingluodlut/BioCreativeVII_DrugProt.
Collapse
Affiliation(s)
- Ling Luo
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, USA
| | - Po-Ting Lai
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, USA
| | - Chih-Hsuan Wei
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), 8600 Rockville Pike, Bethesda, MD 20894, USA
| | - Zhiyong Lu
- *Corresponding author: Tel: 301 594 7089; Fax: 301 480 2288;
| |
Collapse
|
31
|
Almeida T, Antunes R, F. Silva J, Almeida JR, Matos S. Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics. Database (Oxford) 2022; 2022:6625810. [PMID: 35776534 PMCID: PMC9248917 DOI: 10.1093/database/baac047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 05/13/2022] [Accepted: 06/06/2022] [Indexed: 11/14/2022]
Abstract
Abstract
The identification of chemicals in articles has attracted a large interest in the biomedical scientific community, given its importance in drug development research. Most of previous research have focused on PubMed abstracts, and further investigation using full-text documents is required because these contain additional valuable information that must be explored. The manual expert task of indexing Medical Subject Headings (MeSH) terms to these articles later helps researchers find the most relevant publications for their ongoing work. The BioCreative VII NLM-Chem track fostered the development of systems for chemical identification and indexing in PubMed full-text articles. Chemical identification consisted in identifying the chemical mentions and linking these to unique MeSH identifiers. This manuscript describes our participation system and the post-challenge improvements we made. We propose a three-stage pipeline that individually performs chemical mention detection, entity normalization and indexing. Regarding chemical identification, we adopted a deep-learning solution that utilizes the PubMedBERT contextualized embeddings followed by a multilayer perceptron and a conditional random field tagging layer. For the normalization approach, we use a sieve-based dictionary filtering followed by a deep-learning similarity search strategy. Finally, for the indexing we developed rules for identifying the more relevant MeSH codes for each article. During the challenge, our system obtained the best official results in the normalization and indexing tasks despite the lower performance in the chemical mention recognition task. In a post-contest phase we boosted our results by improving our named entity recognition model with additional techniques. The final system achieved 0.8731, 0.8275 and 0.4849 in the chemical identification, normalization and indexing tasks, respectively. The code to reproduce our experiments and run the pipeline is publicly available.
Database URL
https://github.com/bioinformatics-ua/biocreativeVII_track2
Collapse
Affiliation(s)
- Tiago Almeida
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro , Aveiro, Portugal
| | - Rui Antunes
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro , Aveiro, Portugal
| | - João F. Silva
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro , Aveiro, Portugal
| | - João R Almeida
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro , Aveiro, Portugal
- Department of Information and Communications Technologies, University of A Coruña , A Coruña, Spain
| | - Sérgio Matos
- Department of Electronics, Telecommunications and Informatics (DETI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro , Aveiro, Portugal
| |
Collapse
|
32
|
Zhang Y, Wang C, Soukaseum M, Vlachos DG, Fang H. Unleashing the Power of Knowledge Extraction from Scientific Literature in Catalysis. J Chem Inf Model 2022; 62:3316-3330. [PMID: 35772028 DOI: 10.1021/acs.jcim.2c00359] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
Valuable knowledge of catalysis is often hidden in a large amount of scientific literature. There is an urgent need to extract useful knowledge to facilitate scientific discovery. This work takes the first step toward the goal in the field of catalysis. Specifically, we construct the first information extraction benchmark data set that covers the field of catalysis and also develop a general extraction framework that can accurately extract catalysis-related entities from scientific literature with 90% extraction accuracy. We further demonstrate the feasibility of leveraging the extracted knowledge to help users better access relevant information in catalysis through an entity-aware search engine and a correlation analysis system.
Collapse
Affiliation(s)
- Yue Zhang
- Department of Electrical and Computer Engineering, University of Delaware, Newark, Delaware 19711, United States.,Center for Plastics Innovation, University of Delaware, Newark, Delaware 19711, United States
| | - Cong Wang
- Center for Plastics Innovation, University of Delaware, Newark, Delaware 19711, United States.,Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, Delaware 19711, United States
| | - Mya Soukaseum
- Center for Plastics Innovation, University of Delaware, Newark, Delaware 19711, United States.,Department of Chemical and Biological Engineering, Drexel University, Philadelphia, Pennsylvania 19104, United States
| | - Dionisios G Vlachos
- Center for Plastics Innovation, University of Delaware, Newark, Delaware 19711, United States.,Department of Chemical and Biomolecular Engineering, University of Delaware, Newark, Delaware 19711, United States
| | - Hui Fang
- Department of Electrical and Computer Engineering, University of Delaware, Newark, Delaware 19711, United States.,Center for Plastics Innovation, University of Delaware, Newark, Delaware 19711, United States
| |
Collapse
|
33
|
Yang S, Wu X, Ge S, Zhou SK, Xiao L. Knowledge matters: Chest radiology report generation with general and specific knowledge. Med Image Anal 2022; 80:102510. [PMID: 35716558 DOI: 10.1016/j.media.2022.102510] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2021] [Revised: 06/01/2022] [Accepted: 06/06/2022] [Indexed: 10/18/2022]
Abstract
Automatic chest radiology report generation is critical in clinics which can relieve experienced radiologists from the heavy workload and remind inexperienced radiologists of misdiagnosis or missed diagnose. Existing approaches mainly formulate chest radiology report generation as an image captioning task and adopt the encoder-decoder framework. However, in the medical domain, such pure data-driven approaches suffer from the following problems: 1) visual and textual bias problem; 2) lack of expert knowledge. In this paper, we propose a knowledge-enhanced radiology report generation approach introduces two types of medical knowledge: 1) General knowledge, which is input independent and provides the broad knowledge for report generation; 2) Specific knowledge, which is input dependent and provides the fine-grained knowledge for chest X-ray report generation. To fully utilize both the general and specific knowledge, we also propose a knowledge-enhanced multi-head attention mechanism. By merging the visual features of the radiology image with general knowledge and specific knowledge, the proposed model can improve the quality of generated reports. The experimental results on the publicly available IU-Xray dataset show that the proposed knowledge-enhanced approach outperforms state-of-the-art methods in almost all metrics. And the results of MIMIC-CXR dataset show that the proposed knowledge-enhanced approach is on par with state-of-the-art methods. Ablation studies also demonstrate that both general and specific knowledge can help to improve the performance of chest radiology report generation.
Collapse
Affiliation(s)
- Shuxin Yang
- Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology, CAS, Beijing 100190, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Xian Wu
- Tencent Medical AI Lab, Beijing 100094, China.
| | - Shen Ge
- Tencent Medical AI Lab, Beijing 100094, China
| | - S Kevin Zhou
- Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology, CAS, Beijing 100190, China; School of Biomedical Engineering& Suzhou Institute for Advanced Research Center for Medical Imaging, Robotics, and Analytic Computing & LEarning (MIRACLE) University of Science and Technology of China, Suzhou 215123, China.
| | - Li Xiao
- Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology, CAS, Beijing 100190, China; University of Chinese Academy of Sciences, Beijing 100049, China.
| |
Collapse
|
34
|
Chen Z, Peng B, Ioannidis VN, Li M, Karypis G, Ning X. A knowledge graph of clinical trials ([Formula: see text]). Sci Rep 2022; 12:4724. [PMID: 35304504 PMCID: PMC8933553 DOI: 10.1038/s41598-022-08454-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 02/28/2022] [Indexed: 02/05/2023] Open
Abstract
Effective and successful clinical trials are essential in developing new drugs and advancing new treatments. However, clinical trials are very expensive and easy to fail. The high cost and low success rate of clinical trials motivate research on inferring knowledge from existing clinical trials in innovative ways for designing future clinical trials. In this manuscript, we present our efforts on constructing the first publicly available Clinical Trials Knowledge Graph, denoted as [Formula: see text]. [Formula: see text] includes nodes representing medical entities in clinical trials (e.g., studies, drugs and conditions), and edges representing the relations among these entities (e.g., drugs used in studies). Our embedding analysis demonstrates the potential utilities of [Formula: see text] in various applications such as drug repurposing and similarity search, among others.
Collapse
Affiliation(s)
- Ziqi Chen
- The Ohio State University, Columbus, USA
| | - Bo Peng
- The Ohio State University, Columbus, USA
| | | | - Mufei Li
- Amazon Web Services Shanghai AI Lab, Shanghai, China
| | | | - Xia Ning
- The Ohio State University, Columbus, USA
| |
Collapse
|
35
|
Davoudi A, Lee NS, Luong T, Delaney T, Asch E, Chaiyachati K, Mowery D. Identifying Medication-related Intents from a Bidirectional Text Messaging Platform for Hypertension Management: A Pilot Study using a Unsupervised Learning Approach (Preprint). J Med Internet Res 2022; 24:e36151. [PMID: 35767327 PMCID: PMC9280462 DOI: 10.2196/36151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 04/01/2022] [Accepted: 05/17/2022] [Indexed: 12/02/2022] Open
Abstract
Background Free-text communication between patients and providers plays an increasing role in chronic disease management, through platforms varying from traditional health care portals to novel mobile messaging apps. These text data are rich resources for clinical purposes, but their sheer volume render them difficult to manage. Even automated approaches, such as natural language processing, require labor-intensive manual classification for developing training data sets. Automated approaches to organizing free-text data are necessary to facilitate use of free-text communication for clinical care. Objective The aim of this study was to apply unsupervised learning approaches to (1) understand the types of topics discussed and (2) learn medication-related intents from messages sent between patients and providers through a bidirectional text messaging system for managing participant blood pressure (BP). Methods This study was a secondary analysis of deidentified messages from a remote, mobile, text-based employee hypertension management program at an academic institution. We trained a latent Dirichlet allocation (LDA) model for each message type (ie, inbound patient messages and outbound provider messages) and identified the distribution of major topics and significant topics (probability >.20) across message types. Next, we annotated all medication-related messages with a single medication intent. Then, we trained a second medication-specific LDA (medLDA) model to assess how well the unsupervised method could identify more fine-grained medication intents. We encoded each medication message with n-grams (n=1-3 words) using spaCy, clinical named entities using Stanza, and medication categories using MedEx; we then applied chi-square feature selection to learn the most informative features associated with each medication intent. Results In total, 253 participants and 5 providers engaged in the program, generating 12,131 total messages: 46.90% (n=5689) patient messages and 53.10% (n=6442) provider messages. Most patient messages corresponded to BP reporting, BP encouragement, and appointment scheduling; most provider messages corresponded to BP reporting, medication adherence, and confirmatory statements. Most patient and provider messages contained 1 topic and few contained more than 3 topics identified using LDA. In total, 534 medication messages were annotated with a single medication intent. Of these, 282 (52.8%) were patient medication messages: most referred to the medication request intent (n=134, 47.5%). Most of the 252 (47.2%) provider medication messages referred to the medication question intent (n=173, 68.7%). Although the medLDA model could identify a majority intent within each topic, it could not distinguish medication intents with low prevalence within patient or provider messages. Richer feature engineering identified informative lexical-semantic patterns associated with each medication intent class. Conclusions LDA can be an effective method for generating subgroups of messages with similar term usage and facilitating the review of topics to inform annotations. However, few training cases and shared vocabulary between intents precludes the use of LDA for fully automated, deep, medication intent classification. International Registered Report Identifier (IRRID) RR2-10.1101/2021.12.23.21268061
Collapse
Affiliation(s)
- Anahita Davoudi
- Department of Biostatistics, Epidemiology, and Informatics, Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, United States
| | - Natalie S Lee
- Division of General Internal Medicine, Department of Medicine, The Ohio State University Wexner Medical Center, Columbus, OH, United States
| | - ThaiBinh Luong
- Penn Medicine Predictive Healthcare, University of Pennsylvania Health System, Philadelphia, PA, United States
| | - Timothy Delaney
- Center for Health Care Innovation, University of Pennsylvania, Philadelphia, PA, United States
| | - Elizabeth Asch
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, United States
| | - Krisda Chaiyachati
- Center for Health Care Innovation, University of Pennsylvania, Philadelphia, PA, United States
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, United States
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Danielle Mowery
- Department of Biostatistics, Epidemiology, and Informatics, Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
36
|
A Deep Learning Based Approach to Automate Clinical Coding of Electronic Health Records. BIG DATA ANALYTICS 2022. [DOI: 10.1007/978-3-031-24094-2_7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023] Open
|
37
|
Lossio-Ventura JA, Sun R, Boussard S, Hernandez-Boussard T. Clinical concept recognition: Evaluation of existing systems on EHRs. Front Artif Intell 2022; 5:1051724. [PMID: 36714202 PMCID: PMC9880223 DOI: 10.3389/frai.2022.1051724] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 12/15/2022] [Indexed: 01/15/2023] Open
Abstract
Objective The adoption of electronic health records (EHRs) has produced enormous amounts of data, creating research opportunities in clinical data sciences. Several concept recognition systems have been developed to facilitate clinical information extraction from these data. While studies exist that compare the performance of many concept recognition systems, they are typically developed internally and may be biased due to different internal implementations, parameters used, and limited number of systems included in the evaluations. The goal of this research is to evaluate the performance of existing systems to retrieve relevant clinical concepts from EHRs. Methods We investigated six concept recognition systems, including CLAMP, cTAKES, MetaMap, NCBO Annotator, QuickUMLS, and ScispaCy. Clinical concepts extracted included procedures, disorders, medications, and anatomical location. The system performance was evaluated on two datasets: the 2010 i2b2 and the MIMIC-III. Additionally, we assessed the performance of these systems in five challenging situations, including negation, severity, abbreviation, ambiguity, and misspelling. Results For clinical concept extraction, CLAMP achieved the best performance on exact and inexact matching, with an F-score of 0.70 and 0.94, respectively, on i2b2; and 0.39 and 0.50, respectively, on MIMIC-III. Across the five challenging situations, ScispaCy excelled in extracting abbreviation information (F-score: 0.86) followed by NCBO Annotator (F-score: 0.79). CLAMP outperformed in extracting severity terms (F-score 0.73) followed by NCBO Annotator (F-score: 0.68). CLAMP outperformed other systems in extracting negated concepts (F-score 0.63). Conclusions Several concept recognition systems exist to extract clinical information from unstructured data. This study provides an external evaluation by end-users of six commonly used systems across different extraction tasks. Our findings suggest that CLAMP provides the most comprehensive set of annotations for clinical concept extraction tasks and associated challenges. Comparing standard extraction tasks across systems provides guidance to other clinical researchers when selecting a concept recognition system relevant to their clinical information extraction task.
Collapse
Affiliation(s)
- Juan Antonio Lossio-Ventura
- Biomedical Informatics Research, Stanford University, Stanford, CA, United States.,National Institute of Mental Health, National Institutes of Health, Bethesda, MD, United States
| | - Ran Sun
- Biomedical Informatics Research, Stanford University, Stanford, CA, United States
| | | | - Tina Hernandez-Boussard
- Biomedical Informatics Research, Stanford University, Stanford, CA, United States.,Department of Biomedical Data Sciences, Stanford University, Stanford, CA, United States.,Department of Surgery, Stanford University, Stanford, CA, United States
| |
Collapse
|
38
|
Abstract
Electronic health records (EHRs) are becoming a vital source of data for healthcare quality improvement, research, and operations. However, much of the most valuable information contained in EHRs remains buried in unstructured text. The field of clinical text mining has advanced rapidly in recent years, transitioning from rule-based approaches to machine learning and, more recently, deep learning. With new methods come new challenges, however, especially for those new to the field. This review provides an overview of clinical text mining for those who are encountering it for the first time (e.g., physician researchers, operational analytics teams, machine learning scientists from other domains). While not a comprehensive survey, this review describes the state of the art, with a particular focus on new tasks and methods developed over the past few years. It also identifies key barriers between these remarkable technical advances and the practical realities of implementation in health systems and in industry.
Collapse
Affiliation(s)
- Bethany Percha
- Department of Medicine and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10025, USA;
| |
Collapse
|