1
|
Thompson P, Ananiadou S, Basinas I, Brinchmann BC, Cramer C, Galea KS, Ge C, Georgiadis P, Kirkeleit J, Kuijpers E, Nguyen N, Nuñez R, Schlünssen V, Stokholm ZA, Taher EA, Tinnerberg H, Van Tongeren M, Xie Q. Supporting the working life exposome: Annotating occupational exposure for enhanced literature search. PLoS One 2024; 19:e0307844. [PMID: 39146349 PMCID: PMC11326626 DOI: 10.1371/journal.pone.0307844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Accepted: 07/12/2024] [Indexed: 08/17/2024] Open
Abstract
An individual's likelihood of developing non-communicable diseases is often influenced by the types, intensities and duration of exposures at work. Job exposure matrices provide exposure estimates associated with different occupations. However, due to their time-consuming expert curation process, job exposure matrices currently cover only a subset of possible workplace exposures and may not be regularly updated. Scientific literature articles describing exposure studies provide important supporting evidence for developing and updating job exposure matrices, since they report on exposures in a variety of occupational scenarios. However, the constant growth of scientific literature is increasing the challenges of efficiently identifying relevant articles and important content within them. Natural language processing methods emulate the human process of reading and understanding texts, but in a fraction of the time. Such methods can increase the efficiency of both finding relevant documents and pinpointing specific information within them, which could streamline the process of developing and updating job exposure matrices. Named entity recognition is a fundamental natural language processing method for language understanding, which automatically identifies mentions of domain-specific concepts (named entities) in documents, e.g., exposures, occupations and job tasks. State-of-the-art machine learning models typically use evidence from an annotated corpus, i.e., a set of documents in which named entities are manually marked up (annotated) by experts, to learn how to detect named entities automatically in new documents. We have developed a novel annotated corpus of scientific articles to support machine learning based named entity recognition relevant to occupational substance exposures. Through incremental refinements to the annotation process, we demonstrate that expert annotators can attain high levels of agreement, and that the corpus can be used to train high-performance named entity recognition models. The corpus thus constitutes an important foundation for the wider development of natural language processing tools to support the study of occupational exposures.
Collapse
Affiliation(s)
- Paul Thompson
- Department of Computer Science, National Centre for Text Mining, University of Manchester, Manchester, United Kingdom
| | - Sophia Ananiadou
- Department of Computer Science, National Centre for Text Mining, University of Manchester, Manchester, United Kingdom
| | - Ioannis Basinas
- Centre for Occupational and Environmental Health, School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Bendik C Brinchmann
- Federation of Norwegian Industries, Oslo, Norway
- Department of Occupational Medicine and Epidemiology, National Institute of Occupational Health, Oslo, Norway
| | - Christine Cramer
- Department of Public Health, Research Unit for Environment, Occupation and Health, Danish Ramazzini Centre, Aarhus University, Aarhus, Denmark
- Department of Occupational Medicine, Danish Ramazzini Centre, Aarhus University Hospital, Aarhus, Denmark
| | - Karen S Galea
- Institute of Occupational Medicine, Edinburgh, United Kingdom
| | - Calvin Ge
- Netherlands Organisation for Applied Scientific Research, Utrecht, Netherlands
| | - Panagiotis Georgiadis
- Department of Computer Science, National Centre for Text Mining, University of Manchester, Manchester, United Kingdom
| | - Jorunn Kirkeleit
- Federation of Norwegian Industries, Oslo, Norway
- Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway
| | - Eelco Kuijpers
- Netherlands Organisation for Applied Scientific Research, Utrecht, Netherlands
| | - Nhung Nguyen
- Department of Computer Science, National Centre for Text Mining, University of Manchester, Manchester, United Kingdom
| | - Roberto Nuñez
- Occupational Health Group, Institute for Risk Assessment Sciences, Utrecht University, Utrecht, Netherlands
| | - Vivi Schlünssen
- Department of Public Health, Research Unit for Environment, Occupation and Health, Danish Ramazzini Centre, Aarhus University, Aarhus, Denmark
| | - Zara Ann Stokholm
- Department of Occupational Medicine, Danish Ramazzini Centre, Aarhus University Hospital, Aarhus, Denmark
| | - Evana Amir Taher
- Center for Occupational and Environmental Medicine, Stockholm, Sweden
| | - Håkan Tinnerberg
- School of Public Health and Community Medicine, University of Gothenburg, Gothenburg, Sweden
- Institute of Environmental Medicine, Karolinska Institutet, Stockholm, Sweden
| | - Martie Van Tongeren
- Centre for Occupational and Environmental Health, School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Qianqian Xie
- Department of Computer Science, National Centre for Text Mining, University of Manchester, Manchester, United Kingdom
| |
Collapse
|
2
|
Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, Dong H, Poon MTC, Fitzpatrick N, Levine AP, Slater LT, Handy A, Karwath A, Gkoutos GV, Chelala C, Shah AD, Stewart R, Collier N, Alex B, Whiteley W, Sudlow C, Roberts A, Dobson RJB. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med 2022; 5:186. [PMID: 36544046 PMCID: PMC9770568 DOI: 10.1038/s41746-022-00730-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.
Collapse
Affiliation(s)
- Honghan Wu
- Institute of Health Informatics, University College London, London, UK.
| | - Minhong Wang
- Institute of Health Informatics, University College London, London, UK
| | - Jinge Wu
- Institute of Health Informatics, University College London, London, UK
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Farah Francis
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Yun-Hsuan Chang
- Institute of Health Informatics, University College London, London, UK
| | - Alex Shavick
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Hang Dong
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Department of Computer Science, University of Oxford, Oxford, UK
| | | | | | - Adam P Levine
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Luke T Slater
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Alex Handy
- Institute of Health Informatics, University College London, London, UK
- University College London Hospitals NHS Trust, London, UK
| | - Andreas Karwath
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Georgios V Gkoutos
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Claude Chelala
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Anoop Dinesh Shah
- Institute of Health Informatics, University College London, London, UK
| | - Robert Stewart
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK
- South London and Maudsley NHS Foundation Trust, London, UK
| | - Nigel Collier
- Theoretical and Applied Linguistics, Faculty of Modern & Medieval Languages & Linguistics, University of Cambridge, Cambridge, UK
| | - Beatrice Alex
- Edinburgh Futures Institute, University of Edinburgh, Edinburgh, UK
| | | | - Cathie Sudlow
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Angus Roberts
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London, UK
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| |
Collapse
|
3
|
Chen Q, Allot A, Leaman R, Islamaj R, Du J, Fang L, Wang K, Xu S, Zhang Y, Bagherzadeh P, Bergler S, Bhatnagar A, Bhavsar N, Chang YC, Lin SJ, Tang W, Zhang H, Tavchioski I, Pollak S, Tian S, Zhang J, Otmakhova Y, Yepes AJ, Dong H, Wu H, Dufour R, Labrak Y, Chatterjee N, Tandon K, Laleye FAA, Rakotoson L, Chersoni E, Gu J, Friedrich A, Pujari SC, Chizhikova M, Sivadasan N, VG S, Lu Z. Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations. Database (Oxford) 2022; 2022:baac069. [PMID: 36043400 PMCID: PMC9428574 DOI: 10.1093/database/baac069] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 08/02/2022] [Accepted: 08/13/2022] [Indexed: 05/03/2023]
Abstract
The coronavirus disease 2019 (COVID-19) pandemic has been severely impacting global society since December 2019. The related findings such as vaccine and drug development have been reported in biomedical literature-at a rate of about 10 000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200 000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g. Diagnosis and Treatment) to the articles in LitCovid. The annotated topics have been widely used for navigating the COVID literature, rapidly locating articles of interest and other downstream studies. However, annotating the topics has been the bottleneck of manual curation. Despite the continuing advances in biomedical text-mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset-consisting of over 30 000 articles with manually reviewed topics-was created for training and testing. It is one of the largest multi-label classification datasets in biomedical scientific literature. Nineteen teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181 and 0.9394 for macro-F1-score, micro-F1-score and instance-based F1-score, respectively. Notably, these scores are substantially higher (e.g. 12%, higher for macro F1-score) than the corresponding scores of the state-of-art multi-label classification method. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/ for benchmarking and further development. Database URL https://ftp.ncbi.nlm.nih.gov/pub/lu/LitCovid/biocreative/.
Collapse
Affiliation(s)
- Qingyu Chen
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, MD, Bethesda 20892, USA
| | - Alexis Allot
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, MD, Bethesda 20892, USA
| | - Robert Leaman
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, MD, Bethesda 20892, USA
| | - Rezarta Islamaj
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, MD, Bethesda 20892, USA
| | - Jingcheng Du
- School of Biomedical Informatics, UT Health, TX, Houston 77030, USA
| | - Li Fang
- Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Kai Wang
- Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Shuo Xu
- College of Economics and Management, Beijing University of Technology, Beijing, QC, China
| | - Yuefu Zhang
- College of Economics and Management, Beijing University of Technology, Beijing, QC, China
| | | | | | | | | | - Yung-Chun Chang
- Graduate Institute of Data Science, Taipei Medical University, Taipei, Taiwan
| | - Sheng-Jie Lin
- Graduate Institute of Data Science, Taipei Medical University, Taipei, Taiwan
| | - Wentai Tang
- College of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Hongtong Zhang
- College of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Ilija Tavchioski
- Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia
- Jožef Stefan Institute, Ljubljana, Slovenia
| | | | - Shubo Tian
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Jinfeng Zhang
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Yulia Otmakhova
- School of Computing and Information Systems, University of Melbourne, Melbourne, AU-VIC, Australia
| | | | - Hang Dong
- Centre for Medical Informatics, Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Honghan Wu
- Institute of Health Informatics, University College London, London, UK
| | | | | | - Niladri Chatterjee
- Department of Mathematics, Indian Institute of Technology Delhi, New Delhi, India
| | - Kushagri Tandon
- Department of Mathematics, Indian Institute of Technology Delhi, New Delhi, India
| | | | | | - Emmanuele Chersoni
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong, China
| | - Jinghang Gu
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong, China
| | | | - Subhash Chandra Pujari
- Institute of Computer Science, Heidelberg University, Heidelberg, Germany
- Bosch Center for Artificial Intelligence, Renningen, Germany
| | - Mariia Chizhikova
- SINAI Group, Department of Computer Science, Advanced Studies Center in ICT (CEATIC), Universidad de Jaén, Jaén, Spain
| | | | | | - Zhiyong Lu
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, MD, Bethesda 20892, USA
| |
Collapse
|
4
|
A Narrative Literature Review of Natural Language Processing Applied to the Occupational Exposome. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19148544. [PMID: 35886395 PMCID: PMC9316260 DOI: 10.3390/ijerph19148544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 07/07/2022] [Accepted: 07/11/2022] [Indexed: 02/05/2023]
Abstract
The evolution of the Exposome concept revolutionised the research in exposure assessment and epidemiology by introducing the need for a more holistic approach on the exploration of the relationship between the environment and disease. At the same time, further and more dramatic changes have also occurred on the working environment, adding to the already existing dynamic nature of it. Natural Language Processing (NLP) refers to a collection of methods for identifying, reading, extracting and untimely transforming large collections of language. In this work, we aim to give an overview of how NLP has successfully been applied thus far in Exposome research. Methods: We conduct a literature search on PubMed, Scopus and Web of Science for scientific articles published between 2011 and 2021. We use both quantitative and qualitative methods to screen papers and provide insights into the inclusion and exclusion criteria. We outline our approach for article selection and provide an overview of our findings. This is followed by a more detailed insight into selected articles. Results: Overall, 6420 articles were screened for the suitability of this review, where we review 37 articles in depth. Finally, we discuss future avenues of research and outline challenges in existing work. Conclusions: Our results show that (i) there has been an increase in articles published that focus on applying NLP to exposure and epidemiology research, (ii) most work uses existing NLP tools and (iii) traditional machine learning is the most popular approach.
Collapse
|
5
|
Wang Z, Adu-Kumi S, Diamond ML, Guardans R, Harner T, Harte A, Kajiwara N, Klánová J, Liu J, Moreira EG, Muir DCG, Suzuki N, Pinas V, Seppälä T, Weber R, Yuan B. Enhancing Scientific Support for the Stockholm Convention's Implementation: An Analysis of Policy Needs for Scientific Evidence. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2022; 56:2936-2949. [PMID: 35167273 DOI: 10.1021/acs.est.1c06120] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
The Stockholm Convention is key to addressing the global threats of persistent organic pollutants (POPs) to humanity and the environment. It has been successful in identifying new POPs, but its national implementation remains challenging, particularly by low- and middle-income Parties. Concerted action is needed to assist Parties in implementing the Convention's obligations. This analysis aims to identify and recommend research and scientific support needed for timely implementation of the Convention. We aim this analysis at scientists and experts from a variety of natural and social sciences and from all sectors (academia, civil society, industry, and government institutions), as well as research funding agencies. Further, we provide practical guidance to scientists and experts to promote the visibility and accessibility of their work for the Convention's implementation, followed by recommendations for sustaining scientific support to the Convention. This study is the first of a series on analyzing policy needs for scientific evidence under global governance on chemicals and waste.
Collapse
Affiliation(s)
- Zhanyun Wang
- Institute of Environmental Engineering, ETH Zürich, 8093 Zürich, Switzerland
- Empa - Swiss Federal Laboratories for Materials Science and Technology, Technology and Society Laboratory, CH-9014 St. Gallen, Switzerland
| | - Sam Adu-Kumi
- Chemicals Control and Management Centre, Environmental Protection Agency, Ministries, P.O. Box MB 326, Accra GR, Ghana
| | - Miriam L Diamond
- Department of Earth Sciences and School of the Environment, University of Toronto, Toronto, Ontario M5S 3B1, Canada
| | - Ramon Guardans
- WEOG Region Representative for the Global Monitoring Plan of the Stockholm Convention on POPs, Adviser on POPs, Ministry for the Ecological Transition and Demographic Challenge (MITECO), 28046 Madrid, Spain
| | - Tom Harner
- WEOG Region Representative for the Global Monitoring Plan of the Stockholm Convention on POPs, Air Quality Processes Research Section, Environment and Climate Change Canada, Toronto, Ontario M3H 5T4, Canada
| | - Agustín Harte
- National Chemicals and Hazardous Waste Directorate, Secretariat of Environmental Control and Monitoring, Ministry of Environment and Sustainable Development, San Martin 451, Autonomous City of Buenos Aires C1004AAI, Argentina
| | - Natsuko Kajiwara
- Material Cycles Division, National Institute for Environmental Studies, 16-2 Onogawa, Tsukuba, Ibaraki 305-8506, Japan
| | - Jana Klánová
- RECETOX Centre of Masaryk University, the Stockholm Convention Regional Centre for Capacity Building and the Transfer of Technology in Central and Eastern Europe, 611 37 Brno, Czech Republic
| | - Jianguo Liu
- State Key Joint Laboratory for Environmental Simulation and Pollution Control, College of Environmental Sciences and Engineering, Peking University, Beijing 100871, China
| | | | - Derek C G Muir
- Environment and Climate Change Canada, Canada Centre for Inland Waters, Burlington, Ontario L7S 1A1, Canada
| | - Noriyuki Suzuki
- Planning Division, National Institute for Environmental Studies, 16-2 Onogawa, Tsukuba, Ibaraki 305-8506, Japan
| | - Victorine Pinas
- Institute for Graduate Studies and Research, Anton de Kom University of Suriname, P.O.B: 9212, Paramaribo, Suriname
| | - Timo Seppälä
- Finnish Environment Institute, Contaminants Unit, 00790, Helsinki, Finland
| | - Roland Weber
- POPs Environmental Consulting, 73527, Schwäbisch Gmünd Germany
| | - Bo Yuan
- Department of Environmental Science, Stockholm University, 106 91 Stockholm, Sweden
- Department of Chemistry, Norwegian University of Science and Technology, 7491 Trondheim, Norway
| |
Collapse
|
6
|
Majewska O, Collins C, Baker S, Björne J, Brown SW, Korhonen A, Palmer M. BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine. J Biomed Semantics 2021; 12:12. [PMID: 34266499 PMCID: PMC8280585 DOI: 10.1186/s13326-021-00247-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 07/01/2021] [Indexed: 11/10/2022] Open
Abstract
Background Recent advances in representation learning have enabled large strides in natural language understanding; However, verbal reasoning remains a challenge for state-of-the-art systems. External sources of structured, expert-curated verb-related knowledge have been shown to boost model performance in different Natural Language Processing (NLP) tasks where accurate handling of verb meaning and behaviour is critical. The costliness and time required for manual lexicon construction has been a major obstacle to porting the benefits of such resources to NLP in specialised domains, such as biomedicine. To address this issue, we combine a neural classification method with expert annotation to create BioVerbNet. This new resource comprises 693 verbs assigned to 22 top-level and 117 fine-grained semantic-syntactic verb classes. We make this resource available complete with semantic roles and VerbNet-style syntactic frames. Results We demonstrate the utility of the new resource in boosting model performance in document- and sentence-level classification in biomedicine. We apply an established retrofitting method to harness the verb class membership knowledge from BioVerbNet and transform a pretrained word embedding space by pulling together verbs belonging to the same semantic-syntactic class. The BioVerbNet knowledge-aware embeddings surpass the non-specialised baseline by a significant margin on both tasks. Conclusion This work introduces the first large, annotated semantic-syntactic classification of biomedical verbs, providing a detailed account of the annotation process, the key differences in verb behaviour between the general and biomedical domain, and the design choices made to accurately capture the meaning and properties of verbs used in biomedical texts. The demonstrated benefits of leveraging BioVerbNet in text classification suggest the resource could help systems better tackle challenging NLP tasks in biomedicine.
Collapse
Affiliation(s)
- Olga Majewska
- Language Technology Laboratory, MMLL, University of Cambridge, 9 West Road, Cambridge, CB39DB, UK.
| | - Charlotte Collins
- Language Technology Laboratory, MMLL, University of Cambridge, 9 West Road, Cambridge, CB39DB, UK
| | - Simon Baker
- Language Technology Laboratory, MMLL, University of Cambridge, 9 West Road, Cambridge, CB39DB, UK
| | - Jari Björne
- Department of Future Technologies, University of Turku, Vesilinnantie 5, Turku, 20500, Finland
| | - Susan Windisch Brown
- Department of Linguistics, University of Colorado Boulder, 295 UCB, Boulder, 80309-0295, Colorado, USA
| | - Anna Korhonen
- Language Technology Laboratory, MMLL, University of Cambridge, 9 West Road, Cambridge, CB39DB, UK
| | - Martha Palmer
- Department of Linguistics, University of Colorado Boulder, 295 UCB, Boulder, 80309-0295, Colorado, USA
| |
Collapse
|
7
|
Ibrahim MA, Ghani Khan MU, Mehmood F, Asim MN, Mahmood W. GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification. J Biomed Inform 2021; 116:103699. [PMID: 33601013 DOI: 10.1016/j.jbi.2021.103699] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2020] [Revised: 11/30/2020] [Accepted: 02/02/2021] [Indexed: 01/16/2023]
Abstract
Exponential growth of biomedical literature and clinical data demands more robust yet precise computational methodologies to extract useful insights from biomedical literature and to perform accurate assignment of disease-specific codes. Such approaches can largely enhance the effectiveness of diverse biomedicine and bioinformatics applications. State-of-the-art computational biomedical text classification methodologies either solely leverage discrimintaive features extracted through convolution operations performed by deep convolutional neural network or contextual information extracted by recurrent neural network. However, none of the methodology takes advantage of both convolutional and recurrent neural networks. Further, existing methodologies lack to produce decent performance for the classification of different genre biomedical text such as biomedical literature or clinical notes. We, for the very first time, present a generic deep learning based hybrid multi-label classification methodology namely GHS-NET which can be utilized to accurately classify biomedical text of diverse genre. GHS-NET makes use of convolutional neural network to extract most discriminative features and bi-directional Long Short-Term Memory to acquire contextual information. GHS-NET effectiveness is evaluated for extreme multi-label biomedical literature classification and assignment of ICD-9 codes to clinical notes. For the task of extreme multi-label biomedical literature classification, performance comparison of GHS-Net and state-of-the-art deep learning based methodology reveals that GHS-Net marks the increment of 1%, 6%, and 1% for hallmarks of cancer dataset, 10%, 16%, and 11% for chemical exposure dataset in terms of precision, recall, and F1-score. For the task of clinical notes classification, GHS-Net outperforms previous best deep learning based methodology over Medical Information Mart for Intensive Care dataset (MIMIC-III) by the significant margin of 6%, 8% in terms of recall and F1-score. GHS-NET is available as a web service at1 and potentially can be used to accurately classify multi-variate disease and chemical exposure specific text.
Collapse
Affiliation(s)
- Muhammad Ali Ibrahim
- Intelligent Criminology Research Lab, National Center of Artificial Intelligence, Al-Khawarizmi Institute of Computer Science, UET, Lahore, Pakistan; German Research Center for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany
| | - Muhammad Usman Ghani Khan
- Intelligent Criminology Research Lab, National Center of Artificial Intelligence, Al-Khawarizmi Institute of Computer Science, UET, Lahore, Pakistan; Department of Computer Science, University of Engineering and Technology (UET), Lahore, Pakistan
| | - Faiza Mehmood
- Intelligent Criminology Research Lab, National Center of Artificial Intelligence, Al-Khawarizmi Institute of Computer Science, UET, Lahore, Pakistan
| | - Muhammad Nabeel Asim
- Intelligent Criminology Research Lab, National Center of Artificial Intelligence, Al-Khawarizmi Institute of Computer Science, UET, Lahore, Pakistan; German Research Center for Artificial Intelligence (DFKI), 67663 Kaiserslautern, Germany.
| | - Waqar Mahmood
- Intelligent Criminology Research Lab, National Center of Artificial Intelligence, Al-Khawarizmi Institute of Computer Science, UET, Lahore, Pakistan
| |
Collapse
|
8
|
Kromhout H, van Tongeren M, Peters CE, Hall AL. Commentary. Occup Environ Med 2020; 77:513-514. [PMID: 32532844 PMCID: PMC7402456 DOI: 10.1136/oemed-2020-106624] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2020] [Revised: 05/08/2020] [Accepted: 05/13/2020] [Indexed: 01/20/2023]
Affiliation(s)
- Hans Kromhout
- Institute for Risk Assessment Sciences, Utrecht University, Utrecht, The Netherlands
| | - Martie van Tongeren
- Centre for Occupational and Environmental Health, University of Manchester, Manchester, UK
| | - Cheryl E Peters
- Cancer Epidemiology and Prevention Research, Alberta Health Services, Calgary, Alberta, Canada.,Community Health Sciences and Oncology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada
| | - Amy L Hall
- Veterans Affairs, Government of Canada, Charlottetown, Prince Edward Island, Canada
| |
Collapse
|
9
|
Du J, Chen Q, Peng Y, Xiang Y, Tao C, Lu Z. ML-Net: multi-label classification of biomedical texts with deep neural networks. J Am Med Inform Assoc 2019; 26:1279-1285. [PMID: 31233120 PMCID: PMC7647240 DOI: 10.1093/jamia/ocz085] [Citation(s) in RCA: 41] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Revised: 04/21/2019] [Accepted: 05/08/2019] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE In multi-label text classification, each textual document is assigned 1 or more labels. As an important task that has broad applications in biomedicine, a number of different computational methods have been proposed. Many of these methods, however, have only modest accuracy or efficiency and limited success in practical use. We propose ML-Net, a novel end-to-end deep learning framework, for multi-label classification of biomedical texts. MATERIALS AND METHODS ML-Net combines a label prediction network with an automated label count prediction mechanism to provide an optimal set of labels. This is accomplished by leveraging both the predicted confidence score of each label and the deep contextual information (modeled by ELMo) in the target document. We evaluate ML-Net on 3 independent corpora in 2 text genres: biomedical literature and clinical notes. For evaluation, we use example-based measures, such as precision, recall, and the F measure. We also compare ML-Net with several competitive machine learning and deep learning baseline models. RESULTS Our benchmarking results show that ML-Net compares favorably to state-of-the-art methods in multi-label classification of biomedical text. ML-Net is also shown to be robust when evaluated on different text genres in biomedicine. CONCLUSION ML-Net is able to accuractely represent biomedical document context and dynamically estimate the label count in a more systematic and accurate manner. Unlike traditional machine learning methods, ML-Net does not require human effort for feature engineering and is a highly efficient and scalable approach to tasks with a large set of labels, so there is no need to build individual classifiers for each separate label.
Collapse
Affiliation(s)
- Jingcheng Du
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, USA
- The University of Texas School of Biomedical Informatics, Houston, Texas, USA
| | - Qingyu Chen
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, USA
| | - Yifan Peng
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, USA
| | - Yang Xiang
- The University of Texas School of Biomedical Informatics, Houston, Texas, USA
| | - Cui Tao
- The University of Texas School of Biomedical Informatics, Houston, Texas, USA
| | - Zhiyong Lu
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, Maryland, USA
| |
Collapse
|