1
|
Liu Q, Zhang L, Ren G, Zou B. Research on named entity recognition of Traditional Chinese Medicine chest discomfort cases incorporating domain vocabulary features. Comput Biol Med 2023; 166:107466. [PMID: 37742417 DOI: 10.1016/j.compbiomed.2023.107466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 08/20/2023] [Accepted: 09/04/2023] [Indexed: 09/26/2023]
Abstract
OBJECTIVE To promote research on knowledge extraction and knowledge graph construction of chest discomfort medical cases in Traditional Chinese Medicine (TCM), this paper focuses on their named entity recognition (NER). The recognition task includes six entity types: "syndrome", "symptom", "etiology and pathogenesis", "treatment method", "medication", and "prescription". METHODS We annotated data in a BIO (B-begin, I-inside, O-outside) manner. For the characteristics of medical case texts, we proposed a custom dictionary method that can be dynamically updated for word segmentation. To compare the effect of the method on the experimental results, we applied the method in the BiLSTM-CRF model and IDCNN-CRF model, respectively. RESULTS The models using custom dictionaries (BiLSTM-CRF-Loaded and IDCNN-CRF-Loaded) outperformed the models without custom dictionaries (BiLSTM-CRF and IDCNN-CRF) in accuracy, precision, recall, and F1 score. The BiLSTM-CRF-Loaded model yielded F1 scores of 92.59% and 93.23% on the test set and validation set, respectively, surpassing the BERT-BiLSTM-CRF model by 3.59% and 4.87%. Furthermore, when analyzing the results for the six entity categories separately, we found that the use of custom dictionaries has a marked impact, with the categories of "etiology and pathogenesis" and "syndrome" demonstrating the most noticeable improvements. By comparing the F1 scores, we observed that the entity category "medication" yielded the highest performance, reaching F1 scores of 96.04% and 96.48% on the test set and validation set, respectively. CONCLUSION We propose a word segmentation method based on a dynamically updated custom dictionary. The method is combined with the BILSTM-CRF and the IDCNN-CRF models, which enhances the model to recognize domain-specific terms and new entities. It can be widely applied in dealing with complex text structures and texts containing domain-specific terms.
Collapse
Affiliation(s)
- Qingping Liu
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China.
| | - Lunlun Zhang
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China.
| | - Gao Ren
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China.
| | - Beiji Zou
- School of Computer Science and Engineering, Central South University, Changsha, 410083, Hunan, China.
| |
Collapse
|
2
|
Hossain E, Rana R, Higgins N, Soar J, Barua PD, Pisani AR, Turner K. Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review. Comput Biol Med 2023; 155:106649. [PMID: 36805219 DOI: 10.1016/j.compbiomed.2023.106649] [Citation(s) in RCA: 28] [Impact Index Per Article: 28.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 01/04/2023] [Accepted: 02/07/2023] [Indexed: 02/12/2023]
Abstract
BACKGROUND Natural Language Processing (NLP) is widely used to extract clinical insights from Electronic Health Records (EHRs). However, the lack of annotated data, automated tools, and other challenges hinder the full utilisation of NLP for EHRs. Various Machine Learning (ML), Deep Learning (DL) and NLP techniques are studied and compared to understand the limitations and opportunities in this space comprehensively. METHODOLOGY After screening 261 articles from 11 databases, we included 127 papers for full-text review covering seven categories of articles: (1) medical note classification, (2) clinical entity recognition, (3) text summarisation, (4) deep learning (DL) and transfer learning architecture, (5) information extraction, (6) Medical language translation and (7) other NLP applications. This study follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULT AND DISCUSSION EHR was the most commonly used data type among the selected articles, and the datasets were primarily unstructured. Various ML and DL methods were used, with prediction or classification being the most common application of ML or DL. The most common use cases were: the International Classification of Diseases, Ninth Revision (ICD-9) classification, clinical note analysis, and named entity recognition (NER) for clinical descriptions and research on psychiatric disorders. CONCLUSION We find that the adopted ML models were not adequately assessed. In addition, the data imbalance problem is quite important, yet we must find techniques to address this underlining problem. Future studies should address key limitations in studies, primarily identifying Lupus Nephritis, Suicide Attempts, perinatal self-harmed and ICD-9 classification.
Collapse
Affiliation(s)
- Elias Hossain
- School of Engineering & Physical Sciences, North South University, Dhaka 1229, Bangladesh.
| | - Rajib Rana
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Niall Higgins
- School of Management and Enterprise, University of Southern Queensland, Darling Heights QLD 4350, Australia; School of Nursing, Queensland University of Technology, Kelvin Grove, Brisbane, QLD 4000, Australia; Metro North Mental Health, Herston QLD 4029, Australia
| | - Jeffrey Soar
- School of Business, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Prabal Datta Barua
- School of Business, University of Southern Queensland, Springfield Central QLD 4300, Australia
| | - Anthony R Pisani
- Center for the Study and Prevention of Suicide, University of Rochester, Rochester, NY, United States
| | - Kathryn Turner
- School of Nursing, Queensland University of Technology, Kelvin Grove, Brisbane, QLD 4000, Australia
| |
Collapse
|
3
|
Wang Y, Wang Y, Peng Z, Zhang F, Zhou L, Yang F. Medical text classification based on the discriminative pre-training model and prompt-tuning. Digit Health 2023; 9:20552076231193213. [PMID: 37559830 PMCID: PMC10408339 DOI: 10.1177/20552076231193213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 07/18/2023] [Indexed: 08/11/2023] Open
Abstract
Medical text classification, as a fundamental medical natural language processing task, aims to identify the categories to which a short medical text belongs. Current research has focused on performing the medical text classification task using a pre-training language model through fine-tuning. However, this paradigm introduces additional parameters when training extra classifiers. Recent studies have shown that the "prompt-tuning" paradigm induces better performance in many natural language processing tasks because it bridges the gap between pre-training goals and downstream tasks. The main idea of prompt-tuning is to transform binary or multi-classification tasks into mask prediction tasks by fully exploiting the features learned by pre-training language models. This study explores, for the first time, how to classify medical texts using a discriminative pre-training language model called ERNIE-Health through prompt-tuning. Specifically, we attempt to perform prompt-tuning based on the multi-token selection task, which is a pre-training task of ERNIE-Health. The raw text is wrapped into a new sequence with a template in which the category label is replaced by a [UNK] token. The model is then trained to calculate the probability distribution of the candidate categories. Our method is tested on the KUAKE-Question Intention Classification and CHiP-Clinical Trial Criterion datasets and obtains the accuracy values of 0.866 and 0.861. In addition, the loss values of our model decrease faster throughout the training period compared to the fine-tuning. The experimental results provide valuable insights to the community and suggest that prompt-tuning can be a promising approach to improve the performance of pre-training models in domain-specific tasks.
Collapse
Affiliation(s)
- Yu Wang
- School of Biomedical Engineering, Anhui Medical University, Hefei, China
| | - Yuan Wang
- Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, China
| | - Zhenwan Peng
- School of Biomedical Engineering, Anhui Medical University, Hefei, China
| | - Feifan Zhang
- School of Biomedical Engineering, Anhui Medical University, Hefei, China
| | - Luyao Zhou
- School of Biomedical Engineering, Anhui Medical University, Hefei, China
| | - Fei Yang
- School of Biomedical Engineering, Anhui Medical University, Hefei, China
| |
Collapse
|
4
|
Identifying suicidal emotions on social media through transformer-based deep learning. APPL INTELL 2022. [DOI: 10.1007/s10489-022-04060-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
5
|
Singh A, Dargar SK, Gupta A, Kumar A, Srivastava AK, Srivastava M, Kumar Tiwari P, Ullah MA. Evolving Long Short-Term Memory Network-Based Text Classification. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:4725639. [PMID: 35237308 PMCID: PMC8885205 DOI: 10.1155/2022/4725639] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 12/10/2021] [Accepted: 01/12/2022] [Indexed: 11/18/2022]
Abstract
Recently, long short-term memory (LSTM) networks are extensively utilized for text classification. Compared to feed-forward neural networks, it has feedback connections, and thus, it has the ability to learn long-term dependencies. However, the LSTM networks suffer from the parameter tuning problem. Generally, initial and control parameters of LSTM are selected on a trial and error basis. Therefore, in this paper, an evolving LSTM (ELSTM) network is proposed. A multiobjective genetic algorithm (MOGA) is used to optimize the architecture and weights of LSTM. The proposed model is tested on a well-known factory reports dataset. Extensive analyses are performed to evaluate the performance of the proposed ELSTM network. From the comparative analysis, it is found that the LSTM network outperforms the competitive models.
Collapse
Affiliation(s)
- Arjun Singh
- Computer and Communication Engineering, School of Computing and IT, Manipal University Jaipur, Jaipur, India
| | - Shashi Kant Dargar
- Department of Electronics and Communication Engineering, Kalasalingam Academy of Research and Education, Virudhunagar, Tamilnadu, India
| | - Amit Gupta
- Department of Electronics and Communication Engineering, Narasaraopeta Engineering College, Narasaraopeta, Andhra Pradesh, India
| | - Ashish Kumar
- Department of Computer Science and Engineering, School of Computing and IT, Manipal University Jaipur, Jaipur, India
| | | | | | | | - Mohammad Aman Ullah
- Department of Computer Science and Engineering, International Islamic University Chittagong, Chittagong, Bangladesh
| |
Collapse
|
6
|
Roul RK, Satyanath G. A Novel Feature Selection Based Text Classification Using Multi-layer ELM. BIG DATA ANALYTICS 2022. [DOI: 10.1007/978-3-031-24094-2_3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023] Open
|
7
|
Bao T, Ren N, Luo R, Wang B, Shen G, Guo T. A BERT-Based Hybrid Short Text Classification Model Incorporating CNN and Attention-Based BiGRU. J ORGAN END USER COM 2021. [DOI: 10.4018/joeuc.294580] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Short text classification is a research focus for natural language processing (NLP), which is widely used in news classification, sentiment analysis, mail filtering and other fields. In recent years, deep learning techniques are applied to text classification and has made some progress. Different from ordinary text classification, short text has the problem of less vocabulary and feature sparsity, which raise higher request for text semantic feature representation. To address this issue, this paper propose a feature fusion framework based on the Bidirectional Encoder Representations from Transformers (BERT). In this hybrid method, BERT is used to train word vector representation. Convolutional neural network (CNN) capture static features. As a supplement, a bi-gated recurrent neural network (BiGRU) is adopted to capture contextual features. Furthermore, an attention mechanism is introduced to assign the weight of salient words. The experimental results confirmed that the proposed model significantly outperforms the other state-of-the-art baseline methods.
Collapse
Affiliation(s)
- Tong Bao
- Information Center, Jiangsu Academy of Agricultural Sciences & Institute of Science and Technology Information, Jiangsu University, China
| | - Ni Ren
- Information Center, Jiangsu Academy of Agricultural Sciences & Institute of Science and Technology Information, Jiangsu University, China
| | - Rui Luo
- Information Center, Jiangsu Academy of Agricultural Sciences, China
| | - Baojia Wang
- Information Center, Jiangsu Academy of Agricultural Sciences, China
| | - Gengyu Shen
- Information Center, Jiangsu Academy of Agricultural Sciences, China
| | - Ting Guo
- Information Center, Jiangsu Academy of Agricultural Sciences, China
| |
Collapse
|
8
|
Wu P, Li X, Ling C, Ding S, Shen S. Sentiment classification using attention mechanism and bidirectional long short-term memory network. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107792] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|