Improving the Named Entity Recognition of Chinese Electronic Medical Records by Combining Domain Dictionary and Rules

Clinical named entity recognition for percutaneous coronary intervention surgical information with hybrid neural network

Percutaneous coronary intervention (PCI) has become a vital treatment approach for coronary artery disease, but the clinical data of PCI cannot be directly utilized due to its unstructured characteristics. The existing clinical named entity recognition (CNER) has been used to identify specific entities such as body parts, drugs, and diseases, but its specific potential in PCI clinical texts remains largely unexplored. How to effectively use CNER to deeply mine the information in the existing PCI clinical records is worth studying. In this paper, a total of 24 267 corpora are collected from the Cardiovascular Disease Treatment Center of the People's Hospital of Liaoning Province in China. We select three types of clinical record texts of fine-grained PCI surgical information, from which 5.8% of representative surgical records of PCI patients are selected as datasets for labeling. To fully utilize global information and multi-level semantic features, we design a novel character-level vector embedding method and further propose a new hybrid model based on it. Based on the classic Bidirectional Long Short-Term Memory Network (BiLSTM), the model further integrates Convolutional Neural Networks (CNNs) and Bidirectional Encoder Representations from Transformers (BERTs) for feature extraction and representation, and finally uses Conditional Random Field (CRF) for decoding and predicting label sequences. This hybrid model is referred to as BCC-BiLSTM in this paper. In order to verify the performance of the proposed hybrid model for extracting PCI surgical information, we simultaneously compare both representative traditional and intelligent methods. Under the same circumstances, compared with other intelligent methods, the BCC-BiLSTM proposed in this paper reduces the word vector dimension by 15%, and the F1 score reaches 86.2% in named entity recognition of PCI clinical texts, which is 26.4% higher than that of HMM. The improvement is 1.2% higher than BiLSTM + CRF and 0.7% higher than the most popular BERT + BiLSTM + CRF. Compared with the representative models, the hybrid model has better performance and can achieve optimal results faster in the model training process, so it has good clinical application prospects.

A Chinese named entity recognition model incorporating recurrent cell and information state recursion

Chinese is characterized by high syntactic complexity, chaotic annotation granularity, and slow convergence. Joint learning models can effectively improve the accuracy of Chinese Named Entity Recognition (NER), but they focus too much on local feature information and reduce the ability of long sequence feature extraction. To address the limitations of long sequence feature extraction ability, we propose a Chinese NER model called Incorporating Recurrent Cell and Information State Recursion (IRCSR-NER). The model integrates recurrent cells and information state recursion to improve the recognition ability of long entity boundaries. To solve the problem that Chinese and English have different focuses in syntactic analysis. We use the syntactic dependency approach to add lexical relationship information to sentences represented at the word level. The IRCSR-NER is applied to sequence feature extraction to improve the model efficiency and long-text feature extraction ability. The model captures contextual long-distance dependent information while focusing on local feature information. We evaluated our proposed model using four public datasets and compared it with other mainstream models. Experimental results demonstrate that our model outperforms traditional and mainstream models.

A news-based climate policy uncertainty index for China

Climate policies can have a significant impact on the economy. However, these policies have often been associated with uncertainty. Quantitative assessment of the socioeconomic impact of climate policy uncertainty is equally or perhaps more important than looking at the policies themselves. Using a deep learning algorithm-the MacBERT model-this study constructed indices of Chinese climate policy uncertainty (CCPU) at the national, provincial and city levels for the first time. The CCPU indices are based on the text mining of news published by a set of major newspapers in China. A clear upward trend was found in the indices, demonstrating increasing policy uncertainties in China in addressing climate change. There is also evidence of clear regional heterogeneity in subnational indices. The CCPU dataset can provide a useful source of information for government actors, academics and investors in understanding the dynamics of climate policies in China. These indices can also be used to investigate the empirical relationship between climate policy uncertainty and other socioeconomic factors in China.

BIR: Biomedical Information Retrieval System for Cancer Treatment in Electronic Health Record Using Transformers

The rapid growth of electronic health records (EHRs) has led to unprecedented biomedical data. Clinician access to the latest patient information can improve the quality of healthcare. However, clinicians have difficulty finding information quickly and easily due to the sheer data mining volume. Biomedical information retrieval (BIR) systems can help clinicians find the information required by automatically searching EHRs and returning relevant results. However, traditional BIR systems cannot understand the complex relationships between EHR entities. Transformers are a new type of neural network that is very effective for natural language processing (NLP) tasks. As a result, transformers are well suited for tasks such as machine translation and text summarization. In this paper, we propose a new BIR system for EHRs that uses transformers for predicting cancer treatment from EHR. Our system can understand the complex relationships between the different entities in an EHR, which allows it to return more relevant results to clinicians. We evaluated our system on a dataset of EHRs and found that it outperformed state-of-the-art BIR systems on various tasks, including medical question answering and information extraction. Our results show that Transformers are a promising approach for BIR in EHRs, reaching an accuracy and an F1-score of 86.46%, and 0.8157, respectively. We believe that our system can help clinicians find the information they need more quickly and easily, leading to improved patient care.

Enhancing efficiency and capacity of telehealth services with intelligent triage: a bidirectional LSTM neural network model employing character embedding

BACKGROUND

The widespread adoption of telehealth services necessitates accurate online department selection based on patient medical records, a task requiring significant medical knowledge. Incorrect triage results in considerable time wastage for both patients and medical professionals. To address this, we propose an intelligent triage model based on a Bidirectional Long Short-Term Memory (Bi-LSTM) neural network with character embedding to enhance the efficiency and capacity of telehealth services.

METHODS

We gathered a 1.3 GB medical dataset comprising 200,000 records, each including medical history, physical examination data, and other pertinent information found on the electronic medical record homepage. Following data preprocessing, a clinical corpus was established to train character embeddings with a medical context. These character embeddings were then utilized to extract features from patient chief  complaints, and a 2-layer Bi-LSTM neural network was trained to categorize these complaints, enabling intelligent triage for telehealth services.

RESULTS

60,000 chief complaint-department data pairs were extracted from clinical corpus and divided into the training, validation, and test sets of 42,000, 9,000, and 9,000, respectively. The character embedding based Bi-LSTM neural network achieved a macro-precision of 85.50% and an F1 score of 85.45%.

CONCLUSION

The telehealth triage model developed in this study demonstrates strong implementation outcomes and significantly improves the efficiency and capacity of telehealth services. Character embedding outperforms word embedding, and future work will incorporate additional features such as patient age and gender into the chief complaint feature to future enhance model performance.

Named Entity Recognition in Electronic Health Records: A Methodological Review

OBJECTIVES

A substantial portion of the data contained in Electronic Health Records (EHR) is unstructured, often appearing as free text. This format restricts its potential utility in clinical decision-making. Named entity recognition (NER) methods address the challenge of extracting pertinent information from unstructured text. The aim of this study was to outline the current NER methods and trace their evolution from 2011 to 2022.

METHODS

We conducted a methodological literature review of NER methods, with a focus on distinguishing the classification models, the types of tagging systems, and the languages employed in various corpora.

RESULTS

Several methods have been documented for automatically extracting relevant information from EHRs using natural language processing techniques such as NER and relation extraction (RE). These methods can automatically extract concepts, events, attributes, and other data, as well as the relationships between them. Most NER studies conducted thus far have utilized corpora in English or Chinese. Additionally, the bidirectional encoder representation from transformers using the BIO tagging system architecture is the most frequently reported classification scheme. We discovered a limited number of papers on the implementation of NER or RE tasks in EHRs within a specific clinical domain.

CONCLUSIONS

EHRs play a pivotal role in gathering clinical information and could serve as the primary source for automated clinical decision support systems. However, the creation of new corpora from EHRs in specific clinical domains is essential to facilitate the swift development of NER and RE models applied to EHRs for use in clinical practice.

Sentiment Classification of Chinese Tourism Reviews Based on ERNIE-Gram+GCN

Nowadays, tourists increasingly prefer to check the reviews of attractions before traveling to decide whether to visit them or not. To respond to the change in the way tourists choose attractions, it is important to classify the reviews of attractions with high precision. In addition, more and more tourists like to use emojis to express their satisfaction or dissatisfaction with the attractions. In this paper, we built a dataset for Chinese attraction evaluation incorporating emojis (CAEIE) and proposed an explicitly n-gram masking method to enhance the integration of coarse-grained information into a pre-training (ERNIE-Gram) and Text Graph Convolutional Network (textGCN) (E2G) model to classify the dataset with a high accuracy. The E2G preprocesses the text and feeds it to ERNIE-Gram and TextGCN. ERNIE-Gram was trained using its unique mask mechanism to obtain the final probabilities. TextGCN used the dataset to construct heterogeneous graphs with comment text and words, which were trained to obtain a representation of the document output category probabilities. The two probabilities were calculated to obtain the final results. To demonstrate the validity of the E2G model, this paper was compared with advanced models. After experiments, it was shown that E2G had a good classification effect on the CAEIE dataset, and the accuracy of classification was up to 97.37%. Furthermore, the accuracy of E2G was 1.37% and 1.35% ahead of ERNIE-Gram and TextGCN, respectively. In addition, two sets of comparison experiments were conducted to verify the performance of TextGCN and TextGAT on the CAEIE dataset. The final results showed that ERNIE and ERNIE-Gram combined TextGCN and TextGAT, respectively, and TextGCN performed 1.6% and 2.15% ahead. This paper compared the effects of eight activation functions on the second layer of the TextGCN and the activation-function-rectified linear unit 6 (RELU6) with the best results based on experiments.

Extracting clinical named entity for pituitary adenomas from Chinese electronic medical records

OBJECTIVE

Pituitary adenomas are the most common type of pituitary disorders, which usually occur in young adults and often affect the patient's physical development, labor capacity and fertility. Clinical free texts noted in electronic medical records (EMRs) of pituitary adenomas patients contain abundant diagnosis and treatment information. However, this information has not been well utilized because of the challenge to extract information from unstructured clinical texts. This study aims to enable machines to intelligently process clinical information, and automatically extract clinical named entity for pituitary adenomas from Chinese EMRs.

METHODS

The clinical corpus used in this study was from one pituitary adenomas neurosurgery treatment center of a 3A hospital in China. Four types of fine-grained texts of clinical records were selected, which included notes from present illness, past medical history, case characteristics and family history of 500 pituitary adenoma inpatients. The dictionary-based matching, conditional random fields (CRF), bidirectional long short-term memory with CRF (BiLSTM-CRF), and bidirectional encoder representations from transformers with BiLSTM-CRF (BERT-BiLSTM-CRF) were used to extract clinical entities from a Chinese EMRs corpus. A comprehensive dictionary was constructed based on open source vocabularies and a domain dictionary for pituitary adenomas to conduct the dictionary-based matching method. We selected features such as part of speech, radical, document type, and the position of characters to train the CRF-based model. Random character embeddings and the character embeddings pretrained by BERT were used respectively as the input features for the BiLSTM-CRF model and the BERT-BiLSTM-CRF model. Both strict metric and relaxed metric were used to evaluate the performance of these methods.

RESULTS

Experimental results demonstrated that the deep learning and other machine learning methods were able to automatically extract clinical named entities, including symptoms, body regions, diseases, family histories, surgeries, medications, and disease courses of pituitary adenomas from Chinese EMRs. With regard to overall performance, BERT-BiLSTM-CRF has the highest strict F1 value of 91.27% and the highest relaxed F1 value of 95.57% respectively. Additional evaluations showed that BERT-BiLSTM-CRF performed best in almost all entity recognition except surgery and disease course. BiLSTM-CRF performed best in disease course entity recognition, and performed as well as the CRF model for part of speech, radical and document type features, with both strict and relaxed F1 value reaching 96.48%. The CRF model with part of speech, radical and document type features performed best in surgery entity recognition with relaxed F1 value of 95.29%.

CONCLUSIONS

In this study, we conducted four entity recognition methods for pituitary adenomas based on Chinese EMRs. It demonstrates that the deep learning methods can effectively extract various types of clinical entities with satisfying performance. This study contributed to the clinical named entity extraction from Chinese neurosurgical EMRs. The findings could also assist in information extraction in other Chinese medical texts.

A BERT-BiGRU-CRF Model for Entity Recognition of Chinese Electronic Medical Records

Because of difficulty processing the electronic medical record data of patients with cerebrovascular disease, there is little mature recognition technology capable of identifying the named entity of cerebrovascular disease. Excellent research results have been achieved in the field of named entity recognition (NER), but there are several problems in the pre processing of Chinese named entities that have multiple meanings, of which neglecting the combination of contextual information is one. Therefore, to extract five categories of key entity information for diseases, symptoms, body parts, medical examinations, and treatment in electronic medical records, this paper proposes the use of a BERT-BiGRU-CRF named entity recognition method, which is applied to the field of cerebrovascular diseases. The BERT layer first converts the electronic medical record text into a low-dimensional vector, then uses this vector as the input to the BiGRU layer to capture contextual features, and finally uses conditional random fields (CRFs) to capture the dependency between adjacent tags. The experimental results show that the F1 score of the model reaches 90.38%.