Biomedical event trigger detection with convolutional highway neural network and extreme learning machine

SWACG: A Hybrid Neural Network Integrating Sliding Window for Biomedical Event Trigger Extraction

Abstract As an important part of biomedical text mining, biomedical events play a key role in improving biomedical research and disease prevention. Trigger identification, extracting the words describing the event types, is a critical and prerequisite step for biomedical event extraction. Traditional methods excessively rely on natural language processing tools in the feature extraction process, incurring a significant manual cost. In addition, because of the particularity of the biomedical literature, the problem of long-distance dependency is obvious. To solve these problems, we propose a hybrid structure SWACG, which consists of the ReCNN-BiGRU (Residual CNN and Bidirectional Gated Recurrent Unit) hybrid neural network and MH-attention (Multi-Head attention) mechanism. The proposed model uses ReCNN to extract vocabulary-level features and BiGRU to obtain contextual semantic information. Furthermore, sliding window divides long sentences into equal-length short sentences without destroying context information, which can avoid long-distance dependency. Experimental results show that our method advances the state-of-the-art performance on the commonly used Multi-Level Event Extraction (MLEE) corpus, achieving 82.20% F-score.

A transfer learning model with multi-source domains for biomedical event trigger extraction

BACKGROUND

Automatic extraction of biomedical events from literature, that allows for faster update of the latest discoveries automatically, is a heated research topic now. Trigger word recognition is a critical step in the process of event extraction. Its performance directly influences the results of the event extraction. In general, machine learning-based trigger recognition approaches such as neural networks must to be trained on a dataset with plentiful annotations to achieve high performances. However, the problem of the datasets in wide coverage event domains is that their annotations are insufficient and imbalance. One of the methods widely used to deal with this problem is transfer learning. In this work, we aim to extend the transfer learning to utilize multiple source domains. Multiple source domain datasets can be jointly trained to help achieve a higher recognition performance on a target domain with wide coverage events.

RESULTS

Based on the study of previous work, we propose an improved multi-source domain neural network transfer learning architecture and a training approach for biomedical trigger detection task, which can share knowledge between the multi-source and target domains more comprehensively. We extend the ability of traditional adversarial networks to extract common features between source and target domains, when there is more than one dataset in the source domains. Multiple feature extraction channels to simultaneously capture global and local common features are designed. Moreover, under the constraint of an extra classifier, the multiple local common feature sub-channels can extract and transfer more diverse common features from the related multi-source domains effectively. In the experiments, MLEE corpus is used to train and test the proposed model to recognize the wide coverage triggers as a target dataset. Other four corpora with the varying degrees of relevance with MLEE from different domains are used as source datasets, respectively. Our proposed approach achieves recognition improvement compared with traditional adversarial networks. Moreover, its performance is competitive compared with the results of other leading systems on the same MLEE corpus.

CONCLUSIONS

The proposed Multi-Source Transfer Learning-based Trigger Recognizer (MSTLTR) can further improve the performance compared with the traditional method, when the source domains are more than one. The most essential improvement is that our approach represents common features in two aspects: the global common features and the local common features. Hence, these more sharable features improve the performance and generalization of the model on the target domain effectively.