Neural network-based approaches for biomedical relation classification: A review

RIscoper 2.0: A deep learning tool to extract RNA biomedical relation sentences from literature

RNA plays an extensive role in a multi-dimensional regulatory system, and its biomedical relationships are scattered across numerous biological studies. However, text mining works dedicated to the extraction of RNA biomedical relations remain limited. In this study, we established a comprehensive and reliable corpus of RNA biomedical relations, recruiting over 30,000 sentences manually curated from more than 15,000 biomedical literature. We also updated RIscoper 2.0, a BERT-based deep learning tool to extract RNA biomedical relation sentences from literature. Benefiting from approximately 100,000 annotated named entities, we integrated the text classification and named entity recognition tasks in this tool. Additionally, RIscoper 2.0 outperformed the original tool in both tasks and can discover new RNA biomedical relations. Additionally, we provided a user-friendly online search tool that enables rapid scanning of RNA biomedical relationships using local and online resources. Both the online tools and data resources of RIscoper 2.0 are available at http://www.rnainter.org/riscoper.

Large language model based framework for automated extraction of genetic interactions from unstructured data

Extracting biological interactions from published literature helps us understand complex biological systems, accelerate research, and support decision-making in drug or treatment development. Despite efforts to automate the extraction of biological relations using text mining tools and machine learning pipelines, manual curation continues to serve as the gold standard. However, the rapidly increasing volume of literature pertaining to biological relations poses challenges in its manual curation and refinement. These challenges are further compounded because only a small fraction of the published literature is relevant to biological relation extraction, and the embedded sentences of relevant sections have complex structures, which can lead to incorrect inference of relationships. To overcome these challenges, we propose GIX, an automated and robust Gene Interaction Extraction framework, based on pre-trained Large Language models fine-tuned through extensive evaluations on various gene/protein interaction corpora including LLL and RegulonDB. GIX identifies relevant publications with minimal keywords, optimises sentence selection to reduce computational overhead, simplifies sentence structure while preserving meaning, and provides a confidence factor indicating the reliability of extracted relations. GIX's Stage-2 relation extraction method performed well on benchmark protein/gene interaction datasets, assessed using 10-fold cross-validation, surpassing state-of-the-art approaches. We demonstrated that the proposed method, although fully automated, performs as well as manual relation extraction, with enhanced robustness. We also observed GIX's capability to augment existing datasets with new sentences, incorporating newly discovered biological terms and processes. Further, we demonstrated GIX's real-world applicability in inferring E. coli gene circuits.

GlioPredictor: a deep learning model for identification of high-risk adult IDH-mutant glioma towards adjuvant treatment planning

Identification of isocitrate dehydrogenase (IDH)-mutant glioma patients at high risk of early progression is critical for radiotherapy treatment planning. Currently tools to stratify risk of early progression are lacking. We sought to identify a combination of molecular markers that could be used to identify patients who may have a greater need for adjuvant radiation therapy machine learning technology. 507 WHO Grade 2 and 3 glioma cases from The Cancer Genome Atlas, and 1309 cases from AACR GENIE v13.0 datasets were studied for genetic disparities between IDH1-wildtype and IDH1-mutant cohorts, and between different age groups. Genetic features such as mutations and copy number variations (CNVs) correlated with IDH1 mutation status were selected as potential inputs to train artificial neural networks (ANNs) to predict IDH1 mutation status. Grade 2 and 3 glioma cases from the Memorial Sloan Kettering dataset (n = 404) and Grade 3 glioma cases with subtotal resection (STR) from Northwestern University (NU) (n = 21) were used to further evaluate the best performing ANN model as independent datasets. IDH1 mutation is associated with decreased CNVs of EGFR (21% vs. 3%), CDKN2A (20% vs. 6%), PTEN (14% vs. 1.7%), and increased percentage of mutations for TP53 (15% vs. 63%), and ATRX (10% vs. 54%), which were all statistically significant (p < 0.001). Age > 40 was unable to identify high-risk IDH1-mutant with early progression. A glioma early progression risk prediction (GlioPredictor) score generated from the best performing ANN model (6/6/6/6/2/1) with 6 inputs, including CNVs of EGFR, PTEN and CDKN2A, mutation status of TP53 and ATRX, patient's age can predict IDH1 mutation status with over 90% accuracy. The GlioPredictor score identified a subgroup of high-risk IDH1-mutant in TCGA and NU datasets with early disease progression (p = 0.0019, 0.0238, respectively). The GlioPredictor that integrates age at diagnosis, CNVs of EGFR, CDKN2A, PTEN and mutation status of TP53, and ATRX can identify a small cohort of IDH-mutant with high risk of early progression. The current version of GlioPredictor mainly incorporated clinically often tested genetic biomarkers. Considering complexity of clinical and genetic features that correlate with glioma progression, future derivatives of GlioPredictor incorporating more inputs can be a potential supplement for adjuvant radiotherapy patient selection of IDH-mutant glioma patients.

BioEGRE: a linguistic topology enhanced method for biomedical relation extraction based on BioELECTRA and graph pointer neural network

BACKGROUND

Automatic and accurate extraction of diverse biomedical relations from literature is a crucial component of bio-medical text mining. Currently, stacking various classification networks on pre-trained language models to perform fine-tuning is a common framework to end-to-end solve the biomedical relation extraction (BioRE) problem. However, the sequence-based pre-trained language models underutilize the graphical topology of language to some extent. In addition, sequence-oriented deep neural networks have limitations in processing graphical features.

RESULTS

In this paper, we propose a novel method for sentence-level BioRE task, BioEGRE (BioELECTRA and Graph pointer neural net-work for Relation Extraction), aimed at leveraging the linguistic topological features. First, the biomedical literature is preprocessed to retain sentences involving pre-defined entity pairs. Secondly, SciSpaCy is employed to conduct dependency parsing; sentences are modeled as graphs based on the parsing results; BioELECTRA is utilized to generate token-level representations, which are modeled as attributes of nodes in the sentence graphs; a graph pointer neural network layer is employed to select the most relevant multi-hop neighbors to optimize representations; a fully-connected neural network layer is employed to generate the sentence-level representation. Finally, the Softmax function is employed to calculate the probabilities. Our proposed method is evaluated on three BioRE tasks: a multi-class (CHEMPROT) and two binary tasks (GAD and EU-ADR). The results show that our method achieves F1-scores of 79.97% (CHEMPROT), 83.31% (GAD), and 83.51% (EU-ADR), surpassing the performance of existing state-of-the-art models.

CONCLUSION

The experimental results on 3 biomedical benchmark datasets demonstrate the effectiveness and generalization of BioEGRE, which indicates that linguistic topology and a graph pointer neural network layer explicitly improve performance for BioRE tasks.

Biomedical Text Classification Using Augmented Word Representation Based on Distributional and Relational Contexts

Due to the increasing use of information technologies by biomedical experts, researchers, public health agencies, and healthcare professionals, a large number of scientific literatures, clinical notes, and other structured and unstructured text resources are rapidly increasing and being stored in various data sources like PubMed. These massive text resources can be leveraged to extract valuable knowledge and insights using machine learning techniques. Recent advancement in neural network-based classification models has gained popularity which takes numeric vectors (aka word representation) of training data as the input to train classification models. Better the input vectors, more accurate would be the classification. Word representations are learned as the distribution of words in an embedding space, wherein each word has its vector and the semantically similar words based on the contexts appear nearby each other. However, such distributional word representations are incapable of encapsulating relational semantics between distant words. In the biomedical domain, relation mining is a well-studied problem which aims to extract relational words, which associates distant entities generally representing the subject and object of a sentence. Our goal is to capture the relational semantics information between distant words from a large corpus to learn enhanced word representation and employ the learned word representation for various natural language processing tasks such as text classification. In this article, we have proposed an application of biomedical relation triplets to learn word representation through incorporating relational semantic information within the distributional representation of words. In other words, the proposed approach aims to capture both distributional and relational contexts of the words to learn their numeric vectors from text corpus. We have also proposed an application of the learned word representations for text classification. The proposed approach is evaluated over multiple benchmark datasets, and the efficacy of the learned word representations is tested in terms of word similarity and concept categorization tasks. Our proposed approach provides better performance in comparison to the state-of-the-art GloVe model. Furthermore, we have applied the learned word representations to classify biomedical texts using four neural network-based classification models, and the classification accuracy further confirms the effectiveness of the learned word representations by our proposed approach.

Predicting length of stay ranges by using novel deep neural networks

Background and aims

Accurately predicting length of stay (LOS) is considered a challenging task for health care systems globally. In previous studies on LOS range prediction, researchers commonly pre-classified the LOS ranges, which were the same for all patients in the same classification, and then utilized a classifier for prediction. In this study, we innovatively aimed to predict the specific LOS range for each patient (the LOS range was different for each patient).

Methods

In the modified deep neural network (DNN), the overall sample error (root mean square error (RMSE) method), the estimated sample error (ERR_pred method), the probability distribution with different loss functions (Dis_pred_Loss1, Dis_pred_Loss2, and Dis_pred_Loss3 method), and the generative adversarial networks (WGAN-GP for LOS method) are used for LOS range prediction. The Medical Information Mart for Intensive Care III (MIMIC-III) database is used to validate these methods.

Results

The RMSE method is convenient for LOS range prediction, but the predicted ranges are all consistent in the same batch of samples. The ERR_pred method can achieve better prediction results in samples with low errors. However, the prediction effect is worse in samples with larger errors. The Dis_pred_Loss1 method encounters a training instability problem. The Dis_pred_Loss2 and Dis_pred_Loss3 methods perform well in making predictions. Although WGAN-GP for LOS method does not show a substantial advantage over other methods, this method might have the potential to improve the predictive performance.

Conclusion

The results show that it is possible to achieve an acceptable accurate LOS range prediction through a reasonable model design, which may help physicians in the clinic.

Preventable risk factors for type 2 diabetes can be detected using noninvasive spontaneous electroretinogram signals

Given the ever-increasing prevalence of type 2 diabetes and obesity, the pressure on global healthcare is expected to be colossal, especially in terms of blindness. Electroretinogram (ERG) has long been perceived as a first-use technique for diagnosing eye diseases, and some studies suggested its use for preventable risk factors of type 2 diabetes and thereby diabetic retinopathy (DR). Here, we show that in a non-evoked mode, ERG signals contain spontaneous oscillations that predict disease cases in rodent models of obesity and in people with overweight, obesity, and metabolic syndrome but not yet diabetes, using one single random forest-based model. Classification performance was both internally and externally validated, and correlation analysis showed that the spontaneous oscillations of the non-evoked ERG are altered before oscillatory potentials, which are the current gold-standard for early DR. Principal component and discriminant analysis suggested that the slow frequency (0.4-0.7 Hz) components are the main discriminators for our predictive model. In addition, we established that the optimal conditions to record these informative signals, are 5-minute duration recordings under daylight conditions, using any ERG sensors, including ones working with portative, non-mydriatic devices. Our study provides an early warning system with promising applications for prevention, monitoring and even the development of new therapies against type 2 diabetes.

The Clinical Application of Machine Learning Models for Risk Analysis of Ramp Lesions in Anterior Cruciate Ligament Injuries

BACKGROUND

Peripheral tears of the posterior horn of the medial meniscus, known as "ramp lesions," are commonly found in anterior cruciate ligament (ACL)-deficient knees but are frequently missed on routine evaluation.

PURPOSE

To predict the presence of ramp lesions in ACL-deficient knees using machine learning methods with associated risk factors.

STUDY DESIGN

Cohort study (Diagnosis); Level of evidence, 2.

METHODS

This study included 362 patients who underwent ACL reconstruction between June 2010 and March 2019. The exclusion criteria were combined fractures and multiple ligament injuries, except for medial collateral ligament injuries. Patients were grouped according to the presence of ramp lesions on arthroscopic surgery. Binary logistic regression was used to analyze risk factors including age, sex, body mass index, time from injury to surgery (≥3 or <3 months), mechanism of injury (contact or noncontact), side-to-side laxity, pivot-shift grade, medial and lateral tibial/meniscal slope, location of bone contusion, mechanical axis angle, and lateral femoral condyle (LFC) ratio. The receiver operating characteristic curve and area under the curve were also evaluated.

RESULTS

Ramp lesions were identified in 112 patients (30.9%). The risk for ramp lesions increased with steeper medial tibial and meniscal slopes, higher knee laxity, and an increased LFC ratio. Comparing the final performance of all models, the random forest model yielded the best performance (area under the curve: 0.944), although there were no significant differences among the models (P > .05). The cut-off values for the presence of ramp lesions on receiver operating characteristic analysis were as follows: medial tibial slope >5.5° (P < .001), medial meniscal slope >5.0° (P < .001), and LFC ratio >71.3% (P = .033).

CONCLUSION

Steep medial tibial and meniscal slopes, an increased LFC ratio, and higher knee rotatory laxity were observed risk factors for ramp lesions in patients with an ACL injury. The prediction model of this study could be used as a supplementary diagnostic tool for ramp lesions in ACL-injured knees. In general, care should be taken in patients with ramp lesions and its risk factors during ACL reconstruction.

Identification of FERMT1 and SGCD as key marker in acute aortic dissection from the perspective of predictive, preventive, and personalized medicine

Acute aortic dissection (AAD) is a severe aortic injury disease, which is often life-threatening at the onset. However, its early prevention remains a challenge. Therefore, in the context of predictive, preventive, and personalized medicine (PPPM), it is particularly important to identify novel and powerful biomarkers. This study aimed to identify the key markers that may contribute to the predictive early risk of AAD and analyze their role in immune infiltration. Three datasets, including a total of 23 AAD and 20 healthy control aortic samples, were retrieved from the Gene Expression Omnibus (GEO) database, and a total of 519 differentially expressed genes (DEGs) were screened in the training set. Using the least absolute shrinkage and selection operator (LASSO) regression model and the random forest (RF) algorithm, FERMT1 (AUC = 0.886) and SGCD (AUC = 0.876) were identified as key markers of AAD. A novel AAD risk prediction model was constructed using an artificial neural network (ANN), and in the validation set, the AUC = 0.920. Immune infiltration analysis indicated differential gene expression in regulatory T cells, monocytes, γδ T cells, quiescent NK cells, and mast cells in the patients with AAD and the healthy controls. Correlation and ssGSEA analysis showed that two key markers' expression in patients with AAD was correlated with many inflammatory mediators and pathways. In addition, the drug-gene interaction network identified motesanib and pyrazoloacridine as potential therapeutic agents for two key markers, which may provide personalized medical services for AAD patients. These findings highlight FERMT1 and SGCD as key biological targets for AAD and reveal the inflammation-related potential molecular mechanism of AAD, which is helpful for early risk prediction and targeted prevention of AAD. In conclusion, our study provides a new perspective for developing a PPPM method for managing AAD patients.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13167-022-00302-4.

Deep learning joint models for extracting entities and relations in biomedical: a survey and comparison

The rapid development of biomedicine has produced a large number of biomedical written materials. These unstructured text data create serious challenges for biomedical researchers to find information. Biomedical named entity recognition (BioNER) and biomedical relation extraction (BioRE) are the two most fundamental tasks of biomedical text mining. Accurately and efficiently identifying entities and extracting relations have become very important. Methods that perform two tasks separately are called pipeline models, and they have shortcomings such as insufficient interaction, low extraction quality and easy redundancy. To overcome the above shortcomings, many deep learning-based joint name entity recognition and relation extraction models have been proposed, and they have achieved advanced performance. This paper comprehensively summarize deep learning models for joint name entity recognition and relation extraction for biomedicine. The joint BioNER and BioRE models are discussed in the light of the challenges existing in the BioNER and BioRE tasks. Five joint BioNER and BioRE models and one pipeline model are selected for comparative experiments on four biomedical public datasets, and the experimental results are analyzed. Finally, we discuss the opportunities for future development of deep learning-based joint BioNER and BioRE models.

Prediction of Postoperative Survival in Young Colorectal Cancer Patients: A Cohort Study Based on the SEER Database

Objective

Our aim is to make accurate and robust predictions of the risk of postoperative death in young colorectal cancer patients (18-44 years old) by combining tumor characteristics with medical and demographic information about the patient.

Materials and Methods

We used the SEER database to retrieve young patients diagnosed with colorectal cancer who had undergone surgery between 2010 and 2015 as the study cohort. After excluding cases with missing information, the study cohort was divided in a 7 : 3 ratio into a training dataset and a validation dataset. To assess the predictive ability of each predictor on the prognosis of colorectal cancer patients, we used two steps of Cox univariate analysis and Cox stepwise regression to screen variables, and the screened variables were included in a multifactorial Cox proportional risk regression model for modeling. The performance of the model was tested using calibration curves, decision curves, and area under the curve (AUC) for receiver operating characteristic (ROC).

Results

After excluding cases with missing information (n = 23,606), a total of 11,803 patients were included in the study with a median follow-up time of 45 months (1-119). In the training set, we determined that ethnicity, marital status, insurance status, median annual household income, degree of tumor differentiation, type of pathology, degree of infiltration, and tumor location had independent effects on prognosis. In the training dataset, taking 1 year, 3 years, and 5 years as the time nodes, the areas under the working characteristic curve of subjects are 0.825, 0.851, and 0.839, respectively, and in the validation dataset, they are 0.834, 0.837, and 0.829, respectively.

Conclusion

We trained and validated a model using a large multicenter cohort of young colorectal cancer patients with stable and excellent performance in both training and validation datasets.

Discovering Content through Text Mining for a Synthetic Biology Knowledge System

Scientific articles contain a wealth of information about experimental methods and results describing biological designs. Due to its unstructured nature and multiple sources of ambiguity and variability, extracting this information from text is a difficult task. In this paper, we describe the development of the synthetic biology knowledge system (SBKS) text processing pipeline. The pipeline uses natural language processing techniques to extract and correlate information from the literature for synthetic biology researchers. Specifically, we apply named entity recognition, relation extraction, concept grounding, and topic modeling to extract information from published literature to link articles to elements within our knowledge system. Our results show the efficacy of each of the components on synthetic biology literature and provide future directions for further advancement of the pipeline.

A New Competitive Neural Architecture for Object Classification

In this paper, we propose a new neural architecture for object classification, made up from a set of competitive layers whose number and size are dynamically learned from training data using a two-step process that combines unsupervised and supervised learning modes. The first step consists in finding a set of one or more optimal prototypes for each of the c classes that form the training data. For this, it uses the unsupervised learning and prototype generator algorithm called fuzzy learning vector quantization (FLVQ). The second step aims to assess the quality of the learned prototypes in terms of classification results. For this, the c classes are reconstructed by assigning each object to the class represented by its nearest prototype, and the obtained results are compared to the original classes. If one or more constructed classes differ from the original ones, the corresponding prototypes are not validated and the whole process is repeated for all misclassified objects, using additional competitive layers, until no difference persists between the constructed and the original classes or a maximum number of layers is reached. Experimental results show the effectiveness of the proposed method on a variety of well-known benchmark data sets.

Functional Analysis of Bronchopulmonary Dysplasia-Related Neuropeptides in Preterm Infants and miRNA-Based Diagnostic Model Construction

Background

Bronchopulmonary dysplasia (BPD) has a high mortality rate. This study was aimed at identifying and analysing the risk factors associated with BPD using bioinformatic and mechanical analyses and establishing a predictive model to assess the risk of BPD in preterm infants.

Methods

We identified differentially expressed RNAs via the intersection of miRNAs between datasets. Online analysis tools were used to predict genes targeted by differentially expressed miRNAs (DEmiRNAs) and to generate and visualise competing endogenous RNA (ceRNA) coexpression networks. Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) analyses were subsequently performed on the DEmiRNAs. In addition, an intersection analysis was performed on mRNA and neuropeptide-related genes in the ceRNA network. DEmiRNAs associated with BPD and those involved in ceRNA networks were used to establish a diagnostic prediction model. The GSE108604 dataset was used as a validation set to verify the model.

Results

A total of 26 DEmiRNAs were identified from the tracheal aspirates (TAs) of patients with BPD and healthy controls. In addition, a total of 1076 DEmRNAs were obtained from the GSE8586 dataset. Functional enrichment analysis of DEmRNAs revealed an abnormal reduction in mitochondrial-related activity and cellular responses to oxidative stress in patients with BPD. The neuropeptide-related genes OPRL1 and NPPA were found to be upregulated in BPD samples. Eventually, hsa-miR-1258, hsa-miR-298, hsa-miR-483-3p, and hsa-miR-769-5p were screened out and used to establish the prediction model. Calibration curves and detrended correspondence analysis (DCA) revealed that the model had good clinical applicability.

Conclusions

The prediction model provided a simple method for individualised assessment, early diagnosis, and prevention of BPD risk in preterm infants.

A deep learning method and device for bone marrow imaging cell detection

BACKGROUND

Morphological analysis of bone marrow cells is considered as the gold standard for the diagnosis of leukemia. However, due to the diverse morphology of bone marrow cells, extensive experience and patience are needed for morphological examination. automatic diagnosis system through the comprehensive application of image analysis and pattern recognition technology is urgently needed to reduce work intensity, error probability and improves work efficiency.

METHODS

In this article, we establish a new morphological diagnosis system for bone marrow cell detection based on the deep learning object detection framework. The model is based on the Faster Region-Convolutional Neural Network (R-CNN), a classical object detection model. The system automatically detects bone marrow cells and determines their types. As specimens have severe long-tail distribution, i.e., the frequency of different types of cells varies dramatically, we proposed a general score ranking loss to solve such a problem. The general score ranking loss considers the ranking relationship between positive and negative samples and optimizes the positive sample with a higher classification probability value.

RESULTS

We verified this system with 70 bone marrow specimens of leukemia patients, which proved that it can realize intelligent recognition with high efficiency. The software is finally integrated into the microscope system to build an augmented reality system.

CONCLUSIONS

Clinical tests show that the response speed of the newly developed diagnostic system is faster than that of trained diagnostic experts.

Self-Attention-Based Models for the Extraction of Molecular Interactions from Biological Texts

For any molecule, network, or process of interest, keeping up with new publications on these is becoming increasingly difficult. For many cellular processes, the amount molecules and their interactions that need to be considered can be very large. Automated mining of publications can support large-scale molecular interaction maps and database curation. Text mining and Natural-Language-Processing (NLP)-based techniques are finding their applications in mining the biological literature, handling problems such as Named Entity Recognition (NER) and Relationship Extraction (RE). Both rule-based and Machine-Learning (ML)-based NLP approaches have been popular in this context, with multiple research and review articles examining the scope of such models in Biological Literature Mining (BLM). In this review article, we explore self-attention-based models, a special type of Neural-Network (NN)-based architecture that has recently revitalized the field of NLP, applied to biological texts. We cover self-attention models operating either at the sentence level or an abstract level, in the context of molecular interaction extraction, published from 2019 onwards. We conducted a comparative study of the models in terms of their architecture. Moreover, we also discuss some limitations in the field of BLM that identifies opportunities for the extraction of molecular interactions from biological text.

Relation extraction from DailyMed structured product labels by optimally combining crowd, experts and machines

The effectiveness of machine learning models to provide accurate and consistent results in drug discovery and clinical decision support is strongly dependent on the quality of the data used. However, substantive amounts of open data that drive drug discovery suffer from a number of issues including inconsistent representation, inaccurate reporting, and incomplete context. For example, databases of FDA-approved drug indications used in computational drug repositioning studies do not distinguish between treatments that simply offer symptomatic relief from those that target the underlying pathology. Moreover, drug indication sources often lack proper provenance and have little overlap. Consequently, new predictions can be of poor quality as they offer little in the way of new insights. Hence, work remains to be done to establish higher quality databases of drug indications that are suitable for use in drug discovery and repositioning studies. Here, we report on the combination of weak supervision (i.e., programmatic labeling and crowdsourcing) and deep learning methods for relation extraction from DailyMed text to create a higher quality drug-disease relation dataset. The generated drug-disease relation data shows a high overlap with DrugCentral, a manually curated dataset. Using this dataset, we constructed a machine learning model to classify relations between drugs and diseases from text into four categories; treatment, symptomatic relief, contradiction, and effect, exhibiting an improvement of 15.5% with Bi-LSTM (F1 score of 71.8%) over the best performing discrete method. Access to high quality data is crucial to building accurate and reliable drug repurposing prediction models. Our work suggests how the combination of crowds, experts, and machine learning methods can go hand-in-hand to improve datasets and predictive models.

Named Entity Recognition and Relation Detection for Biomedical Information Extraction

The number of scientific publications in the literature is steadily growing, containing our knowledge in the biomedical, health, and clinical sciences. Since there is currently no automatic archiving of the obtained results, much of this information remains buried in textual details not readily available for further usage or analysis. For this reason, natural language processing (NLP) and text mining methods are used for information extraction from such publications. In this paper, we review practices for Named Entity Recognition (NER) and Relation Detection (RD), allowing, e.g., to identify interactions between proteins and drugs or genes and diseases. This information can be integrated into networks to summarize large-scale details on a particular biomedical or clinical problem, which is then amenable for easy data management and further analysis. Furthermore, we survey novel deep learning methods that have recently been introduced for such tasks.

Deep Learning Based Biomedical Literature Classification Using Criteria of Scientific Rigor

A major blockade to support the evidence-based clinical decision-making is accurately and efficiently recognizing appropriate and scientifically rigorous studies in the biomedical literature. We trained a multi-layer perceptron (MLP) model on a dataset with two textual features, title and abstract. The dataset consisting of 7958 PubMed citations classified in two classes: scientific rigor and non-rigor, is used to train the proposed model. We compare our model with other promising machine learning models such as Support Vector Machine (SVM), Decision Tree, Random Forest, and Gradient Boosted Tree (GBT) approaches. Based on the higher cumulative score, deep learning was chosen and was tested on test datasets obtained by running a set of domain-specific queries. On the training dataset, the proposed deep learning model obtained significantly higher accuracy and AUC of 97.3% and 0.993, respectively, than the competitors, but was slightly lower in the recall of 95.1% as compared to GBT. The trained model sustained the performance of testing datasets. Unlike previous approaches, the proposed model does not require a human expert to create fresh annotated data; instead, we used studies cited in Cochrane reviews as a surrogate for quality studies in a clinical topic. We learn that deep learning methods are beneficial to use for biomedical literature classification. Not only do such methods minimize the workload in feature engineering, but they also show better performance on large and noisy data.

Medical Information Extraction in the Age of Deep Learning

OBJECTIVES

We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes-diseases and drugs (or medications)-and relations between them.

METHODS

For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence.

RESULTS

In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies.

CONCLUSIONS

The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.

Incorporating representation learning and multihead attention to improve biomedical cross-sentence n-ary relation extraction

BACKGROUND

Most biomedical information extraction focuses on binary relations within single sentences. However, extracting n-ary relations that span multiple sentences is in huge demand. At present, in the cross-sentence n-ary relation extraction task, the mainstream method not only relies heavily on syntactic parsing but also ignores prior knowledge.

RESULTS

In this paper, we propose a novel cross-sentence n-ary relation extraction method that utilizes the multihead attention and knowledge representation that is learned from the knowledge graph. Our model is built on self-attention, which can directly capture the relations between two words regardless of their syntactic relation. In addition, our method makes use of entity and relation information from the knowledge base to impose assistance while predicting the relation. Experiments on n-ary relation extraction show that combining context and knowledge representations can significantly improve the n-ary relation extraction performance. Meanwhile, we achieve comparable results with state-of-the-art methods.

CONCLUSIONS

We explored a novel method for cross-sentence n-ary relation extraction. Unlike previous approaches, our methods operate directly on the sequence and learn how to model the internal structures of sentences. In addition, we introduce the knowledge representations learned from the knowledge graph into the cross-sentence n-ary relation extraction. Experiments based on knowledge representation learning show that entities and relations can be extracted in the knowledge graph, and coding this knowledge can provide consistent benefits.

Applications of artificial intelligence and machine learning in respiratory medicine

The past 5 years have seen an explosion of interest in the use of artificial intelligence (AI) and machine learning techniques in medicine. This has been driven by the development of deep neural networks (DNNs)-complex networks residing in silico but loosely modelled on the human brain-that can process complex input data such as a chest radiograph image and output a classification such as 'normal' or 'abnormal'. DNNs are 'trained' using large banks of images or other input data that have been assigned the correct labels. DNNs have shown the potential to equal or even surpass the accuracy of human experts in pattern recognition tasks such as interpreting medical images or biosignals. Within respiratory medicine, the main applications of AI and machine learning thus far have been the interpretation of thoracic imaging, lung pathology slides and physiological data such as pulmonary function tests. This article surveys progress in this area over the past 5 years, as well as highlighting the current limitations of AI and machine learning and the potential for future developments.

Chemical–protein interaction extraction via Gaussian probability distribution and external biomedical knowledge

Motivation

The biomedical literature contains a wealth of chemical–protein interactions (CPIs). Automatically extracting CPIs described in biomedical literature is essential for drug discovery, precision medicine, as well as basic biomedical research. Most existing methods focus only on the sentence sequence to identify these CPIs. However, the local structure of sentences and external biomedical knowledge also contain valuable information. Effective use of such information may improve the performance of CPI extraction.

Results

In this article, we propose a novel neural network-based approach to improve CPI extraction. Specifically, the approach first employs BERT to generate high-quality contextual representations of the title sequence, instance sequence and knowledge sequence. Then, the Gaussian probability distribution is introduced to capture the local structure of the instance. Meanwhile, the attention mechanism is applied to fuse the title information and biomedical knowledge, respectively. Finally, the related representations are concatenated and fed into the softmax function to extract CPIs. We evaluate our proposed model on the CHEMPROT corpus. Our proposed model is superior in performance as compared with other state-of-the-art models. The experimental results show that the Gaussian probability distribution and external knowledge are complementary to each other. Integrating them can effectively improve the CPI extraction performance. Furthermore, the Gaussian probability distribution can effectively improve the extraction performance of sentences with overlapping relations in biomedical relation extraction tasks.

Availability and implementation

Data and code are available at https://github.com/CongSun-dlut/CPI_extraction.

Supplementary information

Supplementary data are available at Bioinformatics online.

Modular Characteristics and Mechanism of Action of Herbs for Endometriosis Treatment in Chinese Medicine: A Data Mining and Network Pharmacology-Based Identification

Endometriosis is a common benign disease in women of reproductive age. It has been defined as a disorder characterized by inflammation, compromised immunity, hormone dependence, and neuroangiogenesis. Unfortunately, the mechanisms of endometriosis have not yet been fully elucidated, and available treatment methods are currently limited. The discovery of new therapeutic drugs and improvements in existing treatment schemes remain the focus of research initiatives. Chinese medicine can improve the symptoms associated with endometriosis. Many Chinese herbal medicines could exert antiendometriosis effects via comprehensive interactions with multiple targets. However, these interactions have not been defined. This study used association rule mining and systems pharmacology to discover a method by which potential antiendometriosis herbs can be investigated. We analyzed various combinations and mechanisms of action of medicinal herbs to establish molecular networks showing interactions with multiple targets. The results showed that endometriosis treatment in Chinese medicine is mainly based on methods of supplementation with blood-activating herbs and strengthening qi. Furthermore, we used network pharmacology to analyze the main herbs that facilitate the decoding of multiscale mechanisms of the herbal compounds. We found that Chinese medicine could affect the development of endometriosis by regulating inflammation, immunity, angiogenesis, and other clusters of processes identified by Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway analyses. The antiendometriosis effect of Chinese medicine occurs mainly through nervous system–associated pathways, such as the serotonergic synapse, the neurotrophin signaling pathway, and dopaminergic synapse, among others, to reduce pain. Chinese medicine could also regulate VEGF signaling, toll-like reporter signaling, NF-κB signaling, MAPK signaling, PI3K-Akt signaling, and the HIF-1 signaling pathway, among others. Synergies often exist in herb pairs and herbal prescriptions. In conclusion, we identified some important targets, target pairs, and regulatory networks, using bioinformatics and data mining. The combination of data mining and network pharmacology may offer an efficient method for drug discovery and development from herbal medicines.