Automated recognition of functional compound-protein relationships in literature

Investigating Cross-Domain Binary Relation Classification in Biomedical Natural Language Processing

This paper addresses the challenge of binary relation classification in biomedical Natural Language Processing (NLP), focusing on diverse domains including gene-disease associations, compound protein interactions, and social determinants of health (SDOH). We evaluate different approaches, including fine-tuning Bidirectional Encoder Representations from Transformers (BERT) models and generative Large Language Models (LLMs), and examine their performance in zero and few-shot settings. We also introduce a novel dataset of biomedical text annotated with social and clinical entities to facilitate research into relation classification. Our results underscore the continued complexity of this task for both humans and models. BERT-based models trained on domain-specific data excelled in certain domains and achieved comparable performance and generalization power to generative LLMs in others. Despite these encouraging results, these models are still far from achieving human-level performance. We also highlight the significance of high-quality training data and domain-specific fine-tuning on the performance of all the considered models.

Hypothesis testing procedure for binary and multi-class F1 -scores in the paired design

In modern medicine, medical tests are used for various purposes including diagnosis, disease screening, prognosis, and risk prediction. To quantify the performance of the binary medical test, we often use sensitivity, specificity, and negative and positive predictive values as measures. Additionally, theF 1 $$ {F}_1 $$ -score, which is defined as the harmonic mean of precision (positive predictive value) and recall (sensitivity), has come to be used in the medical field due to its favorable characteristics. TheF 1 $$ {F}_1 $$ -score has been extended for multi-class classification, and two types ofF 1 $$ {F}_1 $$ -scores have been proposed for multi-class classification: a micro-averagedF 1 $$ {F}_1 $$ -score and a macro-averagedF 1 $$ {F}_1 $$ -score. The micro-averagedF 1 $$ {F}_1 $$ -score pools per-sample classifications across classes and then calculates the overallF 1 $$ {F}_1 $$ -score, whereas the macro-averagedF 1 $$ {F}_1 $$ -score computes an arithmetic mean of theF 1 $$ {F}_1 $$ -scores for each class. Additionally, Sokolova and Lapalme1 $$ {}^1 $$ gave an alternative definition of the macro-averagedF 1 $$ {F}_1 $$ -score as the harmonic mean of the arithmetic means of the precision and recall over classes. Although some statistical methods of inference for binary and multi-classF 1 $$ {F}_1 $$ -scores have been proposed, the methodology development of hypothesis testing procedure for them has not been fully progressing yet. Therefore, we aim to develop hypothesis testing procedure for comparing twoF 1 $$ {F}_1 $$ -scores in paired study design based on the large sample multivariate central limit theorem.

Discovering drug-target interaction knowledge from biomedical literature

MOTIVATION

The interaction between drugs and targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedical domain, automatically discovering DTI knowledge from biomedical literature, which are usually triplets about drugs, targets and their interaction, becomes an urgent demand in the industry. Existing methods of discovering biological knowledge are mainly extractive approaches that often require detailed annotations (e.g. all mentions of biological entities, relations between every two entity mentions, etc.). However, it is difficult and costly to obtain sufficient annotations due to the requirement of expert knowledge from biomedical domains.

RESULTS

To overcome these difficulties, we explore an end-to-end solution for this task by using generative approaches. We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations. Further, we propose a semi-supervised method, which leverages the aforementioned end-to-end model to filter unlabeled literature and label them. Experimental results show that our method significantly outperforms extractive baselines on DTI discovery. We also create a dataset, KD-DTI, to advance this task and release it to the community.

AVAILABILITY AND IMPLEMENTATION

Our code and data are available at https://github.com/bert-nmt/BERT-DTI.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

CPRiL: compound-protein relationships in literature

SUMMARY

Newly discovered functional relationships of (bio-)molecules are a key component in molecular biology and life science research. Especially in the drug discovery field, knowledge of how small molecules associated with proteins plays a fundamental role in understanding how drugs or metabolites can affect cells, tissues and human metabolism. Finding relevant information about these relationships among the huge number of published articles is becoming increasingly challenging and time-consuming. On average, more than 25 000 new (bio-)medical articles are added to the literature database PubMed weekly. In this article, we present a new web server [compound-protein relationships in literature (CPRiL)] that provides information on functional relationships between small molecules and proteins in literature. Currently, CPRiL contains ∼465 000 unique names and synonyms of small molecules, ∼100 000 unique proteins and more than 9 million described functional relationships between these entities. The applied BioBERT machine learning model for the determination of functional relationships between small molecules and proteins in texts was extensively trained and tested. On a related benchmark, CPRiL yielded a high performance, with an F1 score of 84.3%, precision of 82.9% and recall of 85.7%.

AVAILABILITY AND IMPLEMENTATION

CPRiL is freely available at https://www.pharmbioinf.uni-freiburg.de/cpril.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.