A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression

A comparison of embedding aggregation strategies in drug-target interaction prediction

The prediction of interactions between novel drugs and biological targets is a vital step in the early stage of the drug discovery pipeline. Many deep learning approaches have been proposed over the last decade, with a substantial fraction of them sharing the same underlying two-branch architecture. Their distinction is limited to the use of different types of feature representations and branches (multi-layer perceptrons, convolutional neural networks, graph neural networks and transformers). In contrast, the strategy used to combine the outputs (embeddings) of the branches has remained mostly the same. The same general architecture has also been used extensively in the area of recommender systems, where the choice of an aggregation strategy is still an open question. In this work, we investigate the effectiveness of three different embedding aggregation strategies in the area of drug-target interaction (DTI) prediction. We formally define these strategies and prove their universal approximator capabilities. We then present experiments that compare the different strategies on benchmark datasets from the area of DTI prediction, showcasing conditions under which specific strategies could be the obvious choice.

Hippocampal Resting-State Functional Connectivity Forecasts Individual Posttraumatic Stress Disorder Symptoms: A Data-Driven Approach

BACKGROUND

Posttraumatic stress disorder (PTSD) is a debilitating disorder, and there is no current accurate prediction of who develops it after trauma. Neurobiologically, individuals with chronic PTSD exhibit aberrant resting-state functional connectivity (rsFC) between the hippocampus and other brain regions (e.g., amygdala, prefrontal cortex, posterior cingulate), and these aberrations correlate with severity of illness. Previous small-scale research (n < 25) has also shown that hippocampal rsFC measured acutely after trauma is predictive of future severity using a region-of-interest-based approach. While this is a promising biomarker, to date, no study has used a data-driven approach to test whole-brain hippocampal FC patterns in forecasting the development of PTSD symptoms.

METHODS

A total of 98 adults at risk of PTSD were recruited from the emergency department after traumatic injury and completed resting-state functional magnetic resonance imaging (8 min) within 1 month; 6 months later, they completed the Clinician-Administered PTSD Scale for DSM-5 for assessment of PTSD symptom severity. Whole-brain rsFC values with bilateral hippocampi were extracted (using CONN) and used in a machine learning kernel ridge regression analysis (PRoNTo); a k-folds (k = 10) and 70/30 testing versus training split approach were used for cross-validation (1000 iterations to bootstrap confidence intervals for significance values).

RESULTS

Acute hippocampal rsFC significantly predicted Clinician-Administered PTSD Scale for DSM-5 scores at 6 months (r = 0.30, p = .006; mean squared error = 120.58, p = .006; R² = 0.09, p = .025). In post hoc analyses, hippocampal rsFC remained significant after controlling for demographics, PTSD symptoms at baseline, and depression, anxiety, and stress severity at 6 months (B = 0.59, SE = 0.20, p = .003).

CONCLUSIONS

Findings suggest that functional connectivity of the hippocampus across the brain acutely after traumatic injury is associated with prospective PTSD symptom severity.

Generalized vec trick for fast learning of pairwise kernel models

Pairwise learning corresponds to the supervised learning setting where the goal is to make predictions for pairs of objects. Prominent applications include predicting drug-target or protein-protein interactions, or customer-product preferences. In this work, we present a comprehensive review of pairwise kernels, that have been proposed for incorporating prior knowledge about the relationship between the objects. Specifically, we consider the standard, symmetric and anti-symmetric Kronecker product kernels, metric-learning, Cartesian, ranking, as well as linear, polynomial and Gaussian kernels. Recently, a $$O(nm+nq)$$ O ( n m + n q ) time generalized vec trick algorithm, where $$n$$ n , $$m$$ m , and $$q$$ q denote the number of pairs, drugs and targets, was introduced for training kernel methods with the Kronecker product kernel. This was a significant improvement over previous $$O(n^2)$$ O ( n 2 ) training methods, since in most real-world applications $$m,q<< n$$ m , q < < n . In this work we show how all the reviewed kernels can be expressed as sums of Kronecker products, allowing the use of generalized vec trick for speeding up their computation. In the experiments, we demonstrate how the introduced approach allows scaling pairwise kernels to much larger data sets than previously feasible, and provide an extensive comparison of the kernels on a number of biological interaction prediction tasks.

Cold-Start Problems in Data-Driven Prediction of Drug-Drug Interaction Effects

Combining drugs, a phenomenon often referred to as polypharmacy, can induce additional adverse effects. The identification of adverse combinations is a key task in pharmacovigilance. In this context, in silico approaches based on machine learning are promising as they can learn from a limited number of combinations to predict for all. In this work, we identify various subtasks in predicting effects caused by drug–drug interaction. Predicting drug–drug interaction effects for drugs that already exist is very different from predicting outcomes for newly developed drugs, commonly called a cold-start problem. We propose suitable validation schemes for the different subtasks that emerge. These validation schemes are critical to correctly assess the performance. We develop a new model that obtains AUC-ROC =0.843 for the hardest cold-start task up to AUC-ROC =0.957 for the easiest one on the benchmark dataset of Zitnik et al. Finally, we illustrate how our predictions can be used to improve post-market surveillance systems or detect drug–drug interaction effects earlier during drug development.

Guiding Mineralization Co-Culture Discovery Using Bayesian Optimization

Many disciplines rely on testing combinations of compounds, materials, proteins, or bacterial species to drive scientific discovery. It is time-consuming and expensive to determine experimentally, via trial-and-error or random selection approaches, which of the many possible combinations will lead to desirable outcomes. Hence, there is a pressing need for more rational and efficient experimental design approaches to reduce experimental effort. In this work, we demonstrate the potential of machine learning methods for the in silico selection of promising co-culture combinations in the application of bioaugmentation. We use the example of pollutant removal in drinking water treatment plants, which can be achieved using co-cultures of a specialized pollutant degrader with combinations of bacterial isolates. To reduce the experimental effort needed to discover high-performing combinations, we propose a data-driven experimental design. Based on a dataset of mineralization performance for all pairs of 13 bacterial species co-cultured with MSH1, we built a Gaussian process regression model to predict the Gompertz mineralization parameters of the co-cultures of two and three species, based on the single-strain parameters. We subsequently used this model in a Bayesian optimization scheme to suggest potentially high-performing combinations of bacteria. We achieved good performance with this approach, both for predicting mineralization parameters and for selecting effective co-cultures, despite the limited dataset. As a novel application of Bayesian optimization in bioremediation, this experimental design approach has promising applications for highlighting co-culture combinations for in vitro testing in various settings, to lessen the experimental burden and perform more targeted screenings.

Improving Generalization via Attribute Selection on Out-of-the-Box Data

Zero-shot learning (ZSL) aims to recognize unseen objects (test classes) given some other seen objects (training classes) by sharing information of attributes between different objects. Attributes are artificially annotated for objects and treated equally in recent ZSL tasks. However, some inferior attributes with poor predictability or poor discriminability may have negative impacts on the ZSL system performance. This letter first derives a generalization error bound for ZSL tasks. Our theoretical analysis verifies that selecting the subset of key attributes can improve the generalization performance of the original ZSL model, which uses all the attributes. Unfortunately, previous attribute selection methods have been conducted based on the seen data, and their selected attributes have poor generalization capability to the unseen data, which is unavailable in the training stage of ZSL tasks. Inspired by learning from pseudo-relevance feedback, this letter introduces out-of-the-box data-pseudo-data generated by an attribute-guided generative model-to mimic the unseen data. We then present an iterative attribute selection (IAS) strategy that iteratively selects key attributes based on the out-of-the-box data. Since the distribution of the generated out-of-the-box data is similar to that of the test data, the key attributes selected by IAS can be effectively generalized to test data. Extensive experiments demonstrate that IAS can significantly improve existing attribute-based ZSL methods and achieve state-of-the-art performance.

Self-Adaptive Deep Multiple Kernel Learning Based on Rademacher Complexity

The deep multiple kernel learning (DMKL) method has caused widespread concern due to its better results compared with shallow multiple kernel learning. However, existing DMKL methods, which have a fixed number of layers and fixed type of kernels, have poor ability to adapt to different data sets and are difficult to find suitable model parameters to improve the test accuracy. In this paper, we propose a self-adaptive deep multiple kernel learning (SA-DMKL) method. Our SA-DMKL method can adapt the model through optimizing the model parameters of each kernel function with a grid search method and change the numbers and types of kernel function in each layer according to the generalization bound that is evaluated with Rademacher chaos complexity. Experiments on the three datasets of University of California—Irvine (UCI) and image dataset Caltech 256 validate the effectiveness of the proposed method on three aspects.

Algebraic shortcuts for leave-one-out cross-validation in supervised network inference

Supervised machine learning techniques have traditionally been very successful at reconstructing biological networks, such as protein-ligand interaction, protein-protein interaction and gene regulatory networks. Many supervised techniques for network prediction use linear models on a possibly nonlinear pairwise feature representation of edges. Recently, much emphasis has been placed on the correct evaluation of such supervised models. It is vital to distinguish between using a model to either predict new interactions in a given network or to predict interactions for a new vertex not present in the original network. This distinction matters because (i) the performance might dramatically differ between the prediction settings and (ii) tuning the model hyperparameters to obtain the best possible model depends on the setting of interest. Specific cross-validation schemes need to be used to assess the performance in such different prediction settings.In this work we discuss a state-of-the-art kernel-based network inference technique called two-step kernel ridge regression. We show that this regression model can be trained efficiently, with a time complexity scaling with the number of vertices rather than the number of edges. Furthermore, this framework leads to a series of cross-validation shortcuts that allow one to rapidly estimate the model performance for any relevant network prediction setting. This allows computational biologists to fully assess the capabilities of their models. The machine learning techniques with the algebraic shortcuts are implemented in the RLScore software package: https://github.com/aatapa/RLScore.