Fast bundle algorithm for multiple-instance learning

Discovering NDM-1 inhibitors using molecular substructure embeddings representations

NDM-1 (New-Delhi-Metallo-β-lactamase-1) is an enzyme developed by bacteria that is implicated in bacteria resistance to almost all known antibiotics. In this study, we deliver a new, curated NDM-1 bioactivities database, along with a set of unifying rules for managing different activity properties and inconsistencies. We define the activity classification problem in terms of Multiple Instance Learning, employing embeddings corresponding to molecular substructures and present an ensemble ranking and classification framework, relaying on a k-fold Cross Validation method employing a per fold hyper-parameter optimization procedure, showing promising generalization ability. The MIL paradigm displayed an improvement up to 45.7 %, in terms of Balanced Accuracy, in comparison to the classical Machine Learning paradigm. Moreover, we investigate different compact molecular representations, based on atomic or bi-atomic substructures. Finally, we scanned the Drugbank for strongly active compounds and we present the top-15 ranked compounds.

Multiple instance neural networks based on sparse attention for cancer detection using T-cell receptor sequences

Early detection of cancers has been much explored due to its paramount importance in biomedical fields. Among different types of data used to answer this biological question, studies based on T cell receptors (TCRs) are under recent spotlight due to the growing appreciation of the roles of the host immunity system in tumor biology. However, the one-to-many correspondence between a patient and multiple TCR sequences hinders researchers from simply adopting classical statistical/machine learning methods. There were recent attempts to model this type of data in the context of multiple instance learning (MIL). Despite the novel application of MIL to cancer detection using TCR sequences and the demonstrated adequate performance in several tumor types, there is still room for improvement, especially for certain cancer types. Furthermore, explainable neural network models are not fully investigated for this application. In this article, we propose multiple instance neural networks based on sparse attention (MINN-SA) to enhance the performance in cancer detection and explainability. The sparse attention structure drops out uninformative instances in each bag, achieving both interpretability and better predictive performance in combination with the skip connection. Our experiments show that MINN-SA yields the highest area under the ROC curve scores on average measured across 10 different types of cancers, compared to existing MIL approaches. Moreover, we observe from the estimated attentions that MINN-SA can identify the TCRs that are specific for tumor antigens in the same T cell repertoire.

IDA-MIL: Classification of Glomerular with Spike-like Projections via Multiple Instance Learning with Instance-level Data Augmentation

BACKGROUND AND OBJECTIVE

Tiny spike-like projections on the basement membrane of glomeruli are the main pathological feature of membranous nephropathy at stage II (MN II), which is the most significant stage for the diagnosis and treatment of renal disease. Pathological technology is the gold standard in the diagnosis of spike-like and other MNs, and automatic classification of spike-like projection is a crucial step in assisting pathologists in their diagnosis. However, owing to hard-to-label spile-like projections and the scarcity of patient data, classification of glomeruli with spike-like projections based on supervised learning methods is a challenging task.

METHOD

To overcome the aforementioned problems, the idea of integrating weakly-supervised learning and data augmentation methods is utilized in designing the classification framework. Specifically, a multiple instance learning with instance-level data augmentation (IDA-MIL) method for the classification of glomeruli with spike-like projections is established in this paper. The proposed classification framework first trains the MIL model on a dataset with image-level labels, and the well-trained MIL model is used to extract instances that include spike-like projections in the whole glomerular image. Then, rather than using an image-level generative adversarial network (ImgGAN), an instance-level generative adversarial network (InsGAN) based on the StyleGAN2-ADA model is trained on the spike-like instances obtained by the MIL model and synthesizes new spike-like projection instances. Finally, the synthesized spike-like instances are extended to the training dataset to further improve the classification performance of MIL.

RESULTS

The designed IDA-MIL model is verified and evaluated from two aspects based on the in-house dataset. On the one hand, the performance comparisons between InsGAN and ImgGAN on five metrics, which involve FID, KID, Precision, Recall and IS, show that InsGAN obtains a better score and can synthesize effective spike-like projections. However, the proposed IDA-MIL model achieves the best classification performance with an accuracy of 0.9405. Then, to make nephrologists believe the inference result of the proposed model, we use heatmap technology to visualize the basis of the model inferences and show the top 4 probability spike-like instances per glomerulus. Furthermore, we analyze the correlation between the disease and the proportion of spike-like instances in bags from historical cases.

CONCLUSION

Compared with the ImgGAN, the InsGAN can synthesize natural and varied spike-like projections, which results in the classification performance of the MIL model achieving great improvement by adding synthesized instance samples into the training dataset. The heatmap of spike-like inferences and the proportion of spike-like instances can help nephrologists to make a preliminary reliable diagnosis in clinical practice. This work provides a valuable reference for medical image classification with limited data and small-scale lesions based on deep learning.

A heuristic approach for multiple instance learning by linear separation

We present a fast heuristic approach for solving a binary multiple instance learning (MIL) problem, which consists in discriminating between two kinds of item sets: the sets are called bags and the items inside them are called instances. Assuming that only two classes of instances are allowed, a common standard hypothesis states that a bag is positive if it contains at least a positive instance and it is negative when all its instances are negative. Our approach constructs a MIL separating hyperplane by preliminary fixing the normal and reducing the learning phase to a univariate nonsmooth optimization problem, which can be quickly solved by simply exploring the kink points. Numerical results are presented on a set of test problems drawn from the literature.

A Semiproximal Support Vector Machine Approach for Binary Multiple Instance Learning

We face a binary multiple instance learning (MIL) problem, whose objective is to discriminate between two kinds of point sets: positive and negative. In the MIL terminology, such sets are called bags, and the points inside each bag are called instances. Considering the case with two classes of instances (positive and negative) and inspired by a well-established instance-space support vector machine (SVM) model, we propose to extend to MIL classification the proximal SVM (PSVM) technique that has revealed very effective for supervised learning, especially in terms of computational time. In particular, our approach is based on a new instance-space model that exploits the benefits coming from both SVM (better accuracy) and PSVM (computational efficiency) paradigms. Starting from the standard MIL assumption, such a model is aimed at generating a hyperplane placed in the middle between two parallel hyperplanes: the first one is a proximal hyperplane that clusters the instances of the positive bags, while the second one constitutes a supporting hyperplane for the instances of the negative bags. Numerical results are presented on a set of MIL test data sets drawn from the literature.

A Lagrangian Relaxation Approach for Binary Multiple Instance Classification

In the standard classification problems, the objective is to categorize points into different classes. Multiple instance learning (MIL), instead, is aimed at classifying bags of points, each point being an instance. The main peculiarity of a MIL problem is that, in the learning phase, only the label of each bag is known whereas the labels of the instances are unknown. We discuss an instance-level learning approach for a binary MIL classification problem characterized by two classes of instances, positive and negative, respectively. In such a problem, a negative bag is constituted only by negative instances, while a bag is positive if it contains at least one positive instance. We start from a mixed integer nonlinear optimization model drawn from the literature and the main result we obtain is to prove that a Lagrangian relaxation approach, equipped with a dual ascent scheme, allows us to obtain an optimal solution of the original problem. The relaxed problem is tackled by means of a block coordinate descent (BCD) algorithm. We provide, finally, the results of our implementation on some benchmark data sets.

Multi-View Multi-Instance Learning Based on Joint Sparse Representation and Multi-View Dictionary Learning

In multi-instance learning (MIL), the relations among instances in a bag convey important contextual information in many applications. Previous studies on MIL either ignore such relations or simply model them with a fixed graph structure so that the overall performance inevitably degrades in complex environments. To address this problem, this paper proposes a novel multi-view multi-instance learning algorithm (MIL) that combines multiple context structures in a bag into a unified framework. The novel aspects are: (i) we propose a sparse -graph model that can generate different graphs with different parameters to represent various context relations in a bag, (ii) we propose a multi-view joint sparse representation that integrates these graphs into a unified framework for bag classification, and (iii) we propose a multi-view dictionary learning algorithm to obtain a multi-view graph dictionary that considers cues from all views simultaneously to improve the discrimination of the MIL. Experiments and analyses in many practical applications prove the effectiveness of the M IL.

A Sphere-Description-Based Approach for Multiple-Instance Learning

Multiple-instance learning (MIL) is a generalization of supervised learning which addresses the classification of bags. Similar to traditional supervised learning, most of the existing MIL work is proposed based on the assumption that a representative training set is available for a proper learning of the classifier. That is to say, the training data can appropriately describe the distribution of positive and negative data in the testing set. However, this assumption may not be always satisfied. In real-world MIL applications, the negative data in the training set may not sufficiently represent the distribution of negative data in the testing set. Hence, how to learn an appropriate MIL classifier when a representative training set is not available becomes a key challenge for real-world MIL applications. To deal with this problem, we propose a novel Sphere-Description-Based approach for Multiple-Instance Learning (SDB-MIL). SDB-MIL learns an optimal sphere by determining a large margin among the instances, and meanwhile ensuring that each positive bag has at least one instance inside the sphere and all negative bags are outside the sphere. Enclosing at least one instance from each positive bag in the sphere enables a more desirable MIL classifier when the negative data in the training set cannot sufficiently represent the distribution of negative data in the testing set. Substantial experiments on the benchmark and real-world MIL datasets show that SDB-MIL obtains statistically better classification performance than the MIL methods compared.

Person Re-Identification by Discriminative Selection in Video Ranking

Current person re-identification (ReID) methods typically rely on single-frame imagery features, whilst ignoring space-time information from image sequences often available in the practical surveillance scenarios. Single-frame (single-shot) based visual appearance matching is inherently limited for person ReID in public spaces due to the challenging visual ambiguity and uncertainty arising from non-overlapping camera views where viewing condition changes can cause significant people appearance variations. In this work, we present a novel model to automatically select the most discriminative video fragments from noisy/incomplete image sequences of people from which reliable space-time and appearance features can be computed, whilst simultaneously learning a video ranking function for person ReID. Using the PRID 2011, iLIDS-VID, and HDA+ image sequence datasets, we extensively conducted comparative evaluations to demonstrate the advantages of the proposed model over contemporary gait recognition, holistic image sequence matching and state-of-the-art single-/multi-shot ReID methods.

General, Nested, and Constrained Wiberg Minimization

Wiberg matrix factorization breaks a matrix Y into low-rank factors U and V by solving for V in closed form given U, linearizing V(U) about U, and iteratively minimizing ||Y - UV(U)||2 with respect to U only. This approach factors the matrix while effectively removing V from the minimization. Recently Eriksson and van den Hengel extended this approach to L1 , minimizing ||Y - UV(U)||1 . We generalize their approach beyond factorization to minimize ||Y - f(U, V)||1 for more general functions f(U, V) that are nonlinear in each of two sets of variables. We demonstrate the idea with a practical Wiberg algorithm for L1 bundle adjustment. One Wiberg minimization can be nested inside another, effectively removing two of three sets of variables from a minimization. We demonstrate this idea with a nested Wiberg algorithm for L1 projective bundle adjustment, solving for camera matrices, points, and projective depths. Wiberg minimization also generalizes to handle nonlinear constraints, and we demonstrate this idea with Constrained Wiberg Minimization for Multiple Instance Learning (CWM-MIL), which removes one set of variables from the constrained optimization. Our experiments emphasize isolating the effect of Wiberg by comparing against the algorithm it modifies, successive linear programming.

The Proximal Trajectory Algorithm in SVM Cross Validation

We propose a bilevel cross-validation scheme for support vector machine (SVM) model selection based on the construction of the entire regularization path. Since such path is a particular case of the more general proximal trajectory concept from nonsmooth optimization, we propose for its construction an algorithm based on solving a finite number of structured linear programs. Our methodology, differently from other approaches, works directly on the primal form of SVM. Numerical results are presented on binary data sets drawn from literature.

A Nonparametric Bayesian Approach to Multiple Instance Learning

Multiple instance learning (MIL) is a type of supervised learning in which labels are available for sets of observations (bags), but not for individual observations (instances). MIL has been applied in different areas, which has led to a large number of algorithms for learning based on MIL data. Many of these approaches focus on maximizing class margins, performing instance selection, or developing distance metrics and kernels suitable for application directly to bags. Although these approaches have shown promise, most require cross-validation-based optimization of hyper parameters or iterative numerical optimization to determine the proper number of target concepts. This work proposes a nonparametric Bayesian approach to learning in MIL scenarios based on Dirichlet process mixture models. The nonparametric nature of the model and the use of noninformative priors remove the need to perform cross-validation-based optimization while variational Bayesian inference allows for rapid parameter learning. The resulting approach generalizes to different applications by easily incorporating alternate data generation models. In a related effort [A. Manandhar et al., IEEE Trans. Geosci. Remote Sensing53(4) (2015) 1737–1745.], the proposed model has been extended to incorporate time-varying data. Results indicate that when the data generation assumption holds, the proposed approach performs competitively with existing MIL and nonMIL methods for several standard MIL datasets and a new MIL dataset introduced in this work.

A similarity-based classification framework for multiple-instance learning

Multiple-instance learning (MIL) is a generalization of supervised learning that attempts to learn useful information from bags of instances. In MIL, the true labels of instances in positive bags are not available for training. This leads to a critical challenge, namely, handling the instances of which the labels are ambiguous (ambiguous instances). To deal with these ambiguous instances, we propose a novel MIL approach, called similarity-based multiple-instance learning (SMILE). Instead of eliminating a number of ambiguous instances in positive bags from training the classifier, as done in some previous MIL works, SMILE explicitly deals with the ambiguous instances by considering their similarity to the positive class and the negative class. Specifically, a subset of instances is selected from positive bags as the positive candidates and the remaining ambiguous instances are associated with two similarity weights, representing the similarity to the positive class and the negative class, respectively. The ambiguous instances, together with their similarity weights, are thereafter incorporated into the learning phase to build an extended SVM-based predictive classifier. A heuristic framework is employed to update the positive candidates and the similarity weights for refining the classification boundary. Experiments on real-world datasets show that SMILE demonstrates highly competitive classification accuracy and shows less sensitivity to labeling noise than the existing MIL methods.

DR-predictor: incorporating flexible docking with specialized electronic reactivity and machine learning techniques to predict CYP-mediated sites of metabolism

Computational methods that can identify CYP-mediated sites of metabolism (SOMs) of drug-like compounds have become required tools for early stage lead optimization. In recent years, methods that combine CYP binding site features with CYP/ligand binding information have been sought in order to increase the prediction accuracy of such hybrid models over those that use only one representation. Two challenges that any hybrid ligand/structure-based method must overcome are (1) identification of the best binding pose for a specific ligand with a given CYP and (2) appropriately incorporating the results of docking with ligand reactivity. To address these challenges we have created Docking-Regioselectivity-Predictor (DR-Predictor)--a method that incorporates flexible docking-derived information with specialized electronic reactivity and multiple-instance-learning methods to predict CYP-mediated SOMs. In this study, the hybrid ligand-structure-based DR-Predictor method was tested on substrate sets for CYP 1A2 and CYP 2A6. For these data, the DR-Predictor model was found to identify the experimentally observed SOM within the top two predicted rank-positions for 86% of the 261 1A2 substrates and 83% of the 100 2A6 substrates. Given the accuracy and extendibility of the DR-Predictor method, we anticipate that it will further facilitate the prediction of CYP metabolism liabilities and aid in in-silico ADMET assessment of novel structures.

The contribution of atom accessibility to site of metabolism models for cytochromes P450

Three different types of atom accessibility descriptors are investigated in relation to site of metabolism predictions. To enable the integration of local accessibility we have constructed 2DSASA, a method for the calculation of the atomic solvent accessible surface area that is independent of 3D coordinates. The method was implemented in the SMARTCyp site of metabolism prediction models and improved the results by up to 4 percentage points for nine cytochrome P450 isoforms. The final models are made available at http://www.farma.ku.dk/smartcyp.

RS-WebPredictor: a server for predicting CYP-mediated sites of metabolism on drug-like molecules

SUMMARY

Regioselectivity-WebPredictor (RS-WebPredictor) is a server that predicts isozyme-specific cytochrome P450 (CYP)-mediated sites of metabolism (SOMs) on drug-like molecules. Predictions may be made for the promiscuous 2C9, 2D6 and 3A4 CYP isozymes, as well as CYPs 1A2, 2A6, 2B6, 2C8, 2C19 and 2E1. RS-WebPredictor is the first freely accessible server that predicts the regioselectivity of the last six isozymes. Server execution time is fast, taking on average 2s to encode a submitted molecule and 1s to apply a given model, allowing for high-throughput use in lead optimization projects.

AVAILABILITY

RS-WebPredictor is accessible for free use at http://reccr.chem.rpi.edu/Software/RS-WebPredictor/

RS-Predictor models augmented with SMARTCyp reactivities: robust metabolic regioselectivity predictions for nine CYP isozymes

RS-Predictor is a tool for creating pathway-independent, isozyme-specific, site of metabolism (SOM) prediction models using any set of known cytochrome P450 (CYP) substrates and metabolites. Until now, the RS-Predictor method was only trained and validated on CYP 3A4 data, but in the present study, we report on the versatility the RS-Predictor modeling paradigm by creating and testing regioselectivity models for substrates of the nine most important CYP isozymes. Through curation of source literature, we have assembled 680 substrates distributed among CYPs 1A2, 2A6, 2B6, 2C19, 2C8, 2C9, 2D6, 2E1, and 3A4, the largest publicly accessible collection of P450 ligands and metabolites released to date. A comprehensive investigation into the importance of different descriptor classes for identifying the regioselectivity mediated by each isozyme is made through the generation of multiple independent RS-Predictor models for each set of isozyme substrates. Two of these models include a density functional theory (DFT) reactivity descriptor derived from SMARTCyp. Optimal combinations of RS-Predictor and SMARTCyp are shown to have stronger performance than either method alone, while also exceeding the accuracy of the commercial regioselectivity prediction methods distributed by Optibrium and Schrödinger, correctly identifying a large proportion of the metabolites in each substrate set within the top two rank-positions: 1A2 (83.0%), 2A6 (85.7%), 2B6 (82.1%), 2C19 (86.2%), 2C8 (83.8%), 2C9 (84.5%), 2D6 (85.9%), 2E1 (82.8%), 3A4 (82.3%), and merged (86.0%). Comprehensive datamining of each substrate set and careful statistical analyses of the predictions made by the different models revealed new insights into molecular features that control metabolic regioselectivity and enable accurate prospective prediction of likely SOMs.

RS-predictor: a new tool for predicting sites of cytochrome P450-mediated metabolism applied to CYP 3A4

This article describes RegioSelectivity-Predictor (RS-Predictor), a new in silico method for generating predictive models of P450-mediated metabolism for drug-like compounds. Within this method, potential sites of metabolism (SOMs) are represented as "metabolophores": A concept that describes the hierarchical combination of topological and quantum chemical descriptors needed to represent the reactivity of potential metabolic reaction sites. RS-Predictor modeling involves the use of metabolophore descriptors together with multiple-instance ranking (MIRank) to generate an optimized descriptor weight vector that encodes regioselectivity trends across all cases in a training set. The resulting pathway-independent (O-dealkylation vs N-oxidation vs Csp(3) hydroxylation, etc.), isozyme-specific regioselectivity model may be used to predict potential metabolic liabilities. In the present work, cross-validated RS-Predictor models were generated for a set of 394 substrates of CYP 3A4 as a proof-of-principle for the method. Rank aggregation was then employed to merge independently generated predictions for each substrate into a single consensus prediction. The resulting consensus RS-Predictor models were shown to reliably identify at least one observed site of metabolism in the top two rank-positions on 78% of the substrates. Comparisons between RS-Predictor and previously described regioselectivity prediction methods reveal new insights into how in silico metabolite prediction methods should be compared.