Robust optimization of SVM hyperparameters in the classification of bioactive compounds

Role of Optimization in RNA-Protein-Binding Prediction

RNA-binding proteins (RBPs) play an important role in regulating biological processes, such as gene regulation. Understanding their behaviors, for example, their binding site, can be helpful in understanding RBP-related diseases. Studies have focused on predicting RNA binding by means of machine learning algorithms including deep convolutional neural network models. One of the integral parts of modeling deep learning is achieving optimal hyperparameter tuning and minimizing a loss function using optimization algorithms. In this paper, we investigate the role of optimization in the RBP classification problem using the CLIP-Seq 21 dataset. Three optimization methods are employed on the RNA-protein binding CNN prediction model; namely, grid search, random search, and Bayesian optimizer. The empirical results show an AUC of 94.42%, 93.78%, 93.23% and 92.68% on the ELAVL1C, ELAVL1B, ELAVL1A, and HNRNPC datasets, respectively, and a mean AUC of 85.30 on 24 datasets. This paper's findings provide evidence on the role of optimizers in improving the performance of RNA-protein binding prediction.

Keras/TensorFlow in Drug Design for Immunity Disorders

Homeostasis of the host immune system is regulated by white blood cells with a variety of cell surface receptors for cytokines. Chemotactic cytokines (chemokines) activate their receptors to evoke the chemotaxis of immune cells in homeostatic migrations or inflammatory conditions towards inflamed tissue or pathogens. Dysregulation of the immune system leading to disorders such as allergies, autoimmune diseases, or cancer requires efficient, fast-acting drugs to minimize the long-term effects of chronic inflammation. Here, we performed structure-based virtual screening (SBVS) assisted by the Keras/TensorFlow neural network (NN) to find novel compound scaffolds acting on three chemokine receptors: CCR2, CCR3, and one CXC receptor, CXCR3. Keras/TensorFlow NN was used here not as a typically used binary classifier but as an efficient multi-class classifier that can discard not only inactive compounds but also low- or medium-activity compounds. Several compounds proposed by SBVS and NN were tested in 100 ns all-atom molecular dynamics simulations to confirm their binding affinity. To improve the basic binding affinity of the compounds, new chemical modifications were proposed. The modified compounds were compared with known antagonists of these three chemokine receptors. Known CXCR3 compounds were among the top predicted compounds; thus, the benefits of using Keras/TensorFlow in drug discovery have been shown in addition to structure-based approaches. Furthermore, we showed that Keras/TensorFlow NN can accurately predict the receptor subtype selectivity of compounds, for which SBVS often fails. We cross-tested chemokine receptor datasets retrieved from ChEMBL and curated datasets for cannabinoid receptors. The NN model trained on the cannabinoid receptor datasets retrieved from ChEMBL was the most accurate in the receptor subtype selectivity prediction. Among NN models trained on the chemokine receptor datasets, the CXCR3 model showed the highest accuracy in differentiating the receptor subtype for a given compound dataset.

TIDAL: Topology-Inferred Drug Addiction Learning

Drug addiction is a global public health crisis, and the design of antiaddiction drugs remains a major challenge due to intricate mechanisms. Since experimental drug screening and optimization are too time-consuming and expensive, there is urgent need to develop innovative artificial intelligence (AI) methods for addressing the challenge. We tackle this challenge by topology-inferred drug addiction learning (TIDAL) built from integrating multiscale topological Laplacians, deep bidirectional transformer, and ensemble-assisted neural networks (EANNs). Multiscale topological Laplacians are a novel class of algebraic topology tools that embed molecular topological invariants and algebraic invariants into its harmonic spectra and nonharmonic spectra, respectively. These invariants complement sequence information extracted from a bidirectional transformer. We validate the proposed TIDAL framework on 22 drug addiction related, 4 hERG, and 12 DAT data sets, which suggests that the proposed TIDAL is a state-of-the-art framework for the modeling and analysis of drug addiction data. We carry out cross-target analysis of the current drug addiction candidates to alert their side effects and identify their repurposing potentials. Our analysis reveals drug-mediated linear and bilinear target correlations. Finally, TIDAL is applied to shed light on relative efficacy, repurposing potential, and potential side effects of 12 existing antiaddiction medications. Our results suggest that TIDAL provides a new computational strategy for pressingly needed antisubstance addiction drug development.

Stepwise decomposition-integration-prediction framework for runoff forecasting considering boundary correction

Predicting river runoff accurately is of substantial significance for flood control, water resource allocation, and basin ecological dispatching. To explore the reasonable and effective application of time series decomposition in runoff forecasting, this study proposed a novel stepwise decomposition-integration-prediction considering boundary correction (SDIPBC) framework by using the stepwise decomposition sampling method and multi-input neural network. On this basis, we implemented a hybrid forecasting model combining seasonal-trend decomposition procedures based on loess (STL) with the long short-term memory (LSTM) network called STL-LSTM (SDIPBC) to estimate mid-long term river runoff. The reliability of the method was assessed using the historical runoff series of the Lianghekou and Jinping I Reservoirs in the Yalong River Basin, China, and developed several single models and hybrid models for comparative experiments. The results show that the existing decomposition-based hybrid forecasting frameworks are not suitable for practical runoff forecasting. The proposed SDIPBC framework can avoid using future information and improve the prediction accuracy of the single prediction model. For the Nash-Sutcliffe efficiency coefficient (NSE), the ten-day runoff forecasting accuracy of STL-LSTM (SDIPBC) in Lianghekou reservoir and Jinping I Reservoirs reached 0.845 and 0.862 respectively, which improved 1.81 % and 2.38 % than the single LSTM model, indicating that this is a practical and reliable decomposition-based hybrid runoff forecasting method.

Learning-Based Clutter Mitigation with Subspace Projection and Sparse Representation in Holographic Subsurface Radar Imaging

The holographic subsurface radar (HSR) is an effective remote sensing modality for surveying shallowly buried objects with high resolution images in plan-view. However, strong reflections from the rough surface and inhomogeneities obscure the detection of stationary targets response. In this paper, a learning-based method is proposed to mitigate the clutter in HSR applications. The proposed method first decomposes the HSR image into raw clutter and target data using an adaptive subspace projection approach. Then, the autoencoder is applied to carry out unsupervised learning to extract the target features and mitigate the clutter. The sparse representation is also combined to further optimize the model and the alternating direction multiplier method (ADMM) is used to solve the optimization problem for precision and efficiency. Experiments using real data were conducted to demonstrate that the proposed method can effectively mitigate the strong clutter with the target preserved. The visual and quantitative results show that the proposed method achieves superior performance on suppressing clutter in HSR images compared with the widely used state-of-the-art clutter mitigation approaches.

Colon cancer prediction on histological images using deep learning features and Bayesian optimized SVM

Colon cancer is one of the highest cancer diagnosis mortality rates worldwide. However, relying on the expertise of pathologists is a demanding and time-consuming process for histopathological analysis. The automated diagnosis of colon cancer from biopsy examination played an important role for patients and prognosis. As conventional handcrafted feature extraction requires specialized experience to select realistic features, deep learning processes have been chosen as abstract high-level features may be extracted automatically. This paper presents the colon cancer detection system using transfer learning architectures to automatically extract high-level features from colon biopsy images for automated diagnosis of patients and prognosis. In this study, the image features are extracted from a pre-trained convolutional neural network (CNN) and used to train the Bayesian optimized Support Vector Machine classifier. Moreover, Alexnet, VGG-16, and Inception-V3 pre-trained neural networks were used to analyze the best network for colon cancer detection. Furthermore, the proposed framework is evaluated using four datasets: two are collected from Indian hospitals (with different magnifications 4X, 10X, 20X, and 40X) and the other two are public colon image datasets. Compared with the existing classifiers and methods using public datasets, the test results evaluated the Inception-V3 network with the accuracy range from 96.5% - 99% as best suited for the proposed framework.

Pharmacoprint: A Combination of a Pharmacophore Fingerprint and Artificial Intelligence as a Tool for Computer-Aided Drug Design

Structural fingerprints and pharmacophore modeling are methodologies that have been used for at least 2 decades in various fields of cheminformatics, from similarity searching to machine learning (ML). Advances in in silico techniques consequently led to combining both these methodologies into a new approach known as the pharmacophore fingerprint. Herein, we propose a high-resolution, pharmacophore fingerprint called Pharmacoprint that encodes the presence, types, and relationships between pharmacophore features of a molecule. Pharmacoprint was evaluated in classification experiments by using ML algorithms (logistic regression, support vector machines, linear support vector machines, and neural networks) and outperformed other popular molecular fingerprints (i.e., ECFP4, Estate, MACCS, PubChem, Substructure, Klekota-Roth, CDK, Extended, and GraphOnly) and the ChemAxon pharmacophoric features fingerprint. Pharmacoprint consisted of 39 973 bits; several methods were applied for dimensionality reduction, and the best algorithm not only reduced the length of the bit string but also improved the efficiency of the ML tests. Further optimization allowed us to define the best parameter settings for using Pharmacoprint in discrimination tests and for maximizing statistical parameters. Finally, Pharmacoprint generated for three-dimensional (3D) structures with defined hydrogens as input data was applied to neural networks with a supervised autoencoder for selecting the most important bits and allowed us to maximize the Matthews correlation coefficient up to 0.962. The results show the potential of Pharmacoprint as a new, perspective tool for computer-aided drug design.

Support Vector Machine as a Supervised Learning for the Prioritization of Novel Potential SARS-CoV-2 Main Protease Inhibitors

In the last year, the COVID-19 pandemic has highly affected the lifestyle of the world population, encouraging the scientific community towards a great effort on studying the infection molecular mechanisms. Several vaccine formulations are nowadays available and helping to reach immunity. Nevertheless, there is a growing interest towards the development of novel anti-covid drugs. In this scenario, the main protease (Mpro) represents an appealing target, being the enzyme responsible for the cleavage of polypeptides during the viral genome transcription. With the aim of sharing new insights for the design of novel Mpro inhibitors, our research group developed a machine learning approach using the support vector machine (SVM) classification. Starting from a dataset of two million commercially available compounds, the model was able to classify two hundred novel chemo-types as potentially active against the viral protease. The compounds labelled as actives by SVM were next evaluated through consensus docking studies on two PDB structures and their binding mode was compared to well-known protease inhibitors. The best five compounds selected by consensus docking were then submitted to molecular dynamics to deepen binding interactions stability. Of note, the compounds selected via SVM retrieved all the most important interactions known in the literature.

Machine Learning Model for Predicting Postoperative Survival of Patients with Colorectal Cancer

Purpose

Machine learning (ML) is a strong candidate for making accurate predictions, as we can use large amount of data with powerful computational algorithms. We developed a ML based model to predict survival of patients with colorectal cancer (CRC) using data from two independent datasets.

Materials and Methods

A total of 364,316 and 1,572 CRC patients were included from the Surveillance, Epidemiology, and End Results (SEER) and a Korean dataset, respectively. As SEER combines data from 18 cancer registries, internal validation was done using 18-Fold-Cross-Validation then external validation was performed by testing the trained model on the Korean dataset. Performance was evaluated using area under the receiver operating characteristic curve (AUROC), sensitivity and positive predictive values.

Results

Clinicopathological characteristics were significantly different between the two datasets and the SEER showed a significant lower 5-year survival rate compared to the Korean dataset (60.1% vs. 75.3%, p < 0.001). The ML-based model using the Light gradient boosting algorithm achieved a better performance in predicting 5-year-survival compared to American Joint Committee on Cancer stage (AUROC, 0.804 vs. 0.736; p < 0.001). The most important features which influenced model performance were age, number of examined lymph nodes, and tumor size. Sensitivity and positive predictive values of predicting 5-year-survival for classes including dead or alive were reported as 68.14%, 77.51% and 49.88%, 88.1% respectively in the validation set. Survival probability can be checked using the web-based survival predictor (http://colorectalcancer.pythonanywhere.com).

Conclusion

ML-based model achieved a much better performance compared to staging in individualized estimation of survival of patients with CRC.

Strain Curve Classification Using Supervised Machine Learning Algorithm with Physiologic Constraints

Speckle tracking echocardiography (STE) enables quantification of myocardial deformation by a generation of spatiotemporal strain curves or time-strain curves (TSCs). Currently, only assessment of peak global longitudinal strain is employed in clinical practice because of the uncertainty in the accuracy of STE. We describe a supervised machine learning, physiologically constrained, fully automatic algorithm, trained with labeled data, for classification of TSCs into physiologic or artifactual classes. The data set of 415 healthy patients, with three cine loops per patient, corresponding to the three standard 2-D longitudinal views, was processed using a previously published, in-house STE software termed K-SAD. We report an accuracy of 86.4% for classifying TSCs as physiologic, artifactual and undetermined curves. The positive predictive value for a physiologic strain curve is 89%. This is as a necessary step for a similar separation of pathologic conditions, to allow full utilization of the temporal information concealed in layer-specific segmental TSCs.

Towards a fully automated algorithm driven platform for biosystems design

Large-scale data acquisition and analysis are often required in the successful implementation of the design, build, test, and learn (DBTL) cycle in biosystems design. However, it has long been hindered by experimental cost, variability, biases, and missed insights from traditional analysis methods. Here, we report the application of an integrated robotic system coupled with machine learning algorithms to fully automate the DBTL process for biosystems design. As proof of concept, we have demonstrated its capacity by optimizing the lycopene biosynthetic pathway. This fully-automated robotic platform, BioAutomata, evaluates less than 1% of possible variants while outperforming random screening by 77%. A paired predictive model and Bayesian algorithm select experiments which are performed by Illinois Biological Foundry for Advanced Biomanufacturing (iBioFAB). BioAutomata excels with black-box optimization problems, where experiments are expensive and noisy and the success of the experiment is not dependent on extensive prior knowledge of biological mechanisms.

Existing efforts have been focused on one of the elements in the automation of the design, build, test, and learn (DBTL) cycle for biosystems design. Here, the authors integrate a robotic system with machine learning algorithms to fully automate the DBTL cycle and apply it in optimizing the lycopene biosynthetic pathway.

Exploring the Potential of Spherical Harmonics and PCVM for Compounds Activity Prediction

Biologically active chemical compounds may provide remedies for several diseases. Meanwhile, Machine Learning techniques applied to Drug Discovery, which are cheaper and faster than wet-lab experiments, have the capability to more effectively identify molecules with the expected pharmacological activity. Therefore, it is urgent and essential to develop more representative descriptors and reliable classification methods to accurately predict molecular activity. In this paper, we investigate the potential of a novel representation based on Spherical Harmonics fed into Probabilistic Classification Vector Machines classifier, namely SHPCVM, to compound the activity prediction task. We make use of representation learning to acquire the features which describe the molecules as precise as possible. To verify the performance of SHPCVM ten-fold cross-validation tests are performed on twenty-one G protein-coupled receptors (GPCRs). Experimental outcomes (accuracy of 0.86) assessed by the classification accuracy, precision, recall, Matthews’ Correlation Coefficient and Cohen’s kappa reveal that using our Spherical Harmonics-based representation which is relatively short and Probabilistic Classification Vector Machines can achieve very satisfactory performance results for GPCRs.

Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems

In this paper, we tackle several problems that appear in robotics and autonomous systems: algorithm tuning, automatic control, and intelligent design. All those problems share in common that they can be mapped to global optimization problems where evaluations are expensive. Bayesian optimization (BO) has become a fundamental global optimization algorithm in many problems where sample efficiency is of paramount importance. BO uses a probabilistic surrogate model to learn the response function and reduce the number of samples required. Gaussian processes (GPs) have become a standard surrogate model for their flexibility to represent a distribution over functions. In a black-box settings, the common assumption is that the underlying function can be modeled with a stationary GP. In this paper, we present a novel kernel function specially designed for BO, that allows nonstationary behavior of the surrogate model in an adaptive local region. This kernel is able to reconstruct nonstationarity even with the irregular sampling distribution that arises from BO. Furthermore, in our experiments, we found that this new kernel results in an improved local search (exploitation), without penalizing the global search (exploration) in many applications. We provide extensive results in well-known optimization benchmarks, machine learning hyperparameter tuning, reinforcement learning, and control problems, and UAV wing optimization. The results show that the new method is able to outperform the state of the art in BO both in stationary and nonstationary problems.

A combined drug discovery strategy based on machine learning and molecular docking

Data mining methods based on machine learning play an increasingly important role in drug design and discovery. In the current work, eight machine learning methods including decision trees, k-Nearest neighbor, support vector machines, random forests, extremely randomized trees, AdaBoost, gradient boosting trees, and XGBoost were evaluated comprehensively through a case study of ACC inhibitor data sets. Internal and external data sets were employed for cross-validation of the eight machine learning methods. Results showed that the extremely randomized trees model performed best and was adopted as the first step of virtual screening. Together with structure-based virtual screening in the second step, this combined strategy obtained desirable results. This work indicates that the combination of machine learning methods with traditional structure-based virtual screening can effectively strengthen the ability in finding potential hits from large compound database for a given target.

Integration of in vitro and in silico Models Using Bayesian Optimization With an Application to Stochastic Modeling of Mesenchymal 3D Cell Migration

Cellular migration plays a crucial role in many aspects of life and development. In this paper, we propose a computational model of 3D migration that is solved by means of the tau-leaping algorithm and whose parameters have been calibrated using Bayesian optimization. Our main focus is two-fold: to optimize the numerical performance of the mechano-chemical model as well as to automate the calibration process of in silico models using Bayesian optimization. The presented mechano-chemical model allows us to simulate the stochastic behavior of our chemically reacting system in combination with mechanical constraints due to the surrounding collagen-based matrix. This numerical model has been used to simulate fibroblast migration. Moreover, we have performed in vitro analysis of migrating fibroblasts embedded in 3D collagen-based fibrous matrices (2 mg/ml). These in vitro experiments have been performed with the main objective of calibrating our model. Nine model parameters have been calibrated testing 300 different parametrizations using a completely automatic approach. Two competing evaluation metrics based on the Bhattacharyya coefficient have been defined in order to fit the model parameters. These metrics evaluate how accurately the in silico model is replicating in vitro measurements regarding the two main variables quantified in the experimental data (number of protrusions and the length of the longest protrusion). The selection of an optimal parametrization is based on the balance between the defined evaluation metrics. Results show how the calibrated model is able to predict the main features observed in the in vitro experiments.

Multiclassification Prediction of Enzymatic Reactions for Oxidoreductases and Hydrolases Using Reaction Fingerprints and Machine Learning Methods

Drug metabolism is a complex procedure in the human body, including a series of enzymatically catalyzed reactions. However, it is costly and time consuming to investigate drug metabolism experimentally; computational methods are hence developed to predict drug metabolism and have shown great advantages. As the first step, classification of metabolic reactions and enzymes is highly desirable for drug metabolism prediction. In this study, we developed multiclassification models for prediction of reaction types catalyzed by oxidoreductases and hydrolases, in which three reaction fingerprints were used to describe the reactions and seven machine learnings algorithms were employed for model building. Data retrieved from KEGG containing 1055 hydrolysis and 2510 redox reactions were used to build the models, respectively. The external validation data consisted of 213 hydrolysis and 512 redox reactions extracted from the Rhea database. The best models were built by neural network or logistic regression with a 2048-bit transformation reaction fingerprint. The predictive accuracies of the main class, subclass, and superclass classification models on external validation sets were all above 90%. This study will be very helpful for enzymatic reaction annotation and further study on metabolism prediction.