Neural networks in drug discovery: have they lived up to their promise?

Comparative approach to different convolutional neural network (CNN) architectures applied to human behavior detection

Medical diagnostics, product classification, surveillance and detection of inappropriate behavior are becoming increasingly sophisticated due to the development of methods based on image analysis using neural networks. Considering this, in this work, we evaluate state-of-the-art convolutional neural network architectures proposed in recent years to classify the driving behavior and distractions of drivers. Our main goal is to measure the performance of such architectures using only free resources (i.e., free graphic processing unit, open source) and to evaluate how much of this technological evolution is available to regular users.

Deep-learning: investigating deep neural networks hyper-parameters and comparison of performance to shallow methods for modeling bioactivity data

Background

In recent years, research in artificial neural networks has resurged, now under the deep-learning umbrella, and grown extremely popular. Recently reported success of DL techniques in crowd-sourced QSAR and predictive toxicology competitions has showcased these methods as powerful tools in drug-discovery and toxicology research. The aim of this work was dual, first large number of hyper-parameter configurations were explored to investigate how they affect the performance of DNNs and could act as starting points when tuning DNNs and second their performance was compared to popular methods widely employed in the field of cheminformatics namely Naïve Bayes, k-nearest neighbor, random forest and support vector machines. Moreover, robustness of machine learning methods to different levels of artificially introduced noise was assessed. The open-source Caffe deep-learning framework and modern NVidia GPU units were utilized to carry out this study, allowing large number of DNN configurations to be explored.

Results

We show that feed-forward deep neural networks are capable of achieving strong classification performance and outperform shallow methods across diverse activity classes when optimized. Hyper-parameters that were found to play critical role are the activation function, dropout regularization, number hidden layers and number of neurons. When compared to the rest methods, tuned DNNs were found to statistically outperform, with p value <0.01 based on Wilcoxon statistical test. DNN achieved on average MCC units of 0.149 higher than NB, 0.092 than kNN, 0.052 than SVM with linear kernel, 0.021 than RF and finally 0.009 higher than SVM with radial basis function kernel. When exploring robustness to noise, non-linear methods were found to perform well when dealing with low levels of noise, lower than or equal to 20%, however when dealing with higher levels of noise, higher than 30%, the Naïve Bayes method was found to perform well and even outperform at the highest level of noise 50% more sophisticated methods across several datasets.

Electronic supplementary material

The online version of this article (doi:10.1186/s13321-017-0226-y) contains supplementary material, which is available to authorized users.

Have artificial neural networks met expectations in drug discovery as implemented in QSAR framework?

INTRODUCTION

Artificial neural networks (ANNs) are highly adaptive nonlinear optimization algorithms that have been applied in many diverse scientific endeavors, ranging from economics, engineering, physics, and chemistry to medical science. Notably, in the past two decades, ANNs have been used widely in the process of drug discovery.

AREAS COVERED

In this review, the authors discuss advantages and disadvantages of ANNs in drug discovery as incorporated into the quantitative structure-activity relationships (QSAR) framework. Furthermore, the authors examine the recent studies, which span over a broad area with various diseases in drug discovery. In addition, the authors attempt to answer the question about the expectations of the ANNs in drug discovery and discuss the trends in this field.

EXPERT OPINION

The old pitfalls of overtraining and interpretability are still present with ANNs. However, despite these pitfalls, the authors believe that ANNs have likely met many of the expectations of researchers and are still considered as excellent tools for nonlinear data modeling in QSAR. It is likely that ANNs will continue to be used in drug development in the future.

Evaluation of new thiadiazoles and benzothiazoles as potential radioprotectors: free radical scavenging activity in vitro and theoretical studies (QSAR, DFT)

Thiol and aminothiol compounds are among the most efficient chemical radioprotectors. To increase their efficiency, we synthesized two new classes of thiol and aminothiol compounds derived from benzothiazole (T1, T2, AM1, AM2) and thiadiazole (T3, T4, AM3) structures. We examined them for their ability to scavenge free radicals (DPPH*, ABTS(*+), *OH). Thiol derivatives with a thiadiazole structure are the most active compounds scavenging DPPH* and ABTS(*+) free radicals, with an IC(50) of 0.053+/-0.006 and 0.023+/-0.002 mM, respectively, for the derivative T3. Moreover, compounds T1, T2, and T3 at 60 microM gave 83% protection against 2-deoxyribose degradation by *OH. The ability of these compounds to protect DNA against *OH produced by a Fenton reaction and gamma-irradiation (15 Gy)-induced strand breaks was also evaluated on pBR322 plasmid DNA. In both tests thiol derivatives were the most efficient compounds. Derivatives T2 and T3 totally inhibit DNA strand breaks at the concentration of 50 microM. The protection afforded by these derivatives was comparatively higher than that of the radioprotectors WR-2721 and WR-1065. Our data indicate that these two compounds are free radical scavengers and potential antioxidant agents. Finally, DFT and QSAR studies were performed to support the experimental observations.

Network models in drug discovery and regenerative medicine

Network motifs and modelling paradigms are attracting increasing attention as modelling tools in drug design and development, and in regenerative medicine. There is a gradual but inexorable convergence between these hitherto disparate disciplines. This review summarizes some very recent work in these areas, leading to an understanding of the complementary roles networks play and factors driving this convergence: network paradigms can be excellent ways of modelling and understanding drug molecules and their action, an understanding of the robustness and vulnerabilities of biological targets may improve the efficacy of drug design and discovery, drug design has an increasingly large role to play in directing stem cell properties, stem cell regulatory networks can be modelled in useful ways using network models at a reasonable level of scale, and the network tools of drug design are also very useful for the design of biomaterials used in regenerative medicine.

Artificial neural networks in biology and chemistry: the evolution of a new analytical tool

Once regarded as an eccentric and unpromising algorithm for the analysis of scientific data, the neural network has been developed in the last decade into a powerful computational tool. Its use now spans all areas of science, from the physical sciences and engineering to the life sciences and allied subjects. Applications range from the assessment of epidemiological data or the deconvolution of spectra to highly practical applications, such as the electronic nose. This introductory chapter considers briefly the growth in the use of neural networks and provides some general background in preparation for the more detailed chapters that follow.

Prediction of atmospheric degradation data for POPs by gene expression programming

Quantitative structure-activity relationship models for the prediction of the mean and the maximum atmospheric degradation half-life values of persistent organic pollutants were developed based on the linear heuristic method (HM) and non-linear gene expression programming (GEP). Molecular descriptors, calculated from the structures alone, were used to represent the characteristics of the compounds. HM was used both to pre-select the whole descriptor sets and to build the linear model. GEP yielded satisfactory prediction results: the square of the correlation coefficient r(2) was 0.80 and 0.81 for the mean and maximum half-life values of the test set, and the root mean square errors were 0.448 and 0.426, respectively. The results of this work indicate that the GEP is a very promising tool for non-linear approximations.

Machine learning approaches for predicting compounds that interact with therapeutic and ADMET related proteins

Computational methods for predicting compounds of specific pharmacodynamic and ADMET (absorption, distribution, metabolism, excretion and toxicity) property are useful for facilitating drug discovery and evaluation. Recently, machine learning methods such as neural networks and support vector machines have been explored for predicting inhibitors, antagonists, blockers, agonists, activators and substrates of proteins related to specific therapeutic and ADMET property. These methods are particularly useful for compounds of diverse structures to complement QSAR methods, and for cases of unavailable receptor 3D structure to complement structure-based methods. A number of studies have demonstrated the potential of these methods for predicting such compounds as substrates of P-glycoprotein and cytochrome P450 CYP isoenzymes, inhibitors of protein kinases and CYP isoenzymes, and agonists of serotonin receptor and estrogen receptor. This article is intended to review the strategies, current progresses and underlying difficulties in using machine learning methods for predicting these protein binders and as potential virtual screening tools. Algorithms for proper representation of the structural and physicochemical properties of compounds are also evaluated.

Synthesis and structure-antibacterial activity relationship investigation of isomeric 2,3,5-substituted perhydropyrrolo[3,4-d]isoxazole-4,6-diones

The synthesis of 2,3,5-substituted perhydropyrrolo[3,4-d]isoxazole-4,6-diones (44 compounds) has been accomplished by the cycloaddition reaction of N-methyl-C-arylnitrones with N-substituted maleimides. The compounds were screened for their antibacterial activities and most of them exhibited activity against Enterococcus faecalis (ATCC 29212) and Staphylococcus aureus (ATCC 25923). cis-3a and cis-3d were found fairly effective against E. faecalis (ATCC 29212) and S. aureus (ATCC 25923) with MIC values of 25 and 50microg/ml. With the changes of cis isomers of the compounds to trans, their antibacterial activities also changed against the bacteria studied. First, pharmacophoric fragments had been calculated in accordance with the rules of the electronic-topological method (ETM). Next, both active compounds and pharmacophores had been projected to the nodes of Kohonen's self-organizing maps (SOM) to obtain the weights of pharmacophore fragments as numerical descriptors, that were used after this for the associative neural networks (ASNN) training. A model for the activity prediction was developed as the result of training the ASNNs.

Robust fuzzy mappings for QSAR studies

This study presents a new robust method of developing quantitative structure-activity relationship (QSAR) models based on fuzzy mappings. An important issue in QSAR modelling is of robustness, i.e., model should not undergo overtraining and model performance should be least sensitive to the modelling errors associated with the chosen descriptors and structure of the model. We establish robust input-output mappings for QSAR studies based on fuzzy "if-then" rules. The identification of these mappings (i.e. the construction of fuzzy rules) is based on a robust criterion that the maximum possible value of energy-gain from modelling errors to the identification errors is minimum. The robustness of proposed approach has been illustrated with simulation studies and QSAR modelling examples. The method of robust fuzzy mappings has been compared with Bayesian regularized neural networks through the QSAR modelling examples of (1) carboquinones' data set, (2) benzodiazepine data set, and (3) predicting the rate constant for hydroxyl radical tropospheric degradation of 460 heterogeneous organic compounds.

MODEL—molecular descriptor lab: A web-based server for computing structural and physicochemical features of compounds

Molecular descriptors represent structural and physicochemical features of compounds. They have been extensively used for developing statistical models, such as quantitative structure activity relationship (QSAR) and artificial neural networks (NN), for computer prediction of the pharmacodynamic, pharmacokinetic, or toxicological properties of compounds from their structure. While computer programs have been developed for computing molecular descriptors, there is a lack of a freely accessible one. We have developed a web-based server, MODEL (Molecular Descriptor Lab), for computing a comprehensive set of 3,778 molecular descriptors, which is significantly more than the approximately 1,600 molecular descriptors computed by other software. Our computational algorithms have been extensively tested and the computed molecular descriptors have been used in a number of published works of statistical models for predicting variety of pharmacodynamic, pharmacokinetic, and toxicological properties of compounds. Several testing studies on the computed molecular descriptors are discussed. MODEL is accessible at http://jing.cz3.nus.edu.sg/cgi-bin/model/model.cgi free of charge for academic use.

Support vector machines for predictive modeling in heterogeneous catalysis: a comprehensive introduction and overfitting investigation based on two real applications

This works provides an introduction to support vector machines (SVMs) for predictive modeling in heterogeneous catalysis, describing step by step the methodology with a highlighting of the points which make such technique an attractive approach. We first investigate linear SVMs, working in detail through a simple example based on experimental data derived from a study aiming at optimizing olefin epoxidation catalysts applying high-throughput experimentation. This case study has been chosen to underline SVM features in a visual manner because of the few catalytic variables investigated. It is shown how SVMs transform original data into another representation space of higher dimensionality. The concepts of Vapnik-Chervonenkis dimension and structural risk minimization are introduced. The SVM methodology is evaluated with a second catalytic application, that is, light paraffin isomerization. Finally, we discuss why SVMs is a strategic method, as compared to other machine learning techniques, such as neural networks or induction trees, and why emphasis is put on the problem of overfitting.

Combined electronic-topological and neural networks study of some hydroxysemicarbazides as potential antitumor agents

Structure-activity relationships study was performed for a series of Schiff bases hydroxysemicarbazide as potential antitumor agents by using the electronic-topological method combined with neural networks (ETM-NN). Data for the approach were obtained from conformational and quantum-chemical calculations and arranged first as matrices called electronic-topological matrices of contiguity, by one for each compound. Then specific molecular fragments were found for active compounds ('activity features') from the ETM application. After this, a system of prognosis was developed as the result of training the Kohonen self-organizing maps (SOM) by the most significant fragments.

Molecular similarity and diversity in chemoinformatics: from theory to applications

This review is dedicated to a survey on molecular similarity and diversity. Key findings reported in recent investigations are selectively highlighted and summarized. Even if this overview is mainly centered in chemoinformatics, applications in other areas (pharmaceutical and medical chemistry, combinatorial chemistry, chemical databases management, etc.) are also introduced. The approaches used to define and describe the concepts of molecular similarity and diversity in the context of chemoinformatics are discussed in the first part of this review. We introduce, in the second and third parts, the descriptions and analyses of different methods and techniques. Finally, current applications and problems are enumerated and discussed in the last part.

QSAR study of 4-phenylpiperidine derivatives as μ opioid agonists by neural network method

A nonlinear QSAR study was conducted on a series of 4-phenylpiperidine derivatives (4PPs) acting as mu opioid agonists by three-layer back-propagation neural network (NN) method. At first a variety of molecular descriptors were calculated and then selected with two-stage least squares combining partial least squares (PLS) method. The selected four molecular descriptors, out of 292 ones, were correlated with the known analgesic activities of 38 4PPs by NN method. The established QSAR model was further validated by five additional 4PPs, as an external testing set. Moreover, a pharmacophore model was hypothesized based on the results, which would be helpful for structural optimization of 4PPs.

Support vector machines-based quantitative structure-property relationship for the prediction of heat capacity

The support vector machine (SVM), as a novel type of learning machine, for the first time, was used to develop a Quantitative Structure-Property Relationship (QSPR) model of the heat capacity of a diverse set of 182 compounds based on the molecular descriptors calculated from the structure alone. Multiple linear regression (MLR) and radial basis function networks (RBFNNs) were also utilized to construct quantitative linear and nonlinear models to compare with the results obtained by SVM. The root-mean-square (rms) errors in heat capacity predictions for the whole data set given by MLR, RBFNNs, and SVM were 4.648, 4.337, and 2.931 heat capacity units, respectively. The prediction results are in good agreement with the experimental value of heat capacity; also, the results reveal the superiority of the SVM over MLR and RBFNNs models.

Accurate Quantitative Structure−Property Relationship Model To Predict the Solubility of C60 in Various Solvents Based on a Novel Approach Using a Least-Squares Support Vector Machine

A least-squares support vector machine (LSSVM) was used for the first time as a novel machine-learning technique for the prediction of the solubility of C60 in a large number of diverse solvents using calculated molecular descriptors from the molecular structure alone and on the basis of the software CODESSA as inputs. The heuristic method of CODESSA was used to select the correlated descriptors and build the linear model. Both the linear and the nonlinear models can give very satisfactory prediction results: the square of the correlation coefficient R(2) was 0.892 and 0.903, and the root-mean-square error was 0.126 and 0.116, respectively, for the whole data set. The prediction result of the LSSVM model is better than that obtained by the heuristic method and the reference, which proved LSSVM was a useful tool in the prediction of the solubility of C60. In addition, this paper provided a new and effective method for predicting the solubility of C60 from its structures and gave some insight into the structural features related to the solubility of C60 in different solvents.

Generation of QSAR sets with a self-organizing map

A Kohonen self-organizing map (SOM) is used to classify a data set consisting of dihydrofolate reductase inhibitors with the help of an external set of Dragon descriptors. The resultant classification is used to generate training, cross-validation (CV) and prediction sets for QSAR modeling using the ADAPT methodology. The results are compared to those of QSAR models generated using sets created by activity binning and a sphere exclusion method. The results indicate that the SOM is able to generate QSAR sets that are representative of the composition of the overall data set in terms of similarity. The resulting QSAR models are half the size of those published and have comparable RMS errors. Furthermore, the RMS errors of the QSAR sets are consistent, indicating good predictive capabilities as well as generalizability.

QSAR and classification models of a novel series of COX-2 selective inhibitors: 1,5-diarylimidazoles based on support vector machines

The support vector machine, which is a novel algorithm from the machine learning community, was used to develop quantitation and classification models which can be used as a potential screening mechanism for a novel series of COX-2 selective inhibitors. Each compound was represented by calculated structural descriptors that encode constitutional, topological, geometrical, electrostatic, and quantum-chemical features. The heuristic method was then used to search the descriptor space and select the descriptors responsible for activity. Quantitative modelling results in a nonlinear, seven-descriptor model based on SVMs with root mean-square errors of 0.107 and 0.136 for training and prediction sets, respectively. The best classification results are found using SVMs: the accuracy for training and test sets is 91.2% and 88.2%, respectively. This paper proposes a new and effective method for drug design and screening.

The prediction of human oral absorption for diffusion rate-limited drugs based on heuristic method and support vector machine

Support vector machine (SVM), as a novel machine learning technique, was used for the prediction of the human oral absorption for a large and diverse data set using the five descriptors calculated from the molecular structure alone. The molecular descriptors were selected by heuristic method (HM) implemented in CODESSA. At the same time, in order to show the influence of different molecular descriptors on absorption and to well understand the absorption mechanism, HM was used to build several multivariable linear models using different numbers of molecular descriptors. Both the linear and non-linear model can give satisfactory prediction results: the square of correlation coefficient R(2) was 0.78 and 0.86 for the training set, and 0.70 and 0.73 for the test set respectively. In addition, this paper provides a new and effective method for predicting the absorption of the drugs from their structures and gives some insight into structural features related to the absorption of the drugs.

Modeling steric and electronic effects in 3D- and 4D-QSAR schemes: predicting benzoic pK(a) values and steroid CBG binding affinities

We conducted a systematic study of the performance of the 3D- and 4D-QSAR schemes in modeling steric and electronic effects. In particular, we compared the CoMFA and Hopfinger's 4D-QSAR schemes, which apply completely different concepts for the generation of the molecular data used for modeling QSAR. Hence, we attempted to predict the pK(a) values of (o-, m-, and p-)benzoic acids which were divided into three subseries in order to simulate different levels of steric and electronic control. The steroids binding to CBG were used as a benchmark series where biological activity is limited by shape factors. Although individual models differ depending upon the individual scheme, generally, both CoMFA and 4D-QSAR appeared to provide comparable results, irrespective of the differences in the coding schemes used for the description. Moreover, a new 4D-QSAR scheme involving a self-organizing neural network was designed. Generally, the SOM scheme that we designed performs comparably to the grid scheme; however, it provides better results for the charge type descriptors, and the robust neuron architecture allows for the decrease of the influence of the molecular superimposition mode.

Quantitative Prediction of logk of Peptides in High-Performance Liquid Chromatography Based on Molecular Descriptors by Using the Heuristic Method and Support Vector Machine

A new method support vector machine (SVM) and the heuristic method (HM) were used to develop the nonlinear and linear models between the capacity factor (logk) and seven molecular descriptors of 75 peptides for the first time. The molecular descriptors representing the structural features of the compounds only included the constitutional and topological descriptors, which can be obtained easily without optimizing the structure of the molecule. The seven molecular descriptors selected by the heuristic method in CODESSA were used as inputs for SVM. The results obtained by SVM were compared with those obtained by the heuristic method. The prediction result of the SVM model is better than that of heuristic method. For the test set, a predictive correlation coefficient R = 0.9801 and root-mean-square error of 0.1523 were obtained. The prediction results are in very good agreement with the experimental values. But the linear model of the heuristic method is easier to understand and ready to use for a chemist. This paper provided a new and effective method for predicting the chromatography retention of peptides and some insight into the structural features which are related to the capacity factor of peptides.

QSAR Models for the Prediction of Binding Affinities to Human Serum Albumin Using the Heuristic Method and a Support Vector Machine

The binding affinities to human serum albumin for 94 diverse drugs and drug-like compounds were modeled with the descriptors calculated from the molecular structure alone using a quantitative structure-activity relationship (QSAR) technique. The heuristic method (HM) and support vector machine (SVM) were utilized to construct the linear and nonlinear prediction models, leading to a good correlation coefficient (R2) of 0.86 and 0.94 and root-mean-square errors (rms) of 0.212 and 0.134 albumin drug binding affinity units, respectively. Furthermore, the models were evaluated by a 10 compound external test set, yielding R2 of 0.71 and 0.89 and rms error of 0.430 and 0.222. The specific information described by the heuristic linear model could give some insights into the factors that are likely to govern the binding affinity of the compounds and be used as an aid to the drug design process; however, the prediction results of the nonlinear SVM model seem to be better than that of the HM.

Artificial Neural Networks and Linear Discriminant Analysis: A Valuable Combination in the Selection of New Antibacterial Compounds

A set of topological descriptors has been used to discriminate between antibacterial and nonantibacterial drugs. Topological descriptors are simple integers calculated from the molecular structure represented in SMILES format. The methods used for antibacterial activity discrimination were linear discriminant analysis (LDA) and artificial neural networks of a multilayer perceptron (MLP) type. The following plot frequency distribution diagrams were used: a function of the number of drugs within a value interval of the discriminant function and the output value of the neural network versus these values. Pharmacological distribution diagrams (PDD) were used as a visualizing technique for the identification of antibacterial agents. The results confirmed the discriminative capacity of the topological descriptors proposed. The combined use of LDA and MLP in the guided search and the selection of new structures with theoretical antibacterial activity proved highly effective, as shown by the in vitro activity and toxicity assays conducted.

Prediction of selectivity of α1-adrenergic antagonists by counterpropagation neural network (CP-ANN)

A quantitative structure-selectivity relationships of series of structurally diverse alpha1-adrenergic antagonists was performed by using counter-propagation neural network (CP-ANN). The theoretical molecular descriptors have been calculated and selected using CODESSA program. The results obtained for a highly non-congeneric set of molecules have confirmed the potential of use of CP-ANN approach in prediction of relative activity (selectivity) of alpha1-adrenergic antagonists.

Study of the Quantitative Structure-Mobility Relationship of Carboxylic Acids in Capillary Electrophoresis Based on Support Vector Machines

The support vector machines (SVM), as a novel type of learning machine, were used to develop a quantitative structure-mobility relationship (QSMR) model of 58 aliphatic and aromatic carboxylic acids based on molecular descriptors calculated from the structure alone. Multiple linear regression (MLR) and radial basis function neural networks (RBFNNs) were also utilized to construct the linear and the nonlinear model to compare with the results obtained by SVM. The root-mean-square errors in absolute mobility predictions for the whole data set given by MLR, RBFNNs, and SVM were 1.530, 1.373, and 0.888 mobility units (10(-5) cm(2) S(-1) V(-1)), respectively, which indicated that the prediction result agrees well with the experimental values of these compounds and also revealed the superiority of SVM over MLR and RBFNNs models for the prediction of the absolute mobility of carboxylic acids. Moreover, the models we proposed could also provide some insight into what structural features are related to the absolute mobility of aliphatic and aromatic carboxylic acids.

An Accurate QSPR Study of O−H Bond Dissociation Energy in Substituted Phenols Based on Support Vector Machines

The support vector machine (SVM), as a novel type of learning machine, was used to develop a Quantitative Structure-Property Relationship (QSPR) model of the O-H bond dissociation energy (BDE) of 78 substituted phenols. The six descriptors calculated solely from the molecular structures of compounds selected by forward stepwise regression were used as inputs for the SVM model. The root-mean-square (rms) errors in BDE predictions for the training, test, and overall data sets were 3.808, 3.320, and 3.713 BDE units (kJ mol(-1)), respectively. The results obtained by Gaussian-kernel SVM were much better than those obtained by multiple linear regression, radial basis function neural networks, linear-kernel SVM, and other QSPR approaches.