Evaluating docking programs: keeping the playing field level

Databases of ligand-binding pockets and protein-ligand interactions

Many research groups and institutions have created a variety of databases curating experimental and predicted data related to protein-ligand binding. The landscape of available databases is dynamic, with new databases emerging and established databases becoming defunct. Here, we review the current state of databases that contain binding pockets and protein-ligand binding interactions. We have compiled a list of such databases, fifty-three of which are currently available for use. We discuss variation in how binding pockets are defined and summarize pocket-finding methods. We organize the fifty-three databases into subgroups based on goals and contents, and describe standard use cases. We also illustrate that pockets within the same protein are characterized differently across different databases. Finally, we assess critical issues of sustainability, accessibility and redundancy.

Structure- and Ligand-Based Virtual Screening on DUD-E+: Performance Dependence on Approximations to the Binding Pocket

Using the DUD-E⁺ benchmark, we explore the impact of using a single protein pocket or ligand for virtual screening compared with using ensembles of alternative pockets, ligands, and sets thereof. For both structure-based and ligand-based approaches, the precise characterization of the binding site in question had a significant impact on screening performance. Using the single original DUD-E protein, Surflex-Dock yielded mean ROC area of 0.81 ± 0.11. Using the cognate ligand instead, with the eSim method for screening, yielded 0.77 ± 0.14. Moving to ensembles of five protein pocket variants increased docking performance to 0.84 ± 0.09. Results for the analogous ligand-based approach (using the five crystallographically aligned cognate ligands) was 0.83 ± 0.11. Using the same ligands, but making use of an automatically generated mutual alignment, yielded mean AUC nearly as good as from single-structure docking: 0.80 ± 0.12. Detailed results and statistical analyses show that structure- and ligand-based methods are complementary and can be fruitfully combined to enhance screening efficiency. A hybrid approach combining ensemble docking with eSim-based screening produced the best and most consistent performance (mean ROC area of 0.89 ± 0.08 and 1% early enrichment of 46-fold). Based on results from both the docking and ligand-similarity approaches, it is clearly unwise to make use of a single arbitrarily chosen protein structure for docking or single ligand query for similarity-based screening.

Electrostatic-field and surface-shape similarity for virtual screening and pose prediction

We introduce a new method for rapid computation of 3D molecular similarity that combines electrostatic field comparison with comparison of molecular surface-shape and directional hydrogen-bonding preferences (called "eSim"). Rather than employing heuristic "colors" or user-defined molecular feature types to represent conformation-dependent molecular electrostatics, eSim calculates the similarity of the electrostatic fields of two molecules (in addition to shape and hydrogen-bonding). We present detailed virtual screening performance data on the standard 102 target DUD-E set. In its moderately fast screening mode, eSim running on a single computing core is capable of processing over 60 molecules per second. In this mode, eSim performed significantly better than all alternate methods for which full DUD-E data were available (mean ROC area of 0.74, p [Formula: see text], by paired t-test, compared with the best performing alternate method). In addition, for 92 targets of the DUD-E set where multiple ligand-bound crystal structures were available, screening performance was assessed using alternate ligands or sets thereof (in their bound poses) as similarity targets. Using the joint alignment of five ligands for each protein target, mean ROC area exceeded 0.82 for the 92 targets. Design-focused application of ligand similarity methods depends on accurate predictions of geometric molecular relationships. We comprehensively assessed pose prediction accuracy by curating nearly 400,000 bound ligand pose pairs across the DUD-E targets. Overall, beginning from agnostic initial poses, we observed an 80% success rate for RMSD [Formula: see text] Å  among the top 20 predicted eSim poses. These examples were split roughly 50/50 into cases with high direct atomic overlap (where a shared scaffold exists between a pair) and low direct atomic overlap (where where a ligand pair has dissimilar scaffolds but largely occupies the same space). Within the high direct atomic overlap subset, the pose prediction success rate was 93%. For the more challenging subset (where dissimilar scaffolds are to be aligned), the success rate was 70%. The eSim approach enables both large-scale screening and rational design of ligands and is rooted in physically meaningful, non-heuristic, molecular comparisons.

Comparing neural-network scoring functions and the state of the art: applications to common library screening

We compare established docking programs, AutoDock Vina and Schrödinger's Glide, to the recently published NNScore scoring functions. As expected, the best protocol to use in a virtual-screening project is highly dependent on the target receptor being studied. However, the mean screening performance obtained when candidate ligands are docked with Vina and rescored with NNScore 1.0 is not statistically different than the mean performance obtained when docking and scoring with Glide. We further demonstrate that the Vina and NNScore docking scores both correlate with chemical properties like small-molecule size and polarizability. Compensating for these potential biases leads to improvements in virtual screen performance. Composite NNScore-based scoring functions suited to a specific receptor further improve performance. We are hopeful that the current study will prove useful for those interested in computer-aided drug design.

How to benchmark methods for structure-based virtual screening of large compound libraries

Structure-based virtual screening is a useful computational technique for ligand discovery. To systematically evaluate different docking approaches, it is important to have a consistent benchmarking protocol that is both relevant and unbiased. Here, we describe the designing of a benchmarking data set for docking screen assessment, a standard docking screening process, and the analysis and presentation of the enrichment of annotated ligands among a background decoy database.

Novel human mPGES-1 inhibitors identified through structure-based virtual screening

Microsomal prostaglandin E synthase-1 (mPGES-1) is an inducible prostaglandin E synthase after exposure to pro-inflammatory stimuli and, therefore, represents a novel target for therapeutic treatment of acute and chronic inflammatory disorders. It is essential to identify mPGES-1 inhibitors with novel scaffolds as new leads or hits for the purpose of drug design and discovery that aim to develop the next-generation anti-inflammatory drugs. Herein we report novel mPGES-1 inhibitors identified through a combination of large-scale structure-based virtual screening, flexible docking, molecular dynamics simulations, binding free energy calculations, and in vitro assays on the actual inhibitory activity of the computationally selected compounds. The computational studies are based on our recently developed three-dimensional (3D) structural model of mPGES-1 in its open state. The combined computational and experimental studies have led to identification of new mPGES-1 inhibitors with new scaffolds. In particular, (Z)-5-benzylidene-2-iminothiazolidin-4-one is a promising novel scaffold for the further rational design and discovery of new mPGES-1 inhibitors. To our best knowledge, this is the first time a 3D structural model of the open state mPGES-1 is used in structure-based virtual screening of a large library of available compounds for the mPGES-1 inhibitor identification. The positive experimental results suggest that our recently modeled trimeric structure of mPGES-1 in its open state is ready for the structure-based drug design and discovery.

Computer- and structure-based lead design for epigenetic targets

The term epigenetics is defined as inheritable changes that influence the outcome of a phenotype without changes in the genome. Epigenetics is based upon DNA methylation and posttranslational histone modifications. While there is much known about reversible acetylation as a posttranslational modification, research on reversible histone methylation is still emerging, especially with regard to drug discovery. As aberrant epigenetic modifications have been linked to many diseases, inhibitors of histone modifying enzymes are very much in demand. This article will summarize the progress on small molecule epigenetic inhibitors identified by structure- and computer-based approaches.

Some Trends in Chem(o)informatics

This introductory chapter gives a brief overview of the history of cheminformatics, and then summarizes some recent trends in computing, cultures, open systems, chemical structure representation, docking, de novo design, fragment-based drug design, molecular similarity, quantitative structure-activity relationships (QSAR), metabolite prediction, the use of phamacophores in drug discovery, data reduction and visualization, and text mining. The aim is to set the scene for the more detailed exposition of these topics in the later chapters.

Comparison of several molecular docking programs: pose prediction and virtual screening accuracy

Molecular docking programs are widely used modeling tools for predicting ligand binding modes and structure based virtual screening. In this study, six molecular docking programs (DOCK, FlexX, GLIDE, ICM, PhDOCK, and Surflex) were evaluated using metrics intended to assess docking pose and virtual screening accuracy. Cognate ligand docking to 68 diverse, high-resolution X-ray complexes revealed that ICM, GLIDE, and Surflex generated ligand poses close to the X-ray conformation more often than the other docking programs. GLIDE and Surflex also outperformed the other docking programs when used for virtual screening, based on mean ROC AUC and ROC enrichment values obtained for the 40 protein targets in the Directory of Useful Decoys (DUD). Further analysis uncovered general trends in accuracy that are specific for particular protein families. Modifying basic parameters in the software was shown to have a significant effect on docking and virtual screening results, suggesting that expert knowledge is critical for optimizing the accuracy of these methods.

Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data

Refined nearest neighbor analysis was recently introduced for the analysis of virtual screening benchmark data sets. It constitutes a technique from the field of spatial statistics and provides a mathematical framework for the nonparametric analysis of mapped point patterns. Here, refined nearest neighbor analysis is used to design benchmark data sets for virtual screening based on PubChem bioactivity data. A workflow is devised that purges data sets of compounds active against pharmaceutically relevant targets from unselective hits. Topological optimization using experimental design strategies monitored by refined nearest neighbor analysis functions is applied to generate corresponding data sets of actives and decoys that are unbiased with regard to analogue bias and artificial enrichment. These data sets provide a tool for Maximum Unbiased Validation (MUV) of virtual screening methods. The data sets and a software package implementing the MUV design workflow are freely available at http://www.pharmchem.tu-bs.de/lehre/baumann/MUV.html.

Effects of protein conformation in docking: improved pose prediction through protein pocket adaptation

Computational methods for docking ligands have been shown to be remarkably dependent on precise protein conformation, where acceptable results in pose prediction have been generally possible only in the artificial case of re-docking a ligand into a protein binding site whose conformation was determined in the presence of the same ligand (the "cognate" docking problem). In such cases, on well curated protein/ligand complexes, accurate dockings can be returned as top-scoring over 75% of the time using tools such as Surflex-Dock. A critical application of docking in modeling for lead optimization requires accurate pose prediction for novel ligands, ranging from simple synthetic analogs to very different molecular scaffolds. Typical results for widely used programs in the "cross-docking case" (making use of a single fixed protein conformation) have rates closer to 20% success. By making use of protein conformations from multiple complexes, Surflex-Dock yields an average success rate of 61% across eight pharmaceutically relevant targets. Following docking, protein pocket adaptation and rescoring identifies single pose families that are correct an average of 67% of the time. Consideration of the best of two pose families (from alternate scoring regimes) yields a 75% mean success rate.

Modelling beta-1,3-exoglucanase-saccharide interactions: structure of the enzyme-substrate complex and enzyme binding to the cell wall

Glycoside hydrolases are a class of enzymes that break/form the bond between sugar monomers (monosaccharides). Candida albicans's beta-1,3-exoglucanase (Exg), a family 5 glycosidase, belongs to this class of enzymes. This small protein is an ideal computational model for its family of enzymes and was used here to create several enzyme-substrate models starting from a crystallographic glucanase-inhibitor structure. A series of enzyme-substrate complexes were generated using molecular docking, ranging from Exg-glucose (Exg-1Glc) to Exg-laminarihexaose (Exg-6Glc). Structure optimizations followed by molecular dynamics provided a picture of the way the enzyme and substrates interact. Molecular dynamics was conducted for each complex to assess the flexibility of the substrate, of the enzyme as a whole, and of enzyme-substrate interactions. The enzyme overall conformation was found to be quite rigid, although most enzyme residues increase mobility upon substrate binding. However, two surface loops stand out by having large fluctuations and becoming less flexible when the substrates were bound. These data point to a possible biological role for the mentioned loops, corresponding to amino acids 36-47 and 101-106. We propose that these loops could bind the enzyme to a glucan chain in the cell wall. The polysaccharide and enzyme structures have very complementary shapes and form numerous interactions; so it appears likely that the flexible loops connect the enzyme to the cell wall and allow it to navigate the wall to shape glucan structure.

Recommendations for evaluation of computational methods

The field of computational chemistry, particularly as applied to drug design, has become increasingly important in terms of the practical application of predictive modeling to pharmaceutical research and development. Tools for exploiting protein structures or sets of ligands known to bind particular targets can be used for binding-mode prediction, virtual screening, and prediction of activity. A serious weakness within the field is a lack of standards with respect to quantitative evaluation of methods, data set preparation, and data set sharing. Our goal should be to report new methods or comparative evaluations of methods in a manner that supports decision making for practical applications. Here we propose a modest beginning, with recommendations for requirements on statistical reporting, requirements for data sharing, and best practices for benchmark preparation and usage.