Evaluation of AlphaFold-Multimer prediction on multi-chain protein complexes

MOTIVATION

Despite near-experimental accuracy on single-chain predictions, there is still scope for improvement among multimeric predictions. Methods like AlphaFold-Multimer and FoldDock can accurately model dimers. However, how well these methods fare on larger complexes is still unclear. Further, evaluation methods of the quality of multimeric complexes are not well established.

RESULTS

We analysed the performance of AlphaFold-Multimer on a homology-reduced dataset of homo- and heteromeric protein complexes. We highlight the differences between the pairwise and multi-interface evaluation of chains within a multimer. We describe why certain complexes perform well on one metric (e.g. TM-score) but poorly on another (e.g. DockQ). We propose a new score, Predicted DockQ version 2 (pDockQ2), to estimate the quality of each interface in a multimer. Finally, we modelled protein complexes (from CORUM) and identified two highly confident structures that do not have sequence homology to any existing structures.

AVAILABILITY AND IMPLEMENTATION

All scripts, models, and data used to perform the analysis in this study are freely available at https://gitlab.com/ElofssonLab/afm-benchmark.

A structural biology community assessment of AlphaFold2 applications

Most proteins fold into 3D structures that determine how they function and orchestrate the biological processes of the cell. Recent developments in computational methods for protein structure predictions have reached the accuracy of experimentally determined models. Although this has been independently verified, the implementation of these methods across structural-biology applications remains to be tested. Here, we evaluate the use of AlphaFold2 (AF2) predictions in the study of characteristic structural elements; the impact of missense variants; function and ligand binding site predictions; modeling of interactions; and modeling of experimental structural data. For 11 proteomes, an average of 25% additional residues can be confidently modeled when compared with homology modeling, identifying structural features rarely seen in the Protein Data Bank. AF2-based predictions of protein disorder and complexes surpass dedicated tools, and AF2 models can be used across diverse applications equally well compared with experimentally determined structures, when the confidence metrics are critically considered. In summary, we find that these advances are likely to have a transformative impact in structural biology and broader life-science research.

Predicting the structure of large protein complexes using AlphaFold and Monte Carlo tree search

AlphaFold can predict the structure of single- and multiple-chain proteins with very high accuracy. However, the accuracy decreases with the number of chains, and the available GPU memory limits the size of protein complexes which can be predicted. Here we show that one can predict the structure of large complexes starting from predictions of subcomponents. We assemble 91 out of 175 complexes with 10–30 chains from predicted subcomponents using Monte Carlo tree search, with a median TM-score of 0.51. There are 30 highly accurate complexes (TM-score ≥0.8, 33% of complete assemblies). We create a scoring function, mpDockQ, that can distinguish if assemblies are complete and predict their accuracy. We find that complexes containing symmetry are accurately assembled, while asymmetrical complexes remain challenging. The method is freely available and accesible as a Colab notebook https://colab.research.google.com/github/patrickbryant1/MoLPC/blob/master/MoLPC.ipynb.

The accuracy of AlphaFold decreases with the number of protein chains and the available GPU memory limits the size of protein complexes that can be predicted. Here, the authors show that complexes with 10–30 chains can be assembled from predicted subcomponents using Monte Carlo tree search.

Scoring of protein–protein docking models utilizing predicted interface residues

Scoring docking solutions is a difficult task, and many methods have been developed for this purpose. In docking, only a handful of the hundreds of thousands of models generated by docking algorithms are acceptable, causing difficulties when developing scoring functions. Today's best scoring functions can significantly increase the number of top‐ranked models but still fail for most targets. Here, we examine the possibility of utilizing predicted interface residues to score docking models generated during the scan stage of a docking algorithm. Many methods have been developed to infer the regions of a protein surface that interact with another protein, but most have not been benchmarked using docking algorithms. This study systematically tests different interface prediction methods for scoring >300.000 low‐resolution rigid‐body template free docking decoys. Overall we find that contact‐based interface prediction by BIPSPI is the best method to score docking solutions, with >12% of first ranked docking models being acceptable. Additional experiments indicated precision as a high‐importance metric when estimating interface prediction quality, focusing on docking constraints production. Finally, we discussed several limitations for adopting interface predictions as constraints in a docking protocol.

Limits and potential of combined folding and docking

MOTIVATION

In the last decade, de novo protein structure prediction accuracy for individual proteins has improved significantly by utilising deep learning (DL) methods for harvesting the co-evolution information from large multiple sequence alignments (MSAs). The same approach can, in principle, also be used to extract information about evolutionary-based contacts across protein-protein interfaces. However, most earlier studies have not used the latest DL methods for inter-chain contact distance prediction. This article introduces a fold-and-dock method based on predicted residue-residue distances with trRosetta.

RESULTS

The method can simultaneously predict the tertiary and quaternary structure of a protein pair, even when the structures of the monomers are not known. The straightforward application of this method to a standard dataset for protein-protein docking yielded limited success. However, using alternative methods for generating MSAs allowed us to dock accurately significantly more proteins. We also introduced a novel scoring function, PconsDock, that accurately separates 98% of correctly and incorrectly folded and docked proteins. The average performance of the method is comparable to the use of traditional, template-based or ab initio shape-complementarity-only docking methods. Moreover, the results of conventional and fold-and-dock approaches are complementary, and thus a combined docking pipeline could increase overall docking success significantly. This methodology contributed to the best model for one of the CASP14 oligomeric targets, H1065.

AVAILABILITY AND IMPLEMENTATION

All scripts for predictions and analysis are available from https://github.com/ElofssonLab/bioinfo-toolbox/ and https://gitlab.com/ElofssonLab/benchmark5/. All models joined alignments, and evaluation results are available from the following figshare repository https://doi.org/10.6084/m9.figshare.14654886.v2.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

On the electrostatic component of protein-protein binding free energy

Calculations of electrostatic properties of protein-protein complexes are usually done within framework of a model with a certain set of parameters. In this paper we present a comprehensive statistical analysis of the sensitivity of the electrostatic component of binding free energy (DeltaDeltaGel) with respect with different force fields (Charmm, Amber, and OPLS), different values of the internal dielectric constant, and different presentations of molecular surface (different values of the probe radius). The study was done using the largest so far set of entries comprising 260 hetero and 2148 homo protein-protein complexes extracted from a previously developed database of protein complexes (ProtCom). To test the sensitivity of the energy calculations with respect to the structural details, all structures were energy minimized with corresponding force field, and the energies were recalculated. The results indicate that the absolute value of the electrostatic component of the binding free energy (DeltaDeltaGel) is very sensitive to the force field parameters, the minimization procedure, the values of the internal dielectric constant, and the probe radius. Nevertheless our results indicate that certain trends in DeltaDeltaGel behavior are much less sensitive to the calculation parameters. For instance, the fraction of the homo-complexes, for which the electrostatics was found to oppose binding, is 80% regardless of the force fields and parameters used. For the hetero-complexes, however, the percentage of the cases for which electrostatics opposed binding varied from 43% to 85%, depending on the protocol and parameters employed. A significant correlation was found between the effects caused by raising the internal dielectric constant and decreasing the probe radius. Correlations were also found among the results obtained with different force fields. However, despite of the correlations found, the absolute DeltaDeltaGel calculated with different force field parameters could differ more than tens of kcal/mol in some cases. Set of rules of obtaining confident predictions of absolute DeltaDeltaGel and DeltaDeltaGel sign are provided in the conclusion section.PACS codes: 87.15.A-, 87.15. km.

Assessing the quality of the homology-modeled 3D structures from electrostatic standpoint: test on bacterial nucleoside monophosphate kinase families

In this study, we address the issue of performing meaningful pK(a) calculations using homology modeled three-dimensional (3D) structures and analyze the possibility of using the calculated pK(a) values to detect structural defects in the models. For this purpose, the 3D structure of each member of five large protein families of a bacterial nucleoside monophosphate kinases (NMPK) have been modeled by means of homology-based approach. Further, we performed pK(a) calculations for the each model and for the template X-ray structures. Each bacterial NMPK family used in the study comprised on average 100 members providing a pool of sequences and 3D models large enough for reliable statistical analysis. It was shown that pK(a) values of titratable groups, which are highly conserved within a family, tend to be conserved among the models too. We demonstrated that homology modeled structures with sequence identity larger than 35% and gap percentile smaller than 10% can be used for meaningful pK(a) calculations. In addition, it was found that some highly conserved titratable groups either exhibit large pK(a) fluctuations among the models or have pK(a) values shifted by several pH units with respect to the pK(a) calculated for the X-ray structure. We demonstrated that such case usually indicates structural errors associated with the model. Thus, we argue that pK(a) calculations can be used for assessing the quality of the 3D models by monitoring fluctuations of the pK(a) values for highly conserved titratable residues within large sets of homologous proteins.

Optimization of electrostatic interactions in protein-protein complexes

In this article, we present a statistical analysis of the electrostatic properties of 298 protein-protein complexes and 356 domain-domain structures extracted from the previously developed database of protein complexes (ProtCom, http://www.ces.clemson.edu/compbio/protcom). For each structure in the dataset we calculated the total electrostatic energy of the binding and its two components, Coulombic and reaction field energy. It was found that in a vast majority of the cases (>90%), the total electrostatic component of the binding energy was unfavorable. At the same time, the Coulombic component of the binding energy was found to favor the complex formation while the reaction field component of the binding energy opposed the binding. It was also demonstrated that the components in a wild-type (WT) structure are optimized/anti-optimized with respect to the corresponding distributions, arising from random shuffling of the charged side chains. The degree of this optimization was assessed through the Z-score of WT energy in respect to the random distribution. It was found that the Z-scores of Coulombic interactions peak at a considerably negative value for all 654 cases considered while the Z-score of the reaction field energy varied among different types of complexes. All these findings indicate that the Coulombic interactions within WT protein-protein complexes are optimized to favor the complex formation while the total electrostatic energy predominantly opposes the binding. This observation was used to discriminate WT structures among sets of structural decoys and showed that the electrostatic component of the binding energy is not a good discriminator of the WT; while, Coulombic or reaction field energies perform better depending upon the decoy set used.

Predicting interacting and interfacial residues using continuous sequence segments

Development of sequence-based methods for predicting putative interfacial residues is an extremely important task in modeling 3D structures of protein-protein complexes. In the present paper we used non-gapped sequence segments to predict both interacting and interfacial residues. We demonstrated that continuous sequence segments do occur at the protein-protein interfaces and showed that continuous interacting interfacial segments (CIIS) of length nine are presented on average, in approximately 37% of the complexes in our dataset. Our results indicate that CIIS consist mostly of interacting strands and/or loops, while the CIIS involving the helixes are scarce. We performed scoring of CIIS using four different scoring mechanisms and found that scores of CIIS differ significantly from the scores calculated for random stretches of residues. We argue that such statistical difference inferred thought the corresponding Z-scores could be used for detecting putative interfacial residue segments without using any structural information. This hypothesis was tested on our dataset and benchmarking resulted to 10-60% prediction accuracy depending on type of benchmarking and scoring scheme used in calculations. Such predictions that do not depend on the availability of the 3D structures of monomers can be quite valuable in modeling 3D structures of obligatory complexes, for which structures of separated monomers do not exist.

BANMOKI: a searchable database of homology-based 3D models and their electrostatic properties of five bacterial nucleoside monophosphate kinase families

The nucleoside monophosphate kinases (NMPK) are important enzymes that control the ratio of mono- and di-phosphate nucleosides and participate in gene regulation and signal transduction in the cell. However, despite their importance only several 3D structures were experimentally determined in contrast to the wealth of sequences available for each of the NMPK families. To fill this gap we present a Web-based database containing structural models for all proteins of the five bacterial nucleoside monophosphate kinase (bNMPK) families. The models were computed by means of homology-based approach using a few experimentally determined bNMPK structures. The database also contains pK(a) values and their components calculated for the homology-based 3D models, which is a unique feature of the database. The BActerial Nucleoside MOnophosphate KInases (BANMOKI) database is freely accessible (http://www.ces.clemson.edu/compbio/banmoki) and offers an easy user-friendly interface for browsing, searching and downloading content of the database. The users can investigate, using the searching tools of the database, the properties of the bNMP kinases in respect to sequence composition, electrostatic interactions and structural differences.

Computation of the Ising partition function for two-dimensional square grids

An improved method for obtaining the Ising partition function of n x n square grids with periodic boundary is presented. Our method applies results from Galois theory in order to split the computation into smaller parts and at the same time avoid the use of numerics. Using this method we have computed the exact partition function for the (320 x 320) grid, the ( 256 x 256 ) grid, and the ( 160 x 160 ) grid, as well as for a number of smaller grids. We obtain scaling parameters and compare with what theory prescribes.