Buccaneer model building with neural network fragment selection

Tracing the backbone is a critical step in protein model building, as incorrect tracing leads to poor protein models. Here, a neural network trained to identify unfavourable fragments and remove them from the model-building process in order to improve backbone tracing is presented. Moreover, a decision tree was trained to select an optimal threshold to eliminate unfavourable fragments. The neural network was tested on experimental phasing data sets from the Joint Center for Structural Genomics (JCSG), recently deposited experimental phasing data sets (from 2015 to 2021) and molecular-replacement data sets. The experimental results show that using the neural network in the Buccaneer protein-model-building software can produce significantly more complete protein models than those built using Buccaneer alone. In particular, Buccaneer with the neural network built protein models with a completeness that was at least 5% higher for 25% and 50% of the original and truncated resolution JCSG experimental phasing data sets, respectively, for 28% of the recently collected experimental phasing data sets and for 43% of the molecular-replacement data sets.

CCP4 Cloud for structure determination and project management in macromolecular crystallography

Nowadays, progress in the determination of three-dimensional macromolecular structures from diffraction images is achieved partly at the cost of increasing data volumes. This is due to the deployment of modern high-speed, high-resolution detectors, the increased complexity and variety of crystallographic software, the use of extensive databases and high-performance computing. This limits what can be accomplished with personal, offline, computing equipment in terms of both productivity and maintainability. There is also an issue of long-term data maintenance and availability of structure-solution projects as the links between experimental observations and the final results deposited in the PDB. In this article, CCP4 Cloud, a new front-end of the CCP4 software suite, is presented which mitigates these effects by providing an online, cloud-based environment for crystallographic computation. CCP4 Cloud was developed for the efficient delivery of computing power, database services and seamless integration with web resources. It provides a rich graphical user interface that allows project sharing and long-term storage for structure-solution projects, and can be linked to data-producing facilities. The system is distributed with the CCP4 software suite version 7.1 and higher, and an online publicly available instance of CCP4 Cloud is provided by CCP4.

IceBreaker: Software for high-resolution single-particle cryo-EM with non-uniform ice

Despite the abundance of available software tools, optimal particle selection is still a vital issue in single-particle cryoelectron microscopy (cryo-EM). Regardless of the method used, most pickers struggle when ice thickness varies on a micrograph. IceBreaker allows users to estimate the relative ice gradient and flatten it by equalizing the local contrast. It allows the differentiation of particles from the background and improves overall particle picking performance. Furthermore, we introduce an additional parameter corresponding to local ice thickness for each particle. Particles with a defined ice thickness can be grouped and filtered based on this parameter during processing. These functionalities are especially valuable for on-the-fly processing to automatically pick as many particles as possible from each micrograph and to select optimal regions for data collection. Finally, estimated ice gradient distributions can be stored separately and used to inspect the quality of prepared samples.

Atomic model validation using the CCP-EM software suite

Recently, there has been a dramatic improvement in the quality and quantity of data derived using cryogenic electron microscopy (cryo-EM). This is also associated with a large increase in the number of atomic models built. Although the best resolutions that are achievable are improving, often the local resolution is variable, and a significant majority of data are still resolved at resolutions worse than 3 Å. Model building and refinement is often challenging at these resolutions, and hence atomic model validation becomes even more crucial to identify less reliable regions of the model. Here, a graphical user interface for atomic model validation, implemented in the CCP-EM software suite, is presented. It is aimed to develop this into a platform where users can access multiple complementary validation metrics that work across a range of resolutions and obtain a summary of evaluations. Based on the validation estimates from atomic models associated with cryo-EM structures from SARS-CoV-2, it was observed that models typically favor adopting the most common conformations over fitting the observations when compared with the model agreement with data. At low resolutions, the stereochemical quality may be favored over data fit, but care should be taken to ensure that the model agrees with the data in terms of resolvable features. It is demonstrated that further re-refinement can lead to improvement of the agreement with data without the loss of geometric quality. This also highlights the need for improved resolution-dependent weight optimization in model refinement and an effective test for overfitting that would help to guide the refinement process.

Predicting the performance of automated crystallographic model-building pipelines

Proteins are macromolecules that perform essential biological functions which depend on their three-dimensional structure. Determining this structure involves complex laboratory and computational work. For the computational work, multiple software pipelines have been developed to build models of the protein structure from crystallographic data. Each of these pipelines performs differently depending on the characteristics of the electron-density map received as input. Identifying the best pipeline to use for a protein structure is difficult, as the pipeline performance differs significantly from one protein structure to another. As such, researchers often select pipelines that do not produce the best possible protein models from the available data. Here, a software tool is introduced which predicts key quality measures of the protein structures that a range of pipelines would generate if supplied with a given crystallographic data set. These measures are crystallographic quality-of-fit indicators based on included and withheld observations, and structure completeness. Extensive experiments carried out using over 2500 data sets show that the tool yields accurate predictions for both experimental phasing data sets (at resolutions between 1.2 and 4.0 Å) and molecular-replacement data sets (at resolutions between 1.0 and 3.5 Å). The tool can therefore provide a recommendation to the user concerning the pipelines that should be run in order to proceed most efficiently to a depositable model.

Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge

This paper describes outcomes of the 2019 Cryo-EM Model Challenge. The goals were to (1) assess the quality of models that can be produced from cryogenic electron microscopy (cryo-EM) maps using current modeling software, (2) evaluate reproducibility of modeling results from different software developers and users and (3) compare performance of current metrics used for model evaluation, particularly Fit-to-Map metrics, with focus on near-atomic resolution. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived by 13 participating teams from four benchmark maps, including three forming a resolution series (1.8 to 3.1 Å). The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual experiments and structure data archives such as the Protein Data Bank. We recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed cryo-EM map density.

Shift-field refinement of macromolecular atomic models

The aim of crystallographic structure solution is typically to determine an atomic model which accurately accounts for an observed diffraction pattern. A key step in this process is the refinement of the parameters of an initial model, which is most often determined by molecular replacement using another structure which is broadly similar to the structure of interest. In macromolecular crystallography, the resolution of the data is typically insufficient to determine the positional and uncertainty parameters for each individual atom, and so stereochemical information is used to supplement the observational data. Here, a new approach to refinement is evaluated in which a `shift field' is determined which describes changes to model parameters affecting whole regions of the model rather than individual atoms only, with the size of the affected region being a key parameter of the calculation which can be changed in accordance with the resolution of the data. It is demonstrated that this approach can improve the radius of convergence of the refinement calculation while also dramatically reducing the calculation time.

Structural barriers to scientific progress

Structural biases, which are intrinsic in the social structures in which we function, play a key role in maintaining boundaries between traditionally privileged and underprivileged groups; however, they are particularly difficult to identify from within those societies. Two instances are highlighted in which the social structures of science appear to have discouraged collaboration, to the disadvantage of software and data users. Possible links are suggested to the strongly hierarchical structure of science and other factors which may in turn also serve to maintain sex and/or gender disparities in participation in the scientific endeavour.

Pairwise running of automated crystallographic model-building pipelines

For the last two decades, researchers have worked independently to automate protein model building, and four widely used software pipelines have been developed for this purpose: ARP/wARP, Buccaneer, Phenix AutoBuild and SHELXE. Here, the usefulness of combining these pipelines to improve the built protein structures by running them in pairwise combinations is examined. The results show that integrating these pipelines can lead to significant improvements in structure completeness and Rfree. In particular, running Phenix AutoBuild after Buccaneer improved structure completeness for 29% and 75% of the data sets that were examined at the original resolution and at a simulated lower resolution, respectively, compared with running Phenix AutoBuild on its own. In contrast, Phenix AutoBuild alone produced better structure completeness than the two pipelines combined for only 7% and 3% of these data sets.

Current approaches for automated model building into cryo-EM maps using Buccaneer with CCP-EM

This work focuses on the use of the existing protein-model-building software Buccaneer to provide structural interpretation of electron cryo-microscopy (cryo-EM) maps. Originally developed for application to X-ray crystallography, the necessary steps to optimise the usage of Buccaneer with cryo-EM maps are shown. This approach has been applied to the data sets of 208 cryo-EM maps with resolutions of better than 4 Å. The results obtained also show an evident improvement in the sequencing step when the initial reference map and model used for crystallographic cases are replaced by a cryo-EM reference. All other necessary changes to settings in Buccaneer are implemented in the model-building pipeline from within the CCP-EM interface (as of version 1.4.0).

Comparison of automated crystallographic model-building pipelines

A comparison of four protein model-building pipelines (ARP/wARP, Buccaneer, PHENIX AutoBuild and SHELXE) was performed using data sets from 202 experimentally phased cases, both with the data as observed and truncated to simulate lower resolutions. All pipelines were run using default parameters. Additionally, an ARP/wARP run was completed using models from Buccaneer. All pipelines achieved nearly complete protein structures and low R_work/R_free at resolutions between 1.2 and 1.9 Å, with PHENIX AutoBuild and ARP/wARP producing slightly lower R factors. At lower resolutions, Buccaneer leads to significantly more complete models.

Macromolecular refinement by model morphing using non-atomic parameterizations

Refinement is a critical step in the determination of a model which explains the crystallographic observations and thus best accounts for the missing phase components. The scattering density is usually described in terms of atomic parameters; however, in macromolecular crystallography the resolution of the data is generally insufficient to determine the values of these parameters for individual atoms. Stereochemical and geometric restraints are used to provide additional information, but produce interrelationships between parameters which slow convergence, resulting in longer refinement times. An alternative approach is proposed in which parameters are not attached to atoms, but to regions of the electron-density map. These parameters can move the density or change the local temperature factor to better explain the structure factors. Varying the size of the region which determines the parameters at a particular position in the map allows the method to be applied at different resolutions without the use of restraints. Potential applications include initial refinement of molecular-replacement models with domain motions, and potentially the use of electron density from other sources such as electron cryo-microscopy (cryo-EM) as the refinement model.

CCP4i2: the new graphical user interface to the CCP4 program suite

The CCP4 (Collaborative Computational Project, Number 4) software suite for macromolecular structure determination by X-ray crystallography groups brings together many programs and libraries that, by means of well established conventions, interoperate effectively without adhering to strict design guidelines. Because of this inherent flexibility, users are often presented with diverse, even divergent, choices for solving every type of problem. Recently, CCP4 introduced CCP4i2, a modern graphical interface designed to help structural biologists to navigate the process of structure determination, with an emphasis on pipelining and the streamlined presentation of results. In addition, CCP4i2 provides a framework for writing structure-solution scripts that can be built up incrementally to create increasingly automatic procedures.

Automating tasks in protein structure determination with the clipper python module

Scripting programming languages provide the fastest means of prototyping complex functionality. Those with a syntax and grammar resembling human language also greatly enhance the maintainability of the produced source code. Furthermore, the combination of a powerful, machine-independent scripting language with binary libraries tailored for each computer architecture allows programs to break free from the tight boundaries of efficiency traditionally associated with scripts. In the present work, we describe how an efficient C++ crystallographic library such as Clipper can be wrapped, adapted and generalized for use in both crystallographic and electron cryo-microscopy applications, scripted with the Python language. We shall also place an emphasis on best practices in automation, illustrating how this can be achieved with this new Python module.

Assessing recent warming using instrumentally homogeneous sea surface temperature records

Sea surface temperature (SST) records are subject to potential biases due to changing instrumentation and measurement practices. Significant differences exist between commonly used composite SST reconstructions from the National Oceanic and Atmospheric Administration's Extended Reconstruction Sea Surface Temperature (ERSST), the Hadley Centre SST data set (HadSST3), and the Japanese Meteorological Agency's Centennial Observation-Based Estimates of SSTs (COBE-SST) from 2003 to the present. The update from ERSST version 3b to version 4 resulted in an increase in the operational SST trend estimate during the last 19 years from 0.07° to 0.12°C per decade, indicating a higher rate of warming in recent years. We show that ERSST version 4 trends generally agree with largely independent, near-global, and instrumentally homogeneous SST measurements from floating buoys, Argo floats, and radiometer-based satellite measurements that have been developed and deployed during the past two decades. We find a large cooling bias in ERSST version 3b and smaller but significant cooling biases in HadSST3 and COBE-SST from 2003 to the present, with respect to most series examined. These results suggest that reported rates of SST warming in recent years have been underestimated in these three data sets.

Automated nucleic acid chain tracing in real time

The crystallographic structure solution of nucleotides and nucleotide complexes is now commonplace. The resulting electron-density maps are often poorer than for proteins, and as a result interpretation in terms of an atomic model can require significant effort, particularly in the case of large structures. While model building can be performed automatically, as with proteins, the process is time-consuming, taking minutes to days depending on the software and the size of the structure. A method is presented for the automatic building of nucleotide chains into electron density which is fast enough to be used in interactive model-building software, with extended chain fragments built around the current view position in a fraction of a second. The speed of the method arises from the determination of the 'fingerprint' of the sugar and phosphate groups in terms of conserved high-density and low-density features, coupled with a highly efficient scoring algorithm. Use cases include the rapid evaluation of an initial electron-density map, addition of nucleotide fragments to prebuilt protein structures, and in favourable cases the completion of the structure while automated model-building software is still running. The method has been incorporated into the Coot software package.

Validation of carbohydrate structures: not just nomenclature

Despite the key implications carbohydrates have in a multitude of pathological processes, a large number of the sugar-containing structures deposited into the Protein Data Bank (PDB) show nomenclature errors [1] that persist even after the remediation of the PDB archive [2]. Here we present the results from a systematic study of the conformation and ring distortion of cyclic carbohydrate models for which structure factors have been deposited into the PDB. These models have also been scored using a real-space correlation coefficient calculated between model and experimental electron density. The results have enabled us to produce a database of well-refined carbohydrate structures for use in the framework of an automated sugar-detecting software, to be announced shortly.

Completion of autobuilt protein models using a database of protein fragments

Two developments in the process of automated protein model building in the Buccaneer software are described: the use of a database of protein fragments in improving the model completeness and the assembly of disconnected chain fragments into complete molecules.

Two developments in the process of automated protein model building in the Buccaneer software are presented. A general-purpose library for protein fragments of arbitrary size is described, with a highly optimized search method allowing the use of a larger database than in previous work. The problem of assembling an autobuilt model into complete chains is discussed. This involves the assembly of disconnected chain fragments into complete molecules and the use of the database of protein fragments in improving the model completeness. Assembly of fragments into molecules is a standard step in existing model-building software, but the methods have not received detailed discussion in the literature.

From crystal to structure with CCP4

An introduction to the proceedings of the CCP4 study weekend is given.

Recent developments in classical density modification

Classical density-modification techniques (as opposed to statistical approaches) offer a computationally cheap method for improving phase estimates in order to provide a good electron-density map for model building. The rise of statistical methods has lead to a shift in focus away from the classical approaches; as a result, some recent developments have not made their way into classical density-modification software. This paper describes the application of some recent techniques, including most importantly the use of prior phase information in the likelihood estimation of phase errors within a classical density-modification framework. The resulting software gives significantly better results than comparable classical methods, while remaining nearly two orders of magnitude faster than statistical methods.

Features and development of Coot

Coot is a molecular-graphics program designed to assist in the building of protein and other macromolecular models. The current state of development and available features are presented.

Coot is a molecular-graphics application for model building and validation of biological macromolecules. The program displays electron-density maps and atomic models and allows model manipulations such as idealization, real-space refinement, manual rotation/translation, rigid-body fitting, ligand search, solvation, mutations, rotamers and Ramachandran idealization. Furthermore, tools are provided for model validation as well as interfaces to external programs for refinement, validation and graphics. The software is designed to be easy to learn for novice users, which is achieved by ensuring that tools for common tasks are ‘discoverable’ through familiar user-interface elements (menus and toolbars) or by intuitive behaviour (mouse controls). Recent developments have focused on providing tools for expert users, with customisable key bindings, extensions and an extensive scripting interface. The software is under rapid development, but has already achieved very widespread use within the crystallographic community. The current state of the software is presented, with a description of the facilities available and of some of the underlying methods employed.

Automated classification of crystallization experiments using wavelets and statistical texture characterization techniques

A method is presented for the classification of protein crystallization images based on image decomposition using the wavelet transform. The distribution of wavelet coefficient values in each sub-band image is modelled by a generalized Gaussian distribution to provide discriminatory variables. These statistical descriptors, together with second-order statistics obtained from joint probability distributions, are used with learning vector quantization to classify protein crystallization images.

Fitting molecular fragments into electron density

A number of techniques for the location of small and medium-sized model fragments in experimentally phased electron-density maps are explored. The application of one of these techniques to automated model building is discussed.

Molecular replacement is a powerful tool for the location of large models using structure-factor magnitudes alone. When phase information is available, it becomes possible to locate smaller fragments of the structure ranging in size from a few atoms to a single domain. The calculation is demanding, requiring a six-dimensional rotation and translation search. A number of approaches have been developed to this problem and a selection of these are reviewed in this paper. The application of one of these techniques to the problem of automated model building is explored in more detail, with particular reference to the problem of sequencing a protein main-chain trace.

The Buccaneer software for automated model building. 1. Tracing protein chains

A new technique for the automated tracing of protein chains in experimental electron-density maps is described. The technique relies on the repeated application of an oriented electron-density likelihood target function to identify likely C(alpha) positions. This function is applied both in the location of a few promising ;seed' positions in the map and to grow those initial C(alpha) positions into extended chain fragments. Techniques for assembling the chain fragments into an initial chain trace are discussed.

Likelihood weighting of partial structure factors using spline coefficients

A method for the weighting of structure factors from an incomplete and inaccurate model is described which relies on the fitting of smooth spline functions of resolution. The use of smooth spline functions avoids the problems of discontinuities introduced when performing calculations in resolution shells. The complexity of the functions to be fit may be varied by changing the number of spline parameters. This approach is used to investigate the stability of the problem when data are limited.

Coot: model-building tools for molecular graphics

CCP4mg is a project that aims to provide a general-purpose tool for structural biologists, providing tools for X-ray structure solution, structure comparison and analysis, and publication-quality graphics. The map-fitting tools are available as a stand-alone package, distributed as 'Coot'.

Developments in the CCP4 molecular-graphics project

Progress towards structure determination that is both high-throughput and high-value is dependent on the development of integrated and automatic tools for electron-density map interpretation and for the analysis of the resulting atomic models. Advances in map-interpretation algorithms are extending the resolution regime in which fully automatic tools can work reliably, but at present human intervention is required to interpret poor regions of macromolecular electron density, particularly where crystallographic data is only available to modest resolution [for example, I/sigma(I) < 2.0 for minimum resolution 2.5 A]. In such cases, a set of manual and semi-manual model-building molecular-graphics tools is needed. At the same time, converting the knowledge encapsulated in a molecular structure into understanding is dependent upon visualization tools, which must be able to communicate that understanding to others by means of both static and dynamic representations. CCP4 mg is a program designed to meet these needs in a way that is closely integrated with the ongoing development of CCP4 as a program suite suitable for both low- and high-intervention computational structural biology. As well as providing a carefully designed user interface to advanced algorithms of model building and analysis, CCP4 mg is intended to present a graphical toolkit to developers of novel algorithms in these fields.

Generic representation and evaluation of properties as a function of position in reciprocal space

A generalized approach is described for evaluating arbitrary functions of position in reciprocal space. This is a generalization which subsumes a whole range of calculations that form a part of almost every crystallographic software application. Examples include scaling of structure factors, the calculation of structure-factor statistics, and some simple likelihood calculations for a single parameter. The generalized approach has a number of advantages: all these calculations may now be performed by a single software routine which need only be debugged and optimized once; the existing approach of dividing reciprocal space into resolution shells with discontinuities at the boundaries is no longer necessary; the implementation provided makes employing the new functionality extremely simple and concise. The calculation is split into three standard components, for which a number of implementations are provided for different tasks. A `basis function' describes some function of position in reciprocal space, the shape of which is determined by a small number of parameters. A `target function' describes the property for which a functional representation is required, for example \langle |F|^2\rangle. An `evaluator' takes a basis and target function and optimizes the parameters of the basis function to fit the target function. Ideally the components should be usable in any combination.

The CCP4 molecular-graphics project

This new package will provide easy-to-use access to crystallographic structure solution, model building and structure analysis. It will be possible for any developer to integrate scientific software into the system.

Fast Fourier feature recognition

Various approaches have been demonstrated for the automatic interpretation of crystallographic data in terms of atomic models. The use of a masked Fourier-based search function has some benefits for this task. The application and optimization of this procedure is discussed in detail. The search function also acquires a statistical significance when used with an appropriate electron-density target and weighting, giving rise to improved results at low resolutions. Methods are discussed for building a library of protein fragments suitable for use with this procedure. These methods are demonstrated with the construction of a statistical target for the identification of short helical fragments in the electron density.

General quadratic functions in real and reciprocal space and their application to likelihood phasing

A general multivariate quadratic function of the structure factors is constructed and transformed to obtain a quadratic function of the continuous electron density. Two special cases, where structure factors are independent and where electron-density values are independent, are examined. These results are related to the new likelihood-based framework of Terwilliger [Terwilliger (1999), Acta Cryst. D55, pp. 1863-1871] for employing structural information which was previously exploited by means of conventional density-modification calculations. The treatment here involves different assumptions and highlights new features of Terwilliger's calculation. The generalization quadratic construction allows the generation of cross terms relating all reflections and electron densities. Other applications of this approach are considered.

Eigensystem analysis of the refinement of a small metalloprotein

The eigenvalues and eigenvectors of the least-squares normal matrix for the full-matrix refinement problem contain a great deal of information about the quality of a model; in particular the precision of the model parameters and correlations between those parameters. They also allow the isolation of those parameters or combinations of parameters which are not determined by the available data. Since a protein refinement is usually under-determined without the application of geometric restraints, such indicators of the reliability of a model offer an important contribution to structural knowledge. Eigensystem analysis is applied to the normal matrices for the refinement of a small metalloprotein using two data sets and models determined at different resolutions. The eigenvalue spectra reveal considerable information about the conditioning of the problem as the resolution varies. In the case of a restrained refinement, it also provides information about the impact of various restraints on the refinement. Initial results support conclusions drawn from the free R factor. Examination of the eigenvectors provides information about which regions of the model are poorly determined. In the case of a restrained refinement, it is also possible to isolate places where X-ray and geometric restraints are in disagreement, usually indicating a problem in the model.

Error estimation and bias correction in phase-improvement calculations

With the rise of Bayesian methods in crystallography, the error estimates attached to estimated phases are becoming as important as the phase estimates themselves. Phase improvement by density modification can cause problems in this environment because the quality of the resulting phases is usually overestimated. This problem is addressed by an extension of the gamma correction [Abrahams (1997). Acta Cryst. D53, 371-376] to arbitrary density-modification techniques. The degree to which the improved phases are biased by the features of the initial map is investigated in order to determine the limits of the resulting procedure and the quality of the phase-error estimates.

Modified phased translation functions and their application to molecular-fragment location

Direct methods at high resolution have depended on the resolution of atomic like features in the map. At data resolutions more typical for protein structures (2-3 A) individual atoms may not be resolved, so larger features must be identified. At one extreme the whole molecule may be located using the diffraction magnitudes alone by the molecular-replacement method. At the other extreme it is possible to locate individual residues in a well phased map. In this paper an intermediate problem is addressed: the location of multi-residue fragments on the basis of weak phase information. An agreement function based on the mean-squared difference between model and map over a masked region is shown to be more effective than a simple overlap integral, and may be efficiently calculated by Fourier methods. The techniques are compared using poorly phased electron-density maps at approximately 3 A for the proteins RNAse and O6-methylguanine-DNA-methyltransferase.

Miscellaneous algorithms for density modification

Various algorithms are described, developed for the dm density modification package, which have not been described elsewhere. Methods are described for the following problems: determination of the absolute scale and overall temperature factor of a data set, by a method which is less dependent on data resolution than Wilson statistics; an efficient interpolation algorithm for averaging and its application to refinement of averaging operators; a method for the automatic determination of averaging masks.