Determination of signal-to-noise ratios and spectral SNRs in cryo-EM low-dose imaging of molecules

Confronting heterogeneity in cryogenic electron microscopy data: Innovative strategies and future perspectives with data-driven methods

The surge in the influx of data from cryogenic electron microscopy (cryo-EM) experiments has intensified the demand for robust algorithms capable of autonomously managing structurally heterogeneous datasets. This presents a wealth of exciting opportunities from a data science viewpoint, inspiring the development of numerous innovative, application-specific methods, many of which leverage contemporary data-driven techniques. However, addressing the challenges posed by heterogeneous datasets remains a paramount yet unresolved issue in the field. Here, we explore the subtleties of this challenge and the array of strategies devised to confront it. We pinpoint the shortcomings of existing methodologies and deliberate on prospective avenues for improvement. Specifically, our discussion focuses on strategies to mitigate model overfitting and manage data noise, as well as the effects of constraints, priors, and invariances on the optimization process.

Zero-shot denoising of microscopy images recorded at high-resolution limits

Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation.

DiffraGAN: a conditional generative adversarial network for phasing single molecule diffraction data to atomic resolution

Introduction

Proteins that adopt multiple conformations pose significant challenges in structural biology research and pharmaceutical development, as structure determination via single particle cryo-electron microscopy (cryo-EM) is often impeded by data heterogeneity. In this context, the enhanced signal-to-noise ratio of single molecule cryo-electron diffraction (simED) offers a promising alternative. However, a significant challenge in diffraction methods is the loss of phase information, which is crucial for accurate structure determination.

Methods

Here, we present DiffraGAN, a conditional generative adversarial network (cGAN) that estimates the missing phases at high resolution from a combination of single particle high-resolution diffraction data and low-resolution image data.

Results

For simulated datasets, DiffraGAN allows effectively determining protein structures at atomic resolution from diffraction patterns and noisy low-resolution images.

Discussion

Our findings suggest that combining single particle cryo-electron diffraction with advanced generative modeling, as in DiffraGAN, could revolutionize the way protein structures are determined, offering an alternative and complementary approach to existing methods.

Cryo-EM images of phase-separated lipid bilayer vesicles analyzed with a machine-learning approach

Lateral lipid heterogeneity (i.e., raft formation) in biomembranes plays a functional role in living cells. Three-component mixtures of low- and high-melting lipids plus cholesterol offer a simplified experimental model for raft domains in which a liquid-disordered (Ld) phase coexists with a liquid-ordered (Lo) phase. Using such models, we recently showed that cryogenic electron microscopy (cryo-EM) can detect phase separation in lipid vesicles based on differences in bilayer thickness. However, the considerable noise within cryo-EM data poses a significant challenge for accurately determining the membrane phase state at high spatial resolution. To this end, we have developed an image-processing pipeline that utilizes machine learning (ML) to predict the bilayer phase in projection images of lipid vesicles. Importantly, the ML method exploits differences in both the thickness and molecular density of Lo compared to Ld, which leads to improved phase identification. To assess accuracy, we used artificial images of phase-separated lipid vesicles generated from all-atom molecular dynamics simulations of Lo and Ld phases. Synthetic ground-truth data sets mimicking a series of compositions along a tieline of Ld + Lo coexistence were created and then analyzed with various ML models. For all tieline compositions, we find that the ML approach can correctly identify the bilayer phase with >90% accuracy, thus providing a means to isolate the intensity profiles of coexisting Ld and Lo phases, as well as accurately determine domain-size distributions, number of domains, and phase-area fractions. The method described here provides a framework for characterizing nanoscopic lateral heterogeneities in membranes and paves the way for a more detailed understanding of raft properties in biological contexts.

Learning structural heterogeneity from cryo-electron sub-tomograms with tomoDRGN

Cryo-electron tomography (cryo-ET) enables observation of macromolecular complexes in their native, spatially contextualized cellular environment. Cryo-ET processing software to visualize such complexes at nanometer resolution via iterative alignment and averaging are well developed but rely upon assumptions of structural homogeneity among the complexes of interest. Recently developed tools allow for some assessment of structural diversity but have limited capacity to represent highly heterogeneous structures, including those undergoing continuous conformational changes. Here we extend the highly expressive cryoDRGN (Deep Reconstructing Generative Networks) deep learning architecture, originally created for single-particle cryo-electron microscopy analysis, to cryo-ET. Our new tool, tomoDRGN, learns a continuous low-dimensional representation of structural heterogeneity in cryo-ET datasets while also learning to reconstruct heterogeneous structural ensembles supported by the underlying data. Using simulated and experimental data, we describe and benchmark architectural choices within tomoDRGN that are uniquely necessitated and enabled by cryo-ET. We additionally illustrate tomoDRGN's efficacy in analyzing diverse datasets, using it to reveal high-level organization of human immunodeficiency virus (HIV) capsid complexes assembled in virus-like particles and to resolve extensive structural heterogeneity among ribosomes imaged in situ.

Quantitatively mapping local quality of super-resolution microscopy by rolling Fourier ring correlation

In fluorescence microscopy, computational algorithms have been developed to suppress noise, enhance contrast, and even enable super-resolution (SR). However, the local quality of the images may vary on multiple scales, and these differences can lead to misconceptions. Current mapping methods fail to finely estimate the local quality, challenging to associate the SR scale content. Here, we develop a rolling Fourier ring correlation (rFRC) method to evaluate the reconstruction uncertainties down to SR scale. To visually pinpoint regions with low reliability, a filtered rFRC is combined with a modified resolution-scaled error map (RSM), offering a comprehensive and concise map for further examination. We demonstrate their performances on various SR imaging modalities, and the resulting quantitative maps enable better SR images integrated from different reconstructions. Overall, we expect that our framework can become a routinely used tool for biologists in assessing their image datasets in general and inspire further advances in the rapidly developing field of computational imaging.

Robust ab initio solution of the cryo-EM reconstruction problem at low resolution with small data sets

Single particle cryo-electron microscopy has become a critical tool in structural biology over the last decade, able to achieve atomic scale resolution in three dimensional models from hundreds of thousands of (noisy) two-dimensional projection views of particles frozen at unknown orientations. This is accomplished by using a suite of software tools to (i) identify particles in large micrographs, (ii) obtain low-resolution reconstructions, (iii) refine those low-resolution structures, and (iv) finally match the obtained electron scattering density to the constituent atoms that make up the macromolecule or macromolecular complex of interest. Here, we focus on the second stage of the reconstruction pipeline: obtaining a low resolution model from picked particle images. Our goal is to create an algorithm that is capable of ab initio reconstruction from small data sets (on the order of a few thousand selected particles). More precisely, we propose an algorithm that is robust, automatic, and fast enough that it can potentially be used to assist in the assessment of particle quality as the data is being generated during the microscopy experiment.

M-Denoiser: Unsupervised image denoising for real-world optical and electron microscopy data

Real-world microscopy data have a large amount of noise due to the limited light/electron that can be used to capture images. The noise of microscopy data is composed of signal-dependent shot noise and signal-independent read noise, and the Poisson-Gaussian noise model is usually used to describe the noise distribution. Meanwhile, the noise is spatially correlated because of the data acquisition process. Due to the lack of clean ground truth, unsupervised and self-supervised denoising algorithms in computer vision shed new light on tackling such tasks by utilizing paired noisy images or one single noisy image. However, they usually make the assumption that the noise is signal-independent or pixel-wise independent, which contradicts with the actual case. Hence, we propose M-Denoiser for denoising real-world microscopy data in an unsupervised manner. Firstly, the shatter module is used to break the dependency and correlation before denoising. Secondly, a novelly designed unsupervised training loss based on a pair of noisy images is proposed for real-world microscopy data. For evaluation, we train our model on optical and electron microscopy datasets. The experimental results show that M-Denoiser achieves the best performance both quantitatively and qualitatively compared with all the baselines.

Learning structural heterogeneity from cryo-electron sub-tomograms with tomoDRGN

Cryo-electron tomography (cryo-ET) allows one to observe macromolecular complexes in their native, spatially contextualized environment. Tools to visualize such complexes at nanometer resolution via iterative alignment and averaging are well-developed but rely on assumptions of structural homogeneity among the complexes under consideration. Recently developed downstream analysis tools allow for some assessment of macromolecular diversity but have limited capacity to represent highly heterogeneous macromolecules, including those undergoing continuous conformational changes. Here, we extend the highly expressive cryoDRGN deep learning architecture, originally created for cryo-electron microscopy single particle analysis, to sub-tomograms. Our new tool, tomoDRGN, learns a continuous low-dimensional representation of structural heterogeneity in cryo-ET datasets while also learning to reconstruct a large, heterogeneous ensemble of structures supported by the underlying data. Using simulated and experimental data, we describe and benchmark architectural choices within tomoDRGN that are uniquely necessitated and enabled by cryo-ET data. We additionally illustrate tomoDRGN's efficacy in analyzing an exemplar dataset, using it to reveal extensive structural heterogeneity among ribosomes imaged in situ.

MEPSi: A tool for simulating tomograms of membrane-embedded proteins

The throughput and fidelity of cryogenic cellular electron tomography (cryo-ET) is constantly increasing through advances in cryogenic electron microscope hardware, direct electron detection devices, and powerful image processing algorithms. However, the need for careful optimization of sample preparations and for access to expensive, high-end equipment, make cryo-ET a costly and time-consuming technique. Generally, only after the last step of the cryo-ET workflow, when reconstructed tomograms are available, it becomes clear whether the chosen imaging parameters were suitable for a specific type of sample in order to answer a specific biological question. Tools for a-priory assessment of the feasibility of samples to answer biological questions and how to optimize imaging parameters to do so would be a major advantage. Here we describe MEPSi (Membrane Embedded Protein Simulator), a simulation tool aimed at rapid and convenient evaluation and optimization of cryo-ET data acquisition parameters for studies of transmembrane proteins in their native environment. We demonstrate the utility of MEPSi by showing how to detangle the influence of different data collection parameters and different orientations in respect to tilt axis and electron beam for two examples: (1) simulated plasma membranes with embedded single-pass transmembrane αIIbβ3 integrin receptors and (2) simulated virus membranes with embedded SARS-CoV-2 spike proteins.

MeasureIce: accessible on-the-fly measurement of ice thickness in cryo-electron microscopy

Ice thickness is arguably one of the most important factors limiting the resolution of protein structures determined by cryo-electron microscopy (cryo-EM). The amorphous atomic structure of the ice that stabilizes and protects biological samples in cryo-EM grids also imprints some additional noise in cryo-EM images. Ice that is too thick jeopardizes the success of particle picking and reconstruction of the biomolecule in the worst case and, at best, deteriorates eventual map resolution. Minimizing the thickness of the ice layer and thus the magnitude of its noise contribution is thus imperative in cryo-EM grid preparation. In this paper we introduce MeasureIce, a simple, easy to use ice thickness measurement tool for screening and selecting acquisition areas of cryo-EM grids. We show that it is possible to simulate thickness-image intensity look-up tables, also usable in SerialEM and Leginon, using elementary scattering physics and thereby adapt the tool to any microscope without time consuming experimental calibration. We benchmark our approach using two alternative techniques: the “ice channel” technique and tilt-series tomography. We also demonstrate the utility of ice thickness measurement for selecting holes in gold grids containing an Equine apoferritin sample, achieving a 1.88 Ångstrom resolution in subsequent refinement of the atomic map.

MeasureIce is an ice thickness measurement tool for screening and selecting acquisition areas of cryo-EM grids.

Gold nanomaterials and their potential use as cryo-electron tomography labels

Rapid advances in cryo-electron tomography (cryo-ET) are driving a revolution in cellular structural biology. However, unambiguous identification of specific biomolecules within cellular tomograms remains challenging. Overcoming this obstacle and reliably identifying targets in the crowded cellular environment is of major importance for the understanding of cellular function and is a pre-requisite for high-resolution structural analysis. The use of highly-specific, readily visualised and adjustable labels would help mitigate this issue, improving both data quality and sample throughput. While progress has been made in cryo-CLEM and in the development of cloneable high-density tags, technical issues persist and a robust 'cryo-GFP' remains elusive. Readily-synthesized gold nanomaterials conjugated to small 'affinity modules' may represent a solution. The synthesis of materials including gold nanoclusters (AuNCs) and gold nanoparticles (AuNPs) is increasingly well understood and is now within the capabilities of non-specialist laboratories. The remarkable chemical and photophysical properties of <3nm diameter nanomaterials and their emergence as tools with widespread biomedical application presents significant opportunities to the cryo-microscopy community. In this review, we will outline developments in the synthesis, functionalisation and labelling uses of both AuNPs and AuNCs in cryo-ET, while discussing their potential as multi-modal probes for cryo-CLEM.

Imaging biological macromolecules in thick specimens: The role of inelastic scattering in cryoEM

We investigate potential improvements in using electron cryomicroscopy to image thick specimens with high-resolution phase contrast imaging. In particular, using model experiments, electron scattering theory, Monte Carlo and multislice simulations, we determine the potential for improving electron cryomicrographs of proteins within a cell using chromatic aberration (C_c) correction. We show that inelastically scattered electrons lose a quantifiable amount of spatial coherence as they transit the specimen, yet can be used to enhance the signal from thick biological specimens (in the 1000 to 5000 Å range) provided they are imaged close to focus with an achromatic lens. This loss of information quantified here, which we call "specimen induced decoherence", is a fundamental limit on imaging biological molecules in situ. We further show that with foreseeable advances in transmission electron microscope technology, it should be possible to directly locate and uniquely identify sub-100 kDa proteins without the need for labels, in a vitrified specimen taken from a cell.

IceBreaker: Software for high-resolution single-particle cryo-EM with non-uniform ice

Despite the abundance of available software tools, optimal particle selection is still a vital issue in single-particle cryoelectron microscopy (cryo-EM). Regardless of the method used, most pickers struggle when ice thickness varies on a micrograph. IceBreaker allows users to estimate the relative ice gradient and flatten it by equalizing the local contrast. It allows the differentiation of particles from the background and improves overall particle picking performance. Furthermore, we introduce an additional parameter corresponding to local ice thickness for each particle. Particles with a defined ice thickness can be grouped and filtered based on this parameter during processing. These functionalities are especially valuable for on-the-fly processing to automatically pick as many particles as possible from each micrograph and to select optimal regions for data collection. Finally, estimated ice gradient distributions can be stored separately and used to inspect the quality of prepared samples.

Real space in cryo-EM: the future is local

Cryo-EM images have extremely low signal-to-noise levels because biological macromolecules are highly radiation-sensitive, requiring low-dose imaging, and because the molecules are poor in contrast. Confident recovery of the signal requires the averaging of many images, the iterative optimization of parameters and the introduction of much prior information. Poor parameter estimates, overfitting and variations in signal strength and resolution across the resulting reconstructions remain frequent issues. Because biological samples are real-space phenomena, exhibiting local variations, real-space measures can be both more reliable and more appropriate than Fourier-space measures. Real-space measures can be calculated separately over each differing region of an image or volume. Real-space filters can be applied according to the local need. Powerful prior information, not available in Fourier space, can be introduced in real space. Priors can be applied in real space in ways that Fourier space precludes. The treatment of biological phenomena remains highly dependent on spatial frequency, however, which would normally be handled in Fourier space. We believe that measures and filters based around real-space operations on extracted frequency bands, i.e. a series of band-pass filtered real-space volumes, and over real-space densities of striding (sequentially increasing or decreasing) resolution through Fourier space are the best way to address this and will perform better than global Fourier-space-based approaches. Future developments in image processing within the field are generally expected to be based on a mixture of both rationally designed and deep-learning approaches, and to incorporate novel prior information from developments such as AlphaFold. Regardless of approach, it is clear that `locality', through real-space measures, filters and processing, will become central to image processing.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

OUP accepted manuscript

Motivation

Cryo-electron microscopy (cryo-EM) is a widely used technology for ultrastructure determination, which constructs the 3D structures of protein and macromolecular complex from a set of 2D micrographs. However, limited by the electron beam dose, the micrographs in cryo-EM generally suffer from the extremely low signal-to-noise ratio (SNR), which hampers the efficiency and effectiveness of downstream analysis. Especially, the noise in cryo-EM is not simple additive or multiplicative noise whose statistical characteristics are quite different from the ones in natural image, extremely shackling the performance of conventional denoising methods.

Results

Here, we introduce the Noise-Transfer2Clean (NT2C), a denoising deep neural network (DNN) for cryo-EM to enhance image contrast and restore specimen signal, whose main idea is to improve the denoising performance by correctly learning the noise distribution of cryo-EM images and transferring the statistical nature of noise into the denoiser. Especially, to cope with the complex noise model in cryo-EM, we design a contrast-guided noise and signal re-weighted algorithm to achieve clean-noisy data synthesis and data augmentation, making our method authentically achieve signal restoration based on noise’s true properties. Our work verifies the feasibility of denoising based on mining the complex cryo-EM noise patterns directly from the noise patches. Comprehensive experimental results on simulated datasets and real datasets show that NT2C achieved a notable improvement in image denoising, especially in background noise removal, compared with the commonly used methods. Moreover, a case study on the real dataset demonstrates that NT2C can greatly alleviate the obstacles caused by the SNR to particle picking and simplify the identifying of particles.

Availabilityand implementation

The code is available at https://github.com/Lihongjia-ict/NoiseTransfer2Clean/.

Supplementary information

Supplementary data are available at Bioinformatics online.

Conquer by cryo-EM without physically dividing

This mini-review provides an update regarding the substantial progress that has been made in using single-particle cryo-EM to obtain high-resolution structures for proteins and other macromolecules whose particle sizes are smaller than 100 kDa. We point out that establishing the limits of what can be accomplished, both in terms of particle size and attainable resolution, serves as a guide for what might be expected when attempting to improve the resolution of small flexible portions of a larger structure using focused refinement approaches. These approaches, which involve computationally ignoring all but a specific, targeted region of interest on the macromolecules, is known as 'masking and refining,' and it thus is the computational equivalent of the 'divide and conquer' approach that has been used so successfully in X-ray crystallography. The benefit of masked refinement, however, is that one is able to determine structures in their native architectural context, without physically separating them from the biological connections that they require for their function. This mini-review also compares where experimental achievements currently stand relative to various theoretical estimates for the smallest particle size that can be successfully reconstructed to high resolution. Since it is clear that a substantial gap still remains between the two, we briefly recap the areas in which further improvement seems possible, both in equipment and in methods.

Cryo-TEM simulations of amorphous radiation-sensitive samples using multislice wave propagation

Image simulation plays a central role in the development and practice of high-resolution electron microscopy, including transmission electron microscopy of frozen-hydrated specimens (cryo-EM). Simulating images with contrast that matches the contrast observed in experimental images remains challenging, especially for amorphous samples. Current state-of-the-art simulators apply post hoc scaling to approximate empirical solvent contrast, attenuated image intensity due to specimen thickness and amplitude contrast. This practice fails for images that require spatially variable scaling, e.g. simulations of a crowded or cellular environment. Modeling both the signal and the noise accurately is necessary to simulate images of biological specimens with contrast that is correct on an absolute scale. The 'frozen plasmon' method is introduced to explicitly model spatially variable inelastic scattering processes in cryo-EM specimens. This approach produces amplitude contrast that depends on the atomic composition of the specimen, reproduces the total inelastic mean free path as observed experimentally and allows for the incorporation of radiation damage in the simulation. These improvements are quantified using the matched filter concept to compare simulation and experiment. The frozen plasmon method, in combination with a new mathematical formulation for accurately sampling the tabulated atomic scattering potentials onto a Cartesian grid, is implemented in the open-source software package cisTEM.

Virus Removal and Inactivation Mechanisms during Iron Electrocoagulation: Capsid and Genome Damages and Electro-Fenton Reactions

Virus destabilization and inactivation are critical considerations in providing safe drinking water. We demonstrate that iron electrocoagulation simultaneously removed (via sweep flocculation) and inactivated a non-enveloped virus surrogate (MS2 bacteriophage) under slightly acidic conditions, resulting in highly effective virus control (e.g., 5-logs at 20 mg Fe/L and pH 6.4 in 30 min). Electrocoagulation simultaneously generated H₂O₂ and Fe(II) that can potentially trigger electro-Fenton reactions to produce reactive oxygen species such as ^•OH and high valent oxoiron(IV) that are capable of inactivating viruses. To date, viral attenuation during water treatment has been largely probed by evaluating infective virions (as plaque forming units) or genomic damage (via the quantitative polymerase chain reaction). In addition to these existing means of assessing virus attenuation, a novel technique of correlating transmission electron micrographs of electrocoagulated MS2 with their computationally altered three-dimensional electron density maps was developed to provide direct visual evidence of capsid morphological damages during electrocoagulation. The majority of coliphages lost at least 10-60% of the capsid protein missing a minimum of one of the 5-fold and two of 3- and 2-fold regions upon electrocoagulation, revealing substantial localized capsid deformation. Attenuated total reflectance-Fourier transform infrared spectroscopy revealed potential oxidation of viral coat proteins and modification of their secondary structures that were attributed to reactive oxygen species. Iron electrocoagulation simultaneously disinfects and coagulates non-enveloped viruses (unlike conventional coagulation), adding to the robustness of multiple barriers necessary for public health protection and appears to be a promising technology for small-scale distributed water treatment.

[Progress in filters for denoising cryo-electron microscopy images]

Cryo-electron microscopy (cryo-EM) imaging has the unique potential to bridge the gap between cellular and molecular biology. Therefore, cryo-EM three-dimensional (3D) reconstruction has been rapidly developed in recent several years and applied widely in life science research to reveal the structures of large macromolecular assemblies and cellular complexes, which is critical to understanding their functions at all scales. Although the technical breakthrough in recent years, for example, the introduction of the direct detection device (DDD) camera and the development of cryo-EM software tools, made the three cryo-EM pioneers share the 2017 Nobel Prize, several bottleneck problems still exist that hamper the further increase of the resolution of single-particle reconstruction and hold back the application of in situ subnanometer structure determination by cryo-tomography. Radiation damage is still the key limiting factor in cryo-EM. In order to minimize the radiation damage and preserve as much resolution as possible, the imaging conditions of a low dose and weak contrast make cryo-EM images extremely noisy with very low signal-to-noise ratios (SNR), generally about 0.1. The high noise will obscure the fine details in cryo-EM images or reconstructed maps. Thus, a method to reduce the level of noise and improve the resolution has become an important issue. In this paper, we systematically reviewed and compared some robust filters in the cryo-EM field of two aspects, single-particle analysis (SPA) and cryo-electron tomography (cryo-ET), and especially studied their applications, such as, 3D reconstruction, visualization, structural analysis, and interpretation. Conventional approaches to noise reduction in cryo-EM imaging include the use of Gaussian, median, and bilateral filters, among other means. A Gaussian filter selects an appropriate filter kernel to conduct spatial convolution with a noisy image. Although noise with larger standard deviations in cryo-EM images can be suppressed and satisfactory performance is achieved in certain cases, this filter also blurs the images and over-smooths small-scale image features. This is especially detrimental when precise quantitative information needs to be extracted. Unlike a Gaussian filter, a median filter is based on the order statistics of the image and selects the median intensity in a window of the adjacent pixels to denoise the image. Although this filter is robust to outliers, it suffers from aliasing problems that possibly result in incorrect information for cryo-EM structure interpretation. A bilateral filter is a nonlinear filter that performs spatial weighted averaging and is more selective in the pixels allowing to contribute to the weighted sum, excluding the high frequency noise from the smoothing process. Thus, this filter can be used to smooth out noise while maintaining the edge details, which is similar to an anisotropic diffusion filter, and distinct from a Gaussian filter but its utility will be limited when the SNR of a cryo-EM image is very low. Generally, spatial filtering methods have the disadvantage of losing image resolution when reducing noise. A wavelet transform can exploit the wavelet's natural ability to separate a signal from noise at multiple image scales to allow for joint resolution in both the spatial and frequency domains, and thus has the potential to outperform existing methods. The modified wavelet shrinkage filter we developed can offer a remarkable improvement in image quality with a good compromise between detail preservation and noise smoothing. We expect that our review study on different filters can provide benefits to cryo-EM applications and the interpretation of biological structures.

DRPnet: automated particle picking in cryo-electron micrographs using deep regression

BACKGROUND

Identification and selection of protein particles in cryo-electron micrographs is an important step in single particle analysis. In this study, we developed a deep learning-based particle picking network to automatically detect particle centers from cryoEM micrographs. This is a challenging task due to the nature of cryoEM data, having low signal-to-noise ratios with variable particle sizes, shapes, distributions, grayscale variations as well as other undesirable artifacts.

RESULTS

We propose a double convolutional neural network (CNN) cascade for automated detection of particles in cryo-electron micrographs. This approach, entitled Deep Regression Picker Network or "DRPnet", is simple but very effective in recognizing different particle sizes, shapes, distributions and grayscale patterns corresponding to 2D views of 3D particles. Particles are detected by the first network, a fully convolutional regression network (FCRN), which maps the particle image to a continuous distance map that acts like a probability density function of particle centers. Particles identified by FCRN are further refined to reduce false particle detections by the second classification CNN. DRPnet's first CNN pretrained with only a single cryoEM dataset can be used to detect particles from different datasets without retraining. Compared to RELION template-based autopicking, DRPnet results in better particle picking performance with drastically reduced user interactions and processing time. DRPnet also outperforms the state-of-the-art particle picking networks in terms of the supervised detection evaluation metrics recall, precision, and F-measure. To further highlight quality of the picked particle sets, we compute and present additional performance metrics assessing the resulting 3D reconstructions such as number of 2D class averages, efficiency/angular coverage, Rosenthal-Henderson plots and local/global 3D reconstruction resolution.

CONCLUSION

DRPnet shows greatly improved time-savings to generate an initial particle dataset compared to manual picking, followed by template-based autopicking. Compared to other networks, DRPnet has equivalent or better performance. DRPnet excels on cryoEM datasets that have low contrast or clumped particles. Evaluating other performance metrics, DRPnet is useful for higher resolution 3D reconstructions with decreased particle numbers or unknown symmetry, detecting particles with better angular orientation coverage.

Enhancing the signal-to-noise ratio and generating contrast for cryo-EM images with convolutional neural networks

In cryogenic electron microscopy (cryo-EM) of radiation-sensitive biological samples, both the signal-to-noise ratio (SNR) and the contrast of images are critically important in the image-processing pipeline. Classic methods improve low-frequency image contrast experimentally, by imaging with high defocus, or computationally, by applying various types of low-pass filter. These contrast improvements typically come at the expense of the high-frequency SNR, which is suppressed by high-defocus imaging and removed by low-pass filtration. Recently, convolutional neural networks (CNNs) trained to denoise cryo-EM images have produced impressive gains in image contrast, but it is not clear how these algorithms affect the information content of the image. Here, a denoising CNN for cryo-EM images was implemented and a quantitative evaluation of SNR enhancement, induced bias and the effects of denoising on image processing and three-dimensional reconstructions was performed. The study suggests that besides improving the visual contrast of cryo-EM images, the enhanced SNR of denoised images may be used in other parts of the image-processing pipeline, such as classification and 3D alignment. These results lay the groundwork for the use of denoising CNNs in the cryo-EM image-processing pipeline beyond particle picking.

Topaz-Denoise: general deep denoising models for cryoEM and cryoET

Cryo-electron microscopy (cryoEM) is becoming the preferred method for resolving protein structures. Low signal-to-noise ratio (SNR) in cryoEM images reduces the confidence and throughput of structure determination during several steps of data processing, resulting in impediments such as missing particle orientations. Denoising cryoEM images can not only improve downstream analysis but also accelerate the time-consuming data collection process by allowing lower electron dose micrographs to be used for analysis. Here, we present Topaz-Denoise, a deep learning method for reliably and rapidly increasing the SNR of cryoEM images and cryoET tomograms. By training on a dataset composed of thousands of micrographs collected across a wide range of imaging conditions, we are able to learn models capturing the complexity of the cryoEM image formation process. The general model we present is able to denoise new datasets without additional training. Denoising with this model improves micrograph interpretability and allows us to solve 3D single particle structures of clustered protocadherin, an elongated particle with previously elusive views. We then show that low dose collection, enabled by Topaz-Denoise, improves downstream analysis in addition to reducing data collection time. We also present a general 3D denoising model for cryoET. Topaz-Denoise and pre-trained general models are now included in Topaz. We expect that Topaz-Denoise will be of broad utility to the cryoEM community for improving micrograph and tomogram interpretability and accelerating analysis.

Non-uniformity of projection distributions attenuates resolution in Cryo-EM

Virtually all single-particle cryo-EM experiments currently suffer from specimen adherence to the air-water interface, leading to a non-uniform distribution in the set of projection views. Whereas it is well accepted that uniform projection distributions can lead to high-resolution reconstructions, non-uniform (anisotropic) distributions can negatively affect map quality, elongate structural features, and in some cases, prohibit interpretation altogether. Although some consequences of non-uniform sampling have been described qualitatively, we know little about how sampling quantitatively affects resolution in cryo-EM. Here, we show how inhomogeneity in any projection distribution scheme attenuates the global Fourier Shell Correlation (FSC) in relation to the number of particles and a single geometrical parameter, which we term the sampling compensation factor (SCF). The reciprocal of the SCF is defined as the average over Fourier shells of the reciprocal of the per-particle sampling and normalized to unity for uniform distributions. The SCF therefore ranges from one to zero, with values close to the latter implying large regions of poorly sampled or completely missing data in Fourier space. Using two synthetic test cases, influenza hemagglutinin and human apoferritin, we demonstrate how any amount of sampling inhomogeneity always attenuates the FSC compared to a uniform distribution. We advocate quantitative evaluation of the SCF criterion to approximate the effect of non-uniform sampling on resolution within experimental single-particle cryo-EM reconstructions.

Analysis of discrete local variability and structural covariance in macromolecular assemblies using Cryo-EM and focused classification

Single-particle electron cryo-microscopy and computational image classification can be used to analyze structural variability in macromolecules and their assemblies. In some cases, a particle may contain different regions that each display a range of distinct conformations. We have developed strategies, implemented within the Frealign and cisTEM image processing packages, to focus-classify on specific regions of a particle and detect potential covariance. The strategies are based on masking the region of interest using either a 2-D mask applied to reference projections and particle images, or a 3-D mask applied to the 3-D volume. We show that focused classification approaches can be used to study structural covariance, a concept that is likely to gain more importance as datasets grow in size, allowing the distinction of more structural states and smaller differences between states. Finally, we apply the approaches to an experimental dataset containing the HIV-1 Transactivation Response (TAR) element RNA fused into the large bacterial ribosomal subunit to deconvolve structural mobility within localized regions of interest, and to a dataset containing assembly intermediates of the large subunit to measure structural covariance.

SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM

Selecting particles from digital micrographs is an essential step in single-particle electron cryomicroscopy (cryo-EM). As manual selection of complete datasets-typically comprising thousands of particles-is a tedious and time-consuming process, numerous automatic particle pickers have been developed. However, non-ideal datasets pose a challenge to particle picking. Here we present the particle picking software crYOLO which is based on the deep-learning object detection system You Only Look Once (YOLO). After training the network with 200-2500 particles per dataset it automatically recognizes particles with high recall and precision while reaching a speed of up to five micrographs per second. Further, we present a general crYOLO network able to pick from previously unseen datasets, allowing for completely automated on-the-fly cryo-EM data preprocessing during data acquisition. crYOLO is available as a standalone program under http://sphire.mpg.de/ and is distributed as part of the image processing workflow in SPHIRE.

Robustness of signal detection in cryo-electron microscopy via a bi-objective-function approach

Background

The detection of weak signals and selection of single particles from low-contrast micrographs of frozen hydrated biomolecules by cryo-electron microscopy (cryo-EM) represents a major practical bottleneck in cryo-EM data analysis. Template-based particle picking by an objective function using fast local correlation (FLC) allows computational extraction of a large number of candidate particles from micrographs. Another independent objective function based on maximum likelihood estimates (MLE) can be used to align the images and verify the presence of a signal in the selected particles. Despite the widespread applications of the two objective functions, an optimal combination of their utilities has not been exploited. Here we propose a bi-objective function (BOF) approach that combines both FLC and MLE and explore the potential advantages and limitations of BOF in signal detection from cryo-EM data.

Results

The robustness of the BOF strategy in particle selection and verification was systematically examined with both simulated and experimental cryo-EM data. We investigated how the performance of the BOF approach is quantitatively affected by the signal-to-noise ratio (SNR) of cryo-EM data and by the choice of initialization for FLC and MLE. We quantitatively pinpointed the critical SNR (~ 0.005), at which the BOF approach starts losing its ability to select and verify particles reliably. We found that the use of a Gaussian model to initialize the MLE suppresses the adverse effects of reference dependency in the FLC function used for template-matching.

Conclusion

The BOF approach, which combines two distinct objective functions, provides a sensitive way to verify particles for downstream cryo-EM structure analysis. Importantly, reference dependency of the FLC does not necessarily transfer to the MLE, enabling the robust detection of weak signals. Our insights into the numerical behavior of the BOF approach can be used to improve automation efficiency in the cryo-EM data processing pipeline for high-resolution structural determination.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2714-8) contains supplementary material, which is available to authorized users.

Structural Variability from Noisy Tomographic Projections

In cryo-electron microscopy, the three-dimensional (3D) electric potentials of an ensemble of molecules are projected along arbitrary viewing directions to yield noisy two-dimensional images. The volume maps representing these potentials typically exhibit a great deal of structural variability, which is described by their 3D covariance matrix. Typically, this covariance matrix is approximately low rank and can be used to cluster the volumes or estimate the intrinsic geometry of the conformation space. We formulate the estimation of this covariance matrix as a linear inverse problem, yielding a consistent least-squares estimator. For n images of size N-by-N pixels, we propose an algorithm for calculating this covariance estimator with computational complexity O ( n N 4 + κ N 6 log N ) , where the condition number κ is empirically in the range 10-200. Its efficiency relies on the observation that the normal equations are equivalent to a deconvolution problem in six dimensions. This is then solved by the conjugate gradient method with an appropriate circulant preconditioner. The result is the first computationally efficient algorithm for consistent estimation of the 3D covariance from noisy projections. It also compares favorably in runtime with respect to previously proposed nonconsistent estimators. Motivated by the recent success of eigenvalue shrinkage procedures for high-dimensional covariance matrix estimation, we incorporate a shrinkage procedure that improves accuracy at lower signal-to-noise ratios. We evaluate our methods on simulated datasets and achieve classification results comparable to state-of-the-art methods in shorter running time. We also present results on clustering volumes in an experimental dataset, illustrating the power of the proposed algorithm for practical determination of structural variability.

Addressing preferred specimen orientation in single-particle cryo-EM through tilting

We present a strategy for tackling preferred specimen orientation in single-particle cryogenic electron microscopy by employing tilts during data collection. We also describe a tool to quantify the resulting directional resolution using 3D Fourier shell correlation volumes. We applied these methods to determine the structures at near-atomic resolution of the influenza hemagglutinin trimer, which adopts a highly preferred specimen orientation, and of ribosomal biogenesis intermediates, which adopt moderately preferred orientations.

Factor Analysis for Spectral Estimation

Power spectrum estimation is an important tool in many applications, such as the whitening of noise. The popular multitaper method enjoys significant success, but fails for short signals with few samples. We propose a statistical model where a signal is given by a random linear combination of fixed, yet unknown, stochastic sources. Given multiple such signals, we estimate the subspace spanned by the power spectra of these fixed sources. Projecting individual power spectrum estimates onto this subspace increases estimation accuracy. We provide accuracy guarantees for this method and demonstrate it on simulated and experimental data from cryo-electron microscopy.

Building Proteins in a Day: Efficient 3D Molecular Structure Estimation with Electron Cryomicroscopy

Discovering the 3D atomic-resolution structure of molecules such as proteins and viruses is one of the foremost research problems in biology and medicine. Electron Cryomicroscopy (cryo-EM) is a promising vision-based technique for structure estimation which attempts to reconstruct 3D atomic structures from a large set of 2D transmission electron microscope images. This paper presents a new Bayesian framework for cryo-EM structure estimation that builds on modern stochastic optimization techniques to allow one to scale to very large datasets. We also introduce a novel Monte-Carlo technique that reduces the cost of evaluating the objective function during optimization by over five orders of magnitude. The net result is an approach capable of estimating 3D molecular structure from large-scale datasets in about a day on a single CPU workstation.

Structural Study of Heterogeneous Biological Samples by Cryoelectron Microscopy and Image Processing

In living organisms, biological macromolecules are intrinsically flexible and naturally exist in multiple conformations. Modern electron microscopy, especially at liquid nitrogen temperatures (cryo-EM), is able to visualise biocomplexes in nearly native conditions and in multiple conformational states. The advances made during the last decade in electronic technology and software development have led to the revelation of structural variations in complexes and also improved the resolution of EM structures. Nowadays, structural studies based on single particle analysis (SPA) suggests several approaches for the separation of different conformational states and therefore disclosure of the mechanisms for functioning of complexes. The task of resolving different states requires the examination of large datasets, sophisticated programs, and significant computing power. Some methods are based on analysis of two-dimensional images, while others are based on three-dimensional studies. In this review, we describe the basic principles implemented in the various techniques that are currently used in the analysis of structural conformations and provide some examples of successful applications of these methods in structural studies of biologically significant complexes.

Story in a sample-the potential (and limitations) of cryo-electron microscopy applied to molecular machines

This article addresses recent developments in cryo-electron microscopy and single-particle reconstruction of macromolecules. With the advent of powerful classification techniques, it is now possible to extract and visualize multiple conformers contained within the same dataset. It is discussed how and to what extent this technique can be used in the study of the dynamics of molecular machines.

A local-optimization refinement algorithm in single particle analysis for macromolecular complex with multiple rigid modules

Single particle analysis, which can be regarded as an average of signals from thousands or even millions of particle projections, is an efficient method to study the three-dimensional structures of biological macromolecules. An intrinsic assumption in single particle analysis is that all the analyzed particles must have identical composition and conformation. Thus specimen heterogeneity in either composition or conformation has raised great challenges for high-resolution analysis. For particles with multiple conformations, inaccurate alignments and orientation parameters will yield an averaged map with diminished resolution and smeared density. Besides extensive classification approaches, here based on the assumption that the macromolecular complex is made up of multiple rigid modules whose relative orientations and positions are in slight fluctuation around equilibriums, we propose a new method called as local optimization refinement to address this conformational heterogeneity for an improved resolution. The key idea is to optimize the orientation and shift parameters of each rigid module and then reconstruct their three-dimensional structures individually. Using simulated data of 80S/70S ribosomes with relative fluctuations between the large (60S/50S) and the small (40S/30S) subunits, we tested this algorithm and found that the resolutions of both subunits are significantly improved. Our method provides a proof-of-principle solution for high-resolution single particle analysis of macromolecular complexes with dynamic conformations.

Single-particle cryoEM analysis at near-atomic resolution from several thousand asymmetric subunits

A single-particle cryoEM reconstruction of the large ribosomal subunit from Saccharomyces cerevisiae was obtained from a dataset of ∼75,000 particles. The gold-standard and frequency-limited approaches to single-particle refinement were each independently used to determine orientation parameters for the final reconstruction. Both approaches showed similar resolution curves and nominal resolution values for the 60S dataset, estimated at 2.9 Å. The amount of over-fitting present during frequency-limited refinement was quantitatively analyzed using the high-resolution phase-randomization test, and the results showed no apparent over-fitting. The number of asymmetric subunits required to reach specific resolutions was subsequently analyzed by refining subsets of the data in an ab initio manner. With our data collection and processing strategies, sub-nanometer resolution was obtained with ∼200 asymmetric subunits (or, equivalently for the ribosomal subunit, particles). Resolutions of 5.6 Å, 4.5 Å, and 3.8 Å were reached with ∼1000, ∼1600, and ∼5000 asymmetric subunits, respectively. At these resolutions, one would expect to detect alpha-helical pitch, separation of beta-strands, and separation of Cα atoms, respectively. Using this map, together with strategies for ab initio model building and model refinement, we built a region of the ribosomal protein eL6, which was missing in previous models of the yeast ribosome. The relevance for more routine high-resolution structure determination is discussed.

Efficient estimation of three-dimensional covariance and its application in the analysis of heterogeneous samples in cryo-electron microscopy

Single-particle cryogenic electron microscopy (cryo-EM) is a powerful tool for the study of macromolecular structures at high resolution. Classification allows multiple structural states to be extracted and reconstructed from the same sample. One classification approach is via the covariance matrix, which captures the correlation between every pair of voxels. Earlier approaches employ computing-intensive resampling and estimate only the eigenvectors of the matrix, which are then used in a separate fast classification step. We propose an iterative scheme to explicitly estimate the covariance matrix in its entirety. In our approach, the flexibility in choosing the solution domain allows us to examine a part of the molecule in greater detail. Three-dimensional covariance maps obtained in this way from experimental data (cryo-EM images of the eukaryotic pre-initiation complex) prove to be in excellent agreement with conclusions derived by using traditional approaches, revealing in addition the interdependencies of ligand bindings and structural changes.

Validation of 3D EM Reconstructions: The Phantom in the Noise

Validation is a necessity to trust the structures solved by electron microscopy by single particle techniques. The impressive achievements in single particle reconstruction fuel its expansion beyond a small community of image processing experts. This poses the risk of inappropriate data processing with dubious results. Nowhere is it more clearly illustrated than in the recovery of a reference density map from pure noise aligned to that map—a phantom in the noise. Appropriate use of existing validating methods such as resolution-limited alignment and the processing of independent data sets (“gold standard”) avoid this pitfall. However, these methods can be undermined by biases introduced in various subtle ways. How can we test that a map is a coherent structure present in the images selected from the micrographs? In stead of viewing the phantom emerging from noise as a cautionary tale, it should be used as a defining baseline. Any map is always recoverable from noise images, provided a sufficient number of images are aligned and used in reconstruction. However, with smaller numbers of images, the expected coherence in the real particle images should yield better reconstructions than equivalent numbers of noise or background images, even without masking or imposing resolution limits as potential biases. The validation test proposed is therefore a simple alignment of a limited number of micrograph and noise images against the final reconstruction as reference, demonstrating that the micrograph images yield a better reconstruction. I examine synthetic cases to relate the resolution of a reconstruction to the alignment error as a function of the signal-to-noise ratio. I also administered the test to real cases of publicly available data. Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

Particle migration analysis in iterative classification of cryo-EM single-particle data

Recently developed classification methods have enabled resolving multiple biological structures from cryo-EM data collected on heterogeneous biological samples. However, there remains the problem of how to base the decisions in the classification on the statistics of the cryo-EM data, to reduce the subjectivity in the process. Here, we propose a quantitative analysis to determine the iteration of convergence and the number of distinguishable classes, based on the statistics of the single particles in an iterative classification scheme. We start the classification with more number of classes than anticipated based on prior knowledge, and then combine the classes that yield similar reconstructions. The classes yielding similar reconstructions can be identified from the migrating particles (jumpers) during consecutive iterations after the iteration of convergence. We therefore termed the method "jumper analysis", and applied it to the output of RELION 3D classification of a benchmark experimental dataset. This work is a step forward toward fully automated single-particle reconstruction and classification of cryo-EM data.

Automatic cryo-EM particle selection for membrane proteins in spherical liposomes

Random spherically constrained (RSC) single particle reconstruction is a method to obtain structures of membrane proteins embedded in lipid vesicles (liposomes). As in all single-particle cryo-EM methods, structure determination is greatly aided by reliable detection of protein "particles" in micrographs. After fitting and subtraction of the membrane density from a micrograph, normalized cross-correlation (NCC) and estimates of the particle signal amplitude are used to detect particles, using as references the projections of a 3D model. At each pixel position, the NCC is computed with only those references that are allowed by the geometric constraint of the particle's embedding in the spherical vesicle membrane. We describe an efficient algorithm for computing this position-dependent correlation, and demonstrate its application to selection of membrane-protein particles, GluA2 glutamate receptors, which present very different views from different projection directions.

Optimod--an automated approach for constructing and optimizing initial models for single-particle electron microscopy

Single-particle cryo-electron microscopy is now well established as a technique for the structural characterization of large macromolecules and macromolecular complexes. The raw data is very noisy and consists of two-dimensional projections, from which the 3D biological object must be reconstructed. The 3D object depends upon knowledge of proper angular orientations assigned to the 2D projection images. Numerous algorithms have been developed for determining relative angular orientations between 2D images, but the transition from 2D to 3D remains challenging and can result in erroneous and conflicting results. Here we describe a general, automated procedure, called OptiMod, for reconstructing and optimizing 3D models using common-lines methodologies. OptiMod approximates orientation angles and reconstructs independent maps from 2D class averages. It then iterates the procedure, while considering each map as a raw solution that needs to be compared with other possible outcomes. We incorporate procedures for 3D alignment, clustering, and refinement to optimize each map, as well as standard scoring metrics to facilitate the selection of the optimal model. We also show that small angle tilt-pair data can be included as one of the scoring metrics to improve the selection of the optimal initial model, and also to provide a validation check. The overall approach is demonstrated using two experimental cryo-EM data sets--the 80S ribosome that represents a relatively straightforward case for ab initio reconstruction, and the Tf-TfR complex that represents a challenging case in that it has previously been shown to provide multiple equally plausible solutions to the initial model problem.

One number does not fit all: mapping local variations in resolution in cryo-EM reconstructions

The resolution of density maps from single particle analysis is usually measured in terms of the highest spatial frequency to which consistent information has been obtained. This calculation represents an average over the entire reconstructed volume. In practice, however, substantial local variations in resolution may occur, either from intrinsic properties of the specimen or for technical reasons such as a non-isotropic distribution of viewing orientations. To address this issue, we propose the use of a space-frequency representation, the short-space Fourier transform, to assess the quality of a density map, voxel-by-voxel, i.e. by local resolution mapping. In this approach, the experimental volume is divided into small subvolumes and the resolution determined for each of them. It is illustrated in applications both to model data and to experimental density maps. Regions with lower-than-average resolution may be mobile components or ones with incomplete occupancy or result from multiple conformational states. To improve the interpretability of reconstructions, we propose an adaptive filtering approach that reconciles the resolution to which individual features are calculated with the results of the local resolution map.

Likelihood-based classification of cryo-EM images using FREALIGN

We describe an implementation of maximum likelihood classification for single particle electron cryo-microscopy that is based on the FREALIGN software. Particle alignment parameters are determined by maximizing a joint likelihood that can include hierarchical priors, while classification is performed by expectation maximization of a marginal likelihood. We test the FREALIGN implementation using a simulated dataset containing computer-generated projection images of three different 70S ribosome structures, as well as a publicly available dataset of 70S ribosomes. The results show that the mixed strategy of the new FREALIGN algorithm yields performance on par with other maximum likelihood implementations, while remaining computationally efficient.