Enhancing the signal-to-noise ratio and generating contrast for cryo-EM images with convolutional neural networks

Refinement of cryo-EM 3D maps with a self-supervised denoising model: crefDenoiser

Cryogenic electron microscopy (cryo-EM) is a pivotal technique for imaging macromolecular structures. However, despite extensive processing of large image sets collected in cryo-EM experiments to amplify the signal-to-noise ratio, the reconstructed 3D protein-density maps are often limited in quality due to residual noise, which in turn affects the accuracy of the macromolecular representation. Here, crefDenoiser is introduced, a denoising neural network model designed to enhance the signal in 3D cryo-EM maps produced with standard processing pipelines. The crefDenoiser model is trained without the need for `clean' ground-truth target maps. Instead, a custom dataset is employed, composed of real noisy protein half-maps sourced from the Electron Microscopy Data Bank repository. Competing with the current state-of-the-art, crefDenoiser is designed to optimize for the theoretical noise-free map during self-supervised training. We demonstrate that our model successfully amplifies the signal across a wide variety of protein maps, outperforming a classic map denoiser and following a network-based sharpening model. Without biasing the map, the proposed denoising method leads to improved visibility of protein structural features, including protein domains, secondary structure elements and modest high-resolution feature restoration.

The advent of preventive high-resolution structural histopathology by artificial-intelligence-powered cryogenic electron tomography

Advances in cryogenic electron microscopy (cryoEM) single particle analysis have revolutionized structural biology by facilitating the in vitro determination of atomic- and near-atomic-resolution structures for fully hydrated macromolecular complexes exhibiting compositional and conformational heterogeneity across a wide range of sizes. Cryogenic electron tomography (cryoET) and subtomogram averaging are rapidly progressing toward delivering similar insights for macromolecular complexes in situ, without requiring tags or harsh biochemical purification. Furthermore, cryoET enables the visualization of cellular and tissue phenotypes directly at molecular, nanometric resolution without chemical fixation or staining artifacts. This forward-looking review covers recent developments in cryoEM/ET and related technologies such as cryogenic focused ion beam milling scanning electron microscopy and correlative light microscopy, increasingly enhanced and supported by artificial intelligence algorithms. Their potential application to emerging concepts is discussed, primarily the prospect of complementing medical histopathology analysis. Machine learning solutions are poised to address current challenges posed by "big data" in cryoET of tissues, cells, and macromolecules, offering the promise of enabling novel, quantitative insights into disease processes, which may translate into the clinic and lead to improved diagnostics and targeted therapeutics.

Joint micrograph denoising and protein localization in cryo-electron microscopy

Cryo-electron microscopy (cryo-EM) is an imaging technique that allows the visualization of proteins and macromolecular complexes at near-atomic resolution. The low electron doses used to prevent radiation damage to the biological samples result in images where the power of noise is 100 times stronger than that of the signal. Accurate identification of proteins from these low signal-to-noise ratio (SNR) images is a critical task, as the detected positions serve as inputs for the downstream 3D structure determination process. Current methods either fail to identify all true positives or result in many false positives, especially when analyzing images from smaller-sized proteins that exhibit extremely low contrast, or require manual labeling that can take days to complete. Acknowledging the fact that accurate protein identification is dependent upon the visual interpretability of micrographs, we propose a framework that can perform denoising and detection in a joint manner and enable particle localization under extremely low SNR conditions using self-supervised denoising and particle identification from sparsely annotated data. We validate our approach on three challenging single-particle cryo-EM datasets and projection images from one cryo-electron tomography dataset with extremely low SNR, showing that it outperforms existing state-of-the-art methods used for cryo-EM image analysis by a significant margin. We also evaluate the performance of our algorithm under decreasing SNR conditions and show that our method is more robust to noise than competing methods.

Overcoming resolution attenuation during tilted cryo-EM data collection

Structural biology efforts using cryogenic electron microscopy are frequently stifled by specimens adopting "preferred orientations" on grids, leading to anisotropic map resolution and impeding structure determination. Tilting the specimen stage during data collection is a generalizable solution but has historically led to substantial resolution attenuation. Here, we develop updated data collection and image processing workflows and demonstrate, using multiple specimens, that resolution attenuation is negligible or significantly reduced across tilt angles. Reconstructions with and without the stage tilted as high as 60° are virtually indistinguishable. These strategies allowed the reconstruction to 3 Å resolution of a bacterial RNA polymerase with preferred orientation, containing an unnatural nucleotide for studying novel base pair recognition. Furthermore, we present a quantitative framework that allows cryo-EM practitioners to define an optimal tilt angle during data acquisition. These results reinforce the utility of employing stage tilt for data collection and provide quantitative metrics to obtain isotropic maps.

Scaling up cryo-EM for biology and chemistry: The journey from niche technology to mainstream method

Cryoelectron microscopy (cryo-EM) methods have made meaningful contributions in a wide variety of scientific research fields. In structural biology, cryo-EM routinely elucidates molecular structure from isolated biological macromolecular complexes or in a cellular context by harnessing the high-resolution power of the electron in order to image samples in a frozen, hydrated environment. For structural chemistry, the cryo-EM method popularly known as microcrystal electron diffraction (MicroED) has facilitated atomic structure generation of peptides and small molecules from their three-dimensional crystal forms. As cryo-EM has grown from an emerging technology, it has undergone modernization to enable multimodal transmission electron microscopy (TEM) techniques becoming more routine, reproducible, and accessible to accelerate research across multiple disciplines. We review recent advances in modern cryo-EM and assess how they are contributing to the future of the field with an eye to the past.

Overcoming Resolution Attenuation During Tilted Cryo-EM Data Collection

Structural biology efforts using cryogenic electron microscopy are frequently stifled by specimens adopting "preferred orientations" on grids, leading to anisotropic map resolution and impeding structure determination. Tilting the specimen stage during data collection is a generalizable solution but has historically led to substantial resolution attenuation. Here, we develop updated data collection and image processing workflows and demonstrate, using multiple specimens, that resolution attenuation is negligible or significantly reduced across tilt angles. Reconstructions with and without the stage tilted as high as 60° are virtually indistinguishable. These strategies allowed the reconstruction to 3 Å resolution of a bacterial RNA polymerase with preferred orientation. Furthermore, we present a quantitative framework that allows cryo-EM practitioners to define an optimal tilt angle for dataset acquisition. These data reinforce the utility of employing stage tilt for data collection and provide quantitative metrics to obtain isotropic maps.

A strategy combining denoising and cryo-EM single particle analysis

In cryogenic electron microscopy (cryo-EM) single particle analysis (SPA), high-resolution three-dimensional structures of biological macromolecules are determined by iteratively aligning and averaging a large number of two-dimensional projections of molecules. Since the correlation measures are sensitive to the signal-to-noise ratio, various parameter estimation steps in SPA will be disturbed by the high-intensity noise in cryo-EM. However, denoising algorithms tend to damage high frequencies and suppress mid- and high-frequency contrast of micrographs, which exactly the precise parameter estimation relies on, therefore, limiting their application in SPA. In this study, we suggest combining a cryo-EM image processing pipeline with denoising and maximizing the signal's contribution in various parameter estimation steps. To solve the inherent flaws of denoising algorithms, we design an algorithm named MScale to correct the amplitude distortion caused by denoising and propose a new orientation determination strategy to compensate for the high-frequency loss. In the experiments on several real datasets, the denoised particles are successfully applied in the class assignment estimation and orientation determination tasks, ultimately enhancing the quality of biomacromolecule reconstruction. The case study on classification indicates that our strategy not only improves the resolution of difficult classes (up to 5 Å) but also resolves an additional class. In the case study on orientation determination, our strategy improves the resolution of the final reconstructed density map by 0.34 Å compared with conventional strategy. The code is available at https://github.com/zhanghui186/Mscale.

Signal enhancement for two-dimensional cryo-EM data processing

Different tasks in the computational pipeline of single-particle cryo-electron microscopy (cryo-EM) require enhancing the quality of the highly noisy raw images. To this end, we develop an efficient algorithm for signal enhancement of cryo-EM images. The enhanced images can be used for a variety of downstream tasks, such as two-dimensional classification, removing uninformative images, constructing ab initio models, generating templates for particle picking, providing a quick assessment of the data set, dimensionality reduction, and symmetry detection. The algorithm includes built-in quality measures to assess its performance and alleviate the risk of model bias. We demonstrate the effectiveness of the proposed algorithm on several experimental data sets. In particular, we show that the quality of the resulting images is high enough to produce ab initio models of Å resolution. The algorithm is accompanied by a publicly available, documented, and easy-to-use code.

Multiple-image super-resolution of cryo-electron micrographs based on deep internal learning

Single-particle cryo-electron microscopy (cryo-EM) is a powerful imaging modality capable of visualizing proteins and macromolecular complexes at near-atomic resolution. The low electron-doses used to prevent radiation damage to the biological samples, however, result in images where the power of the noise is 100 times greater than the power of the signal. To overcome these low signal-to-noise ratios (SNRs), hundreds of thousands of particle projections are averaged to determine the three-dimensional structure of the molecule of interest. The sampling requirements of high-resolution imaging impose limitations on the pixel sizes that can be used for acquisition, limiting the size of the field of view and requiring data collection sessions of several days to accumulate sufficient numbers of particles. Meanwhile, recent image super-resolution (SR) techniques based on neural networks have shown state-of-the-art performance on natural images. Building on these advances, here, we present a multiple-image SR algorithm based on deep internal learning designed specifically to work under low-SNR conditions. Our approach leverages the internal image statistics of cryo-EM movies and does not require training on ground-truth data. When applied to single-particle datasets of apoferritin and T20S proteasome, we show that the resolution of the 3D structure obtained from SR micrographs can surpass the limits imposed by the imaging system. Our results indicate that the combination of low magnification imaging with in silico image SR has the potential to accelerate cryo-EM data collection by virtue of including more particles in each exposure and doing so without sacrificing resolution.

Ab-initio contrast estimation and denoising of cryo-EM images

BACKGROUND AND OBJECTIVE

The contrast of cryo-EM images varies from one to another, primarily due to the uneven thickness of the ice layer. This contrast variation can affect the quality of 2-D class averaging, 3-D ab-initio modeling, and 3-D heterogeneity analysis. Contrast estimation is currently performed during 3-D iterative refinement. As a result, the estimates are not available at the earlier computational stages of class averaging and ab-initio modeling. This paper aims to solve the contrast estimation problem directly from the picked particle images in the ab-initio stage, without estimating the 3-D volume, image rotations, or class averages.

METHODS

The key observation underlying our analysis is that the 2-D covariance matrix of the raw images is related to the covariance of the underlying clean images, the noise variance, and the contrast variability between images. We show that the contrast variability can be derived from the 2-D covariance matrix and we apply the existing Covariance Wiener Filtering (CWF) framework to estimate it. We also demonstrate a modification of CWF to estimate the contrast of individual images.

RESULTS

Our method improves the contrast estimation by a large margin, compared to the previous CWF method. Its estimation accuracy is often comparable to that of an oracle that knows the ground truth covariance of the clean images. The more accurate contrast estimation also improves the quality of image restoration as demonstrated in both synthetic and experimental datasets.

CONCLUSIONS

This paper proposes an effective method for contrast estimation directly from noisy images without using any 3-D volume information. It enables contrast correction in the earlier stage of single particle analysis, and may improve the accuracy of downstream processing.

Cryo-electron tomography related radiation-damage parameters for individual-molecule 3D structure determination

To understand the dynamic structure-function relationship of soft- and biomolecules, the determination of the three-dimensional (3D) structure of each individual molecule (nonaveraged structure) in its native state is sought-after. Cryo-electron tomography (cryo-ET) is a unique tool for imaging an individual object from a series of tilted views. However, due to radiation damage from the incident electron beam, the tolerable electron dose limits image contrast and the signal-to-noise ratio (SNR) of the data, preventing the 3D structure determination of individual molecules, especially at high-resolution. Although recently developed technologies and techniques, such as the direct electron detector, phase plate, and computational algorithms, can partially improve image contrast/SNR at the same electron dose, the high-resolution structure, such as tertiary structure of individual molecules, has not yet been resolved. Here, we review the cryo-electron microscopy (cryo-EM) and cryo-ET experimental parameters to discuss how these parameters affect the extent of radiation damage. This discussion can guide us in optimizing the experimental strategy to increase the imaging dose or improve image SNR without increasing the radiation damage. With a higher dose, a higher image contrast/SNR can be achieved, which is crucial for individual-molecule 3D structure. With 3D structures determined from an ensemble of individual molecules in different conformations, the molecular mechanism through their biochemical reactions, such as self-folding or synthesis, can be elucidated in a straightforward manner.

Low-dose shift- and rotation-invariant diffraction recognition imaging

A low-dose imaging technique which uses recognition rather than recording of a full high-resolution image is proposed. A structural hypothesis is verified by probing the object with only a few particles (photons, electrons). Each scattered particle is detected in the far field and its position on the detector is analysed by applying Bayesian statistics. Already a few detected particles are sufficient to confirm a structural hypothesis at a probability exceeding 95%. As an example, the method is demonstrated as an application in optical character recognition, where a hand-written number is recognized from a set of different written numbers. In other provided examples, the structural hypothesis of a single macromolecule is recognized from a diffraction pattern acquired at an extremely low radiation dose, less than one X-ray photon or electron per Ų, thus leaving the macromolecule practically without any radiation damage. The proposed principle of low-dose recognition can be utilized in various applications, ranging from optical character recognition and optical security elements to recognizing a certain protein or its conformation.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

End-to-end orientation estimation from 2D cryo-EM images

Cryo-electron microscopy (cryo-EM) is a Nobel Prize-winning technique for determining high-resolution 3D structures of biological macromolecules. A 3D structure is reconstructed from hundreds of thousands of noisy 2D projection images. However, existing 3D reconstruction methods are still time-consuming, and one of the major computational bottlenecks is recovering the unknown orientation of the particle in each 2D image. The dominant methods typically exploit an expensive global search on each image to estimate the missing orientations. Here, a novel end-to-end supervised learning method is introduced to directly recover the missing orientations from 2D cryo-EM images. A neural network is used to approximate the mapping from images to orientations. A robust loss function is proposed for optimizing the parameters of the network, which can handle both asymmetric and symmetric 3D structures. Experiments on synthetic data sets with various symmetry types confirm that the neural network is capable of recovering orientations from 2D cryo-EM images, and the results on a real cryo-EM data set further demonstrate its potential under more challenging imaging conditions.

Cryo-EM Analyses Permit Visualization of Structural Polymorphism of Biological Macromolecules

The functions of biological macromolecules are often associated with conformational malleability of the structures. This phenomenon of chemically identical molecules with different structures is coined structural polymorphism. Conventionally, structural polymorphism is observed directly by structural determination at the density map level from X-ray crystal diffraction. Although crystallography approach can report the conformation of a macromolecule with the position of each atom accurately defined in it, the exploration of structural polymorphism and interpreting biological function in terms of crystal structures is largely constrained by the crystal packing. An alternative approach to studying the macromolecule of interest in solution is thus desirable. With the advancement of instrumentation and computational methods for image analysis and reconstruction, cryo-electron microscope (cryo-EM) has been transformed to be able to produce “in solution” structures of macromolecules routinely with resolutions comparable to crystallography but without the need of crystals. Since the sample preparation of single-particle cryo-EM allows for all forms co-existing in solution to be simultaneously frozen, the image data contain rich information as to structural polymorphism. The ensemble of structure information can be subsequently disentangled through three-dimensional (3D) classification analyses. In this review, we highlight important examples of protein structural polymorphism in relation to allostery, subunit cooperativity and function plasticity recently revealed by cryo-EM analyses, and review recent developments in 3D classification algorithms including neural network/deep learning approaches that would enable cryo-EM analyese in this regard. Finally, we brief the frontier of cryo-EM structure determination of RNA molecules where resolving the structural polymorphism is at dawn.

Deep learning approach for an interface structure analysis with a large statistical noise in neutron reflectometry

Neutron reflectometry (NR) allows us to probe into the structure of the surfaces and interfaces of various materials such as soft matters and magnetic thin films with a contrast mechanism dependent on isotopic and magnetic states. The neutron beam flux is relatively low compared to that of other sources such as synchrotron radiation; therefore, there has been a strong limitation in the time-resolved measurement and further advanced experiments such as surface imaging. This study aims at the development of a methodology to enable the structural analysis by the NR data with a large statistical error acquired in a short measurement time. The neural network-based method predicts the true NR profile from the data with a 20-fold lower signal compared to that obtained under the conventional measurement condition. This indicates that the acquisition time in the NR measurement can be reduced by more than one order of magnitude. The current method will help achieve remarkable improvement in temporally and spatially resolved NR methods to gain further insight into the surface and interfaces of materials.