Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities

Accurate size-based protein localization from cryo-ET tomograms

Cryo-electron tomography (cryo-ET) combined with sub-tomogram averaging (STA) allows the determination of protein structures imaged within the native context of the cell at near-atomic resolution. Particle picking is an essential step in the cryo-ET/STA image analysis pipeline that consists in locating the position of proteins within crowded cellular tomograms so that they can be aligned and averaged in 3D to improve resolution. While extensive work in 2D particle picking has been done in the context of single-particle cryo-EM, comparatively fewer strategies have been proposed to pick particles from 3D tomograms, in part due to the challenges associated with working with noisy 3D volumes affected by the missing wedge. While strategies based on 3D template-matching and deep learning are commonly used, these methods are computationally expensive and require either an external template or manual labelling which can bias the results and limit their applicability. Here, we propose a size-based method to pick particles from tomograms that is fast, accurate, and does not require external templates or user provided labels. We compare the performance of our approach against a commonly used algorithm based on deep learning, crYOLO, and show that our method: i) has higher detection accuracy, ii) does not require user input for labeling or time-consuming training, and iii) runs efficiently on non-specialized CPU hardware. We demonstrate the effectiveness of our approach by automatically detecting particles from tomograms representing different types of samples and using these particles to determine the high-resolution structures of ribosomes imaged in vitro and in situ.

Efficient high-resolution refinement in cryo-EM with stochastic gradient descent

Electron cryomicroscopy (cryo-EM) is an imaging technique widely used in structural biology to determine the three-dimensional structure of biological molecules from noisy two-dimensional projections with unknown orientations. As the typical pipeline involves processing large amounts of data, efficient algorithms are crucial for fast and reliable results. The stochastic gradient descent (SGD) algorithm has been used to improve the speed of ab initio reconstruction, which results in a first, low-resolution estimation of the volume representing the molecule of interest, but has yet to be applied successfully in the high-resolution regime, where expectation-maximization algorithms achieve state-of-the-art results, at a high computational cost. In this article, we investigate the conditioning of the optimization problem and show that the large condition number prevents the successful application of gradient descent-based methods at high resolution. Our results include a theoretical analysis of the condition number of the optimization problem in a simplified setting where the individual projection directions are known, an algorithm based on computing a diagonal preconditioner using Hutchinson's diagonal estimator, and numerical experiments showing the improvement in the convergence speed when using the estimated preconditioner with SGD. The preconditioned SGD approach can potentially enable a simple and unified approach to ab initio reconstruction and high-resolution refinement with faster convergence speed and higher flexibility, and our results are a promising step in this direction.

A deep learning method for simultaneous denoising and missing wedge reconstruction in cryogenic electron tomography

Cryogenic electron tomography is a technique for imaging biological samples in 3D. A microscope collects a series of 2D projections of the sample, and the goal is to reconstruct the 3D density of the sample called the tomogram. Reconstruction is difficult as the 2D projections are noisy and can not be recorded from all directions, resulting in a missing wedge of information. Tomograms conventionally reconstructed with filtered back-projection suffer from noise and strong artefacts due to the missing wedge. Here, we propose a deep-learning approach for simultaneous denoising and missing wedge reconstruction called DeepDeWedge. The algorithm requires no ground truth data and is based on fitting a neural network to the 2D projections using a self-supervised loss. DeepDeWedge is simpler than current state-of-the-art approaches for denoising and missing wedge reconstruction, performs competitively and produces more denoised tomograms with higher overall contrast.

Confronting heterogeneity in cryogenic electron microscopy data: Innovative strategies and future perspectives with data-driven methods

The surge in the influx of data from cryogenic electron microscopy (cryo-EM) experiments has intensified the demand for robust algorithms capable of autonomously managing structurally heterogeneous datasets. This presents a wealth of exciting opportunities from a data science viewpoint, inspiring the development of numerous innovative, application-specific methods, many of which leverage contemporary data-driven techniques. However, addressing the challenges posed by heterogeneous datasets remains a paramount yet unresolved issue in the field. Here, we explore the subtleties of this challenge and the array of strategies devised to confront it. We pinpoint the shortcomings of existing methodologies and deliberate on prospective avenues for improvement. Specifically, our discussion focuses on strategies to mitigate model overfitting and manage data noise, as well as the effects of constraints, priors, and invariances on the optimization process.

The advent of preventive high-resolution structural histopathology by artificial-intelligence-powered cryogenic electron tomography

Advances in cryogenic electron microscopy (cryoEM) single particle analysis have revolutionized structural biology by facilitating the in vitro determination of atomic- and near-atomic-resolution structures for fully hydrated macromolecular complexes exhibiting compositional and conformational heterogeneity across a wide range of sizes. Cryogenic electron tomography (cryoET) and subtomogram averaging are rapidly progressing toward delivering similar insights for macromolecular complexes in situ, without requiring tags or harsh biochemical purification. Furthermore, cryoET enables the visualization of cellular and tissue phenotypes directly at molecular, nanometric resolution without chemical fixation or staining artifacts. This forward-looking review covers recent developments in cryoEM/ET and related technologies such as cryogenic focused ion beam milling scanning electron microscopy and correlative light microscopy, increasingly enhanced and supported by artificial intelligence algorithms. Their potential application to emerging concepts is discussed, primarily the prospect of complementing medical histopathology analysis. Machine learning solutions are poised to address current challenges posed by "big data" in cryoET of tissues, cells, and macromolecules, offering the promise of enabling novel, quantitative insights into disease processes, which may translate into the clinic and lead to improved diagnostics and targeted therapeutics.

Joint micrograph denoising and protein localization in cryo-electron microscopy

Cryo-electron microscopy (cryo-EM) is an imaging technique that allows the visualization of proteins and macromolecular complexes at near-atomic resolution. The low electron doses used to prevent radiation damage to the biological samples result in images where the power of noise is 100 times stronger than that of the signal. Accurate identification of proteins from these low signal-to-noise ratio (SNR) images is a critical task, as the detected positions serve as inputs for the downstream 3D structure determination process. Current methods either fail to identify all true positives or result in many false positives, especially when analyzing images from smaller-sized proteins that exhibit extremely low contrast, or require manual labeling that can take days to complete. Acknowledging the fact that accurate protein identification is dependent upon the visual interpretability of micrographs, we propose a framework that can perform denoising and detection in a joint manner and enable particle localization under extremely low SNR conditions using self-supervised denoising and particle identification from sparsely annotated data. We validate our approach on three challenging single-particle cryo-EM datasets and projection images from one cryo-electron tomography dataset with extremely low SNR, showing that it outperforms existing state-of-the-art methods used for cryo-EM image analysis by a significant margin. We also evaluate the performance of our algorithm under decreasing SNR conditions and show that our method is more robust to noise than competing methods.

Self Fourier shell correlation: properties and application to cryo-ET

The Fourier shell correlation (FSC) is a measure of the similarity between two signals computed over corresponding shells in the frequency domain and has broad applications in microscopy. In structural biology, the FSC is ubiquitous in methods for validation, resolution determination, and signal enhancement. Computing the FSC usually requires two independent measurements of the same underlying signal, which can be limiting for some applications. Here, we analyze and extend on an approach to estimate the FSC from a single measurement. In particular, we derive the necessary conditions required to estimate the FSC from downsampled versions of a single noisy measurement. These conditions reveal additional corrections which we implement to increase the applicability of the method. We then illustrate two applications of our approach, first as an estimate of the global resolution from a single 3-D structure and second as a data-driven method for denoising tomographic reconstructions in electron cryo-tomography. These results provide general guidelines for computing the FSC from a single measurement and suggest new applications of the FSC in microscopy.

Self Fourier shell correlation: properties and application to cryo-ET

The Fourier shell correlation (FSC) is a measure of the similarity between two signals computed over corresponding shells in the frequency domain and has broad applications in microscopy. In structural biology, the FSC is ubiquitous in methods for validation, resolution determination, and signal enhancement. Computing the FSC usually requires two independent measurements of the same underlying signal, which can be limiting for some applications. Here, we analyze and extend on an approach proposed by Koho et al. [1] to estimate the FSC from a single measurement. In particular, we derive the necessary conditions required to estimate the FSC from downsampled versions of a single noisy measurement. These conditions reveal additional corrections which we implement to increase the applicability of the method. We then illustrate two applications of our approach, first as an estimate of the global resolution from a single 3-D structure and second as a data-driven method for denoising tomographic reconstructions in electron cryo-tomography. These results provide general guidelines for computing the FSC from a single measurement and suggest new applications of the FSC in microscopy.

An Unsupervised Classification Algorithm for Heterogeneous Cryo-EM Projection Images Based on Autoencoders

Heterogeneous three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is an important but very challenging technique for recovering the conformational heterogeneity of flexible biological macromolecules such as proteins in different functional states. Heterogeneous projection image classification is a feasible solution to solve the structural heterogeneity problem in single-particle cryo-EM. The majority of heterogeneous projection image classification methods are developed using supervised learning technology or require a large amount of a priori knowledge, such as the orientations or common lines of the projection images, which leads to certain limitations in their practical applications. In this paper, an unsupervised heterogeneous cryo-EM projection image classification algorithm based on autoencoders is proposed, which only needs to know the number of heterogeneous 3D structures in the dataset and does not require any labeling information of the projection images or other a priori knowledge. A simple autoencoder with multi-layer perceptrons trained in iterative mode and a complex autoencoder with residual networks trained in one-pass learning mode are implemented to convert heterogeneous projection images into latent variables. The extracted high-dimensional features are reduced to two dimensions using the uniform manifold approximation and projection dimensionality reduction algorithm, and then clustered using the spectral clustering algorithm. The proposed algorithm is applied to two heterogeneous cryo-EM datasets for heterogeneous 3D reconstruction. Experimental results show that the proposed algorithm can effectively extract category features of heterogeneous projection images and achieve high classification and reconstruction accuracy, indicating that the proposed algorithm is effective for heterogeneous 3D reconstruction in single-particle cryo-EM.

Autocorrelation analysis for cryo-EM with sparsity constraints: Improved sample complexity and projection-based algorithms

The number of noisy images required for molecular reconstruction in single-particle cryoelectron microscopy (cryo-EM) is governed by the autocorrelations of the observed, randomly oriented, noisy projection images. In this work, we consider the effect of imposing sparsity priors on the molecule. We use techniques from signal processing, optimization, and applied algebraic geometry to obtain theoretical and computational contributions for this challenging nonlinear inverse problem with sparsity constraints. We prove that molecular structures modeled as sums of Gaussians are uniquely determined by the second-order autocorrelation of their projection images, implying that the sample complexity is proportional to the square of the variance of the noise. This theory improves upon the nonsparse case, where the third-order autocorrelation is required for uniformly oriented particle images and the sample complexity scales with the cube of the noise variance. Furthermore, we build a computational framework to reconstruct molecular structures which are sparse in the wavelet basis. This method combines the sparse representation for the molecule with projection-based techniques used for phase retrieval in X-ray crystallography.

Unsupervised Cryo-EM Images Denoising and Clustering Based on Deep Convolutional Autoencoder and K-Means+

Cryo-electron microscopy (cryo-EM) is a widely used structural determination technique. Because of the extremely low signal-to-noise ratio (SNR) of images captured by cryo-EM, clustering single-particle cryo-EM images with high accuracy is challenging. To address this, we proposed an iterative denoising and clustering method based on a deep convolutional variational autoencoder and K-means++. The proposed method contains two modules: a denoising ResNet variational autoencoder (DRVAE), and Balance size K-means++ (BSK-means++). First, the DRVAE is trained in a fully unsupervised manner to initialize the neural network and obtain preliminary denoised images. Second, BSK-means++ is built for clustering denoised images, and images closer to class centers are divided into reliable samples. Third, the training of DRVAE is continued, while the class-average images are used as pseudo supervision of reliable samples to reserve more detailed information of denoised images. Finally, the second and third steps mentioned above can be performed jointly and iteratively until convergence occurs. The experimental results showed that the proposed method can generate reliable class average images and achieve better clustering accuracy and normalized mutual information than current methods. This study confirmed that DRVAE with BSK-means++ could achieve a good denoise performance on single-particle cryo-EM images, which can help researchers obtain information such as symmetry and heterogeneity of the target particles. In addition, the proposed method avoids the extreme imbalance of class size, which improves the reliability of the clustering result.

Methods for Cryo-EM Single Particle Reconstruction of Macromolecules Having Continuous Heterogeneity

Macromolecules change their shape (conformation) in the process of carrying out their functions. The imaging by cryo-electron microscopy of rapidly-frozen, individual copies of macromolecules (single particles) is a powerful and general approach to understanding the motions and energy landscapes of macromolecules. Widely-used computational methods already allow the recovery of a few distinct conformations from heterogeneous single-particle samples, but the treatment of complex forms of heterogeneity such as the continuum of possible transitory states and flexible regions remains largely an open problem. In recent years there has been a surge of new approaches for treating the more general problem of continuous heterogeneity. This paper surveys the current state of the art in this area.

A strategy combining denoising and cryo-EM single particle analysis

In cryogenic electron microscopy (cryo-EM) single particle analysis (SPA), high-resolution three-dimensional structures of biological macromolecules are determined by iteratively aligning and averaging a large number of two-dimensional projections of molecules. Since the correlation measures are sensitive to the signal-to-noise ratio, various parameter estimation steps in SPA will be disturbed by the high-intensity noise in cryo-EM. However, denoising algorithms tend to damage high frequencies and suppress mid- and high-frequency contrast of micrographs, which exactly the precise parameter estimation relies on, therefore, limiting their application in SPA. In this study, we suggest combining a cryo-EM image processing pipeline with denoising and maximizing the signal's contribution in various parameter estimation steps. To solve the inherent flaws of denoising algorithms, we design an algorithm named MScale to correct the amplitude distortion caused by denoising and propose a new orientation determination strategy to compensate for the high-frequency loss. In the experiments on several real datasets, the denoised particles are successfully applied in the class assignment estimation and orientation determination tasks, ultimately enhancing the quality of biomacromolecule reconstruction. The case study on classification indicates that our strategy not only improves the resolution of difficult classes (up to 5 Å) but also resolves an additional class. In the case study on orientation determination, our strategy improves the resolution of the final reconstructed density map by 0.34 Å compared with conventional strategy. The code is available at https://github.com/zhanghui186/Mscale.

MULTI-TARGET DETECTION WITH ROTATIONS

We consider the multi-target detection problem of estimating a two-dimensional target image from a large noisy measurement image that contains many randomly rotated and translated copies of the target image. Motivated by single-particle cryo-electron microscopy, we focus on the low signal-to-noise regime, where it is difficult to estimate the locations and orientations of the target images in the measurement. Our approach uses autocorrelation analysis to estimate rotationally and translationally invariant features of the target image. We demonstrate that, regardless of the level of noise, our technique can be used to recover the target image when the measurement is sufficiently large.

Signal enhancement for two-dimensional cryo-EM data processing

Different tasks in the computational pipeline of single-particle cryo-electron microscopy (cryo-EM) require enhancing the quality of the highly noisy raw images. To this end, we develop an efficient algorithm for signal enhancement of cryo-EM images. The enhanced images can be used for a variety of downstream tasks, such as two-dimensional classification, removing uninformative images, constructing ab initio models, generating templates for particle picking, providing a quick assessment of the data set, dimensionality reduction, and symmetry detection. The algorithm includes built-in quality measures to assess its performance and alleviate the risk of model bias. We demonstrate the effectiveness of the proposed algorithm on several experimental data sets. In particular, we show that the quality of the resulting images is high enough to produce ab initio models of Å resolution. The algorithm is accompanied by a publicly available, documented, and easy-to-use code.

Cryo-EM image alignment: From pair-wise to joint with deep unsupervised difference learning

Cryo-electron microscopy (cryo-EM) single-particle analysis is a revolutionary imaging technique to resolve and visualize biomacromolecules. Image alignment in cryo-EM is an important and basic step to improve the precision of the image distance calculation. However, it is a very challenging task due to high noise and low signal-to-noise ratio. Therefore, we propose a new deep unsupervised difference learning (UDL) strategy with novel pseudo-label guided learning network architecture and apply it to pair-wise image alignment in cryo-EM. The training framework is fully unsupervised. Furthermore, a variant of UDL called joint UDL (JUDL), is also proposed, which is capable of utilizing the similarity information of the whole dataset and thus further increase the alignment precision. Assessments on both real-world and synthetic cryo-EM single-particle image datasets suggest the new unsupervised joint alignment method can achieve more accurate alignment results. Our method is highly efficient by taking advantages of GPU devices. The source code of our methods is publicly available at "http://www.csbio.sjtu.edu.cn/bioinf/JointUDL/" for academic use.

Multiple-image super-resolution of cryo-electron micrographs based on deep internal learning

Single-particle cryo-electron microscopy (cryo-EM) is a powerful imaging modality capable of visualizing proteins and macromolecular complexes at near-atomic resolution. The low electron-doses used to prevent radiation damage to the biological samples, however, result in images where the power of the noise is 100 times greater than the power of the signal. To overcome these low signal-to-noise ratios (SNRs), hundreds of thousands of particle projections are averaged to determine the three-dimensional structure of the molecule of interest. The sampling requirements of high-resolution imaging impose limitations on the pixel sizes that can be used for acquisition, limiting the size of the field of view and requiring data collection sessions of several days to accumulate sufficient numbers of particles. Meanwhile, recent image super-resolution (SR) techniques based on neural networks have shown state-of-the-art performance on natural images. Building on these advances, here, we present a multiple-image SR algorithm based on deep internal learning designed specifically to work under low-SNR conditions. Our approach leverages the internal image statistics of cryo-EM movies and does not require training on ground-truth data. When applied to single-particle datasets of apoferritin and T20S proteasome, we show that the resolution of the 3D structure obtained from SR micrographs can surpass the limits imposed by the imaging system. Our results indicate that the combination of low magnification imaging with in silico image SR has the potential to accelerate cryo-EM data collection by virtue of including more particles in each exposure and doing so without sacrificing resolution.

Detecting Non-Overlapping Signals with Dynamic Programming

This paper studies the classical problem of detecting the locations of signal occurrences in a one-dimensional noisy measurement. Assuming the signal occurrences do not overlap, we formulate the detection task as a constrained likelihood optimization problem and design a computationally efficient dynamic program that attains its optimal solution. Our proposed framework is scalable, simple to implement, and robust to model uncertainties. We show by extensive numerical experiments that our algorithm accurately estimates the locations in dense and noisy environments, and outperforms alternative methods.

Toward Single Particle Reconstruction without Particle Picking: Breaking the Detection Limit

Single-particle cryo-electron microscopy (cryo-EM) has recently joined X-ray crystallography and NMR spectroscopy as a high-resolution structural method to resolve biological macromolecules. In a cryo-EM experiment, the microscope produces images called micrographs. Projections of the molecule of interest are embedded in the micrographs at unknown locations, and under unknown viewing directions. Standard imaging techniques first locate these projections (detection) and then reconstruct the 3-D structure from them. Unfortunately, high noise levels hinder detection. When reliable detection is rendered impossible, the standard techniques fail. This is a problem, especially for small molecules. In this paper, we pursue a radically different approach: we contend that the structure could, in principle, be reconstructed directly from the micrographs, without intermediate detection. The aim is to bring small molecules within reach for cryo-EM. To this end, we design an autocorrelation analysis technique that allows one to go directly from the micrographs to the sought structures. This involves only one pass over the micrographs, allowing online, streaming processing for large experiments. We show numerical results and discuss challenges that lay ahead to turn this proof-of-concept into a complementary approach to state-of-the-art algorithms.

Deep generative modeling for volume reconstruction in cryo-electron microscopy

Advances in cryo-electron microscopy (cryo-EM) for high-resolution imaging of biomolecules in solution have provided new challenges and opportunities for algorithm development for 3D reconstruction. Next-generation volume reconstruction algorithms that combine generative modelling with end-to-end unsupervised deep learning techniques have shown promise, but many technical and theoretical hurdles remain, especially when applied to experimental cryo-EM images. In light of the proliferation of such methods, we propose here a critical review of recent advances in the field of deep generative modelling for cryo-EM reconstruction. The present review aims to (i) provide a unified statistical framework using terminology familiar to machine learning researchers with no specific background in cryo-EM, (ii) review the current methods in this framework, and (iii) outline outstanding bottlenecks and avenues for improvements in the field.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

Cryo-electron microscopy (cryo-EM) has become a tool of fundamental importance in structural biology, helping us understand the basic building blocks of life. The algorithmic challenge of cryo-EM is to jointly estimate the unknown 3D poses and the 3D electron scattering potential of a biomolecule from millions of extremely noisy 2D images. Existing reconstruction algorithms, however, cannot easily keep pace with the rapidly growing size of cryo-EM datasets due to their high computational and memory cost. We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data. CryoAI combines a learned encoder that predicts the poses of each particle image with a physics-based decoder to aggregate each particle image into an implicit representation of the scattering potential volume. This volume is stored in the Fourier domain for computational efficiency and leverages a modern coordinate network architecture for memory efficiency. Combined with a symmetrized loss function, this framework achieves results of a quality on par with state-of-the-art cryo-EM solvers for both simulated and experimental data, one order of magnitude faster for large datasets and with significantly lower memory requirements than existing methods.

Artificial Intelligence in Cryo-Electron Microscopy

Cryo-electron microscopy (cryo-EM) has become an unrivaled tool for determining the structure of macromolecular complexes. The biological function of macromolecular complexes is inextricably tied to the flexibility of these complexes. Single particle cryo-EM can reveal the conformational heterogeneity of a biochemically pure sample, leading to well-founded mechanistic hypotheses about the roles these complexes play in biology. However, the processing of increasingly large, complex datasets using traditional data processing strategies is exceedingly expensive in both user time and computational resources. Current innovations in data processing capitalize on artificial intelligence (AI) to improve the efficiency of data analysis and validation. Here, we review new tools that use AI to automate the data analysis steps of particle picking, 3D map reconstruction, and local resolution determination. We discuss how the application of AI moves the field forward, and what obstacles remain. We also introduce potential future applications of AI to use cryo-EM in understanding protein communities in cells.

High-resolution structure determination using high-throughput electron cryo-tomography

Tomographic reconstruction of frozen-hydrated specimens followed by extraction and averaging of sub-tomograms has successfully been used to determine the structure of macromolecules in their native environment at resolutions that are high enough to reveal molecular level interactions. The low throughput characteristic of tomographic data acquisition combined with the complex data-analysis pipeline that is required to obtain high-resolution maps, however, has limited the applicability of this technique to favorable samples or to resolutions that are too low to provide useful mechanistic information. Recently, beam image-shift electron cryo-tomography (BISECT), a strategy to significantly accelerate the acquisition of tilt series without sacrificing image quality, was introduced. The ability to produce thousands of high-quality tilt series during a single microscope session, however, introduces significant bottlenecks in the downstream data analysis, which has so far relied on specialized pipelines. Here, recent advances in accurate estimation of the contrast transfer function and self-tuning exposure-weighting routines that contribute to improving the resolution and streamlining the structure-determination process using sub-volume averaging are reviewed. Ultimately, the combination of automated data-driven techniques for image analysis together with high-throughput strategies for tilt-series acquisition will pave the way for tomography to become the technique of choice for in situ structure determination.

Super-resolution multi-reference alignment

We study super-resolution multi-reference alignment, the problem of estimating a signal from many circularly shifted, down-sampled and noisy observations. We focus on the low SNR regime, and show that a signal inℝ M is uniquely determined when the number L of samples per observation is of the order of the square root of the signal's length ( L = O ( M ) ). Phrased more informally, one can square the resolution. This result holds if the number of observations is proportional to 1/SNR3. In contrast, with fewer observations recovery is impossible even when the observations are not down-sampled (L = M). The analysis combines tools from statistical signal processing and invariant theory. We design an expectation-maximization algorithm and demonstrate that it can super-resolve the signal in challenging SNR regimes.

A molecular prior distribution for Bayesian inference based on Wilson statistics

BACKGROUND AND OBJECTIVE

Wilson statistics describe well the power spectrum of proteins at high frequencies. Therefore, it has found several applications in structural biology, e.g., it is the basis for sharpening steps used in cryogenic electron microscopy (cryo-EM). A recent paper gave the first rigorous proof of Wilson statistics based on a formalism of Wilson's original argument. This new analysis also leads to statistical estimates of the scattering potential of proteins that reveal a correlation between neighboring Fourier coefficients. Here we exploit these estimates to craft a novel prior that can be used for Bayesian inference of molecular structures.

METHODS

We describe the properties of the prior and the computation of its hyperparameters. We then evaluate the prior on two synthetic linear inverse problems, and compare against a popular prior in cryo-EM reconstruction at a range of SNRs.

RESULTS

We show that the new prior effectively suppresses noise and fills-in low SNR regions in the spectral domain. Furthermore, it improves the resolution of estimates on the problems considered for a wide range of SNR and produces Fourier Shell Correlation curves that are insensitive to masking effects.

CONCLUSIONS

We analyze the assumptions in the model, discuss relations to other regularization strategies, and postulate on potential implications for structure determination in cryo-EM.

Cryo-Electron Microscopy: the field of 1,000+ methods

Cryo-Electron Microscopy (CryoEM) is currently a well-established method to elucidate a biological macromolecule's three-dimensional (3D) structure. Its success is due to technological and methodological advances in several fronts: sample preparation, electron optics and detection, image acquisition, image processing, and map interpretation. The first methods started in the late 1960s and, since then, new methods on all fronts have continuously been published, maturating the field as we know it now. In terms of publications, we can distinguish several periods, witnessing a substantial acceleration of methodological publications in recent years, pointing out to an increased interest in the domain. On the other hand, this accelerated increase of methods development may confuse practitioners about which method they should be using (and how) and highlight the importance of paying attention to establishing best practices for methods reporting and usage. In this paper, we analyze the trends identified in over 1,000 methodological papers. Our focus is primarily on computational image processing methods. However, our list also covers some aspects of sample preparation and image acquisition. Several interesting ideas stem out from this study: 1) Single Particle Analysis (SPA) has largely accelerated in the last decade and sample preparation methods in the last five years; 2) Electron Tomography is not yet in a rapidly growing phase, but it is foreseeable that it will soon be; 3) the work horses of SPA are 3D classification, 3D reconstruction, and 3D alignment, and there have been many papers on these topics, which are not considered to be solved yet, but ever improving; and 4) since the resolution revolution, atomic modelling has also caught on as a hot topic.

An approximate expectation-maximization for two-dimensional multi-target detection

We consider the two-dimensional multi-target detection (MTD) problem of estimating a target image from a noisy measurement that contains multiple copies of the image, each randomly rotated and translated. The MTD model serves as a mathematical abstraction of the structure reconstruction problem in single-particle cryo-electron microscopy, the chief motivation of this study. We focus on high noise regimes, where accurate detection of image occurrences within a measurement is impossible. To estimate the image, we develop an expectation-maximization framework that aims to maximize an approximation of the likelihood function. We demonstrate image recovery in highly noisy environments, and show that our framework outperforms the previously studied autocorrelation analysis in a wide range of parameters.

Heterogeneous cryo-EM projection image classification using a two-stage spectral clustering based on novel distance measures

Single-particle cryo-electron microscopy (cryo-EM) has become one of the mainstream technologies in the field of structural biology to determine the three-dimensional (3D) structures of biological macromolecules. Heterogeneous cryo-EM projection image classification is an effective way to discover conformational heterogeneity of biological macromolecules in different functional states. However, due to the low signal-to-noise ratio of the projection images, the classification of heterogeneous cryo-EM projection images is a very challenging task. In this paper, two novel distance measures between projection images integrating the reliability of common lines, pixel intensity and class averages are designed, and then a two-stage spectral clustering algorithm based on the two distance measures is proposed for heterogeneous cryo-EM projection image classification. In the first stage, the novel distance measure integrating common lines and pixel intensities of projection images is used to obtain preliminary classification results through spectral clustering. In the second stage, another novel distance measure integrating the first novel distance measure and class averages generated from each group of projection images is used to obtain the final classification results through spectral clustering. The proposed two-stage spectral clustering algorithm is applied on a simulated and a real cryo-EM dataset for heterogeneous reconstruction. Results show that the two novel distance measures can be used to improve the classification performance of spectral clustering, and using the proposed two-stage spectral clustering algorithm can achieve higher classification and reconstruction accuracy than using RELION and XMIPP.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

End-to-end orientation estimation from 2D cryo-EM images

Cryo-electron microscopy (cryo-EM) is a Nobel Prize-winning technique for determining high-resolution 3D structures of biological macromolecules. A 3D structure is reconstructed from hundreds of thousands of noisy 2D projection images. However, existing 3D reconstruction methods are still time-consuming, and one of the major computational bottlenecks is recovering the unknown orientation of the particle in each 2D image. The dominant methods typically exploit an expensive global search on each image to estimate the missing orientations. Here, a novel end-to-end supervised learning method is introduced to directly recover the missing orientations from 2D cryo-EM images. A neural network is used to approximate the mapping from images to orientations. A robust loss function is proposed for optimizing the parameters of the network, which can handle both asymmetric and symmetric 3D structures. Experiments on synthetic data sets with various symmetry types confirm that the neural network is capable of recovering orientations from 2D cryo-EM images, and the results on a real cryo-EM data set further demonstrate its potential under more challenging imaging conditions.

OUP accepted manuscript

Motivation

Cryo-electron microscopy (cryo-EM) is a widely used technology for ultrastructure determination, which constructs the 3D structures of protein and macromolecular complex from a set of 2D micrographs. However, limited by the electron beam dose, the micrographs in cryo-EM generally suffer from the extremely low signal-to-noise ratio (SNR), which hampers the efficiency and effectiveness of downstream analysis. Especially, the noise in cryo-EM is not simple additive or multiplicative noise whose statistical characteristics are quite different from the ones in natural image, extremely shackling the performance of conventional denoising methods.

Results

Here, we introduce the Noise-Transfer2Clean (NT2C), a denoising deep neural network (DNN) for cryo-EM to enhance image contrast and restore specimen signal, whose main idea is to improve the denoising performance by correctly learning the noise distribution of cryo-EM images and transferring the statistical nature of noise into the denoiser. Especially, to cope with the complex noise model in cryo-EM, we design a contrast-guided noise and signal re-weighted algorithm to achieve clean-noisy data synthesis and data augmentation, making our method authentically achieve signal restoration based on noise’s true properties. Our work verifies the feasibility of denoising based on mining the complex cryo-EM noise patterns directly from the noise patches. Comprehensive experimental results on simulated datasets and real datasets show that NT2C achieved a notable improvement in image denoising, especially in background noise removal, compared with the commonly used methods. Moreover, a case study on the real dataset demonstrates that NT2C can greatly alleviate the obstacles caused by the SNR to particle picking and simplify the identifying of particles.

Availabilityand implementation

The code is available at https://github.com/Lihongjia-ict/NoiseTransfer2Clean/.

Supplementary information

Supplementary data are available at Bioinformatics online.

Wavelet invariants for statistically robust multi-reference alignment

We propose a nonlinear, wavelet-based signal representation that is translation invariant and robust to both additive noise and random dilations. Motivated by the multi-reference alignment problem and generalizations thereof, we analyze the statistical properties of this representation given a large number of independent corruptions of a target signal. We prove the nonlinear wavelet-based representation uniquely defines the power spectrum but allows for an unbiasing procedure that cannot be directly applied to the power spectrum. After unbiasing the representation to remove the effects of the additive noise and random dilations, we recover an approximation of the power spectrum by solving a convex optimization problem, and thus reduce to a phase retrieval problem. Extensive numerical experiments demonstrate the statistical robustness of this approximation procedure.

Tomosipo: fast, flexible, and convenient 3D tomography for complex scanning geometries in Python

Tomography is a powerful tool for reconstructing the interior of an object from a series of projection images. Typically, the source and detector traverse a standard path (e.g., circular, helical). Recently, various techniques have emerged that use more complex acquisition geometries. Current software packages require significant handwork, or lack the flexibility to handle such geometries. Therefore, software is needed that can concisely represent, visualize, and compute reconstructions of complex acquisition geometries. We present tomosipo, a Python package that provides these capabilities in a concise and intuitive way. Case studies demonstrate the power and flexibility of tomosipo.

A Fast Image Alignment Approach for 2D Classification of Cryo-EM Images Using Spectral Clustering

Three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is a significant technique for recovering the 3D structure of proteins or other biological macromolecules from their two-dimensional (2D) noisy projection images taken from unknown random directions. Class averaging in single-particle cryo-EM is an important procedure for producing high-quality initial 3D structures, where image alignment is a fundamental step. In this paper, an efficient image alignment algorithm using 2D interpolation in the frequency domain of images is proposed to improve the estimation accuracy of alignment parameters of rotation angles and translational shifts between the two projection images, which can obtain subpixel and subangle accuracy. The proposed algorithm firstly uses the Fourier transform of two projection images to calculate a discrete cross-correlation matrix and then performs the 2D interpolation around the maximum value in the cross-correlation matrix. The alignment parameters are directly determined according to the position of the maximum value in the cross-correlation matrix after interpolation. Furthermore, the proposed image alignment algorithm and a spectral clustering algorithm are used to compute class averages for single-particle 3D reconstruction. The proposed image alignment algorithm is firstly tested on a Lena image and two cryo-EM datasets. Results show that the proposed image alignment algorithm can estimate the alignment parameters accurately and efficiently. The proposed method is also used to reconstruct preliminary 3D structures from a simulated cryo-EM dataset and a real cryo-EM dataset and to compare them with RELION. Experimental results show that the proposed method can obtain more high-quality class averages than RELION and can obtain higher reconstruction resolution than RELION even without iteration.

Advances in Xmipp for Cryo-Electron Microscopy: From Xmipp to Scipion

Xmipp is an open-source software package consisting of multiple programs for processing data originating from electron microscopy and electron tomography, designed and managed by the Biocomputing Unit of the Spanish National Center for Biotechnology, although with contributions from many other developers over the world. During its 25 years of existence, Xmipp underwent multiple changes and updates. While there were many publications related to new programs and functionality added to Xmipp, there is no single publication on the Xmipp as a package since 2013. In this article, we give an overview of the changes and new work since 2013, describe technologies and techniques used during the development, and take a peek at the future of the package.

Assembly and recognition of keratins: A structural perspective

Keratins are one of the major components of cytoskeletal network and assemble into fibrous structures named intermediate filaments (IFs), which are important for maintaining the mechanical properties of cells and tissues. Over the past decades, evidence has shown that the functions of keratins go beyond providing mechanical support for cells, they interact with multiple cellular components and are widely involved in the pathways of cell proliferation, differentiation, motility and death. However, the structural details of keratins and IFs are largely missing and many questions remain regarding the mechanisms of keratin assembly and recognition. Here we briefly review the current structural models and assembly of keratins as well as the interactions of keratins with the binding partners, which may provide a structural view for understanding the mechanisms of keratins in the biological activities and the related diseases.

Fast Cryo-EM Image Alignment Algorithm Using Power Spectrum Features

Cryo-electron microscopy (cryo-EM) single-particle image analysis is a powerful technique to resolve structures of biomacromolecules, while the challenge is that the cryo-EM image is of a low signal-to-noise ratio. For both two-dimensional image analysis and three-dimensional density map analysis, image alignment is an important step to improve the precision of the image distance calculation. In this paper, we introduce a new algorithm for performing two-dimensional pairwise alignment for cryo-EM particle images, which is based on the Fourier transform and power spectrum analysis. Compared to the existing heuristic iterative alignment methods, our method utilizes the signal distribution and signal feature on images' power spectrum to directly compute the alignment parameters. It does not require iterative computations and is robust against the cryo-EM image noise. Both theoretical analysis and experimental results suggest that our power-spectrum-feature-based alignment method is highly computational-efficient and is capable of offering effective alignment results. This new alignment algorithm is publicly available at: www.csbio.sjtu.edu.cn/bioinf/EMAF/for academic use.

ORTHOGONAL TRACE-SUM MAXIMIZATION: APPLICATIONS, LOCAL ALGORITHMS, AND GLOBAL OPTIMALITY

This paper studies the problem of maximizing the sum of traces of matrix quadratic forms on a product of Stiefel manifolds. This orthogonal trace-sum maximization (OTSM) problem generalizes many interesting problems such as generalized canonical correlation analysis (CCA), Procrustes analysis, and cryo-electron microscopy of the Nobel prize fame. For these applications finding global solutions is highly desirable, but it has been unclear how to find even a stationary point, let alone test its global optimality. Through a close inspection of Ky Fan's classical result [Proc. Natl. Acad. Sci. USA, 35 (1949), pp. 652-655] on the variational formulation of the sum of largest eigenvalues of a symmetric matrix, and a semidefinite programming (SDP) relaxation of the latter, we first provide a simple method to certify global optimality of a given stationary point of OTSM. This method only requires testing whether a symmetric matrix is positive semidefinite. A by-product of this analysis is an unexpected strong duality between Shapiro and Botha [SIAM J. Matrix Anal. Appl., 9 (1988), pp. 378-383] and Zhang and Singer [Linear Algebra Appl., 524 (2017), pp. 159-181]. After showing that a popular algorithm for generalized CCA and Procrustes analysis may generate oscillating iterates, we propose a simple fix that provably guarantees convergence to a stationary point. The combination of our algorithm and certificate reveals novel global optima of various instances of OTSM.

Bayesian Random Tomography of Particle Systems

Random tomography is a common problem in imaging science and refers to the task of reconstructing a three-dimensional volume from two-dimensional projection images acquired in unknown random directions. We present a Bayesian approach to random tomography. At the center of our approach is a meshless representation of the unknown volume as a mixture of spherical Gaussians. Each Gaussian can be interpreted as a particle such that the unknown volume is represented by a particle cloud. The particle representation allows us to speed up the computation of projection images and to represent a large variety of structures accurately and efficiently. We develop Markov chain Monte Carlo algorithms to infer the particle positions as well as the unknown orientations. Posterior sampling is challenging due to the high dimensionality and multimodality of the posterior distribution. We tackle these challenges by using Hamiltonian Monte Carlo and a global rotational sampling strategy. We test the approach on various simulated and real datasets.

Guidelines for Regulated Cell Death Assays: A Systematic Summary, A Categorical Comparison, A Prospective

Over the past few years, the field of regulated cell death continues to expand and novel mechanisms that orchestrate multiple regulated cell death pathways are being unveiled. Meanwhile, researchers are focused on targeting these regulated pathways which are closely associated with various diseases for diagnosis, treatment, and prognosis. However, the complexity of the mechanisms and the difficulties of distinguishing among various regulated types of cell death make it harder to carry out the work and delay its progression. Here, we provide a systematic guideline for the fundamental detection and distinction of the major regulated cell death pathways following morphological, biochemical, and functional perspectives. Moreover, a comprehensive evaluation of different assay methods is critically reviewed, helping researchers to make a reliable selection from among the cell death assays. Also, we highlight the recent events that have demonstrated some novel regulated cell death processes, including newly reported biomarkers (e.g., non-coding RNA, exosomes, and proteins) and detection techniques.

Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination

Three-dimensional reconstruction of the electron-scattering potential of biological macromolecules from electron cryo-microscopy (cryo-EM) projection images is an ill-posed problem. The most popular cryo-EM software solutions to date rely on a regularization approach that is based on the prior assumption that the scattering potential varies smoothly over three-dimensional space. Although this approach has been hugely successful in recent years, the amount of prior knowledge that it exploits compares unfavorably with the knowledge about biological structures that has been accumulated over decades of research in structural biology. Here, a regularization framework for cryo-EM structure determination is presented that exploits prior knowledge about biological structures through a convolutional neural network that is trained on known macromolecular structures. This neural network is inserted into the iterative cryo-EM structure-determination process through an approach that is inspired by regularization by denoising. It is shown that the new regularization approach yields better reconstructions than the current state of the art for simulated data, and options to extend this work for application to experimental cryo-EM data are discussed.

Multi-target Detection with an Arbitrary Spacing Distribution

Motivated by the structure reconstruction problem in single-particle cryo-electron microscopy, we consider the multi-target detection model, where multiple copies of a target signal occur at unknown locations in a long measurement, further corrupted by additive Gaussian noise. At low noise levels, one can easily detect the signal occurrences and estimate the signal by averaging. However, in the presence of high noise, which is the focus of this paper, detection is impossible. Here, we propose two approaches-autocorrelation analysis and an approximate expectation maximization algorithm-to reconstruct the signal without the need to detect signal occurrences in the measurement. In particular, our methods apply to an arbitrary spacing distribution of signal occurrences. We demonstrate reconstructions with synthetic data and empirically show that the sample complexity of both methods scales as SNR^-3 in the low SNR regime.