OUP accepted manuscript

Improved Denoising of Cryo-Electron Microscopy Micrographs with Simulation-Aware Pretraining

Cryo-electron microscopy (cryo-EM) has emerged as a potent technique for determining the structure and functionality of biological macromolecules. However, limited by the physical imaging conditions, such as low electron beam dose, micrographs in cryo-EM typically contend with an extremely low signal-to-noise ratio (SNR), impeding the efficiency and efficacy of subsequent analyses. Therefore, there is a growing demand for an efficient denoising algorithm designed for cryo-EM micrographs, aiming to enhance the quality of macromolecular analysis. However, owing to the absence of a comprehensive and well-defined dataset with ground truth images, supervised image denoising methods exhibit limited generalization when applied to experimental micrographs. To tackle this challenge, we introduce a simulation-aware image denoising (SaID) pretrained model designed to enhance the SNR of cryo-EM micrographs where the training is solely based on an accurately simulated dataset. First, we propose a parameter calibration algorithm for simulated dataset generation, aiming to align simulation parameters with those of experimental micrographs. Second, leveraging the accurately simulated dataset, we propose to train a deep general denoising model that can well generalize to real experimental cryo-EM micrographs. Comprehensive experimental results demonstrate that our pretrained denoising model achieves excellent denoising performance on experimental cryo-EM micrographs, significantly streamlining downstream analysis.

Joint micrograph denoising and protein localization in cryo-electron microscopy

Cryo-electron microscopy (cryo-EM) is an imaging technique that allows the visualization of proteins and macromolecular complexes at near-atomic resolution. The low electron doses used to prevent radiation damage to the biological samples result in images where the power of noise is 100 times stronger than that of the signal. Accurate identification of proteins from these low signal-to-noise ratio (SNR) images is a critical task, as the detected positions serve as inputs for the downstream 3D structure determination process. Current methods either fail to identify all true positives or result in many false positives, especially when analyzing images from smaller-sized proteins that exhibit extremely low contrast, or require manual labeling that can take days to complete. Acknowledging the fact that accurate protein identification is dependent upon the visual interpretability of micrographs, we propose a framework that can perform denoising and detection in a joint manner and enable particle localization under extremely low SNR conditions using self-supervised denoising and particle identification from sparsely annotated data. We validate our approach on three challenging single-particle cryo-EM datasets and projection images from one cryo-electron tomography dataset with extremely low SNR, showing that it outperforms existing state-of-the-art methods used for cryo-EM image analysis by a significant margin. We also evaluate the performance of our algorithm under decreasing SNR conditions and show that our method is more robust to noise than competing methods.

Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis

In single-particle cryo-electron microscopy (cryo-EM), efficient determination of orientation parameters for particle images poses a significant challenge yet is crucial for reconstructing 3D structures. This task is complicated by the high noise levels in the datasets, which often include outliers, necessitating several time-consuming 2D clean-up processes. Recently, solutions based on deep learning have emerged, offering a more streamlined approach to the traditionally laborious task of orientation estimation. These solutions employ amortized inference, eliminating the need to estimate parameters individually for each image. However, these methods frequently overlook the presence of outliers and may not adequately concentrate on the components used within the network. This paper introduces a novel method using a 10-dimensional feature vector for orientation representation, extracting orientations as unit quaternions with an accompanying uncertainty metric. Furthermore, we propose a unique loss function that considers the pairwise distances between orientations, thereby enhancing the accuracy of our method. Finally, we also comprehensively evaluate the design choices in constructing the encoder network, a topic that has not received sufficient attention in the literature. Our numerical analysis demonstrates that our methodology effectively recovers orientations from 2D cryo-EM images in an end-to-end manner. Notably, the inclusion of uncertainty quantification allows for direct clean-up of the dataset at the 3D level. Lastly, we package our proposed methods into a user-friendly software suite named cryo-forum, designed for easy access by developers.

In situ cryo-electron tomography: a new method to elucidate cytoplasmic zoning at the molecular level

Cryo-electron microscopy was developed as a powerful tool for imaging biological specimens in near-native conditions. Nowadays, advances in technology, equipment and computations make it possible to obtain structures of biomolecules with near-atomic resolution. Furthermore, cryo-electron tomography combined with continuous specimen tilting allows structural analysis of heterogeneous biological specimens. In particular, when combined with a cryo-focused ion beam scanning electron microscope, it becomes possible to directly analyse the structure of the biomolecules within cells, a process known as in situ cryo-electron tomography. This technique has the potential to visualize cytoplasmic zoning, involving liquid-liquid phase separation, caused by biomolecular networks in aqueous solutions, which has been the subject of recent debate. Here, we review advances in structural studies of biomolecules to study cytoplasmic zoning by in situ cryo-electron tomography.

Self Fourier shell correlation: properties and application to cryo-ET

The Fourier shell correlation (FSC) is a measure of the similarity between two signals computed over corresponding shells in the frequency domain and has broad applications in microscopy. In structural biology, the FSC is ubiquitous in methods for validation, resolution determination, and signal enhancement. Computing the FSC usually requires two independent measurements of the same underlying signal, which can be limiting for some applications. Here, we analyze and extend on an approach to estimate the FSC from a single measurement. In particular, we derive the necessary conditions required to estimate the FSC from downsampled versions of a single noisy measurement. These conditions reveal additional corrections which we implement to increase the applicability of the method. We then illustrate two applications of our approach, first as an estimate of the global resolution from a single 3-D structure and second as a data-driven method for denoising tomographic reconstructions in electron cryo-tomography. These results provide general guidelines for computing the FSC from a single measurement and suggest new applications of the FSC in microscopy.

Self Fourier shell correlation: properties and application to cryo-ET

The Fourier shell correlation (FSC) is a measure of the similarity between two signals computed over corresponding shells in the frequency domain and has broad applications in microscopy. In structural biology, the FSC is ubiquitous in methods for validation, resolution determination, and signal enhancement. Computing the FSC usually requires two independent measurements of the same underlying signal, which can be limiting for some applications. Here, we analyze and extend on an approach proposed by Koho et al. [1] to estimate the FSC from a single measurement. In particular, we derive the necessary conditions required to estimate the FSC from downsampled versions of a single noisy measurement. These conditions reveal additional corrections which we implement to increase the applicability of the method. We then illustrate two applications of our approach, first as an estimate of the global resolution from a single 3-D structure and second as a data-driven method for denoising tomographic reconstructions in electron cryo-tomography. These results provide general guidelines for computing the FSC from a single measurement and suggest new applications of the FSC in microscopy.

Computational methods for in situ structural studies with cryogenic electron tomography

Cryo-electron tomography (cryo-ET) plays a critical role in imaging microorganisms in situ in terms of further analyzing the working mechanisms of viruses and drug exploitation, among others. A data processing workflow for cryo-ET has been developed to reconstruct three-dimensional density maps and further build atomic models from a tilt series of two-dimensional projections. Low signal-to-noise ratio (SNR) and missing wedge are two major factors that make the reconstruction procedure challenging. Because only few near-atomic resolution structures have been reconstructed in cryo-ET, there is still much room to design new approaches to improve universal reconstruction resolutions. This review summarizes classical mathematical models and deep learning methods among general reconstruction steps. Moreover, we also discuss current limitations and prospects. This review can provide software and methods for each step of the entire procedure from tilt series by cryo-ET to 3D atomic structures. In addition, it can also help more experts in various fields comprehend a recent research trend in cryo-ET. Furthermore, we hope that more researchers can collaborate in developing computational methods and mathematical models for high-resolution three-dimensional structures from cryo-ET datasets.

A 2D image 3D reconstruction function adaptive denoising algorithm

To address the issue of image denoising algorithms blurring image details during the denoising process, we propose an adaptive denoising algorithm for the 3D reconstruction of 2D images. This algorithm takes into account the inherent visual characteristics of human eyes and divides the image into regions based on the entropy value of each region. The background region is subject to threshold denoising, while the target region undergoes processing using an adversarial generative network. This network effectively handles 2D target images with noise and generates a 3D model of the target. The proposed algorithm aims to enhance the noise immunity of 2D images during the 3D reconstruction process and ensure that the constructed 3D target model better preserves the original image's detailed information. Through experimental testing on 2D images and real pedestrian videos contaminated with noise, our algorithm demonstrates stable preservation of image details. The reconstruction effect is evaluated in terms of noise reduction and the fidelity of the 3D model to the original target. The results show an average noise reduction exceeding 95% while effectively retaining most of the target's feature information in the original image. In summary, our proposed adaptive denoising algorithm improves the 3D reconstruction process by preserving image details that are often compromised by conventional denoising techniques. This has significant implications for enhancing image quality and maintaining target information fidelity in 3D models, providing a promising approach for addressing the challenges associated with noise reduction in 2D images during 3D reconstruction.

Adaptive differentiable grids for cryo-electron tomography reconstruction and denoising

Motivation

Tilt-series cryo-electron tomography is a powerful tool widely used in structural biology to study 3D structures of micro-organisms, macromolecular complexes, etc. Still, the reconstruction process remains an arduous task due to several challenges: The missing-wedge acquisition, sample misalignment and motion, the need to process large data, and, especially, a low signal-to-noise ratio.

Results

Inspired by the recently introduced neural representations, we propose an adaptive learning-based representation of the density field of the captured sample. This representation consists of an octree structure, where each node represents a 3D density grid optimized from the captured projections during the training process. This optimization is performed using a loss that combines a differentiable image formation model with different regularization terms: total variation, boundary consistency, and a cross-nodes non-local constraint. The final reconstruction is obtained by interpolating the learned density grid at the desired voxel positions. The evaluation of our approach using captured data of viruses and cells shows that our proposed representation is well adapted to handle missing wedges, and improves the signal-to-noise ratio of the reconstructed tomogram. The reconstruction quality is highly improved in comparison to the state-of-the-art methods, while using the lowest computing time footprint.

Availability and implementation

The code is available on Github at https://github.com/yuanhaowang1213/adaptivediffgrid_ex.

Signal enhancement for two-dimensional cryo-EM data processing

Different tasks in the computational pipeline of single-particle cryo-electron microscopy (cryo-EM) require enhancing the quality of the highly noisy raw images. To this end, we develop an efficient algorithm for signal enhancement of cryo-EM images. The enhanced images can be used for a variety of downstream tasks, such as two-dimensional classification, removing uninformative images, constructing ab initio models, generating templates for particle picking, providing a quick assessment of the data set, dimensionality reduction, and symmetry detection. The algorithm includes built-in quality measures to assess its performance and alleviate the risk of model bias. We demonstrate the effectiveness of the proposed algorithm on several experimental data sets. In particular, we show that the quality of the resulting images is high enough to produce ab initio models of Å resolution. The algorithm is accompanied by a publicly available, documented, and easy-to-use code.

A Transfer Learning-Based Classification Model for Particle Pruning in Cryo-Electron Microscopy

The cryo-electron microscopy (cryo-EM) single-particle analysis requires tens of thousands of particle projections to reveal structural information of macromolecular complexes. However, due to the low signal-to-noise ratio and the presence of high contrast artifacts and contaminants in the micrographs, the semiautomatic and fully automatic particle picking algorithms tend to suffer from high false-positive rates, which degrades the confidence of structure determination. In this study, we introduce PickerOptimizer (PO), a transfer learning-based classification neural network for particle pruning in cryo-EM, as an additional strategy to complement the current automated particle picking algorithms. To achieve high classification performance with minimal human intervention, we adopted two key strategies: (1) utilizing the transfer learning techniques to train the convolutional neural network, where the knowledge gained from public classification datasets is applied to the field of cryo-EM. (2) Designing a multiloss strategy, a combination of multiple loss functions, to guide the optimization of the network parameters. To reduce the domain shift between cryo-EM images and natural images for pretraining, we build the first image classification dataset for cryo-EM, which contains positive and negative samples collected from EMPIAR entries. The PO is tested on 14 public experimental datasets, achieving accuracy and F1 scores above 95% in most cases. Furthermore, three case studies are provided to verify the model performance by applying PO on problematic particle selections, showing that our algorithm achieved better or comparable performance compared with other particle pruning strategies.

VP-Detector: A 3D multi-scale dense convolutional neural network for macromolecule localization and classification in cryo-electron tomograms

BACKGROUND AND OBJECTIVE

Cryo-electron tomography (cryo-ET) with subtomogram averaging (STA) is indispensable when studying macromolecule structures and functions in their native environments. Due to the low signal-to-noise ratio, the missing wedge artifacts in tomographic reconstructions, and multiple macromolecules of varied shapes and sizes, macromolecule localization and classification remain challenging. To tackle this bottleneck problem for structural determination by STA, we design an accurate macromolecule localization and classification method named voxelwise particle detector (VP-Detector).

METHODS

VP-Detector is a two-stage particle detection method based on a 3D multiscale dense convolutional neural network (3D MSDNet). The proposed network uses 3D hybrid dilated convolution (3D HDC) to avoid the resolution loss caused by scaling operations. Meanwhile, it uses 3D dense connectivity to encourage the reuse of feature maps to reduce trainable parameters. In addition, the weighted focal loss is proposed to focus more attention on difficult samples and rare classes, which relieves the class imbalance caused by multiple particles of various sizes. The performance of VP-Detector is evaluated on both simulated and real-world tomograms, and it shows that VP-Detector outperforms state-of-the-art methods.

RESULTS

The experiments show that VP-Detector outperforms the state-of-the-art methods on particle localization with an F1-score of 0.951 and a precision of 0.978. In addition, VP-Detector can replace manual particle picking in experiment on the real-world tomograms. Furthermore, it performs well in classifying large-, medium-, and small-weight proteins with accuracies of 1, 0.95, and 0.82, respectively. Finally, ablation studies demonstrate the effectiveness of 3D HDC, 3D dense connectivity, weighted focal loss, and training on small training sets.

CONCLUSIONS

VP-Detector can achieve high accuracy in particle detection with few trainable parameters and support training on small datasets. It can also relieve the class imbalance caused by multiple particles with various shapes and sizes.