Classification of electron sub-tomograms with neural networks and its application to template-matching

Towards unsupervised classification of macromolecular complexes in cryo electron tomography: Challenges and opportunities

BACKGROUND AND OBJECTIVES

Cryo electron tomography visualizes native cells at nanometer resolution, but analysis is challenged by noise and artifacts. Recently, supervised deep learning methods have been applied to decipher the 3D spatial distribution of macromolecules. However, in order to discover unknown objects, unsupervised classification techniques are necessary. In this paper, we provide an overview of unsupervised deep learning techniques, discuss the challenges to analyze cryo-ET data, and provide a proof-of-concept on real data.

METHODS

We propose a weakly supervised subtomogram classification method based on transfer learning. We use a deep neural network to learn a clustering friendly representation able to capture 3D shapes in the presence of noise and artifacts. This representation is learned here from a synthetic data set.

RESULTS

We show that when applying k-means clustering given a learning-based representation, it becomes possible to satisfyingly classify real subtomograms according to structural similarity. It is worth noting that no manual annotation is used for performing classification.

CONCLUSIONS

We describe the advantages and limitations of our proof-of-concept and raise several perspectives for improving classification performance.

FSCC: Few-Shot Learning for Macromolecule Classification Based on Contrastive Learning and Distribution Calibration in Cryo-Electron Tomography

Cryo-electron tomography (Cryo-ET) is an emerging technology for three-dimensional (3D) visualization of macromolecular structures in the near-native state. To recover structures of macromolecules, millions of diverse macromolecules captured in tomograms should be accurately classified into structurally homogeneous subsets. Although existing supervised deep learning–based methods have improved classification accuracy, such trained models have limited ability to classify novel macromolecules that are unseen in the training stage. To adapt the trained model to the macromolecule classification of a novel class, massive labeled macromolecules of the novel class are needed. However, data labeling is very time-consuming and labor-intensive. In this work, we propose a novel few-shot learning method for the classification of novel macromolecules (named FSCC). A two-stage training strategy is designed in FSCC to enhance the generalization ability of the model to novel macromolecules. First, FSCC uses contrastive learning to pre-train the model on a sufficient number of labeled macromolecules. Second, FSCC uses distribution calibration to re-train the classifier, enabling the model to classify macromolecules of novel classes (unseen class in the pre-training). Distribution calibration transfers learned knowledge in the pre-training stage to novel macromolecules with limited labeled macromolecules of novel class. Experiments were performed on both synthetic and real datasets. On the synthetic datasets, compared with the state-of-the-art (SOTA) method based on supervised deep learning, FSCC achieves competitive performance. To achieve such performance, FSCC only needs five labeled macromolecules per novel class. However, the SOTA method needs 1100 ∼ 1500 labeled macromolecules per novel class. On the real datasets, FSCC improves the accuracy by 5% ∼ 16% when compared to the baseline model. These demonstrate good generalization ability of contrastive learning and calibration distribution to classify novel macromolecules with very few labeled macromolecules.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

Assessment of scoring functions to rank the quality of 3D subtomogram clusters from cryo-electron tomography

Cryo-electron tomography provides the opportunity for unsupervised discovery of endogenous complexes in situ. This process usually requires particle picking, clustering and alignment of subtomograms to produce an average structure of the complex. When applied to heterogeneous samples, template-free clustering and alignment of subtomograms can potentially lead to the discovery of structures for unknown endogenous complexes. However, such methods require scoring functions to measure and accurately rank the quality of aligned subtomogram clusters, which can be compromised by contaminations from misclassified complexes and alignment errors. Here, we provide the first study to assess the effectiveness of more than 15 scoring functions for evaluating the quality of subtomogram clusters, which differ in the amount of structural misalignments and contaminations due to misclassified complexes. We assessed both experimental and simulated subtomograms as ground truth data sets. Our analysis showed that the robustness of scoring functions varies largely. Most scores were sensitive to the signal-to-noise ratio of subtomograms and often required Gaussian filtering as preprocessing for improved performance. Two scoring functions, Spectral SNR-based Fourier Shell Correlation and Pearson Correlation in the Fourier domain with missing wedge correction, showed a robust ranking of subtomogram clusters without any preprocessing and irrespective of SNR levels of subtomograms. Of these two scoring functions, Spectral SNR-based Fourier Shell Correlation was fastest to compute and is a better choice for handling large numbers of subtomograms. Our results provide a guidance for choosing an accurate scoring function for template-free approaches to detect complexes from heterogeneous samples.

The promise and the challenges of cryo-electron tomography

Structural biologists have traditionally approached cellular complexity in a reductionist manner in which the cellular molecular components are fractionated and purified before being studied individually. This 'divide and conquer' approach has been highly successful. However, awareness has grown in recent years that biological functions can rarely be attributed to individual macromolecules. Most cellular functions arise from their concerted action, and there is thus a need for methods enabling structural studies performed in situ, ideally in unperturbed cellular environments. Cryo-electron tomography (Cryo-ET) combines the power of 3D molecular-level imaging with the best structural preservation that is physically possible to achieve. Thus, it has a unique potential to reveal the supramolecular architecture or 'molecular sociology' of cells and to discover the unexpected. Here, we review state-of-the-art Cryo-ET workflows, provide examples of biological applications, and discuss what is needed to realize the full potential of Cryo-ET.

LoTToR: An Algorithm for Missing-Wedge Correction of the Low-Tilt Tomographic 3D Reconstruction of a Single-Molecule Structure

A single-molecule three-dimensional (3D) structure is essential for understanding the thermal vibrations and dynamics as well as the conformational changes during the chemical reaction of macromolecules. Individual-particle electron tomography (IPET) is an approach for obtaining a snap-shot 3D structure of an individual macromolecule particle by aligning the tilt series of electron tomographic (ET) images of a targeted particle through a focused iterative 3D reconstruction method. The method can reduce the influence on the 3D reconstruction from large-scale image distortion and deformation. Due to the mechanical tilt limitation, 3D reconstruction often contains missing-wedge artifacts, presented as elongation and an anisotropic resolution. Here, we report a post-processing method to correct the missing-wedge artifact. This low-tilt tomographic reconstruction (LoTToR) method contains a model-free iteration process under a set of constraints in real and reciprocal spaces. A proof of concept is conducted by using the LoTToR on a phantom, i.e., a simulated 3D reconstruction from a low-tilt series of images, including that within a tilt range of ±15°. The method is validated by using both negative-staining (NS) and cryo-electron tomography (cryo-ET) experimental data. A significantly reduced missing-wedge artifact verifies the capability of LoTToR, suggesting a new tool to support the future study of macromolecular dynamics, fluctuation and chemical activity from the viewpoint of single-molecule 3D structure determination.

De Novo Structural Pattern Mining in Cellular Electron Cryotomograms

Electron cryotomography enables 3D visualization of cells in a near-native state at molecular resolution. The produced cellular tomograms contain detailed information about a plethora of macromolecular complexes, their structures, abundances, and specific spatial locations in the cell. However, extracting this information in a systematic way is very challenging, and current methods usually rely on individual templates of known structures. Here, we propose a framework called "Multi-Pattern Pursuit" for de novo discovery of different complexes from highly heterogeneous sets of particles extracted from entire cellular tomograms without using information of known structures. These initially detected structures can then serve as input for more targeted refinement efforts. Our tests on simulated and experimental tomograms show that our automated method is a promising tool for supporting large-scale template-free visual proteomics analysis.

The advent of structural biology in situ by single particle cryo-electron tomography

Single particle tomography (SPT), also known as subtomogram averaging, is a powerful technique uniquely poised to address questions in structural biology that are not amenable to more traditional approaches like X-ray crystallography, nuclear magnetic resonance, and conventional cryoEM single particle analysis. Owing to its potential for in situ structural biology at subnanometer resolution, SPT has been gaining enormous momentum in the last five years and is becoming a prominent, widely used technique. This method can be applied to unambiguously determine the structures of macromolecular complexes that exhibit compositional and conformational heterogeneity, both in vitro and in situ. Here we review the development of SPT, highlighting its applications and identifying areas of ongoing development.

Single particle tomography in EMAN2

Single particle tomography (SPT or subtomogram averaging) offers a powerful alternative to traditional 2-D single particle reconstruction for studying conformationally or compositionally heterogeneous macromolecules. It can also provide direct observation (without labeling or staining) of complexes inside cells at nanometer resolution. The development of computational methods and tools for SPT remains an area of active research. Here we present the EMAN2.1 SPT toolbox, which offers a full SPT processing pipeline, from particle picking to post-alignment analysis of subtomogram averages, automating most steps. Different algorithm combinations can be applied at each step, providing versatility and allowing for procedural cross-testing and specimen-specific strategies. Alignment methods include all-vs-all, binary tree, iterative single-model refinement, multiple-model refinement, and self-symmetry alignment. An efficient angular search, Graphic Processing Unit (GPU) acceleration and both threaded and distributed parallelism are provided to speed up processing. Finally, automated simulations, per particle reconstruction of subtiltseries, and per-particle Contrast Transfer Function (CTF) correction have been implemented. Processing examples using both real and simulated data are shown for several structures.

cellPACK: a virtual mesoscope to model and visualize structural systems biology

cellPACK assembles computational models of the biological mesoscale, an intermediate scale (10⁻⁷–10⁻⁸m) between molecular and cellular biology. cellPACK’s modular architecture unites existing and novel packing algorithms to generate, visualize and analyze comprehensive 3D models of complex biological environments that integrate data from multiple experimental systems biology and structural biology sources. cellPACK is currently available as open source code, with tools for validation of models and with recipes and models for five biological systems: blood plasma, cytoplasm, synaptic vesicles, HIV and a mycoplasma cell. We have applied cellPACK to model distributions of HIV envelope protein to test several hypotheses for consistency with experimental observations. Biologists, educators, and outreach specialists can interact with cellPACK models, develop new recipes and perform packing experiments through scripting and graphical user interfaces at http://cellPACK.org.

M-free: scoring the reference bias in sub-tomogram averaging and template matching

Cryo-electron tomography provides a snapshot of the cellular proteome. With template matching, the spatial positions of various macromolecular complexes within their native cellular context can be detected. However, the growing awareness of the reference bias introduced by the cross-correlation based approaches, and more importantly the lack of a reliable confidence measurement in the selection of these macromolecular complexes, has restricted the use of these applications. Here we propose a heuristic, in which the reference bias is measured in real space in an analogous way to the R-free value in X-ray crystallography. We measure the reference bias within the mask used to outline the area of the template, and do not modify the template itself. The heuristic works by splitting the mask into a working and a testing area in a volume ratio of 9:1. While the working area is used during the calculation of the cross-correlation function, the information from both areas is explored to calculate the M-free score. We show using artificial data, that the M-free score gives a reliable measure for the reference bias. The heuristic can be applied in template matching and in sub-tomogram averaging. We further test the applicability of the heuristic in tomograms of purified macromolecules, and tomograms of whole Mycoplasma cells.

High-resolution cryo-electron microscopy on macromolecular complexes and cell organelles

Cryo-electron microscopy techniques and computational 3-D reconstruction of macromolecular assemblies are tightly linked tools in modern structural biology. This symbiosis has produced vast amounts of detailed information on the structure and function of biological macromolecules. Typically, one of two fundamentally different strategies is used depending on the specimens and their environment. A: 3-D reconstruction based on repetitive and structurally identical unit cells that allow for averaging, and B: tomographic 3-D reconstructions where tilt-series between approximately ± 60 and ± 70° at small angular increments are collected from highly complex and flexible structures that are beyond averaging procedures, at least during the first round of 3-D reconstruction. Strategies of group A are averaging-based procedures and collect large number of 2-D projections at different angles that are computationally aligned, averaged together, and back-projected in 3-D space to reach a most complete 3-D dataset with high resolution, today often down to atomic detail. Evidently, success relies on structurally repetitive particles and an aligning procedure that unambiguously determines the angular relationship of all 2-D projections with respect to each other. The alignment procedure of small particles may rely on their packing into a regular array such as a 2-D crystal, an icosahedral (viral) particle, or a helical assembly. Critically important for cryo-methods, each particle will only be exposed once to the electron beam, making these procedures optimal for highest-resolution studies where beam-induced damage is a significant concern. In contrast, tomographic 3-D reconstruction procedures (group B) do not rely on averaging, but collect an entire dataset from the very same structure of interest. Data acquisition requires collecting a large series of tilted projections at angular increments of 1-2° or less and a tilt range of ± 60° or more. Accordingly, tomographic data collection exposes its specimens to a large electron dose, which is particularly problematic for frozen-hydrated samples. Currently, cryo-electron tomography is a rapidly emerging technology, on one end driven by the newest developments of hardware such as super-stabile microscopy stages as well as the latest generation of direct electron detectors and cameras. On the other end, success also strongly depends on new software developments on all kinds of fronts such as tilt-series alignment and back-projection procedures that are all adapted to the very low-dose and therefore very noisy primary data. Here, we will review the status quo of cryo-electron microscopy and discuss the future of cellular cryo-electron tomography from data collection to data analysis, CTF-correction of tilt-series, post-tomographic sub-volume averaging, and 3-D particle classification. We will also discuss the pros and cons of plunge freezing of cellular specimens to vitrified sectioning procedures and their suitability for post-tomographic volume averaging despite multiple artifacts that may distort specimens to some degree.

Practical workflow for cryo focused-ion-beam milling of tissues and cells for cryo-TEM tomography

Vitreous freezing offers a way to study cells and tissue in a near-native state by cryo-transmission electron microscopy (cryo-TEM), which is important when structural information at the macromolecular level is required. Many cells - especially those in tissue - are too thick to study intact in the cryo-TEM. Cryo focused-ion-beam (cryo-FIB) milling is being used in a few laboratories to thin vitreously frozen specimens, thus avoiding the artifacts and difficulties of cryo-ultramicrotomy. However, the technique is challenging because of the need to avoid devitrification and frost accumulation during the entire process, from the initial step of freezing to the final step of loading the specimen into the cryo-TEM. We present a robust workflow that makes use of custom fixtures and devices that can be used for high-pressure-frozen bulk tissue samples as well as for samples frozen on TEM grids.

A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography

The limitation of using low electron doses in non-destructive cryo-electron tomography of biological specimens can be partially offset via averaging of aligned and structurally homogeneous subsets present in tomograms. This type of sub-volume averaging is especially challenging when multiple species are present. Here, we tackle the problem of conformational separation and alignment with a "collaborative" approach designed to reduce the effect of the "curse of dimensionality" encountered in standard pair-wise comparisons. Our new approach is based on using the nuclear norm as a collaborative similarity measure for alignment of sub-volumes, and by exploiting the presence of symmetry early in the processing. We provide a strict validation of this method by analyzing mixtures of intact simian immunodeficiency viruses SIV mac239 and SIV CP-MAC. Electron microscopic images of these two virus preparations are indistinguishable except for subtle differences in conformation of the envelope glycoproteins displayed on the surface of each virus particle. By using the nuclear norm-based, collaborative alignment method presented here, we demonstrate that the genetic identity of each virus particle present in the mixture can be assigned based solely on the structural information derived from single envelope glycoproteins displayed on the virus surface.

Cryo-electron microscopy--a primer for the non-microscopist

Cryo-electron microscopy (cryo-EM) is increasingly becoming a mainstream technology for studying the architecture of cells, viruses and protein assemblies at molecular resolution. Recent developments in microscope design and imaging hardware, paired with enhanced image processing and automation capabilities, are poised to further advance the effectiveness of cryo-EM methods. These developments promise to increase the speed and extent of automation, and to improve the resolutions that may be achieved, making this technology useful to determine a wide variety of biological structures. Additionally, established modalities for structure determination, such as X-ray crystallography and nuclear magnetic resonance spectroscopy, are being routinely integrated with cryo-EM density maps to achieve atomic-resolution models of complex, dynamic molecular assemblies. In this review, which is directed towards readers who are not experts in cryo-EM methodology, we provide an overview of emerging themes in the application of this technology to investigate diverse questions in biology and medicine. We discuss the ways in which these methods are being used to study structures of macromolecular assemblies that range in size from whole cells to small proteins. Finally, we include a description of how the structural information obtained by cryo-EM is deposited and archived in a publicly accessible database.

Shape-based regularization of electron tomographic reconstruction

We introduce a tomographic reconstruction method implemented using a shape-based regularization technique. Spatial models of known features in the structure being reconstructed are integrated into the reconstruction process as regularizers. Our regularization scheme is driven locally through shape information obtained from segmentation and compared with a known spatial model. We demonstrated our method on tomography data from digital phantoms, simulated data, and experimental electron tomography (ET) data of virus complexes. Our reconstruction showed reduced blurring and an improvement in the resolution of the reconstructed volume was also measured. This method also produced improved demarcation of spike boundaries in viral membranes when compared with popular techniques like weighted back projection and the algebraic reconstruction technique. Improved ET reconstructions will provide better structure elucidation and improved feature visualization, which can aid in solving key biological issues. Our method can also be generalized to other tomographic modalities.

High precision alignment of cryo-electron subtomograms through gradient-based parallel optimization

BACKGROUND

Cryo-electron tomography emerges as an important component for structural system biology. It not only allows the structural characterization of macromolecular complexes, but also the detection of their cellular localizations in near living conditions. However, the method is hampered by low resolution, missing data and low signal-to-noise ratio (SNR). To overcome some of these difficulties and enhance the nominal resolution one can align and average a large set of subtomograms. Existing methods for obtaining the optimal alignments are mostly based on an exhaustive scanning of all but discrete relative rigid transformations (i.e. rotations and translations) of one subtomogram with respect to the other.

RESULTS

In this paper, we propose gradient-guided alignment methods based on two popular subtomogram similarity measures, a real space as well as a Fourier-space constrained score. We also propose a stochastic parallel refinement method that increases significantly the efficiency for the simultaneous refinement of a set of alignment candidates. We estimate that our stochastic parallel refinement is on average about 20 to 40 fold faster in comparison to the standard independent refinement approach. Results on simulated data of model complexes and experimental structures of protein complexes show that even for highly distorted subtomograms and with only a small number of very sparsely distributed initial alignment seeds, our combined methods can accurately recover true transformations with a substantially higher precision than the scanning based alignment methods.

CONCLUSIONS

Our methods increase significantly the efficiency and accuracy for subtomogram alignments, which is a key factor for the systematic classification of macromolecular complexes in cryo-electron tomograms of whole cells.

Computational separation of conformational heterogeneity using cryo-electron tomography and 3D sub-volume averaging

We have previously used cryo-electron tomography combined with sub-volume averaging and classification to obtain 3D structures of macromolecular assemblies in cases where a single dominant species was present, and applied these methods to the analysis of a variety of trimeric HIV-1 and SIV envelope glycoproteins (Env). Here, we extend these studies by demonstrating automated, iterative, missing wedge-corrected 3D image alignment and classification methods to distinguish multiple conformations that are present simultaneously. We present a method for measuring the spatial distribution of the vector elements representing distinct conformational states of Env. We identify data processing strategies that allow clear separation of the previously characterized closed and open conformations, as well as unliganded and antibody-liganded states of Env when they are present in mixtures. We show that identifying and removing spikes with the lowest signal-to-noise ratios improves the overall accuracy of alignment between individual Env sub-volumes, and that alignment accuracy, in turn, determines the success of image classification in assessing conformational heterogeneity in heterogeneous mixtures. We validate these procedures for computational separation by successfully separating and reconstructing distinct 3D structures for unliganded and antibody-liganded as well as open and closed conformations of Env present simultaneously in mixtures.

High-throughput subtomogram alignment and classification by Fourier space constrained fast volumetric matching

Cryo-electron tomography allows the visualization of macromolecular complexes in their cellular environments in close-to-live conditions. The nominal resolution of subtomograms can be significantly increased when individual subtomograms of the same kind are aligned and averaged. A vital step for such a procedure are algorithms that speedup subtomogram alignment and improve its accuracy to allow reference-free subtomogram classifications. Such methods will facilitate automation of tomography analysis and overall high throughput in the data processing. Building on previous work, here we propose a fast rotational alignment method that uses the Fourier equivalent form of a popular constrained correlation measure that considers missing wedge corrections and density variances in the subtomograms. The fast rotational search is based on 3D volumetric matching, which improves the rotational alignment accuracy in particular for highly distorted subtomograms with low SNR and tilt angle ranges in comparison to fast rotational matching of projected 2D spherical images. We further integrate our fast rotational alignment method in a reference-free iterative subtomogram classification scheme, and propose a local feature enhancement strategy in the classification process. As a proof of principle, we can demonstrate that the automatic method can successfully classify a large number of experimental subtomograms without the need of a reference structure.

PyTom: a python-based toolbox for localization of macromolecules in cryo-electron tomograms and subtomogram analysis

Cryo-electron tomography (CET) is a three-dimensional imaging technique for structural studies of macromolecules under close-to-native conditions. In-depth analysis of macromolecule populations depicted in tomograms requires identification of subtomograms corresponding to putative particles, averaging of subtomograms to enhance their signal, and classification to capture the structural variations among them. Here, we introduce the open-source platform PyTom that unifies standard tomogram processing steps in a python toolbox. For subtomogram averaging, we implemented an adaptive adjustment of scoring and sampling that clearly improves the resolution of averages compared to static strategies. Furthermore, we present a novel stochastic classification method that yields significantly more accurate classification results than two deterministic approaches in simulations. We demonstrate that the PyTom workflow yields faithful results for alignment and classification of simulated and experimental subtomograms of ribosomes and GroEL(14)/GroEL(14)GroES(7), respectively, as well as for the analysis of ribosomal 60S subunits in yeast cell lysate. PyTom enables parallelized processing of large numbers of tomograms, but also provides a convenient, sustainable environment for algorithmic development.

Putting structure into context: fitting of atomic models into electron microscopic and electron tomographic reconstructions

A complete understanding of complex dynamic cellular processes such as cell migration or cell adhesion requires the integration of atomic level structural information into the larger cellular context. While direct atomic-level information at the cellular level remains inaccessible, electron microscopy, electron tomography and their associated computational image processing approaches have now matured to a point where sub-cellular structures can be imaged in three dimensions at the nanometer scale. Atomic-resolution information obtained by other means can be combined with this data to obtain three-dimensional models of large macromolecular assemblies in their cellular context. This article summarizes some recent advances in this field.

Evidence for short-range helical order in the 30-nm chromatin fibers of erythrocyte nuclei

Chromatin folding in eukaryotes fits the genome into the limited volume of the cell nucleus. Formation of higher-order chromatin structures attenuates DNA accessibility, thus contributing to the control of essential genome functions such as transcription, DNA replication, and repair. The 30-nm fiber is thought to be the first hierarchical level of chromatin folding, but the nucleosome arrangement in the compact 30-nm fiber was previously unknown. We used cryoelectron tomography of vitreous sections to determine the structure of the compact, native 30-nm fiber of avian erythrocyte nuclei. The predominant geometry of the 30-nm fiber revealed by subtomogram averaging is a left-handed two-start helix with approximately 6.5 nucleosomes per 11 nm, in which the nucleosomes are juxtaposed face-to-face but are shifted off their superhelical axes with an axial translation of approximately 3.4 nm and an azimuthal rotation of approximately 54°. The nucleosomes produce a checkerboard pattern when observed in the direction perpendicular to the fiber axis but are not interdigitated. The nucleosome packing within the fibers shows larger center-to-center internucleosomal distances than previously anticipated, thus excluding the possibility of core-to-core interactions, explaining how transcription and regulation factors can access nucleosomes.