Automatic alignment of transmission electron microscope tilt series without fiducial markers

Automatic marker-based alignment method for a nano-resolution full-field transmission X-ray microscope

Driven by the development of X-ray optics, the spatial resolution of the full-field transmission X-ray microscope (TXM) has reached tens of nanometers and plays an important role in promoting the development of biomedicine and materials science. However, due to the thermal drift and the radial/axial motion error of the rotation stage, TXM computed tomography (CT) data are often associated with random image jitter errors along the horizontal and vertical directions during CT measurement. A nano-resolution 3D structure information reconstruction is almost impossible without a prior appropriate alignment process. To solve this problem, a fully automatic gold particle marker-based alignment approach without human intervention was proposed in this study. It can automatically detect, isolate, and register gold particles for projection image alignment with high efficiency and accuracy, facilitating a high-quality tomographic reconstruction. Simulated and experimental results confirmed the reliability and robustness of this method.

Subtomogram analysis: The sum of a tomogram's particles reveals molecular structure in situ

Cryo-electron tomography is uniquely suited to provide insights into the molecular architecture of cells and tissue in the native state. While frozen hydrated specimens tolerate sufficient electron doses to distinguish different types of particles in a tomogram, the accumulating beam damage does not allow resolving their detailed molecular structure individually. Statistical methods for subtomogram averaging and classification that coherently enhance the signal of particles corresponding to copies of the same type of macromolecular allow obtaining much higher resolution insights into macromolecules. Here, I review the developments in subtomogram analysis at Wolfgang Baumeister's laboratory that make the dream of structural biology in the native cell become reality.

Improvements on marker-free images alignment for electron tomography

Electron tomography is a technique to obtain three-dimensional structural information of samples. However, the technique is limited by shifts occurring during acquisition that need to be corrected before the reconstruction process. In 2009, we proposed an approach for post-acquisition alignment of tilt series images. This approach was marker-free, based on patch tracking and integrated in free software. Here, we present improvements to the method to make it more reliable, stable and accurate. In addition, we modified the image formation model underlying the alignment procedure to include different deformations occurring during acquisition. We propose a new way to correct these computed deformations to obtain reconstructions with reduced artifacts. The new approach has demonstrated to improve the quality of the final 3D reconstruction, giving access to better defined structures for different transmission electron tomography methods: resin embedded STEM-tomography and cryo-TEM tomography. The method is freely available in TomoJ software.

Joint Iterative Fast Projection Matching for Fully Automatic Marker-free Alignment of Nano-tomography Reconstructions

Highly accurate, fully automatic marker-free image alignment plays an important role in nano-tomographic reconstruction, particularly in cases where the spatial resolution of the tomographic system is on the nanometer scale. However, highly accurate marker-free methods such as the projection matching method are computationally complex and time-consuming. Achieving alignment accuracy with reduced computational complexity remains a challenging problem. In this study, we propose an efficient method to achieve marker-free fully automatic alignment. Our method implements three main alignment procedures. First, the frequency-domain common line alignment method is used to correct the in-plane rotational errors of each projection. Second, real-space common line alignment method is used to correct the vertical errors of the projections. Finally, a single layer joint-iterative reconstruction and re-projection method is used to correct the horizontal projection errors. This combined alignment approach significantly reduces the computational complexity of the classical projection matching method, and increases the rate of convergence towards determining the accurate alignment. The total processing time can be reduced by up to 4 orders of magnitude as compared to the classical projection matching method. This suggests that the algorithm can be used to process image alignment of nano-tomographic reconstructions on a conventional personal computer in a reasonable time-frame.

A joint method for marker-free alignment of tilt series in electron tomography

MOTIVATION

Electron tomography (ET) is a widely used technology for 3D macro-molecular structure reconstruction. To obtain a satisfiable tomogram reconstruction, several key processes are involved, one of which is the calibration of projection parameters of the tilt series. Although fiducial marker-based alignment for tilt series has been well studied, marker-free alignment remains a challenge, which requires identifying and tracking the identical objects (landmarks) through different projections. However, the tracking of these landmarks is usually affected by the pixel density (intensity) change caused by the geometry difference in different views. The tracked landmarks will be used to determine the projection parameters. Meanwhile, different projection parameters will also affect the localization of landmarks. Currently, there is no alignment method that takes interrelationship between the projection parameters and the landmarks.

RESULTS

Here, we propose a novel, joint method for marker-free alignment of tilt series in ET, by utilizing the information underlying the interrelationship between the projection model and the landmarks. The proposed method is the first joint solution that combines the extrinsic (track-based) alignment and the intrinsic (intensity-based) alignment, in which the localization of landmarks and projection parameters keep refining each other until convergence. This iterative approach makes our solution robust to different initial parameters and extreme geometric changes, which ensures a better reconstruction for marker-free ET. Comprehensive experimental results on three real datasets show that our new method achieved a significant improvement in alignment accuracy and reconstruction quality, compared to the state-of-the-art methods.

AVAILABILITY AND IMPLEMENTATION

The main program is available at https://github.com/icthrm/joint-marker-free-alignment.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

A fast fiducial marker tracking model for fully automatic alignment in electron tomography

Motivation

Automatic alignment, especially fiducial marker-based alignment, has become increasingly important due to the high demand of subtomogram averaging and the rapid development of large-field electron microscopy. Among the alignment steps, fiducial marker tracking is a crucial one that determines the quality of the final alignment. Yet, it is still a challenging problem to track the fiducial markers accurately and effectively in a fully automatic manner.

Results

In this paper, we propose a robust and efficient scheme for fiducial marker tracking. Firstly, we theoretically prove the upper bound of the transformation deviation of aligning the positions of fiducial markers on two micrographs by affine transformation. Secondly, we design an automatic algorithm based on the Gaussian mixture model to accelerate the procedure of fiducial marker tracking. Thirdly, we propose a divide-and-conquer strategy against lens distortions to ensure the reliability of our scheme. To our knowledge, this is the first attempt that theoretically relates the projection model with the tracking model. The real-world experimental results further support our theoretical bound and demonstrate the effectiveness of our algorithm. This work facilitates the fully automatic tracking for datasets with a massive number of fiducial markers.

Availability and implementation

The C/C ++ source code that implements the fast fiducial marker tracking is available at https://github.com/icthrm/gmm-marker-tracking. Markerauto 1.6 version or later (also integrated in the AuTom platform at http://ear.ict.ac.cn/) offers a complete implementation for fast alignment, in which fast fiducial marker tracking is available by the '-t' option.

Contact

xin.gao@kaust.edu.sa.

Supplementary information

Supplementary data are available at Bioinformatics online.

Analytical registration of vertical image drifts in parallel beam tomographic data

Reconstructing tomographic images of high resolution, as in x-ray microscopy or transmission electron microscopy, is often limited by the stability of the stages or sample drifts, which requires an image alignment prior to reconstruction. Feature-based image registration is routinely used to align images, but this technique relies on strong features in the sample or the application of gold tracer particles, for example. In this Letter, we present an analytic approach for achieving the vertical registration based on the inherent properties of the data acquired for tomographic reconstruction. It is computationally cheap to implement and can be easily integrated into existing reconstruction pipelines.

Improved Three-Dimensional (3D) Resolution of Electron Tomograms Using Robust Mathematical Data-Processing Techniques

Electron tomography has become an essential tool for three-dimensional (3D) characterization of nanomaterials. In recent years, advances have been made in specimen preparation and mounting, acquisition geometries, and reconstruction algorithms. All of these components work together to optimize the resolution and clarity of an electron tomogram. However, one important component of the data-processing has received less attention: the 2D tilt series alignment. This is challenging for a number of reasons, namely because the nature of the data sets and the need to be coherently aligned over the full range of angles. An inaccurate alignment may be difficult to identify, yet can significantly limit the final 3D resolution. In this work, we present an improved center-of-mass alignment model that allows us to overcome discrepancies from unwanted objects that enter the imaging area throughout the tilt series. In particular, we develop an approach to overcome changes in the total mass upon rotation of the imaging area. We apply our approach to accurately recover small Pt nanoparticles embedded in a zeolite that may otherwise go undetected both in the 2D microscopy images and the 3D reconstruction. In addition to this, we highlight the particular effectiveness of the compressed sensing methods with this data set.

Automated angular and translational tomographic alignment and application to phase-contrast imaging

X-ray computerized tomography (CT) is a 3D imaging technique that makes use of x-ray illumination and image reconstruction techniques to reproduce the internal cross-sections of a sample. Tomographic projection data usually require an initial relative alignment or knowledge of the exact object position and orientation with respect to the detector. As tomographic imaging reaches increasingly better resolution, thermal drifts, mechanical instabilities, and equipment limitations are becoming the main dominant factors contributing to sample positioning uncertainties that will further introduce reconstruction artifacts and limit the attained resolution in the final tomographic reconstruction. Alignment algorithms that require manual interaction impede data analysis with ever-increasing data acquisition rates, supplied by more brilliant sources. We present in this paper an iterative reconstruction algorithm for wrapped phase projection data and an alignment algorithm that automatically takes 5 degrees of freedom, including the possible linear and angular motion errors, into consideration. The presented concepts are applied to simulated and real measured phase-contrast data, exhibiting a possible improvement in the reconstruction resolution. A MATLAB implementation is made publicly available and will allow robust analysis of large volumes of phase-contrast tomography data.

Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection

As x-ray and electron tomography is pushed further into the nanoscale, the limitations of rotation stages become more apparent, leading to challenges in the alignment of the acquired projection images. Here we present an approach for rapid post-acquisition alignment of these projections to obtain high quality three-dimensional images. Our approach is based on a joint estimation of alignment errors, and the object, using an iterative refinement procedure. With simulated data where we know the alignment error of each projection image, our approach shows a residual alignment error that is a factor of a thousand smaller, and it reaches the same error level in the reconstructed image in less than half the number of iterations. We then show its application to experimental data in x-ray and electron nanotomography.

AuTom: A novel automatic platform for electron tomography reconstruction

We have developed a software package towards automatic electron tomography (ET): Automatic Tomography (AuTom). The presented package has the following characteristics: accurate alignment modules for marker-free datasets containing substantial biological structures; fully automatic alignment modules for datasets with fiducial markers; wide coverage of reconstruction methods including a new iterative method based on the compressed-sensing theory that suppresses the "missing wedge" effect; and multi-platform acceleration solutions that support faster iterative algebraic reconstruction. AuTom aims to achieve fully automatic alignment and reconstruction for electron tomography and has already been successful for a variety of datasets. AuTom also offers user-friendly interface and auxiliary designs for file management and workflow management, in which fiducial marker-based datasets and marker-free datasets are addressed with totally different subprocesses. With all of these features, AuTom can serve as a convenient and effective tool for processing in electron tomography.

Fast Projection Matching for X-ray Tomography

X-ray 3D tomographic techniques are powerful tools for investigating the morphology and internal structures of specimens. A common strategy for obtaining 3D tomography is to capture a series of 2D projections from different X-ray illumination angles of specimens mounted on a finely calibrated rotational stage. However, the reconstruction quality of 3D tomography relies on the precision and stability of the rotational stage, i.e. the accurate alignment of the 2D projections in the correct three-dimensional positions. This is a crucial problem for nano-tomographic techniques due to the non-negligible mechanical imperfection of the rotational stages at the nanometer level which significantly degrades the spatial resolution of reconstructed 3-D tomography. Even when using an X-ray micro-CT with a highly stabilized rotational stage, thermal effects caused by the CT system are not negligible and may cause sample drift. Here, we propose a markerless image auto-alignment algorithm based on an iterative method. This algorithm reduces the traditional projection matching method into two simplified matching problems and it is much faster and more reliable than traditional methods. This algorithm can greatly decrease hardware requirements for both nano-tomography and data processing and can be easily applied to other tomographic techniques, such as X-ray micro-CT and electron tomography.

The advent of structural biology in situ by single particle cryo-electron tomography

Single particle tomography (SPT), also known as subtomogram averaging, is a powerful technique uniquely poised to address questions in structural biology that are not amenable to more traditional approaches like X-ray crystallography, nuclear magnetic resonance, and conventional cryoEM single particle analysis. Owing to its potential for in situ structural biology at subnanometer resolution, SPT has been gaining enormous momentum in the last five years and is becoming a prominent, widely used technique. This method can be applied to unambiguously determine the structures of macromolecular complexes that exhibit compositional and conformational heterogeneity, both in vitro and in situ. Here we review the development of SPT, highlighting its applications and identifying areas of ongoing development.

Automated tilt series alignment and tomographic reconstruction in IMOD

Automated tomographic reconstruction is now possible in the IMOD software package, including the merging of tomograms taken around two orthogonal axes. Several developments enable the production of high-quality tomograms. When using fiducial markers for alignment, the markers to be tracked through the series are chosen automatically; if there is an excess of markers available, a well-distributed subset is selected that is most likely to track well. Marker positions are refined by applying an edge-enhancing Sobel filter, which results in a 20% improvement in alignment error for plastic-embedded samples and 10% for frozen-hydrated samples. Robust fitting, in which outlying points are given less or no weight in computing the fitting error, is used to obtain an alignment solution, so that aberrant points from the automated tracking can have little effect on the alignment. When merging two dual-axis tomograms, the alignment between them is refined from correlations between local patches; a measure of structure was developed so that patches with insufficient structure to give accurate correlations can now be excluded automatically. We have also developed a script for running all steps in the reconstruction process with a flexible mechanism for setting parameters, and we have added a user interface for batch processing of tilt series to the Etomo program in IMOD. Batch processing is fully compatible with interactive processing and can increase efficiency even when the automation is not fully successful, because users can focus their effort on the steps that require manual intervention.

Physically motivated global alignment method for electron tomography

Electron tomography is widely used for nanoscale determination of 3-D structures in many areas of science. Determining the 3-D structure of a sample from electron tomography involves three major steps: acquisition of sequence of 2-D projection images of the sample with the electron microscope, alignment of the images to a common coordinate system, and 3-D reconstruction and segmentation of the sample from the aligned image data. The resolution of the 3-D reconstruction is directly influenced by the accuracy of the alignment, and therefore, it is crucial to have a robust and dependable alignment method. In this paper, we develop a new alignment method which avoids the use of markers and instead traces the computed paths of many identifiable ‘local’ center-of-mass points as the sample is rotated. Compared with traditional correlation schemes, the alignment method presented here is resistant to cumulative error observed from correlation techniques, has very rigorous mathematical justification, and is very robust since many points and paths are used, all of which inevitably improves the quality of the reconstruction and confidence in the scientific results.

Geometric correction method for 3D in-line X-ray phase contrast image reconstruction

BACKGROUND

Mechanical system with imperfect or misalignment of X-ray phase contrast imaging (XPCI) components causes projection data misplaced, and thus result in the reconstructed slice images of computed tomography (CT) blurred or with edge artifacts. So the features of biological microstructures to be investigated are destroyed unexpectedly, and the spatial resolution of XPCI image is decreased. It makes data correction an essential pre-processing step for CT reconstruction of XPCI.

METHODS

To remove unexpected blurs and edge artifacts, a mathematics model for in-line XPCI is built by considering primary geometric parameters which include a rotation angle and a shift variant in this paper. Optimal geometric parameters are achieved by finding the solution of a maximization problem. And an iterative approach is employed to solve the maximization problem by using a two-step scheme which includes performing a composite geometric transformation and then following a linear regression process. After applying the geometric transformation with optimal parameters to projection data, standard filtered back-projection algorithm is used to reconstruct CT slice images.

RESULTS

Numerical experiments were carried out on both synthetic and real in-line XPCI datasets. Experimental results demonstrate that the proposed method improves CT image quality by removing both blurring and edge artifacts at the same time compared to existing correction methods.

CONCLUSIONS

The method proposed in this paper provides an effective projection data correction scheme and significantly improves the image quality by removing both blurring and edge artifacts at the same time for in-line XPCI. It is easy to implement and can also be extended to other XPCI techniques.

A new field-of-view autotracking method for online tomography reconstruction based on back-projected ray image cross-correlation

We devised a new field-of-view autotracking method for online tomography reconstruction based on a cross-correlation between a pair of neighbours, called back-projected ray images, among a specimen tilt sequence. One ray image is calculated through normal filtered back-projection only in the cross-sectional plane from each projection image. This ray-image matching can reliably track the field-of-view because a pair of neighbouring ray images mostly cross-correlates at the existing three-dimensional object position. Online experiments using real specimens resulted in successful autotracking performance with high accuracy, and online tomograms were obtained immediately after the final tracking at the last tilting angle.

Tomography by point source digital holographic microscopy

We propose a tomographic method for point source inline holographic microscopy. By recording a set of holograms at different illumination angles, shadowing effects are eliminated resulting in three-dimensional images with the same precision at the micrometer-scale in all directions. The advantage of our tomographic approach is that it works for both absorbing and phase objects, regardless of the change of refractive index at interfaces. We develop the method with computer simulations and demonstrate its strength by presenting experimental results for micrometer-sized polystyrene beads and a cotton fiber.

Alternative automatic alignment method for specimen tilt-series images based on back-projected volume data cross-correlations

We devised a new automatic image alignment method for a specimen tilt series; this method is based on the volume data cross-correlation among 3-D cross-sections reconstructed from different sets of projection images (including a single image) for tilt-series alignment or tilt-axis search purposes. This method requires neither markers nor image feature points traceable through the tilt series, and it was examined through simulations and applied to biological thin sections. The method automatically aligned tilt series centred at the correctly detected tilt axis with a precision sufficient for practical applications.

High-precision alignment of electron tomography tilt series using markers formed in helium-ion microscope

Tungsten nanodots formed in a helium-ion microscope (HIM) provide a practical means of aligning markers of electron tomography tilt series with a high degree of precision. The nanodots were formed using a HIM equipped with a W(CO)6 gas injection system, enabling the precise placement of the nanodots at desired locations of a sample. Template matching was applied to the markers formed in the HIM to detect the positions automatically. The relation between the positions of the markers and the accuracy of the alignment was also determined in order to achieve precise alignment. The method was applied to the markers in order to reconstruct three-dimensional (3D) images of a rod-shaped specimen that contained a 65-nm-diameter via structure in a Cu/Low-k interconnect.

Reconstructing virus structures from nanometer to near-atomic resolutions with cryo-electron microscopy and tomography

The past few decades have seen tremendous advances in single-particle electron -cryo-microscopy (cryo-EM). The field has matured to the point that near-atomic resolution density maps can be generated for icosahedral viruses without the need for crystallization. In parallel, substantial progress has been made in determining the structures of nonicosahedrally arranged proteins in viruses by employing either single-particle cryo-EM or cryo-electron tomography (cryo-ET). Implicit in this course have been the availability of a new generation of electron cryo-microscopes and the development of the computational tools that are essential for generating these maps and models. This methodology has enabled structural biologists to analyze structures in increasing detail for virus particles that are in different morphogenetic states. Furthermore, electron imaging of frozen, hydrated cells, in the process of being infected by viruses, has also opened up a new avenue for studying virus structures "in situ". Here we present the common techniques used to acquire and process cryo-EM and cryo-ET data and discuss their implications for structural virology both now and in the future.

New insights into HTLV-1 particle structure, assembly, and Gag-Gag interactions in living cells

Human T-cell leukemia virus type 1 (HTLV-1) has a reputation for being extremely difficult to study in cell culture. The challenges in propagating HTLV-1 has prevented a rigorous analysis of how these viruses replicate in cells, including the detailed steps involved in virus assembly. The details for how retrovirus particle assembly occurs are poorly understood, even for other more tractable retroviral systems. Recent studies on HTLV-1 using state-of-the-art cryo-electron microscopy and fluorescence-based biophysical approaches explored questions related to HTLV-1 particle size, Gag stoichiometry in virions, and Gag-Gag interactions in living cells. These results provided new and exciting insights into fundamental aspects of HTLV-1 particle assembly-which are distinct from those of other retroviruses, including HIV-1. The application of these and other novel biophysical approaches promise to provide exciting new insights into HTLV-1 replication.

Automatic coarse-alignment for TEM tilt series of rod-shaped specimens collected with a full angular range

An automatic coarse-alignment method for a tilt series of rod-shaped specimen collected with a full angular range (from alpha=-90 degrees to +90 degrees, alpha is the tilt angle of the specimen) is presented; this method is based on a cross-correlation method and uses the outline of the specimen shape. Both the rotational angle of the tilt axis and translational value of each image can be detected in the images without the use of markers. This method is performed on the basis of the assumption that the images taken at alpha=-90 degrees and alpha=+90 degrees are symmetric about the tilt axis. In this study, a carbon rod on which gold particles have been deposited is used as a test specimen for the demonstration. This method can be used as an automatic coarse-alignment method prior to the application of a highly accurate alignment method because the alignment procedure can be performed automatically except for the initial setup of some parameters.

Alignator: a GPU powered software package for robust fiducial-less alignment of cryo tilt-series

The robust alignment of tilt-series collected for cryo-electron tomography in the absence of fiducial markers, is a problem that, especially for tilt-series of vitreous sections, still represents a significant challenge. Here we present a complete software package that implements a cross-correlation-based procedure that tracks similar image features that are present in several micrographs and explores them implicitly as substitutes for fiducials like gold beads and quantum dots. The added value compared to previous approaches, is that the algorithm explores a huge number of random positions, which are tracked on several micrographs, while being able to identify trace failures, using a cross-validation procedure based on the 3D marker model of the tilt-series. Furthermore, this method allows the reliable identification of areas which behave as a rigid body during the tilt-series and hence addresses specific difficulties for the alignment of vitreous sections, by correcting practical caveats. The resulting alignments can attain sub-pixel precision at the local level and is able to yield a substantial number of usable tilt-series (around 60%). In principle, the algorithm has the potential to run in a fully automated fashion, and could be used to align any tilt-series directly from the microscope. Finally, we have significantly improved the user interface and implemented the source code on the graphics processing unit (GPU) to accelerate the computations.

Alignment of cryo-electron tomography datasets

Data acquisition of cryo-electron tomography (CET) samples described in previous chapters involves relatively imprecise mechanical motions: the tilt series has shifts, rotations, and several other distortions between projections. Alignment is the procedure of correcting for these effects in each image and requires the estimation of a projection model that describes how points from the sample in three-dimensions are projected to generate two-dimensional images. This estimation is enabled by finding corresponding common features between images. This chapter reviews several software packages that perform alignment and reconstruction tasks completely automatically (or with minimal user intervention) in two main scenarios: using gold fiducial markers as high contrast features or using relevant biological structures present in the image (marker-free). In particular, we emphasize the key decision points in the process that users should focus on in order to obtain high-resolution reconstructions.

Marker-free image registration of electron tomography tilt-series

Background

Tilt series are commonly used in electron tomography as a means of collecting three-dimensional information from two-dimensional projections. A common problem encountered is the projection alignment prior to 3D reconstruction. Current alignment techniques usually employ gold particles or image derived markers to correctly align the images. When these markers are not present, correlation between adjacent views is used to align them. However, sequential pairwise correlation is prone to bias and the resulting alignment is not always optimal.

Results

In this paper we introduce an algorithm to find regions of the tilt series which can be tracked within a subseries of the tilt series. These regions act as landmarks allowing the determination of the alignment parameters. We show our results with synthetic data as well as experimental cryo electron tomography.

Conclusion

Our algorithm is able to correctly align a single-tilt tomographic series without the help of fiducial markers thanks to the detection of thousands of small image patches that can be tracked over a short number of images in the series.

Reprint of "Fiducial-less alignment of cryo-sections" [J. Struct. Biol. 159 (2007) 413-423]

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.

3D electron microscopy of biological nanomachines: principles and applications

Transmission electron microscopy is a powerful technique for studying the three-dimensional (3D) structure of a wide range of biological specimens. Knowledge of this structure is crucial for fully understanding complex relationships among macromolecular complexes and organelles in living cells. In this paper, we present the principles and main application domains of 3D transmission electron microscopy in structural biology. Moreover, we survey current developments needed in this field, and discuss the close relationship of 3D transmission electron microscopy with other experimental techniques aimed at obtaining structural and dynamical information from the scale of whole living cells to atomic structure of macromolecular complexes.

Fiducial-less alignment of cryo-sections

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.

Structure-from-motion without correspondence from tomographic projections by Bayesian inversion theory

In conventional tomography, the interior of an object is reconstructed from tomographic projections such as X-ray or transmission electron microscope images. All the current reconstruction methods assume that projection geometry of the imaging device is either known or solved in advance by using e.g., fiducial or nonfiducial feature points in the images. In this paper, we propose a novel approach where the imaging geometry is solved simultaneously with the volume reconstruction problem while no correspondence information is needed. Our approach is a direct application of Bayesian inversion theory and produces the maximum likelihood or maximum a posteriori estimates for the motion parameters under the selected noise and prior distributions. In this paper, the method is implemented for a two-dimensional model problem with one-dimensional affine projection data. The performance of the method is tested with simulated and measured X-ray projection data.

Three-dimensional reconstruction of chromosomes using electron tomography

The evolvement of preparative methods in structural studies has always been as important as the development of sophisticated equipment. Software development is also a significant part for three-dimensional (3D) structural studies using electron tomography methods (ETMs). Advanced computing makes amenable procedures that relatively recently were only visionary, such as the 3D reconstruction of chromosomes with ETM. Morphological guidelines and beauty are occasionally the only standard for a method to be acceptable in the realms of preparative as well as software development. Bulk isolation of metaphase chromosomes using acetic acid is such an apparent accomplishment in preparative methods. Our ETM with maximum entropy and, more so, the ongoing development toward fully automatic alignments, are contributions in the software line. Furthermore, whole mounting of chromosomes on holey-carbon grids makes it possible to use even yesterday's 80-kV transmission electron microscope with a standard goniometer to collect tilt series. These advances in preparing whole-mount metaphase chromosomes enable laboratories that do not have access to a medium- or high-voltage transmission electron microscope to study complex structures like chromosomes in 3D using today's desktop computers.

Electron tomography of amplified nanogold immunolabelling: Improvement of quality based on alignment of projections with sinograms and use of post-reconstruction deconvolution

Electron tomography of immunolabelled proteins identified with amplified nanogold particles imaged by Scanning and Transmission Electron Microscopy within thick sections is a powerful method to investigate the three-dimensional organization of complex cellular machineries. In order to increase the overall quality of the reconstructed cube, we have developed two methods that improve the tomographic reconstruction process. We first performed a very precise alignment of the projections before reconstruction with a technique using sinograms. After reconstruction, we propose to compute image restoration by calculating the Point Spread Function of the projection/back-projection system and to use it to deblur the reconstructed cubes. Improvement in the quality of the reconstructed cubes is demonstrated on images of nucleolar proteins tagged with EGFP and immunolabelled with nanogold particles.

A procedure to deposit fiducial markers on vitreous cryo-sections for cellular tomography

We describe a novel approach for the accurate alignment of images in electron tomography of vitreous cryo-sections. Quantum dots, suspended in organic solvents at cryo-temperatures, are applied directly onto the sections and are subsequently used as fiducial markers to align the tilt series. Data collection can be performed from different regions of the vitreous sections, even when the sections touch the grid only at a few places. We present high-resolution tomograms of some organelles in cryo-sections of human skin cells using this method. The average error in image alignment was about 1nm and the resolution was estimated to be 5-7nm. Thus, the use of section-attached quantum dots as fiducial markers in electron tomography of vitreous cryo-sections facilitates high-resolution in situ 3D imaging of organelles and macromolecular complexes in their native hydrated state.

Automatic TEM image alignment by trifocal geometry

Here we propose a novel method for automatic, markerless, feature-based alignment of TEM images suitable for electron tomography. The proposed method, termed trifocal alignment, is more accurate than the previous markerless methods. The key components developed are: (1) a reliable multi-resolution algorithm for matching feature points between images; (2) a robust, maximum-likelihood-based estimator for determining the geometry of three views--the trifocal constraint--required for validating the correctness of the matches; and (3) a robust, large-scale optimization framework to compute the alignment parameters from hundreds of thousands of feature point measurements from a few hundred images. The ability to utilize such a large number of measurements successfully compensates for point localization errors. The method was experimentally confirmed with electron tomography tilt series of biological and material sciences samples, consisting of from 40 to 150 images. The results show that, with this feature-based alignment approach, a level of accuracy comparable with fiducial marker alignment can be achieved.

Transform-based backprojection for volume reconstruction of large format electron microscope tilt series

Alignment of the individual images of a tilt series is a critical step in obtaining high-quality electron microscope reconstructions. We report on general methods for producing good alignments, and utilizing the alignment data in subsequent reconstruction steps. Our alignment techniques utilize bundle adjustment. Bundle adjustment is the simultaneous calculation of the position of distinguished markers in the object space and the transforms of these markers to their positions in the observed images, along the bundle of particle trajectories along which the object is projected to each EM image. Bundle adjustment techniques are general enough to encompass the computation of linear, projective or nonlinear transforms for backprojection, and can compensate for curvilinear trajectories through the object, sample warping, and optical aberration. We will also report on new reconstruction codes and describe our results using these codes.

Application of electron tomography to fungal ultrastructure studies

Access to structural information at the nanoscale enables fundamental insights into many complex biological systems. The development of the transmission electron microscope (TEM) has vastly increased our understanding of multiple biological systems. However, when attempting to visualize and understand the organizational and functional complexities that are typical of cells and tissues, the standard 2-D analyses that TEM affords often fall short. In recent years, high-resolution electron tomography methods, coupled with advances in specimen preparation and instrumentation and computational speed, have resulted in a revolution in the biological sciences. Electron tomography is analogous to medical computerized axial tomography (CAT-scan imaging) except at a far finer scale. It utilizes the TEM to assemble multiple projections of an object which are then combined for 3-D analyses. For biological specimens, tomography enables the highest 3-D resolution (5 nm spatial resolution) of internal structures in relatively thick slices of material (0.2-0.4 microm) without requiring the collection and alignment of large numbers of thin serial sections. Thus accurate and revealing 3-D reconstructions of complex cytoplasmic entities and architecture can be obtained. Electron tomography is now being applied to a variety of biological questions with great success. This review gives a brief introduction into cryopreservation and electron tomography relative to aspects of cytoplasmic organization in the hyphal tip of Aspergillus nidulans.

Accurate marker-free alignment with simultaneous geometry determination and reconstruction of tilt series in electron tomography

An image alignment method for electron tomography is presented which is based on cross-correlation techniques and which includes a simultaneous refinement of the tilt geometry. A coarsely aligned tilt series is iteratively refined with a procedure consisting of two steps for each cycle: area matching and subsequent geometry correction. The first step, area matching, brings into register equivalent specimen regions in all images of the tilt series. It determines four parameters of a linear two-dimensional transformation, not just translation and rotation as is done during the preceding coarse alignment with conventional methods. The refinement procedure also differs from earlier methods in that the alignment references are now computed from already aligned images by reprojection of a backprojected volume. The second step, geometry correction, refines the initially inaccurate estimates of the geometrical parameters, including the direction of the tilt axis, a tilt angle offset, and the inclination of the specimen with respect to the support film or specimen holder. The correction values serve as an indicator for the progress of the refinement. For each new iteration, the correction values are used to compute an updated set of geometry parameters by a least squares fit. Model calculations show that it is essential to refine the geometrical parameters as well as the accurate alignment of the images to obtain a faithful map of the original structure.

Three-Dimensional Electron Microscopy at Molecular Resolution

Emerging methods in cryo-electron microscopy allow determination of the three-dimensional architectures of objects ranging in size from small proteins to large eukaryotic cells, spanning a size range of more than 12 orders of magnitude. Advances in determining structures by "single particle" microscopy and by "electron tomography" provide exciting opportunities to describe the structures of subcellular assemblies that are either too large or too heterogeneous to be investigated by conventional crystallographic methods. Here, we review selected aspects of progress in structure determination by cryo-electron microscopy at molecular resolution, with a particular emphasis on topics at the interface of single particle and tomographic approaches. The rapid pace of development in this field suggests that comprehensive descriptions of the structures of whole cells and organelles in terms of the spatial arrangements of their molecular components may soon become routine.