Multiphase method for automatic alignment of transmission electron microscope images using markers

3D imaging of magnetic domains in Nd2Fe14B using scanning hard X-ray nanotomography

Nanoscale structural and electronic heterogeneities are prevalent in condensed matter physics. Investigating these heterogeneities in 3D has become an important task for understanding material properties. To provide a tool to unravel the connection between nanoscale heterogeneity and macroscopic emergent properties in magnetic materials, scanning transmission X-ray microscopy (STXM) is combined with X-ray magnetic circular dichroism. A vector tomography algorithm has been developed to reconstruct the full 3D magnetic vector field without any prior noise assumptions or knowledge about the sample. Two tomographic scans around the vertical axis are acquired on single-crystalline Nd2Fe14B pillars tilted at two different angles, with 2D STXM projections recorded using a focused 120 nm X-ray beam with left and right circular polarization. Image alignment and iterative registration have been implemented based on the 2D STXM projections for the two tilts. Dichroic projections obtained from difference images are used for the tomographic reconstruction to obtain the 3D magnetization distribution at the nanoscale.

Subtomogram averaging from cryo-electron tomograms

Cryo-electron tomography (cryo-ET) allows three-dimensional (3D) visualization of frozen-hydrated biological samples, such as protein complexes and cell organelles, in near-native environments at nanometer scale. Protein complexes that are present in multiple copies in a set of tomograms can be extracted, mutually aligned, and averaged to yield a signal-enhanced 3D structure up to sub-nanometer or even near-atomic resolution. This technique, called subtomogram averaging (StA), is powered by improvements in EM hardware and image processing software. Importantly, StA provides unique biological insights into the structure and function of cellular machinery in close-to-native contexts. In this chapter, we describe the principles and key steps of StA. We briefly cover sample preparation and data collection with an emphasis on image processing procedures related to tomographic reconstruction, subtomogram alignment, averaging, and classification. We conclude by summarizing current limitations and future directions of this technique with a focus on high-resolution StA.

A fast fiducial marker tracking model for fully automatic alignment in electron tomography

Motivation

Automatic alignment, especially fiducial marker-based alignment, has become increasingly important due to the high demand of subtomogram averaging and the rapid development of large-field electron microscopy. Among the alignment steps, fiducial marker tracking is a crucial one that determines the quality of the final alignment. Yet, it is still a challenging problem to track the fiducial markers accurately and effectively in a fully automatic manner.

Results

In this paper, we propose a robust and efficient scheme for fiducial marker tracking. Firstly, we theoretically prove the upper bound of the transformation deviation of aligning the positions of fiducial markers on two micrographs by affine transformation. Secondly, we design an automatic algorithm based on the Gaussian mixture model to accelerate the procedure of fiducial marker tracking. Thirdly, we propose a divide-and-conquer strategy against lens distortions to ensure the reliability of our scheme. To our knowledge, this is the first attempt that theoretically relates the projection model with the tracking model. The real-world experimental results further support our theoretical bound and demonstrate the effectiveness of our algorithm. This work facilitates the fully automatic tracking for datasets with a massive number of fiducial markers.

Availability and implementation

The C/C ++ source code that implements the fast fiducial marker tracking is available at https://github.com/icthrm/gmm-marker-tracking. Markerauto 1.6 version or later (also integrated in the AuTom platform at http://ear.ict.ac.cn/) offers a complete implementation for fast alignment, in which fast fiducial marker tracking is available by the '-t' option.

Contact

xin.gao@kaust.edu.sa.

Supplementary information

Supplementary data are available at Bioinformatics online.

Analytical registration of vertical image drifts in parallel beam tomographic data

Reconstructing tomographic images of high resolution, as in x-ray microscopy or transmission electron microscopy, is often limited by the stability of the stages or sample drifts, which requires an image alignment prior to reconstruction. Feature-based image registration is routinely used to align images, but this technique relies on strong features in the sample or the application of gold tracer particles, for example. In this Letter, we present an analytic approach for achieving the vertical registration based on the inherent properties of the data acquired for tomographic reconstruction. It is computationally cheap to implement and can be easily integrated into existing reconstruction pipelines.

Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection

As x-ray and electron tomography is pushed further into the nanoscale, the limitations of rotation stages become more apparent, leading to challenges in the alignment of the acquired projection images. Here we present an approach for rapid post-acquisition alignment of these projections to obtain high quality three-dimensional images. Our approach is based on a joint estimation of alignment errors, and the object, using an iterative refinement procedure. With simulated data where we know the alignment error of each projection image, our approach shows a residual alignment error that is a factor of a thousand smaller, and it reaches the same error level in the reconstructed image in less than half the number of iterations. We then show its application to experimental data in x-ray and electron nanotomography.

Fast Projection Matching for X-ray Tomography

X-ray 3D tomographic techniques are powerful tools for investigating the morphology and internal structures of specimens. A common strategy for obtaining 3D tomography is to capture a series of 2D projections from different X-ray illumination angles of specimens mounted on a finely calibrated rotational stage. However, the reconstruction quality of 3D tomography relies on the precision and stability of the rotational stage, i.e. the accurate alignment of the 2D projections in the correct three-dimensional positions. This is a crucial problem for nano-tomographic techniques due to the non-negligible mechanical imperfection of the rotational stages at the nanometer level which significantly degrades the spatial resolution of reconstructed 3-D tomography. Even when using an X-ray micro-CT with a highly stabilized rotational stage, thermal effects caused by the CT system are not negligible and may cause sample drift. Here, we propose a markerless image auto-alignment algorithm based on an iterative method. This algorithm reduces the traditional projection matching method into two simplified matching problems and it is much faster and more reliable than traditional methods. This algorithm can greatly decrease hardware requirements for both nano-tomography and data processing and can be easily applied to other tomographic techniques, such as X-ray micro-CT and electron tomography.

Marker Detection in Electron Tomography: A Comparative Study

We conducted a comparative study of three widely used algorithms for the detection of fiducial markers in electron microscopy images. The algorithms were applied to four datasets from different sources. For the purpose of obtaining comparable results, we introduced figures of merit and implemented all three algorithms in a unified code base to exclude software-specific differences. The application of the algorithms revealed that none of the three algorithms is superior to the others in all cases. This leads to the conclusion that the choice of a marker detection algorithm highly depends on the properties of the dataset to be analyzed, even within the narrowed domain of electron tomography.

Physically motivated global alignment method for electron tomography

Electron tomography is widely used for nanoscale determination of 3-D structures in many areas of science. Determining the 3-D structure of a sample from electron tomography involves three major steps: acquisition of sequence of 2-D projection images of the sample with the electron microscope, alignment of the images to a common coordinate system, and 3-D reconstruction and segmentation of the sample from the aligned image data. The resolution of the 3-D reconstruction is directly influenced by the accuracy of the alignment, and therefore, it is crucial to have a robust and dependable alignment method. In this paper, we develop a new alignment method which avoids the use of markers and instead traces the computed paths of many identifiable ‘local’ center-of-mass points as the sample is rotated. Compared with traditional correlation schemes, the alignment method presented here is resistant to cumulative error observed from correlation techniques, has very rigorous mathematical justification, and is very robust since many points and paths are used, all of which inevitably improves the quality of the reconstruction and confidence in the scientific results.

Quantitative immunocytochemistry at the ultrastructural level: a stereology-based approach to molecular nanomorphomics

Biological systems span multiple levels of structural organisation from the macroscopic, via the microscopic, to the nanoscale. Therefore, comprehensive investigation of systems biology requires application of imaging modalities that reveal structure at multiple resolution scales. Nanomorphomics is the part of morphomics devoted to the systematic study of functional morphology at the nanoscale and an important element of its achievement is the combination of immunolabelling and transmission electron microscopy (TEM). The ultimate goal of quantitative immunocytochemistry is to estimate numbers of target molecules (usually peptides, proteins or protein complexes) in biological systems and to map their spatial distributions within them. Immunogold cytochemistry utilises target-specific affinity markers (primary antibodies) and visualisation aids (e.g., colloidal gold particles or silver-enhanced nanogold particles) to detect and localise target molecules at high resolution in intact cells and tissues. In the case of post-embedding labelling of ultrathin sections for TEM, targets are localised as a countable digital readout by using colloidal gold particles. The readout comprises a spatial distribution of gold particles across the section and within the context of biological ultrastructure. The observed distribution across structural compartments (whether volume- or surface-occupying) represents both specific and non-specific labelling; an assessment by eye alone as to whether the distribution is random or non-random is not always possible. This review presents a coherent set of quantitative methods for testing whether target molecules exhibit preferential and specific labelling of compartments and for mapping the same targets in two or more groups of cells as their TEM immunogold-labelling patterns alter after experimental manipulation. The set also includes methods for quantifying colocalisation in multiple-labelling experiments and mapping absolute numbers of colloidal gold particles across compartments at specific positions within cells having a point-like inclusion (e.g., centrosome, nucleolus) and a definable vertical axis. Although developed for quantifying colloidal gold particles, the same methods can in principle be used to quantify other electron-dense punctate nanoparticles, including quantum dots.

Geometric correction method for 3D in-line X-ray phase contrast image reconstruction

BACKGROUND

Mechanical system with imperfect or misalignment of X-ray phase contrast imaging (XPCI) components causes projection data misplaced, and thus result in the reconstructed slice images of computed tomography (CT) blurred or with edge artifacts. So the features of biological microstructures to be investigated are destroyed unexpectedly, and the spatial resolution of XPCI image is decreased. It makes data correction an essential pre-processing step for CT reconstruction of XPCI.

METHODS

To remove unexpected blurs and edge artifacts, a mathematics model for in-line XPCI is built by considering primary geometric parameters which include a rotation angle and a shift variant in this paper. Optimal geometric parameters are achieved by finding the solution of a maximization problem. And an iterative approach is employed to solve the maximization problem by using a two-step scheme which includes performing a composite geometric transformation and then following a linear regression process. After applying the geometric transformation with optimal parameters to projection data, standard filtered back-projection algorithm is used to reconstruct CT slice images.

RESULTS

Numerical experiments were carried out on both synthetic and real in-line XPCI datasets. Experimental results demonstrate that the proposed method improves CT image quality by removing both blurring and edge artifacts at the same time compared to existing correction methods.

CONCLUSIONS

The method proposed in this paper provides an effective projection data correction scheme and significantly improves the image quality by removing both blurring and edge artifacts at the same time for in-line XPCI. It is easy to implement and can also be extended to other XPCI techniques.

A new field-of-view autotracking method for online tomography reconstruction based on back-projected ray image cross-correlation

We devised a new field-of-view autotracking method for online tomography reconstruction based on a cross-correlation between a pair of neighbours, called back-projected ray images, among a specimen tilt sequence. One ray image is calculated through normal filtered back-projection only in the cross-sectional plane from each projection image. This ray-image matching can reliably track the field-of-view because a pair of neighbouring ray images mostly cross-correlates at the existing three-dimensional object position. Online experiments using real specimens resulted in successful autotracking performance with high accuracy, and online tomograms were obtained immediately after the final tracking at the last tilting angle.

Tomography by point source digital holographic microscopy

We propose a tomographic method for point source inline holographic microscopy. By recording a set of holograms at different illumination angles, shadowing effects are eliminated resulting in three-dimensional images with the same precision at the micrometer-scale in all directions. The advantage of our tomographic approach is that it works for both absorbing and phase objects, regardless of the change of refractive index at interfaces. We develop the method with computer simulations and demonstrate its strength by presenting experimental results for micrometer-sized polystyrene beads and a cotton fiber.

Alternative automatic alignment method for specimen tilt-series images based on back-projected volume data cross-correlations

We devised a new automatic image alignment method for a specimen tilt series; this method is based on the volume data cross-correlation among 3-D cross-sections reconstructed from different sets of projection images (including a single image) for tilt-series alignment or tilt-axis search purposes. This method requires neither markers nor image feature points traceable through the tilt series, and it was examined through simulations and applied to biological thin sections. The method automatically aligned tilt series centred at the correctly detected tilt axis with a precision sufficient for practical applications.

Reconstructing virus structures from nanometer to near-atomic resolutions with cryo-electron microscopy and tomography

The past few decades have seen tremendous advances in single-particle electron -cryo-microscopy (cryo-EM). The field has matured to the point that near-atomic resolution density maps can be generated for icosahedral viruses without the need for crystallization. In parallel, substantial progress has been made in determining the structures of nonicosahedrally arranged proteins in viruses by employing either single-particle cryo-EM or cryo-electron tomography (cryo-ET). Implicit in this course have been the availability of a new generation of electron cryo-microscopes and the development of the computational tools that are essential for generating these maps and models. This methodology has enabled structural biologists to analyze structures in increasing detail for virus particles that are in different morphogenetic states. Furthermore, electron imaging of frozen, hydrated cells, in the process of being infected by viruses, has also opened up a new avenue for studying virus structures "in situ". Here we present the common techniques used to acquire and process cryo-EM and cryo-ET data and discuss their implications for structural virology both now and in the future.

Reliable automatic alignment of tomographic projection data by passive auto-focus

PURPOSE

The authors present a robust algorithm that removes the blurring and double-edge artifacts in high-resolution computed tomography (CT) images that are caused by misaligned scanner components. This alleviates the time-consuming process of physically aligning hardware, which is of particular benefit if components are moved or swapped frequently.

METHODS

The proposed method uses the experimental data itself for calibration. A parameterized model of the scanner geometry is constructed and the parameters are varied until the sharpest 3D reconstruction is found. The concept is similar to passive auto-focus algorithms of digital optical instruments. The parameters are used to remap the projection data from the physical detector to a virtual aligned detector. This is followed by a standard reconstruction algorithm, namely the Feldkamp algorithm. Feldkamp et al. [J. Opt. Soc. Am. A 1, 612-619 (1984)].

RESULTS

An example implementation is given for a rabbit liver specimen that was collected with a circular trajectory. The optimal parameters were determined in less computation time than that for a full reconstruction. The example serves to demonstrate that (a) sharpness is an appropriate measure for projection alignment, (b) our parameterization is sufficient to characterize misalignments for cone-beam CT, and (c) the procedure determines parameter values with sufficient precision to remove the associated artifacts.

CONCLUSIONS

The algorithm is fully tested and implemented for regular use at The Australian National University micro-CT facility for both circular and helical trajectories. It can in principle be applied to more general imaging geometries and modalities. It is as robust as manual alignment but more precise since we have quantified the effect of misalignment.

An automatic method of detecting and tracking fiducial markers for alignment in electron tomography

We presented an automatic method for detecting and tracking colloidal gold fiducial markers for alignment in electron tomography (ET). The second-order derivative of direction was used to detect a fiducial marker accurately. The detection was optimized to be selective to the size of fiducial markers. A preliminary tracking result from the normalized correlation coefficient was refined using the detector. A constraint model considering the relationship among the fiducial markers on different images was developed for removing outlier. The three-dimensional positions of the detected fiducial markers and the projection parameters of tilt images were calculated for post process. The accuracy of detection and tracking results was evaluated from the residues by the software IMOD. Application on transmission electron microscopic images also indicated that the presented method could provide a useful approach to automatic alignment in ET.

Automatic coarse-alignment for TEM tilt series of rod-shaped specimens collected with a full angular range

An automatic coarse-alignment method for a tilt series of rod-shaped specimen collected with a full angular range (from alpha=-90 degrees to +90 degrees, alpha is the tilt angle of the specimen) is presented; this method is based on a cross-correlation method and uses the outline of the specimen shape. Both the rotational angle of the tilt axis and translational value of each image can be detected in the images without the use of markers. This method is performed on the basis of the assumption that the images taken at alpha=-90 degrees and alpha=+90 degrees are symmetric about the tilt axis. In this study, a carbon rod on which gold particles have been deposited is used as a test specimen for the demonstration. This method can be used as an automatic coarse-alignment method prior to the application of a highly accurate alignment method because the alignment procedure can be performed automatically except for the initial setup of some parameters.

Alignment of cryo-electron tomography datasets

Data acquisition of cryo-electron tomography (CET) samples described in previous chapters involves relatively imprecise mechanical motions: the tilt series has shifts, rotations, and several other distortions between projections. Alignment is the procedure of correcting for these effects in each image and requires the estimation of a projection model that describes how points from the sample in three-dimensions are projected to generate two-dimensional images. This estimation is enabled by finding corresponding common features between images. This chapter reviews several software packages that perform alignment and reconstruction tasks completely automatically (or with minimal user intervention) in two main scenarios: using gold fiducial markers as high contrast features or using relevant biological structures present in the image (marker-free). In particular, we emphasize the key decision points in the process that users should focus on in order to obtain high-resolution reconstructions.

Marker-free image registration of electron tomography tilt-series

Background

Tilt series are commonly used in electron tomography as a means of collecting three-dimensional information from two-dimensional projections. A common problem encountered is the projection alignment prior to 3D reconstruction. Current alignment techniques usually employ gold particles or image derived markers to correctly align the images. When these markers are not present, correlation between adjacent views is used to align them. However, sequential pairwise correlation is prone to bias and the resulting alignment is not always optimal.

Results

In this paper we introduce an algorithm to find regions of the tilt series which can be tracked within a subseries of the tilt series. These regions act as landmarks allowing the determination of the alignment parameters. We show our results with synthetic data as well as experimental cryo electron tomography.

Conclusion

Our algorithm is able to correctly align a single-tilt tomographic series without the help of fiducial markers thanks to the detection of thousands of small image patches that can be tracked over a short number of images in the series.

Reprint of "Fiducial-less alignment of cryo-sections" [J. Struct. Biol. 159 (2007) 413-423]

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.

A Bayesian reconstruction method for micro-rotation imaging in light microscopy

The authors present a three-dimensional (3D) reconstruction algorithm and reconstruction-based deblurring method for light microscopy using a micro-rotation device. In contrast to conventional 3D optical imaging where the focal plane is shifted along the optical axis, micro-rotation imaging employs dielectric fields to rotate the object inside a fixed optical set-up. To address this entirely new 3D-imaging modality, the authors present a reconstruction algorithm based on Bayesian inversion theory and use the total variation function as a structure prior. The spectral properties of the reconstruction by simulations that illustrate the strengths and the weaknesses of the micro-rotation approach, compared with conventional 3D optical imaging, were studied. The reconstruction from real data sets shows that this method is promising for 3D reconstruction and offers itself as a deblurring method using a reconstruction-based procedure for removing out-of-focus light from the micro-rotation image series.

Markov random field based automatic image alignment for electron tomography

We present a method for automatic full-precision alignment of the images in a tomographic tilt series. Full-precision automatic alignment of cryo electron microscopy images has remained a difficult challenge to date, due to the limited electron dose and low image contrast. These facts lead to poor signal to noise ratio (SNR) in the images, which causes automatic feature trackers to generate errors, even with high contrast gold particles as fiducial features. To enable fully automatic alignment for full-precision reconstructions, we frame the problem probabilistically as finding the most likely particle tracks given a set of noisy images, using contextual information to make the solution more robust to the noise in each image. To solve this maximum likelihood problem, we use Markov Random Fields (MRF) to establish the correspondence of features in alignment and robust optimization for projection model estimation. The resulting algorithm, called Robust Alignment and Projection Estimation for Tomographic Reconstruction, or RAPTOR, has not needed any manual intervention for the difficult datasets we have tried, and has provided sub-pixel alignment that is as good as the manual approach by an expert user. We are able to automatically map complete and partial marker trajectories and thus obtain highly accurate image alignment. Our method has been applied to challenging cryo electron tomographic datasets with low SNR from intact bacterial cells, as well as several plastic section and X-ray datasets.

3D electron microscopy of biological nanomachines: principles and applications

Transmission electron microscopy is a powerful technique for studying the three-dimensional (3D) structure of a wide range of biological specimens. Knowledge of this structure is crucial for fully understanding complex relationships among macromolecular complexes and organelles in living cells. In this paper, we present the principles and main application domains of 3D transmission electron microscopy in structural biology. Moreover, we survey current developments needed in this field, and discuss the close relationship of 3D transmission electron microscopy with other experimental techniques aimed at obtaining structural and dynamical information from the scale of whole living cells to atomic structure of macromolecular complexes.

Fiducial-less alignment of cryo-sections

Cryo-electron tomography of vitreous sections is currently the most promising technique for visualizing arbitrary regions of eukaryotic cells or tissue at molecular resolution. Despite significant progress in the sample preparation techniques over the past few years, the three dimensional reconstruction using electron tomography is not as simple as in plunge frozen samples for various reasons, but mainly due to the effects of irradiation on the sections and the resulting poor alignment. Here, we present a new algorithm, which can provide a useful three-dimensional marker model after investigation of hundreds to thousands of observations calculated using local cross-correlation throughout the tilt series. The observations are chosen according to their coherence to a particular model and assigned to virtual markers. Through this type of measurement a merit figure can be calculated, precisely estimating the quality of the reconstruction. The merit figures of this alignment method are comparable to those obtained with plunge frozen samples using fiducial gold markers. An additional advantage of the algorithm is the implicit detection of areas in the sections that behave as rigid bodies and can thus be properly reconstructed.

Structure-from-motion without correspondence from tomographic projections by Bayesian inversion theory

In conventional tomography, the interior of an object is reconstructed from tomographic projections such as X-ray or transmission electron microscope images. All the current reconstruction methods assume that projection geometry of the imaging device is either known or solved in advance by using e.g., fiducial or nonfiducial feature points in the images. In this paper, we propose a novel approach where the imaging geometry is solved simultaneously with the volume reconstruction problem while no correspondence information is needed. Our approach is a direct application of Bayesian inversion theory and produces the maximum likelihood or maximum a posteriori estimates for the motion parameters under the selected noise and prior distributions. In this paper, the method is implemented for a two-dimensional model problem with one-dimensional affine projection data. The performance of the method is tested with simulated and measured X-ray projection data.

Three-dimensional reconstruction of chromosomes using electron tomography

The evolvement of preparative methods in structural studies has always been as important as the development of sophisticated equipment. Software development is also a significant part for three-dimensional (3D) structural studies using electron tomography methods (ETMs). Advanced computing makes amenable procedures that relatively recently were only visionary, such as the 3D reconstruction of chromosomes with ETM. Morphological guidelines and beauty are occasionally the only standard for a method to be acceptable in the realms of preparative as well as software development. Bulk isolation of metaphase chromosomes using acetic acid is such an apparent accomplishment in preparative methods. Our ETM with maximum entropy and, more so, the ongoing development toward fully automatic alignments, are contributions in the software line. Furthermore, whole mounting of chromosomes on holey-carbon grids makes it possible to use even yesterday's 80-kV transmission electron microscope with a standard goniometer to collect tilt series. These advances in preparing whole-mount metaphase chromosomes enable laboratories that do not have access to a medium- or high-voltage transmission electron microscope to study complex structures like chromosomes in 3D using today's desktop computers.

Automatic TEM image alignment by trifocal geometry

Here we propose a novel method for automatic, markerless, feature-based alignment of TEM images suitable for electron tomography. The proposed method, termed trifocal alignment, is more accurate than the previous markerless methods. The key components developed are: (1) a reliable multi-resolution algorithm for matching feature points between images; (2) a robust, maximum-likelihood-based estimator for determining the geometry of three views--the trifocal constraint--required for validating the correctness of the matches; and (3) a robust, large-scale optimization framework to compute the alignment parameters from hundreds of thousands of feature point measurements from a few hundred images. The ability to utilize such a large number of measurements successfully compensates for point localization errors. The method was experimentally confirmed with electron tomography tilt series of biological and material sciences samples, consisting of from 40 to 150 images. The results show that, with this feature-based alignment approach, a level of accuracy comparable with fiducial marker alignment can be achieved.

Transform-based backprojection for volume reconstruction of large format electron microscope tilt series

Alignment of the individual images of a tilt series is a critical step in obtaining high-quality electron microscope reconstructions. We report on general methods for producing good alignments, and utilizing the alignment data in subsequent reconstruction steps. Our alignment techniques utilize bundle adjustment. Bundle adjustment is the simultaneous calculation of the position of distinguished markers in the object space and the transforms of these markers to their positions in the observed images, along the bundle of particle trajectories along which the object is projected to each EM image. Bundle adjustment techniques are general enough to encompass the computation of linear, projective or nonlinear transforms for backprojection, and can compensate for curvilinear trajectories through the object, sample warping, and optical aberration. We will also report on new reconstruction codes and describe our results using these codes.

Accurate marker-free alignment with simultaneous geometry determination and reconstruction of tilt series in electron tomography

An image alignment method for electron tomography is presented which is based on cross-correlation techniques and which includes a simultaneous refinement of the tilt geometry. A coarsely aligned tilt series is iteratively refined with a procedure consisting of two steps for each cycle: area matching and subsequent geometry correction. The first step, area matching, brings into register equivalent specimen regions in all images of the tilt series. It determines four parameters of a linear two-dimensional transformation, not just translation and rotation as is done during the preceding coarse alignment with conventional methods. The refinement procedure also differs from earlier methods in that the alignment references are now computed from already aligned images by reprojection of a backprojected volume. The second step, geometry correction, refines the initially inaccurate estimates of the geometrical parameters, including the direction of the tilt axis, a tilt angle offset, and the inclination of the specimen with respect to the support film or specimen holder. The correction values serve as an indicator for the progress of the refinement. For each new iteration, the correction values are used to compute an updated set of geometry parameters by a least squares fit. Model calculations show that it is essential to refine the geometrical parameters as well as the accurate alignment of the images to obtain a faithful map of the original structure.

Automatic alignment of transmission electron microscope tilt series without fiducial markers

Accurate image alignment is needed for computing three-dimensional reconstructions from transmission electron microscope tilt series. So far, the best results have been obtained by using colloidal gold beads as fiducial markers. If their use has not been possible for some reason, the only option has been the automatic cross-correlation-based registration methods. However, the latter methods are inaccurate and, as we will show, inappropriate for the whole problem. Conversely, we propose a novel method that uses the actual 3D motion model but works without any fiducial markers in the images. The method is based on matching and tracking some interest points of the intensity surface by first solving the underlying geometrical constraint of consecutive images in the tilt series. The results show that our method is near the gold marker alignment in the level of accuracy and hence opens the way for new opportunities in the analysis of electron tomography reconstructions, especially when markers cannot be used.