Missing wedge computed tomography by iterative algorithm DIRECTT

Simulation-Assisted Augmentation of Missing Wedge and Region-of-Interest Computed Tomography Data

This study reports a strategy to use sophisticated, realistic X-ray Computed Tomography (CT) simulations to reduce Missing Wedge (MW) and Region-of-Interest (RoI) artifacts in FBP (Filtered Back-Projection) reconstructions. A 3D model of the object is used to simulate the projections that include the missing information inside the MW and outside the RoI. Such information augments the experimental projections, thereby drastically improving the reconstruction results. An X-ray CT dataset of a selected object is modified to mimic various degrees of RoI and MW problems. The results are evaluated in comparison to a standard FBP reconstruction of the complete dataset. In all cases, the reconstruction quality is significantly improved. Small inclusions present in the scanned object are better localized and quantified. The proposed method has the potential to improve the results of any CT reconstruction algorithm.

Computational methods for in situ structural studies with cryogenic electron tomography

Cryo-electron tomography (cryo-ET) plays a critical role in imaging microorganisms in situ in terms of further analyzing the working mechanisms of viruses and drug exploitation, among others. A data processing workflow for cryo-ET has been developed to reconstruct three-dimensional density maps and further build atomic models from a tilt series of two-dimensional projections. Low signal-to-noise ratio (SNR) and missing wedge are two major factors that make the reconstruction procedure challenging. Because only few near-atomic resolution structures have been reconstructed in cryo-ET, there is still much room to design new approaches to improve universal reconstruction resolutions. This review summarizes classical mathematical models and deep learning methods among general reconstruction steps. Moreover, we also discuss current limitations and prospects. This review can provide software and methods for each step of the entire procedure from tilt series by cryo-ET to 3D atomic structures. In addition, it can also help more experts in various fields comprehend a recent research trend in cryo-ET. Furthermore, we hope that more researchers can collaborate in developing computational methods and mathematical models for high-resolution three-dimensional structures from cryo-ET datasets.

Automatic Differentiation for Inverse Problems in X-ray Imaging and Microscopy

Computational techniques allow breaking the limits of traditional imaging methods, such as time restrictions, resolution, and optics flaws. While simple computational methods can be enough for highly controlled microscope setups or just for previews, an increased level of complexity is instead required for advanced setups, acquisition modalities or where uncertainty is high; the need for complex computational methods clashes with rapid design and execution. In all these cases, Automatic Differentiation, one of the subtopics of Artificial Intelligence, may offer a functional solution, but only if a GPU implementation is available. In this paper, we show how a framework built to solve just one optimisation problem can be employed for many different X-ray imaging inverse problems.

Precisely Picking Nanoparticles by a "Nano-Scalpel" for 360° Electron Tomography

Electron tomography (ET) has gained increasing attention for the 3D characterization of nanoparticles. However, the missing wedge problem due to a limited tilt angle range is still the main challenge for accurate reconstruction in most experimental TEM setups. Advanced algorithms could in-paint or compensate to some extent the missing wedge artifacts, but cannot recover the missing structural information completely. 360° ET provides an option to solve this problem by tilting a needle-shaped specimen over the full tilt range and thus filling the missing information. However, sample preparation especially for fine powders to perform full-range ET is still challenging, thus limiting its application. In this work, we propose a new universal sample preparation method that enables the transfer of selected individual nanoparticle or a few separated nanoparticles by cutting a piece of carbon film supporting the specimen particles and mounting them onto the full-range tomography holder tip with the help of an easily prepared sharp tungsten tip. This method is demonstrated by 360° ET of Pt@TiO2 hollow cage catalyst showing high quality reconstruction without missing wedge.

Structure Detection in Three-Dimensional Cellular Cryoelectron Tomograms by Reconstructing Two-Dimensional Annotated Tilt Series

The revolutionary technique cryoelectron tomography (cryo-ET) enables imaging of cellular structure and organization in a near-native environment at submolecular resolution, which is vital to subsequent data analysis and modeling. The conventional structure detection process first reconstructs the three-dimensional (3D) tomogram from a series of two-dimensional (2D) projections and then directly detects subcellular components found within the tomogram. However, this process is challenging due to potential structural information loss during the tomographic reconstruction and the limited scope of existing methods since most major state-of-the-art object detection methods are designed for 2D rather than 3D images. Therefore, in this article, as an alternative approach to complement the conventional process, we propose a novel 2D-to-3D framework that detects structures within 2D projection images before reconstructing the results back to 3D. We implemented the proposed framework as three specific algorithms for three individual tasks: semantic segmentation, edge detection, and object localization. As experimental validation of the 2D-to-3D framework for cryo-ET data, we applied the algorithms to the segmentation of mitochondrial calcium phosphate granules, detection of spherical edges, and localization of mitochondria. Quantitative and qualitative results show better performance for prediction tasks of segmentation on the 2D projections and promising performance on object localization and edge detection, paving the way for future studies in the exploration of cryo-ET for in situ structural biology.

IDART: An Improved Discrete Tomography Algorithm for Reconstructing Images With Multiple Gray Levels

The discrete algebraic reconstruction technique has many advantages in computed tomography and electron tomography. However, the number of gray levels and the absolute gray values that should be known in advance are typically not available in experiments especially when there are many gray levels in the image. In this paper, we report an automatic discrete tomography reconstruction algorithm to improve its feasibility in practice, without needing to know these two parameters. In our algorithm, the number of gray levels is estimated by labeling the connected components in the tomogram and the absolute values of them are determined by the modal value of each domain. The proposed algorithm was extensively validated on both simulated and experimental datasets. The results show that our algorithm can accurately recover not only the morphology but also the gray levels of the interested objects, even in the images with multiple gray levels. It is demonstrated that the presented algorithm is robust for eliminating missing wedge artifacts and tolerable for noisy data.

Parakeet: a digital twin software pipeline to assess the impact of experimental parameters on tomographic reconstructions for cryo-electron tomography

In cryo-electron tomography (cryo-ET) of biological samples, the quality of tomographic reconstructions can vary depending on the transmission electron microscope (TEM) instrument and data acquisition parameters. In this paper, we present Parakeet, a 'digital twin' software pipeline for the assessment of the impact of various TEM experiment parameters on the quality of three-dimensional tomographic reconstructions. The Parakeet digital twin is a digital model that can be used to optimize the performance and utilization of a physical instrument to enable in silico optimization of sample geometries, data acquisition schemes and instrument parameters. The digital twin performs virtual sample generation, TEM image simulation, and tilt series reconstruction and analysis within a convenient software framework. As well as being able to produce physically realistic simulated cryo-ET datasets to aid the development of tomographic reconstruction and subtomogram averaging programs, Parakeet aims to enable convenient assessment of the effects of different microscope parameters and data acquisition parameters on reconstruction quality. To illustrate the use of the software, we present the example of a quantitative analysis of missing wedge artefacts on simulated planar and cylindrical biological samples and discuss how data collection parameters can be modified for cylindrical samples where a full 180° tilt range might be measured.

Suppression of Cone-Beam Artefacts with Direct Iterative Reconstruction Computed Tomography Trajectories (DIRECTT)

The reconstruction of cone-beam computed tomography data using filtered back-projection algorithms unavoidably results in severe artefacts. We describe how the Direct Iterative Reconstruction of Computed Tomography Trajectories (DIRECTT) algorithm can be combined with a model of the artefacts for the reconstruction of such data. The implementation of DIRECTT results in reconstructed volumes of superior quality compared to the conventional algorithms.

A novel constrained reconstruction model towards high-resolution subtomogram averaging

MOTIVATION

Electron tomography (ET) offers a unique capacity to image biological structures in situ. However, the resolution of ET reconstructed tomograms is not comparable to that of the single-particle cryo-EM. If many copies of the object of interest are present in the tomograms, their structures can be reconstructed in the tomogram, picked, aligned and averaged to increase the signal-to-noise ratio and improve the resolution, which is known as the subtomogram averaging. To date, the resolution improvement of the subtomogram averaging is still limited because each reconstructed subtomogram is of low reconstruction quality due to the missing wedge issue.

RESULTS

In this article, we propose a novel computational model, the constrained reconstruction model (CRM), to better recover the information from the multiple subtomograms and compensate for the missing wedge issue in each of them. CRM is supposed to produce a refined reconstruction in the final turn of subtomogram averaging after alignment, instead of directly taking the average. We first formulate the averaging method and our CRM as linear systems, and prove that the solution space of CRM is no larger, and in practice much smaller, than that of the averaging method. We then propose a sparse Kaczmarz algorithm to solve the formulated CRM, and further extend the solution to the simultaneous algebraic reconstruction technique (SART). Experimental results demonstrate that CRM can significantly alleviate the missing wedge issue and improve the final reconstruction quality. In addition, our model is robust to the number of images in each tilt series, the tilt range and the noise level.

AVAILABILITY AND IMPLEMENTATION

The codes of CRM-SIRT and CRM-SART are available at https://github.com/icthrm/CRM.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Direct Iterative Reconstruction of Computed Tomography Trajectories Reconstruction from limited number of projections with DIRECTT

X-ray computed tomography has many applications in materials science and non-destructive testing. While the standard filtered back-projection reconstruction of the radiographic datasets is fast and simple, it typically fails in returning accurate results from missing or inconsistent projections. Among the alternative techniques that have been proposed to handle such data is the Direct Iterative REconstruction of Computed Tomography Trajectories (DIRECTT) algorithm. We describe a new approach to the algorithm, which significantly decreases the computational time while achieving a better reconstruction quality than that of other established algorithms.

Tomographic reconstruction with a generative adversarial network

This paper presents a deep learning algorithm for tomographic reconstruction (GANrec). The algorithm uses a generative adversarial network (GAN) to solve the inverse of the Radon transform directly. It works for independent sinograms without additional training steps. The GAN has been developed to fit the input sinogram with the model sinogram generated from the predicted reconstruction. Good quality reconstructions can be obtained during the minimization of the fitting errors. The reconstruction is a self-training procedure based on the physics model, instead of on training data. The algorithm showed significant improvements in the reconstruction accuracy, especially for missing-wedge tomography acquired at less than 180° rotational range. It was also validated by reconstructing a missing-wedge X-ray ptychographic tomography (PXCT) data set of a macroporous zeolite particle, for which only 51 projections over 70° could be collected. The GANrec recovered the 3D pore structure with reasonable quality for further analysis. This reconstruction concept can work universally for most of the ill-posed inverse problems if the forward model is well defined, such as phase retrieval of in-line phase-contrast imaging.

A versatile nanoreactor for complementary in situ X-ray and electron microscopy studies in catalysis and materials science

Two in situ `nanoreactors' for high-resolution imaging of catalysts have been designed and applied at the hard X-ray nanoprobe endstation at beamline P06 of the PETRA III synchrotron radiation source. The reactors house samples supported on commercial MEMS chips, and were applied for complementary hard X-ray ptychography (23 nm spatial resolution) and transmission electron microscopy, with additional X-ray fluorescence measurements. The reactors allow pressures of 100 kPa and temperatures of up to 1573 K, offering a wide range of conditions relevant for catalysis. Ptychographic tomography was demonstrated at limited tilting angles of at least ±35° within the reactors and ±65° on the naked sample holders. Two case studies were selected to demonstrate the functionality of the reactors: (i) annealing of hierarchical nanoporous gold up to 923 K under inert He environment and (ii) acquisition of a ptychographic projection series at ±35° of a hierarchically structured macroporous zeolite sample under ambient conditions. The reactors are shown to be a flexible and modular platform for in situ studies in catalysis and materials science which may be adapted for a range of sample and experiment types, opening new characterization pathways in correlative multimodal in situ analysis of functional materials at work. The cells will presently be made available for all interested users of beamline P06 at PETRA III.

Single-Molecule 3D Images of "Hole-Hole" IgG1 Homodimers by Individual-Particle Electron Tomography

The engineering of immunoglobulin-G molecules (IgGs) is of wide interest for improving therapeutics, for example by modulating the activity or multiplexing the specificity of IgGs to recognize more than one antigen. Optimization of engineered IgG requires knowledge of three-dimensional (3D) structure of synthetic IgG. However, due to flexible nature of the molecules, their structural characterization is challenging. Here, we use our reported individual-particle electron tomography (IPET) method with optimized negative-staining (OpNS) for direct 3D reconstruction of individual IgG hole-hole homodimer molecules. The hole-hole homodimer is an undesired variant generated during the production of a bispecific antibody using the knob-into-hole heterodimer technology. A total of 64 IPET 3D density maps at ~15 Å resolutions were reconstructed from 64 individual molecules, revealing 64 unique conformations. In addition to the known Y-shaped conformation, we also observed an unusual X-shaped conformation. The 3D structure of the X-shaped conformation contributes to our understanding of the structural details of the interaction between two heavy chains in the Fc domain. The IPET approach, as an orthogonal technique to characterize the 3D structure of therapeutic antibodies, provides insight into the 3D structural variety and dynamics of heterogeneous IgG molecules.

Feasible atomic-resolution electron tomography for general crystal surfaces by quantitative reconstruction from a high-resolution image

Whether or not the 3-dimensional surface morphologies of a crystal sample can be reconstructed at atomic-scale from a single 2-dimensional image becomes an interesting issue in high-resolution transmission electron microscopy, after the work by Jia et al. [1]. Here we propose an improved and self-validated algorithm to enhance such an electron tomography method and to make it applicable to more general crystal surfaces even with thin amorphous layers. Our study shows that a resolution in the beam (z) direction and a confidence level have to be defined and estimated after performing tomographic reconstruction in order to evaluate the quality and the reliability of its result. Applying the proposed procedure to the Si[110] image to recover the surface morphologies of a silicon crystal with amorphous contamination, the obtained results show that an atomic-resolution of 0.384 nm in the z-direction and a high confidence level of 95% are achieved for imaging the Si-surface structures, quantitatively described by tomographic parameters, i.e., the height (defocus) and the thickness (atom number) of Si-atomic columns.

Three-dimensional structural dynamics of DNA origami Bennett linkages using individual-particle electron tomography

Scaffolded DNA origami has proven to be a powerful and efficient technique to fabricate functional nanomachines by programming the folding of a single-stranded DNA template strand into three-dimensional (3D) nanostructures, designed to be precisely motion-controlled. Although two-dimensional (2D) imaging of DNA nanomachines using transmission electron microscopy and atomic force microscopy suggested these nanomachines are dynamic in 3D, geometric analysis based on 2D imaging was insufficient to uncover the exact motion in 3D. Here we use the individual-particle electron tomography method and reconstruct 129 density maps from 129 individual DNA origami Bennett linkage mechanisms at ~ 6-14 nm resolution. The statistical analyses of these conformations lead to understanding the 3D structural dynamics of Bennett linkage mechanisms. Moreover, our effort provides experimental verification of a theoretical kinematics model of DNA origami, which can be used as feedback to improve the design and control of motion via optimized DNA sequences and routing.

Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection

As x-ray and electron tomography is pushed further into the nanoscale, the limitations of rotation stages become more apparent, leading to challenges in the alignment of the acquired projection images. Here we present an approach for rapid post-acquisition alignment of these projections to obtain high quality three-dimensional images. Our approach is based on a joint estimation of alignment errors, and the object, using an iterative refinement procedure. With simulated data where we know the alignment error of each projection image, our approach shows a residual alignment error that is a factor of a thousand smaller, and it reaches the same error level in the reconstructed image in less than half the number of iterations. We then show its application to experimental data in x-ray and electron nanotomography.