3D X-Ray Imaging of Continuous Objects beyond the Depth of Focus Limit

Multiview three-dimensional imaging using a Risley-prism-based spatially adaptive virtual camera field

We present a three-dimensional (3D) imaging system that incorporates a stationary camera and Risley prisms. By planning prism rotation to generate a spatially adaptive virtual camera field, the system allows multiple virtual cameras in the field to capture any object from different perspectives for 3D reconstruction. An automatic virtual camera calibration method based on perspective projection and geometric optics is developed to enable virtual camera field construction and characterization. Moreover, a 3D computational reconstruction framework is proposed for multiview information fusion using the virtual camera field. This framework combines nonlinear distortion correction with epipolar geometry computation to efficiently perform image rectification and stereo matching, which can further facilitate 3D object reconstruction through multiview triangulation. The experiments on synthetic and real data validate the feasibility and flexibility of our 3D imaging technique.

Using a modified double deep image prior for crosstalk mitigation in multislice ptychography

Multislice ptychography is a high-resolution microscopy technique used to image multiple separate axial planes using a single illumination direction. However, multislice ptychography reconstructions are often degraded by crosstalk, where some features on one plane erroneously contribute to the reconstructed image of another plane. Here, the use of a modified `double deep image prior' (DDIP) architecture is demonstrated in mitigating crosstalk artifacts in multislice ptychography. Utilizing the tendency of generative neural networks to produce natural images, a modified DDIP method yielded good results on experimental data. For one of the datasets, it is shown that using DDIP could remove the need of using additional experimental data, such as from X-ray fluorescence, to suppress the crosstalk. This method may help X-ray multislice ptychography work for more general experimental scenarios.

Adorym: a multi-platform generic X-ray image reconstruction framework based on automatic differentiation

We describe and demonstrate an optimization-based X-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-performance computing systems. We demonstrate its use on several experimental datasets to show improved image quality through parameter refinement.

Upscaling X-ray nanoimaging to macroscopic specimens

Upscaling X-ray nanoimaging to macroscopic specimens has the potential for providing insights across multiple length scales, but its feasibility has long been an open question. By combining the imaging requirements and existing proof-of-principle examples in large-specimen preparation, data acquisition and reconstruction algorithms, the authors provide imaging time estimates for howX-ray nanoimaging can be scaled to macroscopic specimens. To arrive at this estimate, a phase contrast imaging model that includes plural scattering effects is used to calculate the required exposure and corresponding radiation dose. The coherent X-ray flux anticipated from upcoming diffraction-limited light sources is then considered. This imaging time estimation is in particular applied to the case of the connectomes of whole mouse brains. To image the connectome of the whole mouse brain, electron microscopy connectomics might require years, whereas optimized X-ray microscopy connectomics could reduce this to one week. Furthermore, this analysis points to challenges that need to be overcome (such as increased X-ray detector frame rate) and opportunities that advances in artificial-intelligence-based 'smart' scanning might provide. While the technical advances required are daunting, it is shown that X-ray microscopy is indeed potentially applicable to nanoimaging of millimetre- or even centimetre-size specimens.

Comparison of distributed memory algorithms for X-ray wave propagation in inhomogeneous media

Calculations of X-ray wave propagation in large objects are needed for modeling diffractive X-ray optics and for optimization-based approaches to image reconstruction for objects that extend beyond the depth of focus. We describe three methods for calculating wave propagation with large arrays on parallel computing systems with distributed memory: (1) a full-array Fresnel multislice approach, (2) a tiling-based short-distance Fresnel multislice approach, and (3) a finite difference approach. We find that the first approach suffers from internode communication delays when the transverse array size becomes large, while the second and third approaches have similar scaling to large array size problems (with the second approach offering about three times the compute speed).

Overcoming information reduced data and experimentally uncertain parameters in ptychography with regularized optimization

The overdetermination of the mathematical problem underlying ptychography is reduced by a host of experimentally more desirable settings. Furthermore, reconstruction of the sample-induced phase shift is typically limited by uncertainty in the experimental parameters and finite sample thicknesses. Presented is a conjugate gradient descent algorithm, regularized optimization for ptychography (ROP), that recovers the partially known experimental parameters along with the phase shift, improves resolution by incorporating the multislice formalism to treat finite sample thicknesses, and includes regularization in the optimization process, thus achieving reliable results from noisy data with severely reduced and underdetermined information.

The achievable resolution for X-ray imaging of cells and other soft biological material

X-ray imaging of soft materials is often difficult because of the low contrast of the components. This particularly applies to frozen hydrated biological cells where the feature of interest can have a similar density to the surroundings. As a consequence, a high dose is often required to achieve the desired resolution. However, the maximum dose that a specimen can tolerate is limited by radiation damage. Results from 3D coherent diffraction imaging (CDI) of frozen hydrated specimens have given resolutions of ∼80 nm compared with the expected resolution of 10 nm predicted from theoretical considerations for identifying a protein embedded in water. Possible explanations for this include the inapplicability of the dose-fractionation theorem, the difficulty of phase determination, an overall object-size dependence on the required fluence and dose, a low contrast within the biological cell, insufficient exposure, and a variety of practical difficulties such as scattering from surrounding material. A recent article [Villaneuva-Perez et al. (2018), Optica, 5, 450-457] concluded that imaging by Compton scattering gave a large dose advantage compared with CDI because of the object-size dependence for CDI. An object-size dependence would severely limit the applicability of CDI and perhaps related coherence-based methods for structural studies. This article specifically includes the overall object size in the analysis of the fluence and dose requirements for coherent imaging in order to investigate whether there is a dependence on object size. The applicability of the dose-fractionation theorem is also discussed. The analysis is extended to absorption-based imaging and imaging by incoherent scattering (Compton) and fluorescence. This article includes analysis of the dose required for imaging specific low-contrast cellular organelles as well as for protein against water. This article concludes that for both absorption-based and coherent diffraction imaging, the dose-fractionation theorem applies and the required dose is independent of the overall size of the object. For incoherent-imaging methods such as Compton scattering, the required dose depends on the X-ray path length through the specimen. For all three types of imaging, the dependence of fluence and dose on a resolution d goes as 1/d 4 when imaging uniform-density voxels. The independence of CDI on object size means that there is no advantage for Compton scattering over coherent-based imaging methods. The most optimistic estimate of achievable resolution is 3 nm for imaging protein molecules in water/ice using lensless imaging methods in the water window. However, the attainable resolution depends on a variety of assumptions including the model for radiation damage as a function of resolution, the efficiency of any phase-retrieval process, the actual contrast of the feature of interest within the cell and the definition of resolution itself. There is insufficient observational information available regarding the most appropriate model for radiation damage in frozen hydrated biological material. It is advocated that, in order to compare theory with experiment, standard methods of reporting results covering parameters such as the feature examined (e.g. which cellular organelle), resolution, contrast, depth of the material (for 2D), estimate of noise and dose should be adopted.

Three dimensions, two microscopes, one code: Automatic differentiation for x-ray nanotomography beyond the depth of focus limit

Conventional tomographic reconstruction algorithms assume that one has obtained pure projection images, involving no within-specimen diffraction effects nor multiple scattering. Advances in x-ray nanotomography are leading toward the violation of these assumptions, by combining the high penetration power of x-rays, which enables thick specimens to be imaged, with improved spatial resolution that decreases the depth of focus of the imaging system. We describe a reconstruction method where multiple scattering and diffraction effects in thick samples are modeled by multislice propagation and the 3D object function is retrieved through iterative optimization. We show that the same proposed method works for both full-field microscopy and for coherent scanning techniques like ptychography. Our implementation uses the optimization toolbox and the automatic differentiation capability of the open-source deep learning package TensorFlow, demonstrating a straightforward way to solve optimization problems in computational imaging with flexibility and portability.

Using automatic differentiation as a general framework for ptychographic reconstruction

Coherent diffraction imaging methods enable imaging beyond lens-imposed resolution limits. In these methods, the object can be recovered by minimizing an error metric that quantifies the difference between diffraction patterns as observed, and those calculated from a present guess of the object. Efficient minimization methods require analytical calculation of the derivatives of the error metric, which is not always straightforward. This limits our ability to explore variations of basic imaging approaches. In this paper, we propose to substitute analytical derivative expressions with the automatic differentiation method, whereby we can achieve object reconstruction by specifying only the physics-based experimental forward model. We demonstrate the generality of the proposed method through straightforward object reconstruction for a variety of complex ptychographic experimental models.

Gridrec-MS: an algorithm for multi-slice tomography

Advances in imaging systems and modeling allow for depth information to be retrieved from projections via virtual sectioning of the imaged object. Here we introduce a regridding method that explicitly and directly incorporates this information into a general and non-iterative tomographic reconstruction algorithm. The method is applicable to any imaging scheme that provides depth-resolved projections. Additionally, we show, via numerical simulations, that with this method the required number of projections for adequate angular sampling can be reduced.

Joint ptycho-tomography reconstruction through alternating direction method of multipliers

We present the extension of ptychography for three-dimensional object reconstruction in a tomography setting. We describe the alternating direction method of multipliers (ADMM) as a generic reconstruction framework to efficiently solve the nonlinear optimization problem. In this framework, the ADMM breaks the joint reconstruction problem into two well-defined subproblems: ptychographic phase retrieval and tomographic reconstruction. In this paper, we use the gradient descent algorithm to solve both problems and demonstrate the efficiency of the proposed approach through numerical simulations. Further, we show that the proposed joint approach relaxes existing requirements for lateral probe overlap in conventional ptychography. Thus, it can allow more flexible data acquisition.

Resolving 500 nm axial separation by multi-slice X-ray ptychography

Combining multi-slice ptychography with multi-modality scanning probe microscopy reconstructs two planes of nanostructures separated by 500 nm with sub-20 nm lateral resolution, assisted by simultaneously measured fluorescence maps for decoupling low-spatial-frequency features.

Multi-slice X-ray ptychography offers an approach to achieve images with a nanometre-scale resolution from samples with thicknesses larger than the depth of field of the imaging system by modeling a thick sample as a set of thin slices and accounting for the wavefront propagation effects within the specimen. Here, we present an experimental demonstration that resolves two layers of nanostructures separated by 500 nm along the axial direction, with sub-10 nm and sub-20 nm resolutions on two layers, respectively. Fluorescence maps are simultaneously measured in the multi-modality imaging scheme to assist in decoupling the mixture of low-spatial-frequency features across different slices. The enhanced axial sectioning capability using correlative signals obtained from multi-modality measurements demonstrates the great potential of the multi-slice ptychography method for investigating specimens with extended dimensions in 3D with high resolution.