Invited article: a [corrected] unified evaluation of iterative projection algorithms for phase retrieval

Crystal structure via fluctuation scattering

Crystallography is a quintessential method for determining the atomic structure of crystals. The most common implementation of crystallography uses single crystals that must be of sufficient size, typically tens of micrometres or larger, depending on the complexity of the crystal structure. The emergence of serial data-collection methods in crystallography, particularly for time-resolved experiments, opens up opportunities to develop new routes to structure determination for nanocrystals and ensembles of crystals. Fluctuation X-ray scattering is a correlation-based approach for single-particle imaging from ensembles of identical particles, but has yet to be applied to crystal structure determination. Here, an iterative algorithm is presented that recovers crystal structure-factor intensities from fluctuation X-ray scattering correlations. The capabilities of this algorithm are demonstrated by recovering the structure of three small-molecule crystals and a protein crystal from simulated fluctuation X-ray scattering correlations. This method could facilitate the recovery of structure-factor intensities from crystals in serial crystallography experiments and relax sample requirements for crystallography experiments.

Strain Heterogeneity and Extended Defects in Halide Perovskite Devices

Strain is an important property in halide perovskite semiconductors used for optoelectronic applications because of its ability to influence device efficiency and stability. However, descriptions of strain in these materials are generally limited to bulk averages of bare films, which miss important property-determining heterogeneities that occur on the nanoscale and at interfaces in multilayer device stacks. Here, we present three-dimensional nanoscale strain mapping using Bragg coherent diffraction imaging of individual grains in Cs0.1FA0.9Pb(I0.95Br0.05)3 and Cs0.15FA0.85SnI3 (FA = formamidinium) halide perovskite absorbers buried in full solar cell devices. We discover large local strains and striking intragrain and grain-to-grain strain heterogeneity, identifying distinct islands of tensile and compressive strain inside grains. Additionally, we directly image dislocations with surprising regularity in Cs0.15FA0.85SnI3 grains and find evidence for dislocation-induced antiphase boundary formation. Our results shine a rare light on the nanoscale strains in these materials in their technologically relevant device setting.

Imaging extended single crystal lattice distortion fields with multi-peak Bragg ptychography

Recent advances in phase-retrieval-based x-ray imaging methods have demonstrated the ability to reconstruct 3D distortion vector fields within a nanocrystal by using coherent diffraction information from multiple crystal Bragg reflections. However, these works do not provide a solution to the challenges encountered in imaging lattice distortions in crystals with significant defect content that result in phase wrapping. Moreover, these methods only apply to isolated crystals smaller than the x-ray illumination, and therefore cannot be used for imaging of distortions in extended crystals. We introduce multi-peak Bragg ptychography which addresses both challenges via an optimization framework that combines stochastic gradient descent and phase unwrapping methods for robust image reconstruction of lattice distortions and defects in extended crystals. Our work uses modern automatic differentiation toolsets so that the method is easy to extend to other settings and easy to implement in high-performance computers. This work is particularly timely given the broad interest in using the increased coherent flux in fourth-generation synchrotrons for innovative material research.

Recovery of spatial frequencies in coherent diffraction imaging in the presence of a central obscuration

Coherent diffraction imaging (CDI) and its scanning version, ptychography, are lensless imaging approaches used to iteratively retrieve a sample's complex scattering amplitude from its measured diffraction patterns. These imaging methods are most useful in extreme ultraviolet (EUV) and X-ray regions of the electromagnetic spectrum, where efficient imaging optics are difficult to manufacture. CDI relies on high signal-to-noise ratio diffraction data to recover the phase, but increasing the flux can cause saturation effects on the detector. A conventional solution to this problem is to place a beam stop in front of the detector. The pixel masking method is a common solution to the problem of missing frequencies due to a beam stop. This paper describes the information redundancy in the recorded data set and expands on how the reconstruction algorithm can exploit this redundancy to estimate the missing frequencies. Thereafter, we modify the size of the beam stop in experimental and simulation data to assess the impact of the missing frequencies, investigate the extent to which the lost portion of the diffraction spectrum can be recovered, and quantify the effect of the beam stop on the image quality. The experimental findings and simulations conducted for EUV imaging demonstrate that when using a beam stop, the numerical aperture of the condenser is a crucial factor in the recovery of lost frequencies. Our thorough investigation of the reconstructed images provides information on the overall quality of reconstruction and highlights the vulnerable frequencies if the beam stop size is larger than the extent of the illumination NA. The outcome of this study can be applied to other sources of frequency loss, and it will contribute to the improvement of experiments and reconstruction algorithms in CDI.

A workflow for single-particle structure determination via iterative phasing of rotational invariants in fluctuation X-ray scattering

Fluctuation X-ray scattering (FXS) offers a complementary approach for nano- and bioparticle imaging with an X-ray free-electron laser (XFEL), by extracting structural information from correlations in scattered XFEL pulses. Here a workflow is presented for single-particle structure determination using FXS. The workflow includes procedures for extracting the rotational invariants from FXS patterns, performing structure reconstructions via iterative phasing of the invariants, and aligning and averaging multiple reconstructions. The reconstruction pipeline is implemented in the open-source software xFrame and its functionality is demonstrated on several simulated structures.

Ultrafast X-ray Diffraction Probe of Coherent Spin-State Dynamics in Molecules

We propose an approach to probe coherent spin-state dynamics of molecules using circularly polarized hard X-ray pulses. For the dynamically aligned nitric oxide molecules in a coherent superposition spin-orbit coupled electronic state that can be prepared through stimulated Raman scattering, we demonstrate the capability of ultrafast X-ray diffraction to not only reveal the quantum beating of the coherent spin-state wave packet but also image the spatial spin density of the molecule. With a circularly polarized ultrafast X-ray diffraction signal, we show that the electronic density matrix can be retrieved. The spatiotemporal resolving power of ultrafast X-ray diffraction paves the way for tracking transient spatial wave function in molecular dynamics involving the spin degree of freedom.

Nanoscale Three-Dimensional Network Structure of a Mesoporous Particle Unveiled via Adaptive Multidistance Coherent X-ray Tomography

Mesoporous nanoparticles provide rich platforms to devise functional materials by customizing the three-dimensional (3D) structures of nanopores. With the pore network as a key tuning parameter, the noninvasive and quantitative characterization of these 3D structures is crucial for the rational design of functional materials. This has prompted researchers to develop versatile nanoprobes with a high penetration power to inspect various specimens sized a few micrometers at nanoscale 3D resolutions. Here, with adaptive phase retrievals on independent data sets with different sampling frequencies, we introduce multidistance coherent X-ray tomography as a noninvasive and quantitative nanoprobe to realize high-resolution 3D imaging of micrometer-sized specimens. The 3D density distribution of an entire mesoporous silica nanoparticle was obtained at 13 nm 3D resolution for quantitative physical and morphological analyses of its 3D pore structure. The morphological features of the whole 3D pore network and pore connectivity were examined to gain insight into the potential functions of the particles. The proposed multidistance tomographic imaging scheme with quantitative structural analyses is expected to advance studies of functional materials by facilitating their structure-based rational design.

Imaging Light-Induced Migration of Dislocations in Halide Perovskites with 3D Nanoscale Strain Mapping

In recent years, halide perovskite materials have been used to make high-performance solar cells and light-emitting devices. However, material defects still limit device performance and stability. Here, synchrotron-based Bragg coherent diffraction imaging is used to visualize nanoscale strain fields, such as those local to defects, in halide perovskite microcrystals. Significant strain heterogeneity within MAPbBr3 (MA = CH3 NH3 + ) crystals is found in spite of their high optoelectronic quality, and both 〈100〉 and 〈110〉 edge dislocations are identified through analysis of their local strain fields. By imaging these defects and strain fields in situ under continuous illumination, dramatic light-induced dislocation migration across hundreds of nanometers is uncovered. Further, by selectively studying crystals that are damaged by the X-ray beam, large dislocation densities and increased nanoscale strains are correlated with material degradation and substantially altered optoelectronic properties assessed using photoluminescence microscopy measurements. These results demonstrate the dynamic nature of extended defects and strain in halide perovskites, which will have important consequences for device performance and operational stability.

Three-dimensional imaging using coherent x rays at grazing incidence geometry

We have developed a three-dimensional coherent diffraction imaging algorithm to retrieve phases of diffraction patterns of samples in grazing incidence small angle x-ray scattering experiments. The algorithm interprets the diffraction patterns using the distorted-wave Born approximation instead of the Born approximation, as in this case, the existence of a reflected beam from the substrate causes the diffraction pattern to deviate significantly from the simple Fourier transform of the object. Detailed computer simulations show that the algorithm works. Verification with real experiments is planned.

Phase retrieval for arbitrary complex-valued objects using structured illumination

A method to solve of the phase retrieval problem in a non-convex formulation for complex-valued objects with a support constraint is proposed. It is shown that two coded diffraction patterns (CDPs) obtained in the same Fresnel or Fraunhofer diffraction plane by masking an object with two, direct and inverse, random binary amplitude masks, are sufficient to reconstruct an arbitrary complex-valued object up to the global phase. The general solution of the problem was found as the sum of two mutually phase-consistent partial solutions obtained by applying the modified error-reduction or hybrid input-output algorithm to each of two "mask+CDP" pairs. The results of model experiments confirmed the possibility of noise-resistant and high-accuracy retrieval of complex-valued objects of various types with the oversampling ratio σ ≥ 2 making use of a small number of iterations. The method is applicable to coherent radiation of any kind.

High-precision phase retrieval method for speckle suppression based on optimized modulation masks

Traditional methods of coherent diffraction imaging using random masks result in an insufficient difference between the diffraction patterns, making it challenging to form a strong amplitude constraint, causing significant speckle noise in the measurement results. Hence, this study proposes an optimized mask design method combining random and Fresnel masks. Increasing the difference between diffraction intensity patterns enhances the amplitude constraint, suppresses the speckle noise effectively, and improves the phase recovery accuracy. The numerical distribution of the modulation masks is optimized by adjusting the combination ratio of the two mask modes. The simulation and physical experiments show that the reconstruction results of PSNR and SSIM using the proposed method are higher than those using random masks, and the speckle noises are effectively reduced.

Direct high-resolution X-ray imaging exploiting pseudorandomness

Owing to its unique penetrating power and high-resolution capability, X-ray imaging has been an irreplaceable tool since its discovery. Despite the significance, the resolution of X-ray imaging has largely been limited by the technical difficulties on X-ray lens making. Various lensless imaging methods have been proposed, but are yet relying on multiple measurements or additional constraints on measurements or samples. Here we present coherent speckle-correlation imaging (CSI) using a designed X-ray diffuser. CSI has no prerequisites for samples or measurements. Instead, from a single shot measurement, the complex sample field is retrieved based on the pseudorandomness of the speckle intensity pattern, ensured through a diffuser. We achieve a spatial resolution of 13.9 nm at 5.46 keV, beating the feature size of the diffuser used (300 nm). The high-resolution imaging capability is theoretically explained based on fundamental and practical limits. We expect the CSI to be a versatile tool for navigating the unexplored world of nanometer.

Three-dimensional femtosecond snapshots of isolated faceted nanostructures

The structure and dynamics of isolated nanosamples in free flight can be directly visualized via single-shot coherent diffractive imaging using the intense and short pulses of x-ray free-electron lasers. Wide-angle scattering images encode three-dimensional (3D) morphological information of the samples, but its retrieval remains a challenge. Up to now, effective 3D morphology reconstructions from single shots were only achieved via fitting with highly constrained models, requiring a priori knowledge about possible geometries. Here, we present a much more generic imaging approach. Relying on a model that allows for any sample morphology described by a convex polyhedron, we reconstruct wide-angle diffraction patterns from individual silver nanoparticles. In addition to known structural motives with high symmetries, we retrieve imperfect shapes and agglomerates that were not previously accessible. Our results open unexplored routes toward true 3D structure determination of single nanoparticles and, ultimately, 3D movies of ultrafast nanoscale dynamics.

Physical optics simulations for synchrotron radiation sources

We describe approaches to high-accuracy physical optics calculations used for the development of x-ray beamlines at synchrotron radiation sources, as well as simulation of experiments and processing of experimental data at some of these beamlines. We pay special attention to the treatment of the partial coherence of x rays, a topic of high practical importance for modern low-emittance high-brightness synchrotron radiation facilities. The approaches are based, to a large extent, on the works of Emil Wolf and co-authors, including the basic scalar diffraction theory and the coherent mode decomposition method. The presented simulation examples are related to the case of the novel Coherent Diffractive Imaging beamline that is currently under development at the National Synchrotron Light Source II at the Brookhaven National Laboratory.

Embedded parallel Fourier ptychographic microscopy reconstruction system

Fourier ptychographic microscopy (FPM) has attracted a wide range of focus for its ability of large space-bandwidth product and quantitative phase imaging. It is a typical computational imaging technique that jointly optimizes imaging hardware and reconstruction algorithms. The data redundancy and inverse problem algorithms are the sources of FPM's excellent performance. But at the same time, this large amount of data processing and complex algorithms also evidently reduce the imaging speed. To accelerate the FPM reconstruction speed, we proposed a fast FPM reconstruction framework consisting of three levels of parallel computation and implemented it with an embedded computing module. In the conventional FPM framework, the sample image is divided into multiple sub-regions to process separately because the illumination angles and defocus distances for different sub-regions may also be different. Our parallel framework first performs digital refocusing and high-resolution reconstruction for each sub-region separately and then stitches the complex sub-regions together to obtain the final high-resolution complex image. The feasibility of the proposed parallel FPM reconstruction framework is verified with different experimental results acquired with the system we built.

A general method for directly phasing diffraction data from high-solvent-content protein crystals

A procedure is described for direct phase determination in protein crystallography, applicable to crystals with high solvent content. The procedure requires only the diffraction data and an estimate of the solvent content as input. Direct phase determination is treated as a constraint satisfaction problem, in which an image is sought that is consistent with both the diffraction data and generic constraints on the density distribution in the crystal. The problem is solved using an iterative projection algorithm, the Difference Map algorithm, which has good global convergence properties, and can locate the correct solution without any initial phase information. Computational efficiency is improved by breaking the problem down into two stages; initial approximation of the molecular envelope at low resolution, followed by subsequent phase determination using all of the data. The molecular envelope is continually updated during the phase determination step. At both stages, the algorithm is initiated with many different and random phase sets, which are evolved subject to the constraints. A clustering procedure is used to identify consistent results across multiple runs, which are then averaged to generate consensus envelopes or phase sets. The emergence of highly consistent phase sets is diagnostic of success. The effectiveness of the procedure is demonstrated by application to 42 known structures of solvent fraction 0.60-0.85. The procedure works robustly at intermediate resolutions (1.9-3.5 Å) but is strongly dependent on crystal solvent content, only working routinely with solvent fractions greater than 0.70.

Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization

Deep learning-based image reconstruction methods have achieved remarkable success in phase recovery and holographic imaging. However, the generalization of their image reconstruction performance to new types of samples never seen by the network remains a challenge. Here we introduce a deep learning framework, termed Fourier Imager Network (FIN), that can perform end-to-end phase recovery and image reconstruction from raw holograms of new types of samples, exhibiting unprecedented success in external generalization. FIN architecture is based on spatial Fourier transform modules that process the spatial frequencies of its inputs using learnable filters and a global receptive field. Compared with existing convolutional deep neural networks used for hologram reconstruction, FIN exhibits superior generalization to new types of samples, while also being much faster in its image inference speed, completing the hologram reconstruction task in ~0.04 s per 1 mm² of the sample area. We experimentally validated the performance of FIN by training it using human lung tissue samples and blindly testing it on human prostate, salivary gland tissue and Pap smear samples, proving its superior external generalization and image reconstruction speed. Beyond holographic microscopy and quantitative phase imaging, FIN and the underlying neural network architecture might open up various new opportunities to design broadly generalizable deep learning models in computational imaging and machine vision fields.

Classification of diffraction patterns using a convolutional neural network in single-particle-imaging experiments performed at X-ray free-electron lasers

Single particle imaging (SPI) at X-ray free-electron lasers is particularly well suited to determining the 3D structure of particles at room temperature. For a successful reconstruction, diffraction patterns originating from a single hit must be isolated from a large number of acquired patterns. It is proposed that this task could be formulated as an image-classification problem and solved using convolutional neural network (CNN) architectures. Two CNN configurations are developed: one that maximizes the F1 score and one that emphasizes high recall. The CNNs are also combined with expectation-maximization (EM) selection as well as size filtering. It is observed that the CNN selections have lower contrast in power spectral density functions relative to the EM selection used in previous work. However, the reconstruction of the CNN-based selections gives similar results. Introducing CNNs into SPI experiments allows the reconstruction pipeline to be streamlined, enables researchers to classify patterns on the fly, and, as a consequence, enables them to tightly control the duration of their experiments. Incorporating non-standard artificial-intelligence-based solutions into an existing SPI analysis workflow may be beneficial for the future development of SPI experiments.

High-resolution 3D Fourier ptychographic reconstruction using a hemispherical illumination source with multiplexed-coded strategy

Fourier ptychography is a promising and flexible imaging technique that can achieve 2D quantitative reconstruction with higher resolution beyond the limitation of the system. Meanwhile, by using different imaging models, the same platform can be applied to achieve 3D refractive index reconstruction. To improve the illumination NA as much as possible while reducing the intensity attenuation problem caused by the LED board used in the traditional FP platform, we apply a hemispherical lighting structure and design a new LED arrangement according to 3D Fourier diffraction theory. Therefore, we could obtain the illumination of 0.98NA using 187 LEDs and achieve imaging half-pitch resolutions of ∼174 nm and ∼524 nm for the lateral and axial directions respectively, using a 40×/0.6NA objective lens. Furthermore, to reduce the number of captured images required and realize real-time data collection, we apply the multiplexed-coded illumination strategy and compare several coded patterns through simulation and experiment. Through comparison, we determined a radial-coded illumination pattern that could achieve more similar results as sequential scanning and increase the acquisition speed to above 1 Hz. Therefore, this paper provides the possibility of this technique in real-time 3D observation of in vitro live samples.

Adaptive Detection of Wave Aberrations Based on the Multichannel Filter

An adaptive method for determining the type and magnitude of aberration in a wide range is proposed on the basis of an optical processing of the analyzed wavefront using a multichannel filter matched to the adjustable Zernike phase functions. The approach is based on an adaptive (or step-by-step) compensation of wavefront aberrations based on a dynamically tunable multichannel filter implemented on a spatial light modulator. For adaptive filter adjustment, a set of criteria is proposed that takes into account not only the magnitude of the correlation peak, but also the maximum intensity, compactness, and orientation of the distribution in each diffraction order. The experimental results have shown the efficiency of the proposed approach for detecting wavefront aberrations in a wide range (from 0.1λ to λ).

Ultrafast Imaging of Molecular Dynamics Using Ultrafast Low-Frequency Lasers, X-ray Free Electron Lasers, and Electron Pulses

The requirement of high space-time resolution and brightness is a great challenge for imaging atomic motion and making molecular movies. Important breakthroughs in ultrabright tabletop laser, X-ray, and electron sources have enabled the direct imaging of evolving molecular structures in chemical processes, and recent experimental advances in preparing ultrafast laser and electron pulses resulted in molecular imaging with femtosecond time resolution. This Perspective presents an overview of the versatile imaging methods of molecular dynamics. High-order harmonic generation imaging and photoelectron diffraction imaging are based on laser-induced ionization and rescattering processes. Coulomb explosion imaging retrieves molecular structural information by detecting the momentum vectors of fragmented ions. Diffraction imaging encodes molecular structural and electronic information in reciprocal space. We also present various applications of these ultrafast imaging methods in resolving laser-induced nuclear and electronic dynamics.

Radiographic imaging and tomography

Radiographic imaging and tomography (RadIT) come in many types such as x-ray imaging and tomography (IT), proton IT, neutron IT, muon IT, and more. We identify five RadIT themes: physics, sources, detectors, methods, and data science, which are integral parts of image interpretation and 3D tomographic reconstruction. Traditionally, RadIT has been driven by medicine, non-destructive testing, material sciences, and security applications. The latest thrusts of growth come from automation, machine vision, additive manufacturing, and virtual reality (the "metaverse"). The five RadIT themes parallel their counterparts in optical IT. Synergies among different forms of RadIT and with optical IT motivate further advances towards multi-modal IT and quantum IT.

Advances in laboratory-scale ptychography using high harmonic sources [Invited]

Extreme ultraviolet microscopy and wavefront sensing are key elements for next-generation ultrafast applications, such as chemically-resolved imaging, focal spot diagnostics in pump-and-probe experiments, and actinic metrology for the state-of-the-art lithography node at 13.5 nm wavelength. Ptychography offers a robust solution to the aforementioned challenges. Originally adapted by the electron and synchrotron communities, advances in the stability and brightness of high-harmonic tabletop sources have enabled the transfer of ptychography to the laboratory. This review covers the state of the art in tabletop ptychography with high harmonic generation sources. We consider hardware options such as illumination optics and detector concepts as well as algorithmic aspects in the analysis of multispectral ptychography data. Finally, we review technological application cases such as multispectral wavefront sensing, attosecond pulse characterization, and depth-resolved imaging.

Understanding and correcting wavenumber error in interference pattern structured illumination imaging

The impacts of uncertainty in mirror movements in mechanically scanned interference pattern structured illumination imaging (IPSII) are discussed. It is shown that uncertainty in IPSII mirror movements causes errors in both the phase and amplitude of the Fourier transform of the resulting imaging. Finally, we demonstrate that iterative phase retrieval algorithms can improve the quality of IPSII images by correcting the phase errors caused by mirror movement uncertainties.

Methods of Coherent X-Ray Diffraction Imaging

Abstract

Methods of coherent X-ray diffraction imaging of the spatial structure of noncrystalline objects and nanocrystals (nanostructures) are considered. Particular attention is paid to the methods of scanning-based coherent diffraction imaging (ptychography), visualization based on coherent surface scattering with application of correlation spectroscopy approaches, and specific features of visualization using X-ray free-electron laser radiation. The corresponding data in the literature are analyzed to demonstrate the state of the art of the methods of coherent diffraction imaging and fields of their application.

Modern Types of Axicons: New Functions and Applications

Axicon is a versatile optical element for forming a zero-order Bessel beam, including high-power laser radiation schemes. Nevertheless, it has drawbacks such as the produced beam's parameters being dependent on a particular element, the output beam's intensity distribution being dependent on the quality of element manufacturing, and uneven axial intensity distribution. To address these issues, extensive research has been undertaken to develop nondiffracting beams using a variety of advanced techniques. We looked at four different and special approaches for creating nondiffracting beams in this article. Diffractive axicons, meta-axicons-flat optics, spatial light modulators, and photonic integrated circuit-based axicons are among these approaches. Lately, there has been noteworthy curiosity in reducing the thickness and weight of axicons by exploiting diffraction. Meta-axicons, which are ultrathin flat optical elements made up of metasurfaces built up of arrays of subwavelength optical antennas, are one way to address such needs. In addition, when compared to their traditional refractive and diffractive equivalents, meta-axicons have a number of distinguishing advantages, including aberration correction, active tunability, and semi-transparency. This paper is not intended to be a critique of any method. We have outlined the most recent advancements in this field and let readers determine which approach best meets their needs based on the ease of fabrication and utilization. Moreover, one section is devoted to applications of axicons utilized as sensors of optical properties of devices and elements as well as singular beams states and wavefront features.

Dynamic coherent diffractive imaging with a physics-driven untrained learning method

Reconstruction of a complex field from one single diffraction measurement remains a challenging task among the community of coherent diffraction imaging (CDI). Conventional iterative algorithms are time-consuming and struggle to converge to a feasible solution because of the inherent ambiguities. Recently, deep-learning-based methods have shown considerable success in computational imaging, but they require large amounts of training data that in many cases are difficult to obtain. Here, we introduce a physics-driven untrained learning method, termed Deep CDI, which addresses the above problem and can image a dynamic process with high confidence and fast reconstruction. Without any labeled data for pretraining, the Deep CDI can reconstruct a complex-valued object from a single diffraction pattern by combining a conventional artificial neural network with a real-world physical imaging model. To our knowledge, we are the first to demonstrate that the support region constraint, which is widely used in the iteration-algorithm-based method, can be utilized for loss calculation. The loss calculated from support constraint and free propagation constraint are summed up to optimize the network's weights. As a proof of principle, numerical simulations and optical experiments on a static sample are carried out to demonstrate the feasibility of our method. We then continuously collect 3600 diffraction patterns and demonstrate that our method can predict the dynamic process with an average reconstruction speed of 228 frames per second (FPS) using only a fraction of the diffraction data to train the weights.

Coherent modulation imaging using unknown modulators

Coherent modulation imaging (CMI) is an effective lensless diffraction imaging method with fast algorithmic convergence and high robustness to data defects. In the reported algorithms for CMI, one important requirement is that the modulator function need to be known a priori; and an additional step for the modulator characterization is required to be carried out in advance by other methods, such as ptychography, which could be cumbersome in practice. Here, we propose an improved algorithm that allows for the transmission function of a completely unknown modulator to be recovered during the same iterative process of image reconstruction. We have verified the method in both simulations and optical experiments. This improvement would turn CMI into a more practical and standalone technique for broader applications in biology and materials science.

Fast digital refocusing and depth of field extended Fourier ptychography microscopy

Fourier ptychography microscopy (FPM) shares its roots with the synthetic aperture technique and phase retrieval method, and is a recently developed computational microscopic super-resolution technique. By turning on the light-emitting diode (LED) elements sequentially and acquiring the corresponding images that contain different spatial frequencies, FPM can achieve a wide field-of-view (FOV), high-spatial-resolution imaging and phase recovery simultaneously. Conventional FPM assumes that the sample is sufficiently thin and strictly in focus. Nevertheless, even for a relatively thin sample, the non-planar distribution characteristics and the non-ideal position/posture of the sample will cause all or part of FOV to be defocused. In this paper, we proposed a fast digital refocusing and depth-of-field (DOF) extended FPM strategy by taking the advantages of image lateral shift caused by sample defocusing and varied-angle illuminations. The lateral shift amount is proportional to the defocus distance and the tangent of the illumination angle. Instead of searching the optimal defocus distance with the optimization search strategy, which is time consuming, the defocus distance of each subregion of the sample can be precisely and quickly obtained by calculating the relative lateral shift amounts corresponding to different oblique illuminations. And then, digital refocusing strategy rooting in the angular spectrum (AS) method is integrated into FPM framework to achieve the high-resolution and phase information reconstruction for each part of the sample, which means the DOF of the FPM can be effectively extended. The feasibility of the proposed method in fast digital refocusing and DOF extending is verified in the actual experiments with the USAF chart and biological samples.

Generalized optimization framework for pixel super-resolution imaging in digital holography

The imaging quality of in-line digital holography is challenged by the twin-image and aliasing effects because sensors only respond to intensity and pixels are of finite size. As a result, phase retrieval and pixel super-resolution techniques serve as two essential ingredients for high-fidelity and high-resolution holographic imaging. In this work, we combine the two as a unified optimization problem and propose a generalized algorithmic framework for pixel-super-resolved phase retrieval. In particular, we introduce the iterative projection algorithms and gradient descent algorithms for solving this problem. The basic building blocks, namely the projection operator and the Wirtinger gradient, are derived and analyzed. As an example, the Wirtinger gradient descent algorithm for pixel-super-resolved phase retrieval, termed as Wirtinger-PSR, is proposed and compared with the classical error-reduction algorithm. The Wirtinger-PSR algorithm is verified with both simulated and experimental data. The proposed framework generalizes well to various physical settings and helps bridging the gap between empirical studies and theoretical analyses.

Scattering Matrix Determination in Crystalline Materials from 4D Scanning Transmission Electron Microscopy at a Single Defocus Value

Recent work has revived interest in the scattering matrix formulation of electron scattering in transmission electron microscopy as a stepping stone toward atomic-resolution structure determination in the presence of multiple scattering. We discuss ways of visualizing the scattering matrix that make its properties clear. Through a simulation-based case study incorporating shot noise, we shown how regularizing on this continuity enables the scattering matrix to be reconstructed from 4D scanning transmission electron microscopy (STEM) measurements from a single defocus value. Intriguingly, for crystalline samples, this process also yields the sample thickness to nanometer accuracy with no a priori knowledge about the sample structure. The reconstruction quality is gauged by using the reconstructed scattering matrix to simulate STEM images at defocus values different from that of the data from which it was reconstructed.

Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm

The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained "blind" ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.

Flexible ptychography platform to expand the potential of imaging at free electron lasers

Ptychography, a scanning coherent diffraction imaging method, can produce a high-resolution reconstruction of a sample and, at the same time, of the illuminating beam. The emergence of vacuum ultraviolet and X-ray free electron lasers (FELs) has brought sources with unprecedented characteristics that enable X-ray ptychography with highly intense and ultra-fast short-wavelength pulses. However, the shot-to-shot pulse fluctuations typical for FEL pulses and particularly the partial spatial coherence of self-amplified spontaneous emission (SASE) FELs lead to numerical complexities in the ptychographic algorithms and ultimately restrict the application of ptychography at FELs. We present a general adaptive forward model for ptychography based on automatic differentiation, which is able to perform reconstructions even under these conditions. We applied this model to the first ptychography experiment at FLASH, the Free electron LASer in Hamburg, and obtained a high-resolution reconstruction of the sample as well as the complex wavefronts of individual FLASH pulses together with their coherence properties. This is not possible with more common ptychography algorithms.

Exploring the 3D structure and defects of a self-assembled gold mesocrystal by coherent X-ray diffraction imaging

Mesocrystals are nanostructured materials consisting of individual nanocrystals having a preferred crystallographic orientation. On mesoscopic length scales, the properties of mesocrystals are strongly affected by structural heterogeneity. Here, we report the detailed structural characterization of a faceted mesocrystal grain self-assembled from 60 nm sized gold nanocubes. Using coherent X-ray diffraction imaging, we determined the structure of the mesocrystal with the resolution sufficient to resolve each gold nanoparticle. The reconstructed electron density of the gold mesocrystal reveals its intrinsic structural heterogeneity, including local deviations of lattice parameters, and the presence of internal defects. The strain distribution shows that the average superlattice obtained by angular X-ray cross-correlation analysis and the real, "multidomain" structure of a mesocrystal are very close to each other, with a deviation less than 10%. These results will provide an important impact to understanding the fundamental principles of structuring and self-assembly including ensuing properties of mesocrystals.

Holographic optical field recovery using a regularized untrained deep decoder network

Image reconstruction using minimal measured information has been a long-standing open problem in many computational imaging approaches, in particular in-line holography. Many solutions are devised based on compressive sensing (CS) techniques with handcrafted image priors or supervised deep neural networks (DNN). However, the limited performance of CS methods due to lack of information about the image priors and the requirement of an enormous amount of per-sample-type training resources for DNNs has posed new challenges over the primary problem. In this study, we propose a single-shot lensless in-line holographic reconstruction method using an untrained deep neural network which is incorporated with a physical image formation algorithm. We demonstrate that by modifying a deep decoder network with simple regularizers, a Gabor hologram can be inversely reconstructed via a minimization process that is constrained by a deep image prior. The outcoming model allows to accurately recover the phase and amplitude images without any training dataset, excess measurements, or specific assumptions about the object's or the measurement's characteristics.

Adorym: a multi-platform generic X-ray image reconstruction framework based on automatic differentiation

We describe and demonstrate an optimization-based X-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-performance computing systems. We demonstrate its use on several experimental datasets to show improved image quality through parameter refinement.

Macromolecular phasing using diffraction from multiple crystal forms

A phasing algorithm for protein crystallography using diffraction data from multiple crystal forms is proposed. The algorithm is evaluated by simulation, and practical aspects and potential for ab initio phasing are discussed.

A phasing algorithm for macromol­ecular crystallography is proposed that utilizes diffraction data from multiple crystal forms – crystals of the same mol­ecule with different unit-cell packings (different unit-cell parameters or space-group symmetries). The approach is based on the method of iterated projections, starting with no initial phase information. The practicality of the method is demonstrated by simulation using known structures that exist in multiple crystal forms, assuming some information on the mol­ecular envelope and positional relationships between the mol­ecules in the different unit cells. With incorporation of new or existing methods for determination of these parameters, the approach has potential as a method for ab initio phasing.

In Situ Strain Evolution on Pt Nanoparticles during Hydrogen Peroxide Decomposition

Fundamental understanding of structural changes during catalytic reactions is crucial to understanding the underlying mechanisms and optimizing efficiencies. Surface energy and related catalytic mechanisms are widely studied. However, the catalyst lattice deformation induced by catalytic processes is not well understood. Here, we study the strain in an individual platinum (Pt) nanoparticle (NP) using Bragg coherent diffraction imaging under in situ oxidation and reduction reactions. When Pt NPs are exposed to H₂O₂, a typical oxidizer and an intermediate during the oxygen reduction reaction process, alternating overall strain distribution near the surface and inside the NP is observed at the (111) Bragg reflection. In contrast, relatively insignificant changes appear in the (200) reflection. Density functional theory calculations are employed to rationalize the anisotropic lattice strain in terms of induced stress by H₂O₂ adsorption and decomposition on the Pt NP surface. Our study provides deeper insight into the activity-structure relationship in this system.

An advanced workflow for single-particle imaging with the limited data at an X-ray free-electron laser

An improved analysis for single-particle imaging (SPI) experiments, using the limited data, is presented here. Results are based on a study of bacteriophage PR772 performed at the Atomic, Molecular and Optical Science instrument at the Linac Coherent Light Source as part of the SPI initiative. Existing methods were modified to cope with the shortcomings of the experimental data: inaccessibility of information from half of the detector and a small fraction of single hits. The general SPI analysis workflow was upgraded with the expectation-maximization based classification of diffraction patterns and mode decomposition on the final virus-structure determination step. The presented processing pipeline allowed us to determine the 3D structure of bacteriophage PR772 without symmetry constraints with a spatial resolution of 6.9 nm. The obtained resolution was limited by the scattering intensity during the experiment and the relatively small number of single hits.

PyNX: high-performance computing toolkit for coherent X-ray imaging based on operators

The open-source PyNX toolkit has been extended to provide tools for coherent X-ray imaging data analysis and simulation. All calculations can be executed on graphical processing units (GPUs) to achieve high-performance computing speeds. The toolkit can be used for coherent diffraction imaging (CDI), ptychography and wavefront propagation, in the far- or near-field regime. Moreover, all imaging operations (propagation, projections, algorithm cycles…) can be implemented in Python as simple mathematical operators, an approach which can be used to easily combine basic algorithms in a tailored chain. Calculations can also be distributed to multiple GPUs, e.g. for large ptychography data sets. Command-line scripts are available for on-line CDI and ptychography analysis, either from raw beamline data sets or using the coherent X-ray imaging data format.

Modulator refinement algorithm for coherent modulation imaging

Coherent modulation imaging (CMI) has been shown to be an effective lensless diffraction approach to imaging general extended samples with fast algorithmic convergence and high robustness to data imperfection. In the reported phasing algorithms of CMI, an exact knowledge of modulator is used as a priori. Extra characterization of the modulator is thus required before the CMI experiments are conducted and this can be cumbersome. Here we propose a modulator refinement algorithm that allows for modulator refinement in the same iterative process of image reconstruction. We demonstrate the method for both near-field and far-field geometries in simulations and for a far-field experiment. A relaxed requirement on exactly knowing the modulator would turn CMI into a standalone technique and make it much easier to implement, thus open up its wider applications in biology and materials science.

Phase retrieval algorithms for lensless imaging using diffractive shearing interferometry

Diffractive shearing interferometry (DSI) is a method that has recently been developed to perform lensless imaging using extreme ultraviolet radiation generated by high-harmonic generation. In this paper, we investigate the uniqueness of the DSI solution and the requirements for the support constraint size. We find that there can be multiple solutions to the DSI problem that consist of displaced copies of the actual object. These alternative solutions can be eliminated by enforcing a sufficiently tight support constraint, or by introducing additional synthetic constraints. We furthermore propose a new DSI algorithm inspired by the analogy with coherent diffractive imaging (CDI) algorithms: the original DSI algorithm is in a way analogous to the hybrid input-output algorithm as used in CDI, and we propose a new algorithm that is more analogous to the error reduction algorithm as used in CDI. We find that the newly proposed algorithm is suitable for final refinement of the reconstruction.

A phase-retrieval toolbox for X-ray holography and tomography

Propagation-based phase-contrast X-ray imaging is by now a well established imaging technique, which - as a full-field technique - is particularly useful for tomography applications. Since it can be implemented with synchrotron radiation and at laboratory micro-focus sources, it covers a wide range of applications. A limiting factor in its development has been the phase-retrieval step, which was often performed using methods with a limited regime of applicability, typically based on linearization. In this work, a much larger set of algorithms, which covers a wide range of cases (experimental parameters, objects and constraints), is compiled into a single toolbox - the HoloTomoToolbox - which is made publicly available. Importantly, the unified structure of the implemented phase-retrieval functions facilitates their use and performance test on different experimental data.

Imaging around corners in the mid-infrared using speckle correlations

Speckle correlation imaging offers the ability to see objects through diffusive materials and around corners. Imaging self-illuminating thermal objects in non-line-of-sight scenarios is of particular interest. Here, using bispectrum and phase retrieval methods, we demonstrate speckle correlation imaging of mid-infrared objects through diffusers and around corners at resolutions near the diffraction limit. The images agree well with those recorded by conventional cameras with line-of-sight to the same objects.

General approaches for shear-correcting coordinate transformations in Bragg coherent diffraction imaging. Part I

This two-part article series provides a generalized description of the scattering geometry of Bragg coherent diffraction imaging (BCDI) experiments, the shear distortion effects inherent in the 3D image obtained from presently used methods and strategies to mitigate this distortion. Part I starts from fundamental considerations to present the general real-space coordinate transformation required to correct this shear, in a compact operator formulation that easily lends itself to implementation with available software packages. Such a transformation, applied as a final post-processing step following phase retrieval, is crucial for arriving at an undistorted, correctly oriented and physically meaningful image of the 3D crystalline scatterer. As the relevance of BCDI grows in the field of materials characterization, the available sparse literature that addresses the geometric theory of BCDI and the subsequent analysis methods are generalized here. This geometrical aspect, specific to coherent Bragg diffraction and absent in 2D transmission CDI experiments, gains particular importance when it comes to spatially resolved characterization of 3D crystalline materials in a reliable nondestructive manner. This series of articles describes this theory, from the diffraction in Bragg geometry to the corrections needed to obtain a properly rendered digital image of the 3D scatterer. Part I of this series provides the experimental BCDI community with the general form of the 3D real-space distortions in the phase-retrieved object, along with the necessary post-retrieval correction method. Part II builds upon the geometric theory developed in Part I with the formalism to correct the shear distortions directly on an orthogonal grid within the phase-retrieval algorithm itself, allowing more physically realistic constraints to be applied. Taken together, Parts I and II provide the X-ray science community with a set of generalized BCDI shear-correction techniques crucial to the final rendering of a 3D crystalline scatterer and for the development of new BCDI methods and experiments.

Single-shot sequential projection phase retrieval and 3D localization from chromatic aberration

A phase retrieval method based on sequential projection and chromatic aberration is reported. Construction of this method includes a red, green and blue (RGB) LED, an objective and a color camera. Owing to the chromatic aberration characteristics of the objective, three color images obtained by the color camera correspond to three equivalent propagation planes. Equivalent relative distances among these planes can be obtained by defining and iteratively minimizing the convergence index. Then, sequential projection strategy is used for phase retrieval in each image plane. Based on the recovered phase information and angular spectrum propagation principle, digital refocusing and 3D localization can be achieved for each subregion of the sample. Finally, the feasibility of our method is demonstrated by simulations and experiments.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Extreme ultraviolet lensless imaging without object support through rotational diversity in diffractive shearing interferometry

We report on a method that allows microscopic image reconstruction from extreme-ultraviolet diffraction patterns without the need for object support constraints or other prior knowledge about the object structure. This is achieved by introducing additional diversity through rotation of an object in a rotationally asymmetric probe beam, produced by the spatial interference between two phase-coherent high-harmonic beams. With this rotational diffractive shearing interferometry method, we demonstrate robust image reconstruction of microscopic objects at wavelengths around 30 nm, using images recorded at only three to five different object rotations.

Free log-likelihood as an unbiased metric for coherent diffraction imaging

Coherent Diffraction Imaging (CDI), a technique where an object is reconstructed from a single (2D or 3D) diffraction pattern, recovers the lost diffraction phases without a priori knowledge of the extent (support) of the object. The uncertainty of the object support can lead to over-fitting and prevents an unambiguous metric evaluation of solutions. We propose to use a ‘free’ log-likelihood indicator, where a small percentage of points are masked from the reconstruction algorithms, as an unbiased metric to evaluate the validity of computed solutions, independent of the sample studied. We also show how a set of solutions can be analysed through an eigen-decomposition to yield a better estimate of the real object. Example analysis on experimental data is presented both for a test pattern dataset, and the diffraction pattern from a live cyanobacteria cell. The method allows the validation of reconstructions on a wide range of materials (hard condensed or biological), and should be particularly relevant for 4th generation synchrotrons and X-ray free electron lasers, where large, high-throughput datasets require a method for unsupervised data evaluation.

Singleshot polychromatic coherent diffractive imaging with a high-order harmonic source

Singleshot polychromatic coherent diffractive imaging is performed with a high-intensity high-order harmonic generation source. The coherence properties are analyzed and several reconstructions show the shot-to-shot fluctuations of the incident beam wavefront. The method is based on a multi-step approach. First, the spectrum is extracted from double-slit diffraction data. The spectrum is used as input to extract the monochromatic sample diffraction pattern, then phase retrieval is performed on the quasi-monochromatic data to obtain the sample's exit surface wave. Reconstructions based on guided error reduction (ER) and alternating direction method of multipliers (ADMM) are compared. ADMM allows additional penalty terms to be included in the cost functional to promote sparsity within the reconstruction.