Real-time coherent diffraction inversion using deep generative networks

Proton Conducting Neuromorphic Materials and Devices

Neuromorphic computing and artificial intelligence hardware generally aims to emulate features found in biological neural circuit components and to enable the development of energy-efficient machines. In the biological brain, ionic currents and temporal concentration gradients control information flow and storage. It is therefore of interest to examine materials and devices for neuromorphic computing wherein ionic and electronic currents can propagate. Protons being mobile under an external electric field offers a compelling avenue for facilitating biological functionalities in artificial synapses and neurons. In this review, we first highlight the interesting biological analog of protons as neurotransmitters in various animals. We then discuss the experimental approaches and mechanisms of proton doping in various classes of inorganic and organic proton-conducting materials for the advancement of neuromorphic architectures. Since hydrogen is among the lightest of elements, characterization in a solid matrix requires advanced techniques. We review powerful synchrotron-based spectroscopic techniques for characterizing hydrogen doping in various materials as well as complementary scattering techniques to detect hydrogen. First-principles calculations are then discussed as they help provide an understanding of proton migration and electronic structure modification. Outstanding scientific challenges to further our understanding of proton doping and its use in emerging neuromorphic electronics are pointed out.

AI-NERD: Elucidation of relaxation dynamics beyond equilibrium through AI-informed X-ray photon correlation spectroscopy

Understanding and interpreting dynamics of functional materials in situ is a grand challenge in physics and materials science due to the difficulty of experimentally probing materials at varied length and time scales. X-ray photon correlation spectroscopy (XPCS) is uniquely well-suited for characterizing materials dynamics over wide-ranging time scales. However, spatial and temporal heterogeneity in material behavior can make interpretation of experimental XPCS data difficult. In this work, we have developed an unsupervised deep learning (DL) framework for automated classification of relaxation dynamics from experimental data without requiring any prior physical knowledge of the system. We demonstrate how this method can be used to accelerate exploration of large datasets to identify samples of interest, and we apply this approach to directly correlate microscopic dynamics with macroscopic properties of a model system. Importantly, this DL framework is material and process agnostic, marking a concrete step towards autonomous materials discovery.

Sub-photon accuracy noise reduction of a single shot coherent diffraction pattern with an atomic model trained autoencoder

Single-shot imaging with femtosecond X-ray lasers is a powerful measurement technique that can achieve both high spatial and temporal resolution. However, its accuracy has been severely limited by the difficulty of applying conventional noise-reduction processing. This study uses deep learning to validate noise reduction techniques, with autoencoders serving as the learning model. Focusing on the diffraction patterns of nanoparticles, we simulated a large dataset treating the nanoparticles as composed of many independent atoms. Three neural network architectures are investigated: neural network, convolutional neural network and U-net, with U-net showing superior performance in noise reduction and subphoton reproduction. We also extended our models to apply to diffraction patterns of particle shapes different from those in the simulated data. We then applied the U-net model to a coherent diffractive imaging study, wherein a nanoparticle in a microfluidic device is exposed to a single X-ray free-electron laser pulse. After noise reduction, the reconstructed nanoparticle image improved significantly even though the nanoparticle shape was different from the training data, highlighting the importance of transfer learning.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Noise-robust latent vector reconstruction in ptychography using deep generative models

Computational imaging is increasingly vital for a broad spectrum of applications, ranging from biological to material sciences. This includes applications where the object is known and sufficiently sparse, allowing it to be described with a reduced number of parameters. When no explicit parameterization is available, a deep generative model can be trained to represent an object in a low-dimensional latent space. In this paper, we harness this dimensionality reduction capability of autoencoders to search for the object solution within the latent space rather than the object space. We demonstrate what we believe to be a novel approach to ptychographic image reconstruction by integrating a deep generative model obtained from a pre-trained autoencoder within an automatic differentiation ptychography (ADP) framework. This approach enables the retrieval of objects from highly ill-posed diffraction patterns, offering an effective method for noise-robust latent vector reconstruction in ptychography. Moreover, the mapping into a low-dimensional latent space allows us to visualize the optimization landscape, which provides insight into the convexity and convergence behavior of the inverse problem. With this work, we aim to facilitate new applications for sparse computational imaging such as when low radiation doses or rapid reconstructions are essential.

Complex Oxides for Brain-Inspired Computing: A Review

The fields of brain-inspired computing, robotics, and, more broadly, artificial intelligence (AI) seek to implement knowledge gleaned from the natural world into human-designed electronics and machines. In this review, the opportunities presented by complex oxides, a class of electronic ceramic materials whose properties can be elegantly tuned by doping, electron interactions, and a variety of external stimuli near room temperature, are discussed. The review begins with a discussion of natural intelligence at the elementary level in the nervous system, followed by collective intelligence and learning at the animal colony level mediated by social interactions. An important aspect highlighted is the vast spatial and temporal scales involved in learning and memory. The focus then turns to collective phenomena, such as metal-to-insulator transitions (MITs), ferroelectricity, and related examples, to highlight recent demonstrations of artificial neurons, synapses, and circuits and their learning. First-principles theoretical treatments of the electronic structure, and in situ synchrotron spectroscopy of operating devices are then discussed. The implementation of the experimental characteristics into neural networks and algorithm design is then revewed. Finally, outstanding materials challenges that require a microscopic understanding of the physical mechanisms, which will be essential for advancing the frontiers of neuromorphic computing, are highlighted.

Investigating the robustness of a deep learning-based method for quantitative phase retrieval from propagation-based x-ray phase contrast measurements under laboratory conditions

Objective.Quantitative phase retrieval (QPR) in propagation-based x-ray phase contrast imaging of heterogeneous and structurally complicated objects is challenging under laboratory conditions due to partial spatial coherence and polychromaticity. A deep learning-based method (DLBM) provides a nonlinear approach to this problem while not being constrained by restrictive assumptions about object properties and beam coherence. The objective of this work is to assess a DLBM for its applicability under practical scenarios by evaluating its robustness and generalizability under typical experimental variations.Approach.Towards this end, an end-to-end DLBM was employed for QPR under laboratory conditions and its robustness was investigated across various system and object conditions. The robustness of the method was tested via varying propagation distances and its generalizability with respect to object structure and experimental data was also tested.Main results.Although the end-to-end DLBM was stable under the studied variations, its successful deployment was found to be affected by choices pertaining to data pre-processing, network training considerations and system modeling.Significance.To our knowledge, we demonstrated for the first time, the potential applicability of an end-to-end learning-based QPR method, trained on simulated data, to experimental propagation-based x-ray phase contrast measurements acquired under laboratory conditions with a commercial x-ray source and a conventional detector. We considered conditions of polychromaticity, partial spatial coherence, and high noise levels, typical to laboratory conditions. This work further explored the robustness of this method to practical variations in propagation distances and object structure with the goal of assessing its potential for experimental use. Such an exploration of any DLBM (irrespective of its network architecture) before practical deployment provides an understanding of its potential behavior under experimental settings.

Phase Contrast Image Restoration by Formulating Its Imaging Principle and Reversing the Formulation With Deep Neural Networks

Phase contrast microscopy, as a noninvasive imaging technique, has been widely used to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle of the specifically-designed microscope, phase contrast microscopy images contain artifacts such as halo and shade-off which hinder the cell segmentation and detection tasks. Some previous works developed simplified computational imaging models for phase contrast microscopes by linear approximations and convolutions. The approximated models do not exactly reflect the imaging principle of the phase contrast microscope and accordingly the image restoration by solving the corresponding deconvolution process is not perfect. In this paper, we revisit the optical principle of the phase contrast microscope to precisely formulate its imaging model without any approximation. Based on this model, we propose an image restoration procedure by reversing this imaging model with a deep neural network, instead of mathematically deriving the inverse operator of the model which is technically impossible. Extensive experiments are conducted to demonstrate the superiority of the newly derived phase contrast microscopy imaging model and the power of the deep neural network on modeling the inverse imaging procedure. Moreover, the restored images enable that high quality cell segmentation task can be easily achieved by simply thresholding methods. Implementations of this work are publicly available at https://github.com/LiangHann/Phase-Contrast-Microscopy-Image-Restoration.

Finding the semantic similarity in single-particle diffraction images using self-supervised contrastive projection learning

Single-shot coherent diffraction imaging of isolated nanosized particles has seen remarkable success in recent years, yielding in-situ measurements with ultra-high spatial and temporal resolution. The progress of high-repetition-rate sources for intense X-ray pulses has further enabled recording datasets containing millions of diffraction images, which are needed for the structure determination of specimens with greater structural variety and dynamic experiments. The size of the datasets, however, represents a monumental problem for their analysis. Here, we present an automatized approach for finding semantic similarities in coherent diffraction images without relying on human expert labeling. By introducing the concept of projection learning, we extend self-supervised contrastive learning to the context of coherent diffraction imaging and achieve a dimensionality reduction producing semantically meaningful embeddings that align with physical intuition. The method yields substantial improvements compared to previous approaches, paving the way toward real-time and large-scale analysis of coherent diffraction experiments at X-ray free-electron lasers.

Artifact identification in X-ray diffraction data using machine learning methods

In situ synchrotron high-energy X-ray powder diffraction (XRD) is highly utilized by researchers to analyze the crystallographic structures of materials in functional devices (e.g. battery materials) or in complex sample environments (e.g. diamond anvil cells or syntheses reactors). An atomic structure of a material can be identified by its diffraction pattern along with a detailed analysis of the Rietveld refinement which yields rich information on the structure and the material, such as crystallite size, microstrain and defects. For in situ experiments, a series of XRD images is usually collected on the same sample under different conditions (e.g. adiabatic conditions) yielding different states of matter, or is simply collected continuously as a function of time to track the change of a sample during a chemical or physical process. In situ experiments are usually performed with area detectors and collect images composed of diffraction patterns. For an ideal powder, the diffraction pattern should be a series of concentric Debye-Scherrer rings with evenly distributed intensities in each ring. For a realistic sample, one may observe different characteristics other than the typical ring pattern, such as textures or preferred orientations and single-crystal diffraction spots. Textures or preferred orientations usually have several parts of a ring that are more intense than the rest, whereas single-crystal diffraction spots are localized intense spots owing to diffraction of large crystals, typically >10 µm. In this work, an investigation of machine learning methods is presented for fast and reliable identification and separation of the single-crystal diffraction spots in XRD images. The exclusion of artifacts during an XRD image integration process allows a precise analysis of the powder diffraction rings of interest. When it is trained with small subsets of highly diverse datasets, the gradient boosting method can consistently produce high-accuracy results. The method dramatically decreases the amount of time spent identifying and separating single-crystal diffraction spots in comparison with the conventional method.

Linking scientific instruments and computation: Patterns, technologies, and experiences

Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly discarding some data elements or by directing instruments to relevant areas of experimental space. Thus, methods are required for configuring and running distributed computing pipelines—what we call flows—that link instruments, computers (e.g., for analysis, simulation, artificial intelligence [AI] model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. We review common patterns associated with such flows and describe methods for instantiating these patterns. We present experiences with the application of these methods to the processing of data from five different scientific instruments, each of which engages powerful computers for data inversion,model training, or other purposes. We also discuss implications of such methods for operators and users of scientific facilities.

•

Patterns for linking instruments and computers for online analysis are reviewed

•

Methods are presented for capturing such “flows” in reusable forms

•

The use of Globus automation services to run flows is described

•

Implications of these methods for scientists and facilities are discussed

The industrial revolution transformed society via large-scale automation of manufacturing. Today, AI- and robotics-driven automation of scientific research seems set to usher in a new era of accelerated discovery. But just as the industrial revolution depended on new replicable and scalable manufacturing processes and methods for delivering the copious mechanical power required by those processes, so the automated discovery revolution demands new methods for implementing research automation processes and for connecting those processes to computing and data power. We present here new methods that address these essential needs by allowing scientists to capture common automation patterns in reusable flows and to embed such flows in a global trust, data, and computing fabric that enables instant access to powerful AI, simulation, and other computational capabilities. We use examples from synchrotron light sources to show how these methods can be realized in software and applied at scale.

Gwaihir: Jupyter Notebook graphical user interface for Bragg coherent diffraction imaging

In a world where data are steadily made more available, Gwaihir is a tool that overcomes multiple issues by bridging remote access, cluster computing and a user-friendly interface, consequentially improving the link between synchrotrons and their users for Bragg coherent diffraction imaging.

Bragg coherent X-ray diffraction is a nondestructive method for probing material structure in three dimensions at the nanoscale, with unprecedented resolution in displacement and strain fields. This work presents Gwaihir, a user-friendly and open-source tool to process and analyze Bragg coherent X-ray diffraction data. It integrates the functionalities of the existing packages bcdi and PyNX in the same toolbox, creating a natural workflow and promoting data reproducibility. Its graphical interface, based on Jupyter Notebook widgets, combines an interactive approach for data analysis with a powerful environment designed to link large-scale facilities and scientists.

Cryptanalysis of an optical cryptosystem with uncertainty quantification in a probabilistic model

In this paper, a modified probabilistic deep learning method is proposed to attack the double random phase encryption by modeling the conditional distribution of plaintext. The well-trained probabilistic model gives both predictions of plaintext and uncertainty quantification, the latter of which is first introduced to optical cryptanalysis. Predictions of the model are close to real plaintexts, showing the success of the proposed model. Uncertainty quantification reveals the level of reliability of each pixel in the prediction of plaintext without ground truth. Subsequent simulation experiments demonstrate that uncertainty quantification can effectively identify poor-quality predictions to avoid the risk of unreliability from deep learning models.

Performance Evaluation of Deep Neural Network Model for Coherent X-ray Imaging

We present a supervised deep neural network model for phase retrieval of coherent X-ray imaging and evaluate the performance. A supervised deep-learning-based approach requires a large amount of pre-training datasets. In most proposed models, the various experimental uncertainties are not considered when the input dataset, corresponding to the diffraction image in reciprocal space, is generated. We explore the performance of the deep neural network model, which is trained with an ideal quality of dataset, when it faces real-like corrupted diffraction images. We focus on three aspects of data qualities such as a detection dynamic range, a degree of coherence and noise level. The investigation shows that the deep neural network model is robust to a limited dynamic range and partially coherent X-ray illumination in comparison to the traditional phase retrieval, although it is more sensitive to the noise than the iteration-based method. This study suggests a baseline capability of the supervised deep neural network model for coherent X-ray imaging in preparation for the deployment to the laboratory where diffraction images are acquired.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

Noise reduction and mask removal neural network for X-ray single-particle imaging

Free-electron lasers could enable X-ray imaging of single biological macromolecules and the study of protein dynamics, paving the way for a powerful new imaging tool in structural biology, but a low signal-to-noise ratio and missing regions in the detectors, colloquially termed 'masks', affect data collection and hamper real-time evaluation of experimental data. In this article, the challenges posed by noise and masks are tackled by introducing a neural network pipeline that aims to restore diffraction intensities. For training and testing of the model, a data set of diffraction patterns was simulated from 10 900 different proteins with molecular weights within the range of 10-100 kDa and collected at a photon energy of 8 keV. The method is compared with a simple low-pass filtering algorithm based on autocorrelation constraints. The results show an improvement in the mean-squared error of roughly two orders of magnitude in the presence of masks compared with the noisy data. The algorithm was also tested at increasing mask width, leading to the conclusion that demasking can achieve good results when the mask is smaller than half of the central speckle of the pattern. The results highlight the competitiveness of this model for data processing and the feasibility of restoring diffraction intensities from unknown structures in real time using deep learning methods. Finally, an example is shown of this preprocessing making orientation recovery more reliable, especially for data sets containing very few patterns, using the expansion-maximization-compression algorithm.

Cryptographic analysis on an optical random-phase-encoding cryptosystem for complex targets based on physics-informed learning

Optical cryptanalysis based on deep learning (DL) has grabbed more and more attention. However, most DL methods are purely data-driven methods, lacking relevant physical priors, resulting in generalization capabilities restrained and limiting practical applications. In this paper, we demonstrate that the double-random phase encoding (DRPE)-based optical cryptosystems are susceptible to preprocessing ciphertext-only attack (pCOA) based on DL strategies, which can achieve high prediction fidelity for complex targets by using only one random phase mask (RPM) for training. After preprocessing the ciphertext information to procure substantial intrinsic information, the physical knowledge DL method based on physical priors is exploited to further learn the statistical invariants in different ciphertexts. As a result, the generalization ability has been significantly improved by increasing the number of training RPMs. This method also breaks the image size limitation of the traditional COA method. Optical experiments demonstrate the feasibility and the effectiveness of the proposed learning-based pCOA method.

Dynamic coherent diffractive imaging with a physics-driven untrained learning method

Reconstruction of a complex field from one single diffraction measurement remains a challenging task among the community of coherent diffraction imaging (CDI). Conventional iterative algorithms are time-consuming and struggle to converge to a feasible solution because of the inherent ambiguities. Recently, deep-learning-based methods have shown considerable success in computational imaging, but they require large amounts of training data that in many cases are difficult to obtain. Here, we introduce a physics-driven untrained learning method, termed Deep CDI, which addresses the above problem and can image a dynamic process with high confidence and fast reconstruction. Without any labeled data for pretraining, the Deep CDI can reconstruct a complex-valued object from a single diffraction pattern by combining a conventional artificial neural network with a real-world physical imaging model. To our knowledge, we are the first to demonstrate that the support region constraint, which is widely used in the iteration-algorithm-based method, can be utilized for loss calculation. The loss calculated from support constraint and free propagation constraint are summed up to optimize the network's weights. As a proof of principle, numerical simulations and optical experiments on a static sample are carried out to demonstrate the feasibility of our method. We then continuously collect 3600 diffraction patterns and demonstrate that our method can predict the dynamic process with an average reconstruction speed of 228 frames per second (FPS) using only a fraction of the diffraction data to train the weights.

PhaseGAN: a deep-learning phase-retrieval approach for unpaired datasets

Phase retrieval approaches based on deep learning (DL) provide a framework to obtain phase information from an intensity hologram or diffraction pattern in a robust manner and in real-time. However, current DL architectures applied to the phase problem rely on i) paired datasets, i. e., they are only applicable when a satisfactory solution of the phase problem has been found, and ii) the fact that most of them ignore the physics of the imaging process. Here, we present PhaseGAN, a new DL approach based on Generative Adversarial Networks, which allows the use of unpaired datasets and includes the physics of image formation. The performance of our approach is enhanced by including the image formation physics and a novel Fourier loss function, providing phase reconstructions when conventional phase retrieval algorithms fail, such as ultra-fast experiments. Thus, PhaseGAN offers the opportunity to address the phase problem in real-time when no phase reconstructions but good simulations or data from other experiments are available.

Recovering the phase and amplitude of X-ray FEL pulses using neural networks and differentiable models

Dynamics experiments are an important use-case for X-ray free-electron lasers (XFELs), but time-domain measurements of the X-ray pulses themselves remain a challenge. Shot-by-shot X-ray diagnostics could enable a new class of simpler and potentially higher-resolution pump-probe experiments. Here, we report training neural networks to combine low-resolution measurements in both the time and frequency domains to recover X-ray pulses at high-resolution. Critically, we also recover the phase, opening the door to coherent-control experiments with XFELs. The model-based generative neural-network architecture can be trained directly on unlabeled experimental data and is fast enough for real-time analysis on the new generation of MHz XFELs.

Reconstruction of nanoscale particles from single-shot wide-angle free-electron-laser diffraction patterns with physics-informed neural networks

Single-shot wide-angle diffraction imaging is a widely used method to investigate the structure of noncrystallizing objects such as nanoclusters, large proteins, or even viruses. Its main advantage is that information about the three-dimensional structure of the object is already contained in a single image. This makes it useful for the reconstruction of fragile and nonreproducible particles without the need for tomographic measurements. However, currently there is no efficient numerical inversion algorithm available that is capable of determining the object's structure in real time. Neural networks, on the other hand, excel in image processing tasks suited for such purpose. Here we show how a physics-informed deep neural network can be used to reconstruct complete three-dimensional object models of uniform, convex particles on a voxel grid from single two-dimensional wide-angle scattering patterns. We demonstrate its universal reconstruction capabilities for silver nanoclusters, where the network uncovers novel geometric structures that reproduce the experimental scattering data with very high precision.

Bragg Coherent Diffraction Imaging for In Situ Studies in Electrocatalysis

Electrocatalysis is at the heart of a broad range of physicochemical applications that play an important role in the present and future of a sustainable economy. Among the myriad of different electrocatalysts used in this field, nanomaterials are of ubiquitous importance. An increased surface area/volume ratio compared to bulk makes nanoscale catalysts the preferred choice to perform electrocatalytic reactions. Bragg coherent diffraction imaging (BCDI) was introduced in 2006 and since has been applied to obtain 3D images of crystalline nanomaterials. BCDI provides information about the displacement field, which is directly related to strain. Lattice strain in the catalysts impacts their electronic configuration and, consequently, their binding energy with reaction intermediates. Even though there have been significant improvements since its birth, the fact that the experiments can only be performed at synchrotron facilities and its relatively low resolution to date (∼10 nm spatial resolution) have prevented the popularization of this technique. Herein, we will briefly describe the fundamentals of the technique, including the electrocatalysis relevant information that we can extract from it. Subsequently, we review some of the computational experiments that complement the BCDI data for enhanced information extraction and improved understanding of the underlying nanoscale electrocatalytic processes. We next highlight success stories of BCDI applied to different electrochemical systems and in heterogeneous catalysis to show how the technique can contribute to future studies in electrocatalysis. Finally, we outline current challenges in spatiotemporal resolution limits of BCDI and provide our perspectives on recent developments in synchrotron facilities as well as the role of machine learning and artificial intelligence in addressing them.

Real-time phase-retrieval and wavefront sensing enabled by an artificial neural network

In this manuscript we demonstrate a method to reconstruct the wavefront of focused beams from a measured diffraction pattern behind a diffracting mask in real-time. The phase problem is solved by means of a neural network, which is trained with simulated data and verified with experimental data. The neural network allows live reconstructions within a few milliseconds, which previously with iterative phase retrieval took several seconds, thus allowing the adjustment of complex systems and correction by adaptive optics in real time. The neural network additionally outperforms iterative phase retrieval with high noise diffraction patterns.

Complex imaging of phase domains by deep neural networks

The reconstruction of a single-particle image from the modulus of its Fourier transform, by phase-retrieval methods, has been extensively applied in X-ray structural science. Particularly for strong-phase objects, such as the phase domains found inside crystals by Bragg coherent diffraction imaging (BCDI), conventional iteration methods are time consuming and sensitive to their initial guess because of their iterative nature. Here, a deep-neural-network model is presented which gives a fast and accurate estimate of the complex single-particle image in the form of a universal approximator learned from synthetic data. A way to combine the deep-neural-network model with conventional iterative methods is then presented to refine the accuracy of the reconstructed results from the proposed deep-neural-network model. Improved convergence is also demonstrated with experimental BCDI data.

Deep neural networks in real-time coherent diffraction imaging

The potential for convolutional neural networks to provide real-time imaging capabilities for coherent diffraction imaging experiments at XFELs is discussed.

Fast reconstruction of single-shot wide-angle diffraction images through deep learning

Single-shot x-ray imaging of short-lived nanostructures such as clusters and nanoparticles near a phase transition or non-crystalizing objects such as large proteins and viruses is currently the most elegant method for characterizing their structure. Using hard x-ray radiation provides scattering images that encode two-dimensional projections, which can be combined to identify the full three-dimensional object structure from multiple identical samples. Wide-angle scattering using XUV or soft x-rays, despite yielding lower resolution, provides three-dimensional structural information in a single shot and has opened routes towards the characterization of non-reproducible objects in the gas phase. The retrieval of the structural information contained in wide-angle scattering images is highly non-trivial, and currently no efficient rigorous algorithm is known. Here we show that deep learning networks, trained with simulated scattering data, allow for fast and accurate reconstruction of shape and orientation of nanoparticles from experimental images. The gain in speed compared to conventional retrieval techniques opens the route for automated structure reconstruction algorithms capable of real-time discrimination and pre-identification of nanostructures in scattering experiments with high repetition rate—thus representing the enabling technology for fast femtosecond nanocrystallography.

Single-shot characterization of strongly focused coherent XUV and soft X-ray beams

In this Letter, we present a novel, to the best of our knowledge, single-shot method for characterizing focused coherent beams. We utilize a dedicated amplitude-only mask, in combination with an iterative phase retrieval algorithm, to reconstruct the amplitude and phase of a focused beam from a single measured far-field diffraction pattern alone. In a proof-of-principle experiment at a wavelength of 13.5 nm, we demonstrate our new method and obtain an RMS phase error of better than λ/70. This method will find applications in the alignment of complex optical systems, real-time feedback to adaptive optics, and single-shot beam characterization, e.g., at free-electron lasers or high-order harmonic beamlines.

Multi-resolution convolutional neural networks for inverse problems

Inverse problems in image processing, phase imaging, and computer vision often share the same structure of mapping input image(s) to output image(s) but are usually solved by different application-specific algorithms. Deep convolutional neural networks have shown great potential for highly variable tasks across many image-based domains, but sometimes can be challenging to train due to their internal non-linearity. We propose a novel, fast-converging neural network architecture capable of solving generic image(s)-to-image(s) inverse problems relevant to a diverse set of domains. We show this approach is useful in recovering wavefronts from direct intensity measurements, imaging objects from diffusely reflected images, and denoising scanning transmission electron microscopy images, just by using different training datasets. These successful applications demonstrate the proposed network to be an ideal candidate solving general inverse problems falling into the category of image(s)-to-image(s) translation.