Very-High Dynamic Range, 10,000 Frames/Second Pixel Array Detector for Electron Microscopy

Ultra-fast Digital DPC Yielding High Spatio-temporal Resolution for Low-Dose Phase Characterization

In the scanning transmission electron microscope, both phase imaging of beam-sensitive materials and characterization of a material's functional properties using in situ experiments are becoming more widely available. As the practicable scan speed of 4D-STEM detectors improves, so too does the temporal resolution achievable for both differential phase contrast (DPC) and ptychography. However, the read-out burden of pixelated detectors, and the size of the gigabyte to terabyte sized data sets, remain a challenge for both temporal resolution and their practical adoption. In this work, we combine ultra-fast scan coils and detector signal digitization to show that a high-fidelity DPC phase reconstruction can be achieved from an annular segmented detector. Unlike conventional analog data phase reconstructions from digitized DPC-segment images yield reliable data, even at the fastest scan speeds. Finally, dose fractionation by fast scanning and multi-framing allows for postprocess binning of frame streams to balance signal-to-noise ratio and temporal resolution for low-dose phase imaging for in situ experiments.

High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the total data. Lay abstract: Four-dimensional scanning transmission electron microscopy (4-D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2-dimensional signal is acquired at each 2-D probe location, equating to a 4-D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4-D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100-1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4-D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10-100 times. The acquired data is then processed to fill-in or inpaint the missing data, taking advantage of the inherently low-complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4-D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4-D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100-300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

Atomic-level imaging of beam-sensitive COFs and MOFs by low-dose electron microscopy

Electron microscopy, an important technique that allows for the precise determination of structural information with high spatiotemporal resolution, has become indispensable in unravelling the complex relationships between material structure and properties ranging from mesoscale morphology to atomic arrangement. However, beam-sensitive materials, particularly those comprising organic components such as metal-organic frameworks (MOFs) and covalent organic frameworks (COFs), would suffer catastrophic damage from the high energy electrons, hindering the determination of atomic structures. A low-dose approach has arisen as a possible solution to this problem based on the integration of advancements in several aspects: electron optical system, detector, image processing, and specimen preservation. This article summarizes the transmission electron microscopy characterization of MOFs and COFs, including local structures, host-guest interactions, and interfaces at the atomic level. Revolutions in advanced direct electron detectors, algorithms in image acquisition and processing, and emerging methodology for high quality low-dose imaging are also reviewed. Finally, perspectives on the future development of electron microscopy methodology with the support of computer science are presented.

Partial hard occluded target reconstruction of Fourier single pixel imaging guided through range slice

Fourier single pixel imaging utilizes pre-programmed patterns for laser spatial distribution modulation to reconstruct intensity image of the target through reconstruction algorithms. The approach features non-locality and high anti-interference performance. However, Poor image quality is induced when the target of interest is occluded in Fourier single pixel imaging. To address the problem, a deep learning-based image inpainting algorithm is employed within Fourier single pixel imaging to reconstruct partially obscured targets with high quality. It applies a distance-based segmentation method to segment obscured regions and the target of interest. Additionally, it utilizes an image inpainting network that combines multi-scale sparse convolution and transformer architecture, along with a reconstruction network that integrates Channel Attention Mechanism and Attention Gate modules to reconstruct complete and clear intensity images of the target of interest. The proposed method significantly expands the application scenarios and improves the imaging quality of Fourier single pixel imaging. Simulation and real-world experimental results demonstrate that the proposed method exhibits the high inpainting and reconstruction capacity in the conditions of hard occlusion and down-sampling.

Spin disorder control of topological spin texture

Stabilization of topological spin textures in layered magnets has the potential to drive the development of advanced low-dimensional spintronics devices. However, achieving reliable and flexible manipulation of the topological spin textures beyond skyrmion in a two-dimensional magnet system remains challenging. Here, we demonstrate the introduction of magnetic iron atoms between the van der Waals gap of a layered magnet, Fe3GaTe2, to modify local anisotropic magnetic interactions. Consequently, we present direct observations of the order-disorder skyrmion lattices transition. In addition, non-trivial topological solitons, such as skyrmioniums and skyrmion bags, are realized at room temperature. Our work highlights the influence of random spin control of non-trivial topological spin textures.

Quantification of Hybrid Topological Spin Textures and Their Nanoscale Fluctuations in Ferrimagnets

Noncolinear spin textures, including chiral stripes and skyrmions, have shown great potential in spintronics. Basic configurations of spin textures are either Bloch or Néel types, and the intermediate hybrid type has rarely been reported. A major challenge in identifying hybrid spin textures is to quantitatively determine the hybrid angle, especially in ferrimagnets with weak net magnetization. Here, we develop an approach to quantify magnetic parameters, including chirality, saturation magnetization, domain wall width, and hybrid angle with sub-5 nm spatial resolution, based on Lorentz four-dimensional scanning transmission electron microscopy (Lorentz 4D-STEM). We find strong nanometer-scale variations in the hybrid angle and domain wall width within structurally and chemically homogeneous FeGd ferrimagnetic films. These variations fluctuate during different magnetization circles, revealing intrinsic local magnetization inhomogeneities. Furthermore, hybrid skyrmions can also be nucleated in FeGd films. These analyses demonstrate that the Lorentz 4D-STEM is a quantitative tool for exploring complex spin textures.

Deep learning at the edge enables real-time streaming ptychographic imaging

Coherent imaging techniques provide an unparalleled multi-scale view of materials across scientific and technological fields, from structural materials to quantum devices, from integrated circuits to biological cells. Driven by the construction of brighter sources and high-rate detectors, coherent imaging methods like ptychography are poised to revolutionize nanoscale materials characterization. However, these advancements are accompanied by significant increase in data and compute needs, which precludes real-time imaging, feedback and decision-making capabilities with conventional approaches. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz. The proposed AI-enabled workflow eliminates the oversampling constraints, allowing low-dose imaging using orders of magnitude less data than required by traditional methods.

Multi-scale time-resolved electron diffraction: A case study in moiré materials

Ultrafast-optical-pump - structural-probe measurements, including ultrafast electron and x-ray scattering, provide direct experimental access to the fundamental timescales of atomic motion, and are thus foundational techniques for studying matter out of equilibrium. High-performance detectors are needed in scattering experiments to obtain maximum scientific value from every probe particle. We deploy a hybrid pixel array direct electron detector to perform ultrafast electron diffraction experiments on a WSe2/MoSe2 2D heterobilayer, resolving the weak features of diffuse scattering and moiré superlattice structure without saturating the zero order peak. Enabled by the detector's high frame rate, we show that a chopping technique provides diffraction difference images with signal-to-noise at the shot noise limit. Finally, we demonstrate that a fast detector frame rate coupled with a high repetition rate probe can provide continuous time resolution from femtoseconds to seconds, enabling us to perform a scanning ultrafast electron diffraction experiment that maps thermal transport in WSe2/MoSe2 and resolves distinct diffusion mechanisms in space and time.

Geometric phase correction: A direct phase correction method to register low contrast noisy TEM images

A major challenge in the emerging field of low-dose electron microscopy lies in the development of drift correction algorithms against beam-induced specimen motion and compatible with highly noisy transmission electron microscopy (TEM) images. We report here a new drift correction method, namely geometric phase correlation (GPC), to correlate the specimen motion in real space by directly measuring the unwrapped geometric phase shift in the spatial frequency domain of the TEM image (especially from the intensive Bragg spots for crystalline materials) with sub-pixel precision. The GPC method outperforms cross-correlation-based methods in both accuracy of specimen motion prediction from highly noisy TEM movies and computational efficiency of drift calculation from abundant image frames, which holds great promise for diverse applications in low-dose TEM imaging of beam-sensitive materials, such as metal-organic frameworks (MOFs) and covalent organic frameworks (COFs).

Defect Contrast with 4D-STEM: Understanding Crystalline Order with Virtual Detectors and Beam Modification

Material properties strongly depend on the nature and concentration of defects. Characterizing these features may require nano- to atomic-scale resolution to establish structure-property relationships. 4D-STEM, a technique where diffraction patterns are acquired at a grid of points on the sample, provides a versatile method for highlighting defects. Computational analysis of the diffraction patterns with virtual detectors produces images that can map material properties. Here, using multislice simulations, we explore different virtual detectors that can be applied to the diffraction patterns that go beyond the binary response functions that are possible using ordinary STEM detectors. Using graphene and lead titanate as model systems, we investigate the application of virtual detectors to study local order and in particular defects. We find that using a small convergence angle with a rotationally varying detector most efficiently highlights defect signals. With experimental graphene data, we demonstrate the effectiveness of these detectors in characterizing atomic features, including vacancies, as suggested in simulations. Phase and amplitude modification of the electron beam provides another process handle to change image contrast in a 4D-STEM experiment. We demonstrate how tailored electron beams can enhance signals from short-range order and how a vortex beam can be used to characterize local symmetry.

Cryogenic electron ptychographic single particle analysis with wide bandwidth information transfer

Advances in cryogenic transmission electron microscopy have revolutionised the determination of many macromolecular structures at atomic or near-atomic resolution. This method is based on conventional defocused phase contrast imaging. However, it has limitations of weaker contrast for small biological molecules embedded in vitreous ice, in comparison with cryo-ptychography, which shows increased contrast. Here we report a single-particle analysis based on the use of ptychographic reconstruction data, demonstrating that three dimensional reconstructions with a wide information transfer bandwidth can be recovered by Fourier domain synthesis. Our work suggests future applications in otherwise challenging single particle analyses, including small macromolecules and heterogeneous or flexible particles. In addition structure determination in situ within cells without the requirement for protein purification and expression may be possible.

Three-dimensional electron ptychography of organic-inorganic hybrid nanostructures

Three dimensional scaffolded DNA origami with inorganic nanoparticles has been used to create tailored multidimensional nanostructures. However, the image contrast of DNA is poorer than those of the heavy nanoparticles in conventional transmission electron microscopy at high defocus so that the biological and non-biological components in 3D scaffolds cannot be simultaneously resolved using tomography of samples in a native state. We demonstrate the use of electron ptychography to recover high contrast phase information from all components in a DNA origami scaffold without staining. We further quantitatively evaluate the enhancement of contrast in comparison with conventional transmission electron microscopy. In addition, We show that for ptychography post-reconstruction focusing simplifies the workflow and reduces electron dose and beam damage.

The authors demonstrate electron ptychographic computed tomography by simultaneously recording high contrast data from both the organic- and inorganic components in a 3D DNA-origami framework hybrid nanostructure.

Automatic parameter selection for electron ptychography via Bayesian optimization

Electron ptychography provides new opportunities to resolve atomic structures with deep sub-angstrom spatial resolution and to study electron-beam sensitive materials with high dose efficiency. In practice, obtaining accurate ptychography images requires simultaneously optimizing multiple parameters that are often selected based on trial-and-error, resulting in low-throughput experiments and preventing wider adoption. Here, we develop an automatic parameter selection framework to circumvent this problem using Bayesian optimization with Gaussian processes. With minimal prior knowledge, the workflow efficiently produces ptychographic reconstructions that are superior to those processed by experienced experts. The method also facilitates better experimental designs by exploring optimized experimental parameters from simulated data.