High speed/low dose analytical electron microscopy with dynamic sampling

Framework of compressive sensing and data compression for 4D-STEM

Four-dimensional Scanning Transmission Electron Microscopy (4D-STEM) is a powerful technique for high-resolution and high-precision materials characterization at multiple length scales, including the characterization of beam-sensitive materials. However, the field of view of 4D-STEM is relatively small, which in absence of live processing is limited by the data size required for storage. Furthermore, the rectilinear scan approach currently employed in 4D-STEM places a resolution- and signal-dependent dose limit for the study of beam sensitive materials. Improving 4D-STEM data and dose efficiency, by keeping the data size manageable while limiting the amount of electron dose, is thus critical for broader applications. Here we introduce a general method for reconstructing 4D-STEM data with subsampling in both real and reciprocal spaces at high fidelity. The approach is first tested on the subsampled datasets created from a full 4D-STEM dataset, and then demonstrated experimentally using random scan in real-space. The same reconstruction algorithm can also be used for compression of 4D-STEM datasets, leading to a large reduction (100 times or more) in data size, while retaining the fine features of 4D-STEM imaging, for crystalline samples.

Demonstration of an AI-driven workflow for autonomous high-resolution scanning microscopy

Modern scanning microscopes can image materials with up to sub-atomic spatial and sub-picosecond time resolutions, but these capabilities come with large volumes of data, which can be difficult to store and analyze. We report the Fast Autonomous Scanning Toolkit (FAST) that addresses this challenge by combining a neural network, route optimization, and efficient hardware controls to enable a self-driving experiment that actively identifies and measures a sparse but representative data subset in lieu of the full dataset. FAST requires no prior information about the sample, is computationally efficient, and uses generic hardware controls with minimal experiment-specific wrapping. We test FAST in simulations and a dark-field X-ray microscopy experiment of a WSe2 film. Our studies show that a FAST scan of <25% is sufficient to accurately image and analyze the sample. FAST is easy to adapt for any scanning microscope; its broad adoption will empower general multi-level studies of materials evolution with respect to time, temperature, or other parameters.

Alignment-invariant signal reality reconstruction in hyperspectral imaging using a deep convolutional neural network architecture

The energy resolution in hyperspectral imaging techniques has always been an important matter in data interpretation. In many cases, spectral information is distorted by elements such as instruments' broad optical transfer function, and electronic high frequency noises. In the past decades, advances in artificial intelligence methods have provided robust tools to better study sophisticated system artifacts in spectral data and take steps towards removing these artifacts from the experimentally obtained data. This study evaluates the capability of a recently developed deep convolutional neural network script, EELSpecNet, in restoring the reality of a spectral data. The particular strength of the deep neural networks is to remove multiple instrumental artifacts such as random energy jitters of the source, signal convolution by the optical transfer function and high frequency noise at once using a single training data set. Here, EELSpecNet performance in reducing noise, and restoring the original reality of the spectra is evaluated for near zero-loss electron energy loss spectroscopy signals in Scanning Transmission Electron Microscopy. EELSpecNet demonstrates to be more efficient and more robust than the currently widely used Bayesian statistical method, even in harsh conditions (e.g. high signal broadening, intense high frequency noise).

Scanning Transmission Electron Microscopy in a Scanning Electron Microscope for the High-Throughput Imaging of Biological Assemblies

Electron microscopy of soft and biological materials, or "soft electron microscopy", is essential to the characterization of macromolecules. Soft microscopy is governed by enhancing contrast while maintaining low electron doses, and sample preparation and imaging methodologies are driven by the length scale of features of interest. While cryo-electron microscopy offers the highest resolution, larger structures can be characterized efficiently and with high contrast using low-voltage electron microscopy by performing scanning transmission electron microscopy in a scanning electron microscope (STEM-in-SEM). Here, STEM-in-SEM is demonstrated for a four-lobed protein assembly where the arrangement of the proteins in the construct must be examined. STEM image simulations show the theoretical contrast enhancement at SEM-level voltages for unstained structures, and experimental images with multiple STEM modes exhibit the resolution possible for negative-stained proteins. This technique can be extended to complex protein assemblies, larger structures such as cell sections, and hybrid materials, making STEM-in-SEM a valuable high-throughput imaging method.

Making the Most of your Electrons: Challenges and Opportunities in Characterizing Hybrid Interfaces with STEM

Inspired by the unique architectures composed of hard and soft materials in natural and biological systems, synthetic hybrid structures and associated soft-hard interfaces have recently evoked significant interest. Soft matter is typically dominated by fluctuations even at room temperature, while hard matter (which often serves as the substrate or anchor for the soft component) is governed by rigid mechanical behavior. This dichotomy offers considerable opportunities to leverage the disparate properties offered by these components across a wide spectrum spanning from basic science to engineering insights with significant technological overtones. Such hybrid structures, which include polymer nanocomposites, DNA functionalized nanoparticle superlattices and metal organic frameworks to name a few, have delivered promising insights into the areas of catalysis, environmental remediation, optoelectronics, medicine, and beyond. The interfacial structure between these hard and soft phases exists across a variety of length scales and often strongly influence the functionality of hybrid systems. While scanning/transmission electron microscopy (S/TEM) has proven to be a valuable tool for acquiring intricate molecular and nanoscale details of these interfaces, the unusual nature of hybrid composites presents a suite of challenges that make assessing or establishing the classical structure-property relationships especially difficult. These include challenges associated with preparing electron-transparent samples and obtaining sufficient contrast to resolve the interface between dissimilar materials given the dose sensitivity of soft materials. We discuss each of these challenges and supplement a review of recent developments in the field with additional experimental investigations and simulations to present solutions for attaining a nano or atomic-level understanding of these interfaces. These solutions present a host of opportunities for investigating and understanding the role interfaces play in this unique class of functional materials.

An adaptive microscope for the imaging of biological surfaces

Scanning fluorescence microscopes are now able to image large biological samples at high spatial and temporal resolution. This comes at the expense of an increased light dose which is detrimental to fluorophore stability and cell physiology. To highly reduce the light dose, we designed an adaptive scanning fluorescence microscope with a scanning scheme optimized for the unsupervised imaging of cell sheets, which underly the shape of many embryos and organs. The surface of the tissue is first delineated from the acquisition of a very small subset (~0.1%) of sample space, using a robust estimation strategy. Two alternative scanning strategies are then proposed to image the tissue with an improved photon budget, without loss in resolution. The first strategy consists in scanning only a thin shell around the estimated surface of interest, allowing high reduction of light dose when the tissue is curved. The second strategy applies when structures of interest lie at the cell periphery (e.g. adherens junctions). An iterative approach is then used to propagate scanning along cell contours. We demonstrate the benefit of our approach imaging live epithelia from Drosophila melanogaster. On the examples shown, both approaches yield more than a 20-fold reduction in light dose -and up to more than 80-fold- compared to a full scan of the volume. These smart-scanning strategies can be easily implemented on most scanning fluorescent imaging modality. The dramatic reduction in light exposure of the sample should allow prolonged imaging of the live processes under investigation.

Fast reconstruction of atomic-scale STEM-EELS images from sparse sampling

This paper discusses the reconstruction of partially sampled spectrum-images to accelerate the acquisition in scanning transmission electron microscopy (STEM). The problem of image reconstruction has been widely considered in the literature for many imaging modalities, but only a few attempts handled 3D data such as spectral images acquired by STEM electron energy loss spectroscopy (EELS). Besides, among the methods proposed in the microscopy literature, some are fast but inaccurate while others provide accurate reconstruction but at the price of a high computation burden. Thus none of the proposed reconstruction methods fulfills our expectations in terms of accuracy and computation complexity. In this paper, we propose a fast and accurate reconstruction method suited for atomic-scale EELS. This method is compared to popular solutions such as beta process factor analysis (BPFA) which is used for the first time on STEM-EELS images. Experiments based on real as synthetic data will be conducted.

A literature review of in situ transmission electron microscopy technique in corrosion studies

One of the biggest challenges in corrosion investigation is foreseeing precisely how and where materials will degenerate in a designated condition owing to scarceness of accurate corrosion mechanisms. Recent fast development of in situ transmission electron microscopy (TEM) technique makes it achievable to better understand the corrosion mechanism and physicochemical processes at the interfaces between samples and gases or electrolytes by dynamical capture the microstructural and chemical changes with high resolution within a realistic or near-realistic environment. However, a detailed and in-depth account summing up the development and latest achievements of in situ TEM techniques, especially the application of emerging liquid and electrochemical cells in the community of corrosion study in the last several years is lacking and is urgently needed for its heathy development. To fill this gap, this critical review summarizes firstly the key scientific issues in corrosion research, followed by introducing the configurations of several typical closed-type cells. Then, the achievements of in situ TEM using open-type or closed-type cells in corrosion study are presented in detail. The study directions in the future are commented finally in terms of spatial and temporal resolution, electron radiation, and linkage between microstructure and electrochemical performance in corrosion community.