Suppressing Electron Exposure Artifacts: An Electron Scanning Paradigm with Bayesian Machine Learning

High-throughput determination of grain size distributions by EBSD with low-discrepancy sampling

Because microstructure plays an important role in the mechanical properties of structural materials, developing the capability to quantify microstructures rapidly is important to enabling high-throughput screening of structural materials. Electron backscatter diffraction (EBSD) is a common method for studying microstructures and extracting information such as grain size distributions (GSDs), but is not particularly fast and thus could be a bottleneck in high-throughput systems. One approach to accelerating EBSD is to reduce the number of points that must be scanned. In this work, we describe an iterative method for reducing the number of scan points needed to measure GSDs using incremental low-discrepancy sampling, including on-the-fly grain size calculations and a convergence test for the resulting GSD based on the Kolmogorov-Smirnov test. We demonstrate this method on five real EBSD maps collected from magnesium AZ31B specimens and compare the effectiveness of sampling according to two different low discrepancy sequences, the Sobol and R2 sequences, and random sampling. We find that R2 sampling is able to produce GSDs that are statistically very similar to the GSDs of the full density grids using, on average, only 52% of the total scan points. For EBSD maps that contained monodisperse GSDs and over 1000 grains, R2 sampling only required an average of 39% of the total EBSD points.

Deep Learning Approach for Dynamic Sampling for Multichannel Mass Spectrometry Imaging

Mass Spectrometry Imaging (MSI), using traditional rectilinear scanning, takes hours to days for high spatial resolution acquisitions. Given that most pixels within a sample's field of view are often neither relevant to underlying biological structures nor chemically informative, MSI presents as a prime candidate for integration with sparse and dynamic sampling algorithms. During a scan, stochastic models determine which locations probabilistically contain information critical to the generation of low-error reconstructions. Decreasing the number of required physical measurements thereby minimizes overall acquisition times. A Deep Learning Approach for Dynamic Sampling (DLADS), utilizing a Convolutional Neural Network (CNN) and encapsulating molecular mass intensity distributions within a third dimension, demonstrates a simulated 70% throughput improvement for Nanospray Desorption Electrospray Ionization (nano-DESI) MSI tissues. Evaluations are conducted between DLADS, a Supervised Learning Approach for Dynamic Sampling, with Least-Squares regression (SLADS-LS), and a Multi-Layer Perceptron (MLP) network (SLADS-Net). When compared with SLADS-LS, limited to a single m / z channel, as well as multichannel SLADS-LS and SLADS-Net, DLADS respectively improves regression performance by 36.7%, 7.0%, and 6.2%, resulting in gains to reconstruction quality of 6.0%, 2.1%, and 3.4% for acquisition of targeted m / z .

Adaptive partial scanning transmission electron microscopy with reinforcement learning

Compressed sensing can decrease scanning transmission electron microscopy electron dose and scan time with minimal information loss. Traditionally, sparse scans used in compressed sensing sample a static set of probing locations. However, dynamic scans that adapt to specimens are expected to be able to match or surpass the performance of static scans as static scans are a subset of possible dynamic scans. Thus, we present a prototype for a contiguous sparse scan system that piecewise adapts scan paths to specimens as they are scanned. Sampling directions for scan segments are chosen by a recurrent neural network (RNN) based on previously observed scan segments. The RNN is trained by reinforcement learning to cooperate with a feedforward convolutional neural network that completes the sparse scans. This paper presents our learning policy, experiments, and example partial scans, and discusses future research directions. Source code, pretrained models, and training data is openly accessible at https://github.com/Jeffrey-Ede/adaptive-scans.

Two-Dimensional Polymers and Polymerizations

Synthetic chemists have developed robust methods to synthesize discrete molecules, linear and branched polymers, and disordered cross-linked networks. However, two-dimensional polymers (2DPs) prepared from designed monomers have been long missing from these capabilities, both as objects of chemical synthesis and in nature. Recently, new polymerization strategies and characterization methods have enabled the unambiguous realization of covalently linked macromolecular sheets. Here we review 2DPs and 2D polymerization methods. Three predominant 2D polymerization strategies have emerged to date, which produce 2DPs either as monolayers or multilayer assemblies. We discuss the fundamental understanding and scope of each of these approaches, including: the bond-forming reactions used, the synthetic diversity of 2DPs prepared, their multilayer stacking behaviors, nanoscale and mesoscale structures, and macroscale morphologies. Additionally, we describe the analytical tools currently available to characterize 2DPs in their various isolated forms. Finally, we review emergent 2DP properties and the potential applications of planar macromolecules. Throughout, we highlight achievements in 2D polymerization and identify opportunities for continued study.

Deep Learning Approach for Dynamic Sparse Sampling for High-Throughput Mass Spectrometry Imaging

A Supervised Learning Approach for Dynamic Sampling (SLADS) addresses traditional issues with the incorporation of stochastic processes into a compressed sensing method. Statistical features, extracted from a sample reconstruction, estimate entropy reduction with regression models, in order to dynamically determine optimal sampling locations. This work introduces an enhanced SLADS method, in the form of a Deep Learning Approach for Dynamic Sampling (DLADS), showing reductions in sample acquisition times for high-fidelity reconstructions between ~ 70-80% over traditional rectilinear scanning. These improvements are demonstrated for dimensionally asymmetric, high-resolution molecular images of mouse uterine and kidney tissues, as obtained using Nanospray Desorption ElectroSpray Ionization (nano-DESI) Mass Spectrometry Imaging (MSI). The methodology for training set creation is adjusted to mitigate stretching artifacts generated when using prior SLADS approaches. Transitioning to DLADS removes the need for feature extraction, further advanced with the employment of convolutional layers to leverage inter-pixel spatial relationships. Additionally, DLADS demonstrates effective generalization, despite dissimilar training and testing data. Overall, DLADS is shown to maximize potential experimental throughput for nano-DESI MSI.

Making the Most of your Electrons: Challenges and Opportunities in Characterizing Hybrid Interfaces with STEM

Inspired by the unique architectures composed of hard and soft materials in natural and biological systems, synthetic hybrid structures and associated soft-hard interfaces have recently evoked significant interest. Soft matter is typically dominated by fluctuations even at room temperature, while hard matter (which often serves as the substrate or anchor for the soft component) is governed by rigid mechanical behavior. This dichotomy offers considerable opportunities to leverage the disparate properties offered by these components across a wide spectrum spanning from basic science to engineering insights with significant technological overtones. Such hybrid structures, which include polymer nanocomposites, DNA functionalized nanoparticle superlattices and metal organic frameworks to name a few, have delivered promising insights into the areas of catalysis, environmental remediation, optoelectronics, medicine, and beyond. The interfacial structure between these hard and soft phases exists across a variety of length scales and often strongly influence the functionality of hybrid systems. While scanning/transmission electron microscopy (S/TEM) has proven to be a valuable tool for acquiring intricate molecular and nanoscale details of these interfaces, the unusual nature of hybrid composites presents a suite of challenges that make assessing or establishing the classical structure-property relationships especially difficult. These include challenges associated with preparing electron-transparent samples and obtaining sufficient contrast to resolve the interface between dissimilar materials given the dose sensitivity of soft materials. We discuss each of these challenges and supplement a review of recent developments in the field with additional experimental investigations and simulations to present solutions for attaining a nano or atomic-level understanding of these interfaces. These solutions present a host of opportunities for investigating and understanding the role interfaces play in this unique class of functional materials.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Partial Scanning Transmission Electron Microscopy with Deep Learning

Compressed sensing algorithms are used to decrease electron microscope scan time and electron beam exposure with minimal information loss. Following successful applications of deep learning to compressed sensing, we have developed a two-stage multiscale generative adversarial neural network to complete realistic 512 × 512 scanning transmission electron micrographs from spiral, jittered gridlike, and other partial scans. For spiral scans and mean squared error based pre-training, this enables electron beam coverage to be decreased by 17.9× with a 3.8% test set root mean squared intensity error, and by 87.0× with a 6.2% error. Our generator networks are trained on partial scans created from a new dataset of 16227 scanning transmission electron micrographs. High performance is achieved with adaptive learning rate clipping of loss spikes and an auxiliary trainer network. Our source code, new dataset, and pre-trained models are publicly available.