Electron dose dependence of signal-to-noise ratio, atom contrast and resolution in transmission electron microscope images

Electron beam and thermal stabilities of MFM-300(M) metal-organic frameworks

This work reports the thermal and electron beam stabilities of a series of isostructural metal-organic frameworks (MOFs) of type MFM-300(M) (M = Al, Ga, In, Cr). MFM-300(Cr) was most stable under the electron beam, having an unusually high critical electron fluence of 1111 e- Å-2 while the Group 13 element MOFs were found to be less stable. Within Group 13, MFM-300(Al) had the highest critical electron fluence of 330 e- Å-2, compared to 189 e- Å-2 and 147 e- Å-2 for the Ga and In MOFs, respectively. For all four MOFs, electron beam-induced structural degradation was independent of crystal size and was highly anisotropic, although both the length and width of the channels decreased during electron beam irradiation. Notably, MFM-300(Cr) was found to retain crystallinity while shrinking up to 10%. Thermal stability was studied using in situ synchrotron X-ray diffraction at elevated temperature, which revealed critical temperatures for crystal degradation to be 605, 570, 490 and 480 °C for Al, Cr, Ga, and In, respectively. The pore channel diameters contracted by ≈0.5% on desorption of solvent species, but thermal degradation at higher temperatures was isotropic. The observed electron stabilities were found to scale with the relative inertness of the cations and correlate well to the measured lifetime of the materials when used as photocatalysts.

Interpretability of high-resolution transmission electron microscopy images

High-resolution electron microscopy is a well-suited tool for characterizing the nanoscale structure of materials. However, the interaction of the sample and the high-energy electrons of the beam can often have a detrimental impact on the sample structure. This effect can only be alleviated by decreasing the number of electrons to which the sample is exposed but will come at the cost of a decreased signal-to-noise ratio in the resulting image. Images with low signal to noise ratios are often challenging to interpret as parts of the sample with a low interaction with the electron beam are reproduced with very low contrast. Here we suggest simple measures as alternatives to the conventional signal-to-noise ratio and investigate how these can be used to predict the interpretability of the electron microscopy images. We test the models on a sample consisting of gold nanoparticles supported on a cerium dioxide substrate. The models are evaluated based on series of images acquired at varying electron dose.

Zero-shot denoising of microscopy images recorded at high-resolution limits

Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation.

An Efficient Electron Ptychography Method for Retrieving the Object Spectrum from Only a Few Iterations

We present an efficient approach for electron ptychography based on a mathematical relationship that differs from that underlying the established algorithms of the ptychography iterative engine or the noniterative algorithms like the Wigner-distribution-deconvolution or the single-side-band method. Three variables are handled in this method-the transfer function of the objective lens, the object spectrum, and the diffraction wave whose phase is unknown. In the case of an aberration-corrected electron microscope, one is able to obtain a well-estimated transfer function of the lens. After reducing the number of three variables down to two, we construct an iterative loop between the object spectrum and the diffraction wave, which retrieves the object spectrum within a small number of iterations. We tested this object spectrum retrieval method on both a calculated and an experimental 4D-STEM datasets. By applying this method, we explore the influence of sampling, dose, and the size of illumination aperture on the reconstructed phase images.

Quantifying noise limitations of neural network segmentations in high-resolution transmission electron microscopy

Motivated by the need for low electron dose transmission electron microscopy imaging, we report the optimal frame dose (i.e.e-/Å2) range for object detection and segmentation tasks with neural networks. The MSD-net architecture shows promising abilities over the industry standard U-net architecture in generalising to frame doses below the range included in the training set, for both simulated and experimental images. It also presents a heightened ability to learn from lower dose images. The MSD-net displays mild visibility of a Au nanoparticle at 20-30 e-/Å2, and converges at 200 e-/Å2 where a full segmentation of the nanoparticle is achieved. Between 30 and 200 e-/Å2 object detection applications are still possible. This work also highlights the importance of modelling the modulation transfer function when training with simulated images for applications on images acquired with scintillator based detectors such as the Gatan Oneview camera. A parametric form of the modulation transfer function is applied with varying ranges of parameters, and the effects on low electron dose segmentation is presented.

Extracellular Vesicles: Techniques and Biomedical Applications Related to Single Vesicle Analysis

Extracellular vesicles (EVs) are extensively dispersed lipid bilayer membrane vesicles involved in the delivery and transportation of molecular payloads to certain cell types to facilitate intercellular interactions. Their significant roles in physiological and pathological processes make EVs outstanding biomarkers for disease diagnosis and treatment monitoring as well as ideal candidates for drug delivery. Nevertheless, differences in the biogenesis processes among EV subpopulations have led to a diversity of biophysical characteristics and molecular cargos. Additionally, the prevalent heterogeneity of EVs has been found to substantially hamper the sensitivity and accuracy of disease diagnosis and therapeutic monitoring, thus impeding the advancement of clinical applications. In recent years, the evolution of single EV (SEV) analysis has enabled an in-depth comprehension of the physical properties, molecular composition, and biological roles of EVs at the individual vesicle level. This review examines the sample acquisition tactics prior to SEV analysis, i.e., EV isolation techniques, and outlines the current state-of-the-art label-free and label-based technologies for SEV identification. Furthermore, the challenges and prospects of biomedical applications based on SEV analysis are systematically discussed.

Direct Observation of Transient Structural Dynamics of Atomically Thin Halide Perovskite Nanowires

Halide perovskite is a unique dynamical system, whose structural and chemical processes happening across different timescales have significant impact on its physical properties and device-level performance. However, due to its intrinsic instability, real-time investigation of the structure dynamics of halide perovskite is challenging, which hinders the systematic understanding of the chemical processes in the synthesis, phase transition, and degradation of halide perovskite. Here, we show that atomically thin carbon materials can stabilize ultrathin halide perovskite nanostructures against otherwise detrimental conditions. Moreover, the protective carbon shells enable atomic-level visualization of the vibrational, rotational, and translational movement of halide perovskite unit cells. Albeit atomically thin, protected halide perovskite nanostructures can maintain their structural integrity up to an electron dose rate of 10,000 e^-/Ų·s while exhibiting unusual dynamical behaviors pertaining to the lattice anharmonicity and nanoscale confinement. Our work demonstrates an effective method to protect beam-sensitive materials during in situ observation, unlocking new solutions to study new modes of structure dynamics of nanomaterials.

Learning-based defect recognition for quasi-periodic HRSTEM images

Controlling crystalline material defects is crucial, as they affect properties of the material that may be detrimental or beneficial for the final performance of a device. Defect analysis on the sub-nanometer scale is enabled by high-resolution scanning transmission electron microscopy (HRSTEM), where the identification of defects is currently carried out based on human expertise. However, the process is tedious, highly time consuming and, in some cases, yields ambiguous results. Here we propose a semi-supervised machine learning method that assists in the detection of lattice defects from atomic resolution HRSTEM images. It involves a convolutional neural network that classifies image patches as defective or non-defective, a graph-based heuristic that chooses one non-defective patch as a model, and finally an automatically generated convolutional filter bank, which highlights symmetry breaking such as stacking faults, twin defects and grain boundaries. Additionally, we suggest a variance filter to segment amorphous regions and beam defects. The algorithm is tested on III-V/Si crystalline materials and successfully evaluated against different metrics and a baseline approach, showing promising results even for extremely small training data sets and for noise compromised images. By combining the data-driven classification generality, robustness and speed of deep learning with the effectiveness of image filters in segmenting faulty symmetry arrangements, we provide a valuable open-source tool to the microscopist community that can streamline future HRSTEM analyses of crystalline materials.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Ultra-high contrast STEM imaging for segmented/pixelated detectors by maximizing the signal-to-noise ratio

Atomic-resolution low-dose imaging for beam-sensitive materials is one of the most challenging topics in electron microscopy research. In this study, we theoretically developed a new scanning transmission electron microscopy (STEM) imaging technique by maximizing the signal-to-noise ratio of an obtainable image under the weak phase object approximation (WPOA), which we will call optimum bright-field (OBF) imaging. OBF images are obtained by processing multiple images acquired by segmented/pixelated detectors through complex frequency filtering. This method has been confirmed through a systematic image simulation to be highly dose-efficient. Furthermore, we experimentally demonstrate the high dose efficiency of the OBF technique by visualizing the atomic structure in a lithium-ion battery material using a high-speed segmented detector. Furthermore, it was shown that OBF imaging is usable for real-time imaging, which makes low-dose observations of beam-sensitive materials much easier to achieve.

Near-atomic-scale observation of grain boundaries in a layer-stacked two-dimensional polymer

Two-dimensional (2D) polymers hold great promise in the rational materials design tailored for next-generation applications. However, little is known about the grain boundaries in 2D polymers, not to mention their formation mechanisms and potential influences on the material's functionalities. Using aberration-corrected high-resolution transmission electron microscopy, we present a direct observation of the grain boundaries in a layer-stacked 2D polyimine with a resolution of 2.3 Å, shedding light on their formation mechanisms. We found that the polyimine growth followed a "birth-and-spread" mechanism. Antiphase boundaries implemented a self-correction to the missing-linker and missing-node defects, and tilt boundaries were formed via grain coalescence. Notably, we identified grain boundary reconstructions featuring closed rings at tilt boundaries. Quantum mechanical calculations revealed that boundary reconstruction is energetically allowed and can be generalized into different 2D polymer systems. We envisage that these results may open up the opportunity for future investigations on defect-property correlations in 2D polymers.

Quantification and optimization of ADF-STEM image contrast for beam-sensitive materials

Many functional materials are difficult to analyse by scanning transmission electron microscopy (STEM) on account of their beam sensitivity and low contrast between different phases. The problem becomes even more severe when thick specimens need to be investigated, a situation that is common for materials that are ordered from the nanometre to micrometre length scales or when performing dynamic experiments in a TEM liquid cell. Here we report a method to optimize annular dark-field (ADF) STEM imaging conditions and detector geometries for a thick and beam-sensitive low-contrast specimen using the example of a carbon nanotube/polymer nanocomposite. We carried out Monte Carlo simulations as well as quantitative ADF-STEM imaging experiments to predict and verify optimum contrast conditions. The presented method is general, can be easily adapted to other beam-sensitive and/or low-contrast materials, as shown for a polymer vesicle within a TEM liquid cell, and can act as an expert guide on whether an experiment is feasible and to determine the best imaging conditions.

Accuracy of surface strain measurements from transmission electron microscopy images of nanoparticles

Strain analysis from high-resolution transmission electron microscopy (HRTEM) images offers a convenient tool for measuring strain in materials at the atomic scale. In this paper we present a theoretical study of the precision and accuracy of surface strain measurements directly from aberration-corrected HRTEM images. We examine the influence of defocus, crystal tilt and noise, and find that absolute errors of at least 1–2% strain should be expected. The model structures include surface relaxations determined using molecular dynamics, and we show that this is important for correctly evaluating the errors introduced by image aberrations.

Atomic-Scale Analytical Tomography

The concept of atomic-scale tomography has been proposed in the past decade as a technique that could deliver the position of all atoms with high precision and their elemental (isotopic) identity. The technique was never intended to be limited to merely structural information and there is clearly a rich array of additional analytical information that can be brought to bear on such tomographs. In this paper, some of these types of information are considered and the implications are explored. The fuller realm of this analytical and structural information may be called atomic-scale analytical tomography.

Statistical analysis of support thickness and particle size effects in HRTEM imaging of metal nanoparticles

High-resolution transmission electron microscopy (HRTEM) examination of nanoparticles requires their placement on some manner of support - either TEM grid membranes or part of the material itself, as in many heterogeneous catalyst systems - but a systematic quantification of the practical imaging limits of this approach has been lacking. Here we address this issue through a statistical evaluation of how nanoparticle size and substrate thickness affects the ability to resolve structural features of interest in HRTEM images of metallic nanoparticles on common support membranes. The visibility of lattice fringes from crystalline Au nanoparticles on amorphous carbon and silicon supports of varying thickness was investigated with both conventional and aberration-corrected TEM. Over the 1-4nm nanoparticle size range examined, the probability of successfully resolving lattice fringes differed significantly as a function both of nanoparticle size and support thickness. Statistical analysis was used to formulate guidelines for the selection of supports and to quantify the impact a given support would have on HRTEM imaging of crystalline structure. For nanoparticles ≥1nm, aberration-correction was found to provide limited benefit for the purpose of visualizing lattice fringes; electron dose is more predictive of lattice fringe visibility than aberration correction. These results confirm that the ability to visualize lattice fringes is ultimately dependent on the signal-to-noise ratio of the HRTEM images, rather than the point-to-point resolving power of the microscope. This study provides a benchmark for HRTEM imaging of crystalline supported metal nanoparticles and is extensible to a wide variety of supports and nanostructures.

Aberration-Corrected Scanning Transmission Electron Microscope (STEM) Through-Focus Imaging for Three-Dimensional Atomic Analysis of Bismuth Segregation on Copper [001]/33° Twist Bicrystal Grain Boundaries

Scanning transmission electron microscope (STEM) through-focus imaging (TFI) has been used to determine the three-dimensional atomic structure of Bi segregation-induced brittle Cu grain boundaries (GBs). With TFI, it is possible to observe single Bi atom distributions along Cu [001] twist GBs using an aberration-corrected STEM operating at 200 kV. The depth resolution is ~5 nm. Specimens with GBs intentionally inclined with respect to the microscope's optic axis were used to investigate Bi segregant atom distributions along and through the Cu GB. It was found that Bi atoms exist at most once per Cu unit cell along the GB, meaning that no continuous GB film is present. Therefore, the reduced fracture toughness of this particular Bi-doped Cu boundary would not be caused by fracture of Bi-Bi bonds.

Design and implementation of a micron-sized electron column fabricated by focused ion beam milling

We have designed, fabricated and tested a micron-sized electron column with an overall length of about 700 microns comprising two electron lenses; a micro-lens with a minimal bore of 1 micron followed by a second lens with a bore of up to 50 microns in diameter to shape a coherent low-energy electron wave front. The design criteria follow the notion of scaling down source size, lens-dimensions and kinetic electron energy for minimizing spherical aberrations to ensure a parallel coherent electron wave front. All lens apertures have been milled employing a focused ion beam and could thus be precisely aligned within a tolerance of about 300 nm from the optical axis. Experimentally, the final column shapes a quasi-planar wave front with a minimal full divergence angle of 4 mrad and electron energies as low as 100 eV.

In quest of perfection in electron optics: a biographical sketch of Harald Rose on the occasion of his 80th birthday

This brief biographical sketch of Harald Rose on occasion of his 80th birthday describes some of the key events in an extraordinarily successful scientific life. Many of the theoretical concepts developed by him over the last 50 years have been fundamental for electron optics. Indeed, some of them have changed the whole complexion of this field and are fundamental to modern electron microscopy, both in TEM and in STEM mode. With this dedicated issue of Ultramicroscopy, the members of the electron microscopy community would like to thank Harald Rose for dedicating his professional life to their field and thereby enriching the life of those active in it.

Non-invasive transmission electron microscopy of vacancy defects in graphene produced by ion irradiation

Irradiation with high-energy ions has been widely suggested as a tool to engineer properties of graphene. Experiments show that it indeed has a strong effect on graphene's transport, magnetic and mechanical characteristics. However, to use ion irradiation as an engineering tool requires understanding of the type and detailed characteristics of the produced defects which is still lacking, as the use of high-resolution transmission microscopy (HRTEM)--the only technique allowing direct imaging of atomic-scale defects--often modifies or even creates defects during imaging, thus making it impossible to determine the intrinsic atomic structure. Here we show that encapsulating the studied graphene sample between two other (protective) graphene sheets allows non-invasive HRTEM imaging and reliable identification of atomic-scale defects. Using this simple technique, we demonstrate that proton irradiation of graphene produces reconstructed monovacancies, which explains the profound effect that such defects have on graphene's magnetic and transport properties. This finding resolves the existing uncertainty with regard to the effect of ion irradiation on the electronic structure of graphene.