A new approach to the investigation of nanoparticles: electron tomography with compressed sensing

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

Making the Most of your Electrons: Challenges and Opportunities in Characterizing Hybrid Interfaces with STEM

Inspired by the unique architectures composed of hard and soft materials in natural and biological systems, synthetic hybrid structures and associated soft-hard interfaces have recently evoked significant interest. Soft matter is typically dominated by fluctuations even at room temperature, while hard matter (which often serves as the substrate or anchor for the soft component) is governed by rigid mechanical behavior. This dichotomy offers considerable opportunities to leverage the disparate properties offered by these components across a wide spectrum spanning from basic science to engineering insights with significant technological overtones. Such hybrid structures, which include polymer nanocomposites, DNA functionalized nanoparticle superlattices and metal organic frameworks to name a few, have delivered promising insights into the areas of catalysis, environmental remediation, optoelectronics, medicine, and beyond. The interfacial structure between these hard and soft phases exists across a variety of length scales and often strongly influence the functionality of hybrid systems. While scanning/transmission electron microscopy (S/TEM) has proven to be a valuable tool for acquiring intricate molecular and nanoscale details of these interfaces, the unusual nature of hybrid composites presents a suite of challenges that make assessing or establishing the classical structure-property relationships especially difficult. These include challenges associated with preparing electron-transparent samples and obtaining sufficient contrast to resolve the interface between dissimilar materials given the dose sensitivity of soft materials. We discuss each of these challenges and supplement a review of recent developments in the field with additional experimental investigations and simulations to present solutions for attaining a nano or atomic-level understanding of these interfaces. These solutions present a host of opportunities for investigating and understanding the role interfaces play in this unique class of functional materials.

Imaging Beam-Sensitive Materials by Electron Microscopy

Electron microscopy allows the extraction of multidimensional spatiotemporally correlated structural information of diverse materials down to atomic resolution, which is essential for figuring out their structure-property relationships. Unfortunately, the high-energy electrons that carry this important information can cause damage by modulating the structures of the materials. This has become a significant problem concerning the recent boost in materials science applications of a wide range of beam-sensitive materials, including metal-organic frameworks, covalent-organic frameworks, organic-inorganic hybrid materials, 2D materials, and zeolites. To this end, developing electron microscopy techniques that minimize the electron beam damage for the extraction of intrinsic structural information turns out to be a compelling but challenging need. This article provides a comprehensive review on the revolutionary strategies toward the electron microscopic imaging of beam-sensitive materials and associated materials science discoveries, based on the principles of electron-matter interaction and mechanisms of electron beam damage. Finally, perspectives and future trends in this field are put forward.

Removing Stripes, Scratches, and Curtaining with Nonrecoverable Compressed Sensing

Highly-directional image artifacts such as ion mill curtaining, mechanical scratches, or image striping from beam instability degrade the interpretability of micrographs. These unwanted, aperiodic features extend the image along a primary direction and occupy a small wedge of information in Fourier space. Deleting this wedge of data replaces stripes, scratches, or curtaining, with more complex streaking and blurring artifacts-known within the tomography community as "missing wedge" artifacts. Here, we overcome this problem by recovering the missing region using total variation minimization, which leverages image sparsity-based reconstruction techniques-colloquially referred to as compressed sensing (CS)-to reliably restore images corrupted by stripe-like features. Our approach removes beam instability, ion mill curtaining, mechanical scratches, or any stripe features and remains robust at low signal-to-noise. The success of this approach is achieved by exploiting CS's inability to recover directional structures that are highly localized and missing in Fourier Space.

Characterization techniques for nanoparticles: comparison and complementarity upon studying nanoparticle properties

Nanostructures have attracted huge interest as a rapidly growing class of materials for many applications. Several techniques have been used to characterize the size, crystal structure, elemental composition and a variety of other physical properties of nanoparticles. In several cases, there are physical properties that can be evaluated by more than one technique. Different strengths and limitations of each technique complicate the choice of the most suitable method, while often a combinatorial characterization approach is needed. In addition, given that the significance of nanoparticles in basic research and applications is constantly increasing, it is necessary that researchers from separate fields overcome the challenges in the reproducible and reliable characterization of nanomaterials, after their synthesis and further process (e.g. annealing) stages. The principal objective of this review is to summarize the present knowledge on the use, advances, advantages and weaknesses of a large number of experimental techniques that are available for the characterization of nanoparticles. Different characterization techniques are classified according to the concept/group of the technique used, the information they can provide, or the materials that they are destined for. We describe the main characteristics of the techniques and their operation principles and we give various examples of their use, presenting them in a comparative mode, when possible, in relation to the property studied in each case.

Electron tomography simulator with realistic 3D phantom for evaluation of acquisition, alignment and reconstruction methods

Because of the significance of electron microscope tomography in the investigation of biological structure at nanometer scales, ongoing improvement efforts have been continuous over recent years. This is particularly true in the case of software developments. Nevertheless, verification of improvements delivered by new algorithms and software remains difficult. Current analysis tools do not provide adaptable and consistent methods for quality assessment. This is particularly true with images of biological samples, due to image complexity, variability, low contrast and noise. We report an electron tomography (ET) simulator with accurate ray optics modeling of image formation that includes curvilinear trajectories through the sample, warping of the sample and noise. As a demonstration of the utility of our approach, we have concentrated on providing verification of the class of reconstruction methods applicable to wide field images of stained plastic-embedded samples. Accordingly, we have also constructed digital phantoms derived from serial block face scanning electron microscope images. These phantoms are also easily modified to include alignment features to test alignment algorithms. The combination of more realistic phantoms with more faithful simulations facilitates objective comparison of acquisition parameters, alignment and reconstruction algorithms and their range of applicability. With proper phantoms, this approach can also be modified to include more complex optical models, including distance-dependent blurring and phase contrast functions, such as may occur in cryotomography.

Biological application of Compressed Sensing Tomography in the Scanning Electron Microscope

The three-dimensional tomographic reconstruction of a biological sample, namely collagen fibrils in human dermal tissue, was obtained from a set of projection-images acquired in the Scanning Electron Microscope. A tailored strategy for the transmission imaging mode was implemented in the microscope and proved effective in acquiring the projections needed for the tomographic reconstruction. Suitable projection alignment and Compressed Sensing formulation were used to overcome the limitations arising from the experimental acquisition strategy and to improve the reconstruction of the sample. The undetermined problem of structure reconstruction from a set of projections, limited in number and angular range, was indeed supported by exploiting the sparsity of the object projected in the electron microscopy images. In particular, the proposed system was able to preserve the reconstruction accuracy even in presence of a significant reduction of experimental projections.

3D Visualization of the Iron Oxidation State in FeO/Fe3O4 Core-Shell Nanocubes from Electron Energy Loss Tomography

The physicochemical properties used in numerous advanced nanostructured devices are directly controlled by the oxidation states of their constituents. In this work we combine electron energy-loss spectroscopy, blind source separation, and computed tomography to reconstruct in three dimensions the distribution of Fe(2+) and Fe(3+) ions in a FeO/Fe3O4 core/shell cube-shaped nanoparticle with nanometric resolution. The results highlight the sharpness of the interface between both oxides and provide an average shell thickness, core volume, and average cube edge length measurements in agreement with the magnetic characterization of the sample.

Advanced electron crystallography through model-based imaging

The increasing need for precise determination of the atomic arrangement of non-periodic structures in materials design and the control of nanostructures explains the growing interest in quantitative transmission electron microscopy. The aim is to extract precise and accurate numbers for unknown structure parameters including atomic positions, chemical concentrations and atomic numbers. For this purpose, statistical parameter estimation theory has been shown to provide reliable results. In this theory, observations are considered purely as data planes, from which structure parameters have to be determined using a parametric model describing the images. As such, the positions of atom columns can be measured with a precision of the order of a few picometres, even though the resolution of the electron microscope is still one or two orders of magnitude larger. Moreover, small differences in average atomic number, which cannot be distinguished visually, can be quantified using high-angle annular dark-field scanning transmission electron microscopy images. In addition, this theory allows one to measure compositional changes at interfaces, to count atoms with single-atom sensitivity, and to reconstruct atomic structures in three dimensions. This feature article brings the reader up to date, summarizing the underlying theory and highlighting some of the recent applications of quantitative model-based transmisson electron microscopy.