Deep data analysis via physically constrained linear unmixing: universal framework, domain examples, and a community-wide platform

Non-negative matrix factorization-aided phase unmixing and trace element quantification of STEM-EDXS data

Energy-dispersive X-ray spectroscopy (EDXS) mapping with a scanning transmission electron microscope (STEM) is commonly used for chemical characterization of materials. However, STEM-EDXS quantification becomes challenging when the phases constituting the sample under investigation share common elements and overlap spatially. In this paper, we present a methodology to identify, segment, and unmix phases with a substantial spectral and spatial overlap in a semi-automated fashion through combining non-negative matrix factorization with a priori knowledge of the sample. We illustrate the methodology using a sample taken from an electron beam-sensitive mineral assemblage representing Earth's deep mantle. With it, we retrieve the true EDX spectra of the constituent phases and their corresponding phase abundance maps. It further enables us to achieve a reliable quantification for trace elements having concentration levels of ∼100 ppm. Our approach can be adapted to aid the analysis of many materials systems that produce STEM-EDXS datasets having phase overlap and/or limited signal-to-noise ratio (SNR) in spatially-integrated spectra.

Deep learning analysis on transmission electron microscope imaging of atomic defects in two-dimensional materials

Defects are prevalent in two-dimensional (2D) materials due to thermal equilibrium and processing kinetics. The presence of various defect types can affect material properties significantly. With the development of the advanced transmission electron microscopy (TEM), the property-related structures could be investigated in multiple dimensions. It produces TEM datasets containing a large amount of information. Traditional data analysis is influenced by the subjectivity of researchers, and manual analysis is inefficient and imprecise. Recent developments in deep learning provide robust methods for the quantitative identification of defects in 2D materials efficiently and precisely. Taking advantage of big data, it breaks the limitations of TEM as a local characterization tool, making TEM an intelligent macroscopic analysis method. In this review, the recent developments in the TEM data analysis of defects in 2D materials using deep learning technology are summarized. Initially, an in-depth examination of the distinctions between TEM and natural images is presented. Subsequently, a comprehensive exploration of TEM data analysis ensues, encompassing denoising, point defects, line defects, planar defects, quantitative analysis, and applications. Furthermore, an exhaustive assessment of the significant obstacles encountered in the accurate identification of distinct structures is also provided.

Armoring of MgO by a Passivation Layer Impedes Direct Air Capture of CO2

It has been proposed to use magnesium oxide (MgO) to separate carbon dioxide directly from the atmosphere at the gigaton level. We show experimental results on MgO single crystals reacting with the atmosphere for longer (decades) and shorter (days to months) periods with the goal of gauging reaction rates. Here, we find a substantial slowdown of an initially fast reaction as a result of mineral armoring by reaction products (surface passivation). In short-term experiments, we observe fast hydroxylation, carbonation, and formation of amorphous hydrated magnesium carbonate at early stages, leading to the formation of crystalline hydrated Mg carbonates. The preferential location of Mg carbonates along the atomic steps on the crystal surface of MgO indicates the importance of the reactive site density for carbonation kinetics. The analysis of 27-year-old single-crystal MgO samples demonstrates that the thickness of the reacted layer is limited to ∼1.5 μm on average, which is thinner than expected and indicates surface passivation. Thus, if MgO is to be employed for direct air capture of CO2, surface passivation must be circumvented.

Deep Learning-Assisted Multivariate Analysis for Nanoscale Characterization of Heterogeneous Beam-Sensitive Materials

Nanoscale materials characterization often uses highly energetic probes which can rapidly damage beam-sensitive materials, such as hybrid organic-inorganic compounds. Reducing the probe dose minimizes the damage, but often at the cost of lower signal-to-noise ratio (SNR) in the acquired data. This work reports the optimization and validation of principal component analysis (PCA) and nonnegative matrix factorization for the postprocessing of low-dose nanoscale characterization data. PCA is found to be the best approach for data denoising. However, the popular scree plot-based method for separation of principal and noise components results in inaccurate or excessively noisy models of the heterogeneous original data, even after Poissonian noise weighting. Manual separation of principal and noise components produces a denoised model which more accurately reproduces physical features present in the raw data while improving SNR by an order of magnitude. However, manual selection is time-consuming and potentially subjective. To suppress these disadvantages, a deep learning-based component classification method is proposed. The neural network model can examine PCA components and automatically classify them with an accuracy of >99% and a rate of ∼2 component/s. Together, multivariate analysis and deep learning enable a deeper analysis of nanoscale materials' characterization, allowing as much information as possible to be extracted.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

Why it is Unfortunate that Linear Machine Learning "Works" so well in Electromechanical Switching of Ferroelectric Thin Films

Machine learning (ML) is relied on for materials spectroscopy. It is challenging to make ML models fail because statistical correlations can mimic the physics without causality. Here, using a benchmark band-excitation piezoresponse force microscopy polarization spectroscopy (BEPS) dataset the pitfalls of the so-called "better", "faster", and "less-biased" ML of electromechanical switching are demonstrated and overcome. Using a toy and real experimental dataset, it is demonstrated how linear nontemporal ML methods result in physically reasonable embedding (eigenvalues) while producing nonsensical eigenvectors and generated spectra, promoting misleading interpretations. A new method of unsupervised multimodal hyperspectral analysis of BEPS is demonstrated using long-short-term memory (LSTM) β-variational autoencoders (β-VAEs) . By including LSTM neurons, the ordinal nature of ferroelectric switching is considered. To improve the interpretability of the latent space, a variational Kullback-Leibler-divergency regularization is imposed . Finally, regularization scheduling of β as a disentanglement metric is leveraged to reduce user bias. Combining these experiment-inspired modifications enables the automated detection of ferroelectric switching mechanisms, including a complex two-step, three-state one. Ultimately, this work provides a robust ML method for the rapid discovery of electromechanical switching mechanisms in ferroelectrics and is applicable to other multimodal hyperspectral materials spectroscopies.

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. The discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.

Automated Hyperspectral 2D/3D Raman Analysis Using the Learner-Predictor Strategy: Machine Learning-Based Inline Raman Data Analytics

Synchronously detecting multiple Raman spectral signatures in two-dimensional/three-dimensional (2D/3D) hyperspectral Raman analysis is a daunting challenge. The underlying reasons notwithstanding the enormous volume of the data and also the complexities involved in the end-to-end Raman analytics pipeline: baseline removal, cosmic noise elimination, and extraction of trusted spectral signatures and abundance maps. Elimination of cosmic noise is the bottleneck in the entire Raman analytics pipeline. Unless this issue is addressed, the realization of autonomous Raman analytics is impractical. Here, we present a learner-predictor strategy-based "automated hyperspectral Raman analysis framework" to rapidly fingerprint the molecular variations in the hyperspectral 2D/3D Raman dataset. We introduce the spectrum angle mapper (SAM) technique to eradicate the cosmic noise from the hyperspectral Raman dataset. The learner-predictor strategy eludes the necessity of human inference, and analytics can be done in autonomous mode. The learner owns the ability to learn; it automatically eliminates the baseline and cosmic noise from the Raman dataset, extracts the predominant spectral signatures, and renders the respective abundance maps. In a nutshell, the learner precisely learned the spectral features space during the hyperspectral Raman analysis. Afterward, the learned spectral features space was translated into a neural network (LNN) model. In the predictor, machine-learned intelligence (LNN) is utilized to predict the alternate batch specimen's abundance maps in real time. The qualitative/quantitative evaluation of abundance maps implicitly lays the foundation for monitoring the offline/inline industrial qualitative/quantitative quality control (QA/QC) process. The present strategy is best suited for 2D/3D/four-dimensional (4D) hyperspectral Raman spectroscopic techniques. The proposed ML framework is intuitive because it obviates human intelligence, sophisticated computational hardware, and solely a personal computer is enough for the end-to-end pipeline.

Propagation of priors for more accurate and efficient spectroscopic functional fits and their application to ferroelectric hysteresis

Multi-dimensional spectral-imaging is a mainstay of the scanning probe and electron microscopies, micro-Raman, and various forms of chemical imaging. In many cases, individual spectra can be fit to a specific functional form, with the model parameter maps, providing direct insight into material properties. Since spectra are often acquired across a spatial grid of points, spatially adjacent spectra are likely to be similar to one another; yet, this fact is almost never used when considering parameter estimation for functional fits. On datasets tried here, we show that by utilizing proximal information, whether it be in the spatial or spectral domains, it is possible to improve the reliability and increase the speed of such functional fits by ∼2–3×, as compared to random priors. We explore and compare three distinct new methods: (a) spatially averaging neighborhood spectra, and propagating priors based on functional fits to the averaged case, (b) hierarchical clustering-based methods where spectra are grouped hierarchically based on response, with the priors propagated progressively down the hierarchy, and (c) regular clustering without hierarchical methods with priors propagated from fits to cluster means. Our results highlight that utilizing spatial and spectral neighborhood information is often critical for accurate parameter estimation in noisy environments, which we show for ferroelectric hysteresis loops acquired on a prototypical PbTiO3 thin film with piezoresponse spectroscopy. This method is general and applicable to any spatially measured spectra where functional forms are available. Examples include exploring the superconducting gap with tunneling spectroscopy, using the Dynes formula, or current–voltage curve fits in conductive atomic force microscopy mapping. Here we explore the problem for ferroelectric hysteresis, which, given its large parameter space, constitutes a more difficult task than, for example, fitting current–voltage curves with a Schottky emission formula (Chiu 2014 Adv. Mater. Sci. Eng. 2014 578168).

Fast Grain Mapping with Sub-Nanometer Resolution Using 4D-STEM with Grain Classification by Principal Component Analysis and Non-Negative Matrix Factorization

High-throughput grain mapping with sub-nanometer spatial resolution is demonstrated using scanning nanobeam electron diffraction (also known as 4D scanning transmission electron microscopy, or 4D-STEM) combined with high-speed direct-electron detection. An electron probe size down to 0.5 nm in diameter is used and the sample investigated is a gold–palladium nanoparticle catalyst. Computational analysis of the 4D-STEM data sets is performed using a disk registration algorithm to identify the diffraction peaks followed by feature learning to map the individual grains. Two unsupervised feature learning techniques are compared: principal component analysis (PCA) and non-negative matrix factorization (NNMF). The characteristics of the PCA versus NNMF output are compared and the potential of the 4D-STEM approach for statistical analysis of grain orientations at high spatial resolution is discussed.

Revealing the Chemical Bonding in Adatom Arrays via Machine Learning of Hyperspectral Scanning Tunneling Spectroscopy Data

The adatom arrays on surfaces offer an ideal playground to explore the mechanisms of chemical bonding via changes in the local electronic tunneling spectra. While this information is readily available in hyperspectral scanning tunneling spectroscopy data, its analysis has been considerably impeded by a lack of suitable analytical tools. Here we develop a machine learning based workflow combining supervised feature identification in the spatial domain and unsupervised clustering in the energy domain to reveal the details of structure-dependent changes of the electronic structure in adatom arrays on the Co₃Sn₂S₂ cleaved surface. This approach, in combination with first-principles calculations, provides insight for using artificial neural networks to detect adatoms and classifies each based on their local neighborhood comprised of other adatoms. These structurally classified adatoms are further spectrally deconvolved. The unexpected inhomogeneity of electronic structures among adatoms in similar configurations is unveiled using this method, suggesting there is not a single atomic species of adatoms, but rather multiple types of adatoms on the Co₃Sn₂S₂ surface. This is further supported by a slight contrast difference in the images (or slight size variation) of the topography of the adatoms.

Role of Decomposition Product Ions in Hysteretic Behavior of Metal Halide Perovskite

Ion migration is one of the most debated mechanisms and credited with multiple observed phenomena and performance in metal halide perovskites (MHPs) semiconductor devices. However, to date, the migration of ions and their effects on MHPs are not still fully understood, largely due to a lack of direct observations of temporal ion migration. In this work, using direct observation of ion migration in-operando, we observe the hysteretic migration behavior of intrinsic ions (i.e., CH₃NH₃⁺ and I^-) as well as reveal the migration behavior of CH₃NH₃⁺ decomposition ions. We find that CH₃NH₃⁺ decomposition products can be affected by light and accumulate at the interfaces under bias. These MHP decomposition products are tightly related to the device performance and stability. Complementary results of time-resolved Kelvin probe force microscopy (tr-KPFM) demonstrate a correlation between dynamics of these interfacial ions and charge carriers. Overall, we find that there are a number of mobile ions including CH₃NH₃⁺ decomposition products in MHPs that need to be taken into account when measuring MHP device responses (e.g., charge dynamics) and should be considered in future optimization studies of MHP semiconductor devices.

Machine Learning Method Reveals Hidden Strong Metal-Support Interaction in Microscopy Datasets

Forming an ultra-thin, permeable encapsulation oxide-support layer on a metal catalyst surface is considered an effective strategy for achieving a balance between high stability and high activity in heterogenous catalysts. The success of such a design relies not only on the thickness, ideally one to two atomic layers thick, but also on the morphology and chemistry of the encapsulation layer. Reliably identifying the presence and chemical nature of such a trace layer has been challenging. Electron energy-loss spectroscopy (EELS) performed in a scanning transmission electron microscope (STEM), the primary technique utilized for such studies, is limited by a weak signal on overlayers when using conventional analysis methods, often leading to misinterpreted or missed information. Here, a robust, unsupervised machine learning data analysis method is developed to reveal trace encapsulation layers that are otherwise overlooked in STEM-EELS datasets. This method provides a reliable tool for analyzing encapsulation of catalysts and is generally applicable to any spectroscopic analysis of materials and devices where revealing a trace signal and its spatial distribution is challenging.

Directly Probing the Local Coordination, Charge State, and Stability of Single Atom Catalysts by Advanced Electron Microscopy: A Review

The drive for atom efficient catalysts with carefully controlled properties has motivated the development of single atom catalysts (SACs), aided by a variety of synthetic methods, characterization techniques, and computational modeling. The distinct capabilities of SACs for oxidation, hydrogenation, and electrocatalytic reactions have led to the optimization of activity and selectivity through composition variation. However, characterization methods such as infrared and X-ray spectroscopy are incapable of direct observations at atomic scale. Advances in transmission electron microscopy (TEM) including aberration correction, monochromators, environmental TEM, and micro-electro-mechanical system based in situ holders have improved catalysis study, allowing researchers to peer into regimes previously unavailable, observing critical structural and chemical information at atomic scale. This review presents recent development and applications of TEM techniques to garner information about the location, bonding characteristics, homogeneity, and stability of SACs. Aberration corrected TEM imaging routinely achieves sub-Ångstrom resolution to reveal the atomic structure of materials. TEM spectroscopy provides complementary information about local composition, chemical bonding, electronic properties, and atomic/molecular vibration with superior spatial resolution. In situ/operando TEM directly observe the evolution of SACs under reaction conditions. This review concludes with remarks on the challenges and opportunities for further development of TEM to study SACs.

Toward Decoding the Relationship between Domain Structure and Functionality in Ferroelectrics via Hidden Latent Variables

Polarization switching mechanisms in ferroelectric materials are fundamentally linked to local domain structure and the presence of the structural defects, which both can act as nucleation and pinning centers and create local electrostatic and mechanical depolarization fields affecting wall dynamics. However, the general correlative mechanisms between domain structure and polarization dynamics are only weakly explored, precluding insight into the associated physical mechanisms. Here, the correlation between local domain structures and switching behavior in ferroelectric materials is explored using convolutional encoder-decoder networks, enabling image to spectral (im2spec) and spectral to image (spec2im) translations via encoding of latent variables. The latter reflect the assumption that the relationship between domain structure and polarization switching is parsimonious, i.e., is based upon a small number of local mechanisms. The analysis of latent variables distributions and their real-space representations provides insight into the predictability of the local switching behavior and hence associated physical mechanisms. We further pose that the regions where these correlative relationships are violated, i.e., predictability of the polarization dynamics from domain structure is reduced, represent the obvious target for detailed studies, e.g., in the context of automated experiments. This approach provides a workflow to establish the presence of correlation between local spectral responses and local structure and can be universally applied to spectral imaging techniques such as piezoresponse force microscopy (PFM), scanning tunneling microscopy (STM) and spectroscopy, and electron energy loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM).

Hysteretic Ion Migration and Remanent Field in Metal Halide Perovskites

The gap in understanding how underlying chemical dynamics impact the functionality of metal halide perovskites (MHPs) leads to the controversy about the origin of many phenomena associated with ion migration in MHPs. In particular, the debate regarding the impact of ion migration on current-voltage (I-V) hysteresis of MHPs devices has lasted for many years, where the difficulty lies in directly uncovering the chemical dynamics, as well as identifying and separating the impact of specific ions. In this work, using a newly developed time-resolved time-of-flight secondary ion mass spectrometry CH3NH3 + and I- migrations in CH3NH3PbI3 are directly observed, revealing hysteretic CH3NH3 + and I- migrations. Additionally, hysteretic CH3NH3 + migration is illumination-dependent. Correlating these results with the I-V characterization, this work uncovers that CH3NH3 + redistribution can induce a remanent field leading to a spontaneous current in the device. It unveils that the CH3NH3 + migration is responsible for the illumination-associated I-V hysteresis in MHPs. Hysteretic ion migration has not been uncovered and the contribution of any ions (e.g., CH3NH3 +) has not been specified before. Such insightful and detailed information has up to now been missing, which is critical to improving MHPs photovoltaic performance and developing MHPs-based memristors and synaptic devices.

Data fusion by joint non-negative matrix factorization for hypothesizing pseudo-chemistry using Bayesian networks

We infer reaction networks and chemistry using data fusion of spectroscopic sensors.

Unsupervised machine learning applied to scanning precession electron diffraction data

Scanning precession electron diffraction involves the acquisition of a two-dimensional precession electron diffraction pattern at every probe position in a two-dimensional scan. The data typically comprise many more diffraction patterns than the number of distinct microstructural volume elements (e.g. crystals) in the region sampled. A dimensionality reduction, ideally to one representative diffraction pattern per distinct element, may then be sought. Further, some diffraction patterns will contain contributions from multiple crystals sampled along the beam path, which may be unmixed by harnessing this oversampling. Here, we report on the application of unsupervised machine learning methods to achieve both dimensionality reduction and signal unmixing. Potential artefacts are discussed and precession electron diffraction is demonstrated to improve results by reducing the impact of bending and dynamical diffraction so that the data better approximate the case in which each crystal yields a given diffraction pattern.