Machine Learning to Reveal Nanoparticle Dynamics from Liquid-Phase TEM Videos

Imaging Locally Inhomogeneous Properties of Metal Halide Perovskites

Metal halide perovskites (MHPs) are a perfect example of state-of-the-art photovoltaic materials whose compositional and structural diversity, coupled with utilization of low-temperature processing, can undesirably result in spatially inhomogeneous properties that locally vary within the material. This complexity of MHPs requires sensitive imaging characterization methods at the microscopic level to gauge the impact of such inhomogeneities on device performance and to formulate mitigation strategies. This review consolidates properties of MHPs that are susceptible to local variations and highlights appropriate imaging techniques that can be employed to map them. Inhomogeneities in morphology, emission, electrical response, and chemical composition of MHP thin films are specifically considered, and possible microscopic techniques for their visualization are reviewed. For each type of microscopy, a short discussion about spatial resolution, sample requirements, advantages, and limitations is provided, thus leaving the reader with a guide of available imaging characterization tools to evaluate inhomogeneities of their MHPs.

When Metal Nanoclusters Meet Smart Synthesis

Atomically precise metal nanoclusters (MNCs) represent a fascinating class of ultrasmall nanoparticles with molecule-like properties, bridging conventional metal-ligand complexes and nanocrystals. Despite their potential for various applications, synthesis challenges such as a precise understanding of varied synthetic parameters and property-driven synthesis persist, hindering their full exploitation and wider application. Incorporating smart synthesis methodologies, including a closed-loop framework of automation, data interpretation, and feedback from AI, offers promising solutions to address these challenges. In this perspective, we summarize the closed-loop smart synthesis that has been demonstrated in various nanomaterials and explore the research frontiers of smart synthesis for MNCs. Moreover, the perspectives on the inherent challenges and opportunities of smart synthesis for MNCs are discussed, aiming to provide insights and directions for future advancements in this emerging field of AI for Science, while the integration of deep learning algorithms stands to substantially enrich research in smart synthesis by offering enhanced predictive capabilities, optimization strategies, and control mechanisms, thereby extending the potential of MNC synthesis.

Segmentation study of nanoparticle topological structures based on synthetic data

Nanoparticles exhibit broad applications in materials mechanics, medicine, energy and other fields. The ordered arrangement of nanoparticles is very important to fully understand their properties and functionalities. However, in materials science, the acquisition of training images requires a large number of professionals and the labor cost is extremely high, so there are usually very few training samples in the field of materials. In this study, a segmentation method of nanoparticle topological structure based on synthetic data (SD) is proposed, which aims to solve the issue of small data in the field of materials. Our findings reveal that the combination of SD generated by rendering software with merely 15% Authentic Data (AD) shows better performance in training deep learning model. The trained U-Net model shows that Miou of 0.8476, accuracy of 0.9970, Kappa of 0.8207, and Dice of 0.9103, respectively. Compared with data enhancement alone, our approach yields a 1% improvement in the Miou metric. These results show that our proposed strategy can achieve better prediction performance without increasing the cost of data acquisition.

Review of machine learning for lipid nanoparticle formulation and process development

Lipid nanoparticles (LNPs) are a subset of pharmaceutical nanoparticulate formulations designed to encapsulate, stabilize, and deliver nucleic acid cargoes in vivo. Applications for LNPs include new interventions for genetic disorders, novel classes of vaccines, and alternate modes of intracellular delivery for therapeutic proteins. In the pharmaceutical industry, establishing a robust formulation and process to achieve target product performance is a critical component of drug development. Fundamental understanding of the processes for making LNPs and their interactions with biological systems have advanced considerably in the wake of the COVID-19 pandemic. Nevertheless, LNP formulation research remains largely empirical and resource intensive due to the multitude of input parameters and the complex physical phenomena that govern the processes of nanoparticle precipitation, self-assembly, structure evolution, and stability. Increasingly, artificial intelligence and machine learning (AI/ML) are being applied to improve the efficiency of research activities through in silico models and predictions, and to drive deeper fundamental understanding of experimental inputs to functional outputs. This review will identify current challenges and opportunities in the development of robust LNP formulations of nucleic acids, review studies that apply machine learning methods to experimental datasets, and provide discussion on associated data science challenges to facilitate collaboration between formulation and data scientists, aiming to accelerate the advancement of AI/ML applied to LNP formulation and process optimization.

Investigation of the Octahedral Network Structure in Formamidinium Lead Bromide Nanocrystals by Low-Dose Scanning Transmission Electron Microscopy

Metal halide perovskites (MHP) are highly promising semiconductors. In this study, we focus on FAPbBr3 nanocrystals, which are of great interest for green light-emitting diodes. Structural parameters significantly impact the properties of MHPs and are linked to phase instability, which hampers long-term applications. Clearly, there is a need for local and precise characterization techniques at the atomic scale, such as transmission electron microscopy. Because of the high electron beam sensitivity of MHPs, these investigations are extremely challenging. Here, we applied a low-dose method based on four-dimensional scanning transmission electron microscopy. We quantified the observed elongation of the projections of the Br atomic columns, suggesting an alternation in the position of the Br atoms perpendicular to the Pb-Br-Pb bonds. Together with molecular dynamics simulations, these results remarkably reveal local distortions in an on-average cubic structure. Additionally, this study provides an approach to prospectively investigating the fundamental degradation mechanisms of MHPs.

Determining the orderliness of carbon materials with nanoparticle imaging and explainable machine learning

Carbon materials have paramount importance in various fields of materials science, from electronic devices to industrial catalysts. The properties of these materials are strongly related to the distribution of defects-irregularities in electron density on their surfaces. Different materials have various distributions and quantities of these defects, which can be imaged using a procedure that involves depositing palladium nanoparticles. The resulting scanning electron microscopy (SEM) images can be characterized by a key descriptor-the ordering of nanoparticle positions. This work presents a highly interpretable machine learning approach for distinguishing between materials with ordered and disordered arrangements of defects marked by nanoparticle attachment. The influence of the degree of ordering was experimentally evaluated on the example of catalysis via chemical reactions involving carbon-carbon bond formation. This represents an important step toward automated analysis of SEM images in materials science.

Nanocrystal Assemblies: Current Advances and Open Problems

We explore the potential of nanocrystals (a term used equivalently to nanoparticles) as building blocks for nanomaterials, and the current advances and open challenges for fundamental science developments and applications. Nanocrystal assemblies are inherently multiscale, and the generation of revolutionary material properties requires a precise understanding of the relationship between structure and function, the former being determined by classical effects and the latter often by quantum effects. With an emphasis on theory and computation, we discuss challenges that hamper current assembly strategies and to what extent nanocrystal assemblies represent thermodynamic equilibrium or kinetically trapped metastable states. We also examine dynamic effects and optimization of assembly protocols. Finally, we discuss promising material functions and examples of their realization with nanocrystal assemblies.

Atomic-level imaging of beam-sensitive COFs and MOFs by low-dose electron microscopy

Electron microscopy, an important technique that allows for the precise determination of structural information with high spatiotemporal resolution, has become indispensable in unravelling the complex relationships between material structure and properties ranging from mesoscale morphology to atomic arrangement. However, beam-sensitive materials, particularly those comprising organic components such as metal-organic frameworks (MOFs) and covalent organic frameworks (COFs), would suffer catastrophic damage from the high energy electrons, hindering the determination of atomic structures. A low-dose approach has arisen as a possible solution to this problem based on the integration of advancements in several aspects: electron optical system, detector, image processing, and specimen preservation. This article summarizes the transmission electron microscopy characterization of MOFs and COFs, including local structures, host-guest interactions, and interfaces at the atomic level. Revolutions in advanced direct electron detectors, algorithms in image acquisition and processing, and emerging methodology for high quality low-dose imaging are also reviewed. Finally, perspectives on the future development of electron microscopy methodology with the support of computer science are presented.

Observing the Dynamics of an Electrochemically Driven Active Material with Liquid Electron Microscopy

Electrochemical liquid electron microscopy has revolutionized our understanding of nanomaterial dynamics by allowing for direct observation of their electrochemical production. This technique, primarily applied to inorganic materials, is now being used to explore the self-assembly dynamics of active molecular materials. Our study examines these dynamics across various scales, from the nanoscale behavior of individual fibers to the micrometer-scale hierarchical evolution of fiber clusters. To isolate the influences of the electron beam and electrical potential on material behavior, we conducted thorough beam-sample interaction analyses. Our findings reveal that the dynamics of these active materials at the nanoscale are shaped by their proximity to the electrode and the applied electrical current. By integrating electron microscopy observations with reaction-diffusion simulations, we uncover that local structures and their formation history play a crucial role in determining assembly rates. This suggests that the emergence of nonequilibrium structures can locally accelerate further structural development, offering insights into the behavior of active materials under electrochemical conditions.

Automatic center identification of electron diffraction with multi-scale transformer networks

Selected area electron diffraction (SAED) is a widely used technique for characterizing the structure and measuring lattice parameters of materials. An autonomous analytic method has become an urgent demand for the large-scale SAED data produced from in-situ experiments. In this work, we realize the automatic processing for center identification with a proposed deep segmentation model named the multi-scale Transformer (MS-Trans) network. This algorithm enables robust segmentation of the central spots by combining a novel gated axial-attention module and multi-scale feature fusion. The proposed MS-Trans model shows high precision and robustness, enabling autonomous processing of SAED patterns without any prior knowledge. The application on in-situ SAED data of the oxidation process of FeNi alloy demonstrates its capability of implementing autonomous quantitative processing. © 2017 Elsevier Inc. All rights reserved.

Electron videography of a lipid-protein tango

Biological phenomena, from enzymatic catalysis to synaptic transmission, originate in the structural transformations of biomolecules and biomolecular assemblies in liquid water. However, directly imaging these nanoscopic dynamics without probes or labels has been a fundamental methodological challenge. Here, we developed an approach for "electron videography"-combining liquid phase electron microscopy with molecular modeling-with which we filmed the nanoscale structural fluctuations of individual, suspended, and unlabeled membrane protein nanodiscs in liquid. Systematic comparisons with biochemical data and simulation indicate the graphene encapsulation involved can afford sufficiently reduced effects of the illuminating electron beam for these observations to yield quantitative fingerprints of nanoscale lipid-protein interactions. Our results suggest that lipid-protein interactions delineate dynamically modified membrane domains across unexpectedly long ranges. Moreover, they contribute to the molecular mechanics of the nanodisc as a whole in a manner specific to the protein within. Overall, this work illustrates an experimental approach to film, quantify, and understand biomolecular dynamics at the nanometer scale.

4D-STEM Mapping of Nanocrystal Reaction Dynamics and Heterogeneity in a Graphene Liquid Cell

Chemical reaction kinetics at the nanoscale are intertwined with heterogeneity in structure and composition. However, mapping such heterogeneity in a liquid environment is extremely challenging. Here we integrate graphene liquid cell (GLC) transmission electron microscopy and four-dimensional scanning transmission electron microscopy to image the etching dynamics of gold nanorods in the reaction media. Critical to our experiment is the small liquid thickness in a GLC that allows the collection of high-quality electron diffraction patterns at low dose conditions. Machine learning-based data-mining of the diffraction patterns maps the three-dimensional nanocrystal orientation, groups spatial domains of various species in the GLC, and identifies newly generated nanocrystallites during reaction, offering a comprehensive understanding on the reaction mechanism inside a nanoenvironment. This work opens opportunities in probing the interplay of structural properties such as phase and strain with solution-phase reaction dynamics, which is important for applications in catalysis, energy storage, and self-assembly.

Generalization Across Experimental Parameters in Neural Network Analysis of High-Resolution Transmission Electron Microscopy Datasets

Neural networks are promising tools for high-throughput and accurate transmission electron microscopy (TEM) analysis of nanomaterials, but are known to generalize poorly on data that is "out-of-distribution" from their training data. Given the limited set of image features typically seen in high-resolution TEM imaging, it is unclear which images are considered out-of-distribution from others. Here, we investigate how the choice of metadata features in the training dataset influences neural network performance, focusing on the example task of nanoparticle segmentation. We train and validate neural networks across curated, experimentally collected high-resolution TEM image datasets of nanoparticles under various imaging and material parameters, including magnification, dosage, nanoparticle diameter, and nanoparticle material. Overall, we find that our neural networks are not robust across microscope parameters, but do generalize across certain sample parameters. Additionally, data preprocessing can have unintended consequences on neural network generalization. Our results highlight the need to understand how dataset features affect deployment of data-driven algorithms.

Machine Learning Refinement of In Situ Images Acquired by Low Electron Dose LC-TEM

We have studied a machine learning (ML) technique for refining images acquired during in situ observation using liquid-cell transmission electron microscopy. Our model is constructed using a U-Net architecture and a ResNet encoder. For training our ML model, we prepared an original image dataset that contained pairs of images of samples acquired with and without a solution present. The former images were used as noisy images, and the latter images were used as corresponding ground truth images. The number of pairs of image sets was 1,204, and the image sets included images acquired at several different magnifications and electron doses. The trained model converted a noisy image into a clear image. The time necessary for the conversion was on the order of 10 ms, and we applied the model to in situ observations using the software Gatan DigitalMicrograph (DM). Even if a nanoparticle was not visible in a view window in the DM software because of the low electron dose, it was visible in a successive refined image generated by our ML model.

Correlating electrochemical stimulus to structural change in liquid electron microscopy videos using the structural dissimilarity metric

In-situ liquid cell transmission electron microscopy (LCTEM) with electrical biasing capabilities has emerged as an invaluable tool for directly imaging electrode processes with high temporal and spatial resolution. However, accurately quantifying structural changes that occur on the electrode and subsequently correlating them to the applied stimulus remains challenging. Here, we present structural dissimilarity (DSSIM) analysis as segmentation-free video processing algorithm for locally detecting and quantifying structural change occurring in LCTEM videos. In this study, DSSIM analysis is applied to two in-situ LCTEM videos to demonstrate how to implement this algorithm and interpret the results. We show DSSIM analysis can be used as a visualization tool for qualitative data analysis by highlighting structural changes which are easily missed when viewing the raw data. Furthermore, we demonstrate how DSSIM analysis can serve as a quantitative metric and efficiently convert 3-dimensional microscopy videos to 1-dimenional plots which makes it easy to interpret and compare events occurring at different timepoints in a video. In the analyses presented here, DSSIM is used to directly correlate the magnitude and temporal scale of structural change to the features of the applied electrical bias. ImageJ, Python, and MATLAB programs, including a user-friendly interface and accompanying documentation, are published alongside this manuscript to make DSSIM analysis easily accessible to the scientific community.

Revealing microscopic dynamics: in situ liquid-phase TEM for live observations of soft materials and quantitative analysis via deep learning

In various domains spanning materials synthesis, chemical catalysis, life sciences, and energy materials, in situ transmission electron microscopy (TEM) methods exert a profound influence. These methodologies enable the real-time observation and manipulation of gas-phase and liquid-phase reactions at the nanoscale, facilitating the exploration of pivotal reaction mechanisms. Fundamental research areas like crystal nucleation, growth, etching, and self-assembly have greatly benefited from these techniques. Additionally, their applications extend across diverse fields such as catalysis, batteries, bioimaging, and drug delivery kinetics. However, the intricate nature of 'soft matter' presents a challenge due to the unique molecular properties and dynamic behavior of these substances that remain insufficiently understood. Investigating soft matter within in situ liquid-phase TEM settings demands further exploration and advancement compared to other research domains. This research harnesses the potential of in situ liquid-phase TEM technology while integrating deep learning methodologies to comprehensively analyze the quantitative aspects of soft matter dynamics. This study centers on diverse phenomena, encompassing surfactant molecule nucleation, block copolymer behavior, confinement-driven self-assembly, and drying processes. Furthermore, deep learning techniques are employed to precisely analyze Ostwald ripening and digestive ripening dynamics. The outcomes of this study not only deepen the understanding of soft matter at its fundamental level but also serve as a pivotal foundation for developing innovative functional materials and cutting-edge devices.

Nanoparticle Assembly: From Self-Organization to Controlled Micropatterning for Enhanced Functionalities

Nanoparticles form long-range micropatterns via self-assembly or directed self-assembly with superior mechanical, electrical, optical, magnetic, chemical, and other functional properties for broad applications, such as structural supports, thermal exchangers, optoelectronics, microelectronics, and robotics. The precisely defined particle assembly at the nanoscale with simultaneously scalable patterning at the microscale is indispensable for enabling functionality and improving the performance of devices. This article provides a comprehensive review of nanoparticle assembly formed primarily via the balance of forces at the nanoscale (e.g., van der Waals, colloidal, capillary, convection, and chemical forces) and nanoparticle-template interactions (e.g., physical confinement, chemical functionalization, additive layer-upon-layer). The review commences with a general overview of nanoparticle self-assembly, with the state-of-the-art literature review and motivation. It subsequently reviews the recent progress in nanoparticle assembly without the presence of surface templates. Manufacturing techniques for surface template fabrication and their influence on nanoparticle assembly efficiency and effectiveness are then explored. The primary focus is the spatial organization and orientational preference of nanoparticles on non-templated and pre-templated surfaces in a controlled manner. Moreover, the article discusses broad applications of micropatterned surfaces, encompassing various fields. Finally, the review concludes with a summary of manufacturing methods, their limitations, and future trends in nanoparticle assembly.

Perspectives on Theoretical Models and Molecular Simulations of Polymer Brushes

Polymer brushes have witnessed extensive utilization and progress, driven by their distinct attributes in surface modification, tethered group functionality, and tailored interactions at the nanoscale, enabling them for various scientific and industrial applications of coatings, sensors, switchable/responsive materials, nanolithography, and lab-on-a-chips. Despite the wealth of experimental investigations into polymer brushes, this review primarily focuses on computational studies of antifouling polymer brushes with a strong emphasis on achieving a molecular-level understanding and structurally designing antifouling polymer brushes. Computational exploration covers three realms of thermotical models, molecular simulations, and machine-learning approaches to elucidate the intricate relationship between composition, structure, and properties concerning polymer brushes in the context of nanotribology, surface hydration, and packing conformation. Upon acknowledging the challenges currently faced, we extend our perspectives toward future research directions by delineating potential avenues and unexplored territories. Our overarching objective is to advance our foundational comprehension and practical utilization of polymer brushes for antifouling applications, leveraging the synergy between computational methods and materials design to drive innovation in this crucial field.

Direct Imaging of "Patch-Clasping" and Relaxation in Robust and Flexible Nanoparticle Assemblies

Polymer patching on inorganic nanoparticles (NPs) enables multifunctionality and directed self-assembly into nonclosely packed optical and mechanical metamaterials. However, experimental demonstration of such assemblies has been scant due to challenges in leveraging patch-induced NP-NP attractions and understanding NP self-assembly dynamics. Here we use low-dose liquid-phase transmission electron microscopy to visualize the dynamic behaviors of tip-patched triangular nanoprisms upon patch-clasping, where polymer patches interpenetrate to form cohesive bonds that connect NPs. Notably, these bonds are longitudinally robust but rotationally flexible. Patch-clasping is found to allow highly selective tip-tip assembly, interconversion between dimeric bowtie and sawtooth configurations, and collective structural relaxation of NP networks. The integration of single particle tracking, polymer physics theory, and molecular dynamics simulation reveals the macromolecular origin of patch-clasping-induced NP dynamics. Our experiment-computation integration can aid the design of stimuli-responsive nanomaterials, such as topological metamaterials for chiral sensors, waveguides, and nanoantennas.

Observing the Evolution of Metal Oxides in Liquids

Metal oxides with diverse compositions and structures have garnered considerable interest from researchers in various reactions, which benefits from transmission electron microscopy (TEM) in determining their morphologies, phase, structural and chemical information. Recent breakthroughs have made liquid-phase TEM a promising imaging platform for tracking the dynamic structure, morphology, and composition evolution of metal oxides in solution under work conditions. Herein, this review introduces the recent advances in liquid cells, especially closed liquid cell chips. Subsequently, the recent progress including particle growth, phase transformation, self-assembly, core-shell nanostructure growth, and chemical etching are introduced. With the late technical advances in TEM and liquid cells, liquid-phase TEM is used to characterize many fundamental processes of metal oxides for CO2 reduction and water-splitting reactions. Finally, the outlook and challenges in this research field are discussed. It is believed this compilation inspires and stimulates more efforts in developing and utilizing in situ liquid-phase TEM for metal oxides at the atomic scale for different applications.

UTILE-Gen: Automated Image Analysis in Nanoscience Using Synthetic Dataset Generator and Deep Learning

This work presents the development and implementation of a deep learning-based workflow for autonomous image analysis in nanoscience. A versatile, agnostic, and configurable tool was developed to generate instance-segmented imaging datasets of nanoparticles. The synthetic generator tool employs domain randomization to expand the image/mask pairs dataset for training supervised deep learning models. The approach eliminates tedious manual annotation and allows training of high-performance models for microscopy image analysis based on convolutional neural networks. We demonstrate how the expanded training set can significantly improve the performance of the classification and instance segmentation models for a variety of nanoparticle shapes, ranging from spherical-, cubic-, to rod-shaped nanoparticles. Finally, the trained models were deployed in a cloud-based analytics platform for the autonomous particle analysis of microscopy images.

Exploration of Organic Nanomaterials with Liquid-Phase Transmission Electron Microscopy

ConspectusOrganic, soft materials with solution-phase nanoscale structures, such as emulsions, hydrogels, and thermally responsive materials, are inherently difficult to directly image via dry state and cryogenic-transmission electron microscopy (TEM). Therefore, we lack a routine microscopy method with sufficient resolution that can, in tandem with scattering techniques, probe the morphology and dynamics of these and many related systems. These challenges motivate liquid cell (LC) TEM method development, aimed at making the technique generally available and routine. To date, the field has been and continues to be dominantly focused on analyzing solution-phase inorganic materials. These mostly metallic nanoparticles have been studied at electron fluxes that can allow for high-resolution imaging, in the range of hundreds to thousands of e- Å-2 s-1. Despite excellent contrast, in these cases, one often contends with knock-on damage, direct radiolysis, and sensitization of the solvent by virtue of enhanced secondary electron production by the impinging electron beam. With an interest in soft materials, we face both related and distinct challenges, especially in achieving a high-enough contrast within solvated liquid cells. Additionally, we must be aware of artifacts associated with high-flux imaging conditions in terms of direct radiolysis of the solvent and the sensitive materials themselves. Regardless, with care, it has become possible to gain real insight into both static and dynamic organic nanomaterials in solution. This is due, in large part, to key advances that have been made, including improved sample preparation protocols, image capture technologies, and image analysis, which have allowed LCTEM to have utility. To enable solvated soft matter characterization by LCTEM, a generalizable multimodal workflow was developed by leveraging both experimental and theoretical precedents from across the LCTEM field and adjacent works concerned with solution radiolysis and nanoparticle tracking analyses. This workflow consists of (1) modeling electron beam-solvent interactions, (2) studying electron beam-sample interactions via LCTEM coupled with post-mortem analysis, (3) the construction of "damage plots" displaying sample integrity under varied imaging and sample conditions, (4) optimized LCTEM imaging, (5) image processing, and (6) correlative analysis via X-ray or light scattering. In this Account, we present this outlook and the challenges we continue to overcome in the direct imaging of dynamic solvated nanoscale soft materials.

In Situ and Emerging Transmission Electron Microscopy for Catalysis Research

Catalysts are the primary facilitator in many dynamic processes. Therefore, a thorough understanding of these processes has vast implications for a myriad of energy systems. The scanning/transmission electron microscope (S/TEM) is a powerful tool not only for atomic-scale characterization but also in situ catalytic experimentation. Techniques such as liquid and gas phase electron microscopy allow the observation of catalysts in an environment conducive to catalytic reactions. Correlated algorithms can greatly improve microscopy data processing and expand multidimensional data handling. Furthermore, new techniques including 4D-STEM, atomic electron tomography, cryogenic electron microscopy, and monochromated electron energy loss spectroscopy (EELS) push the boundaries of our comprehension of catalyst behavior. In this review, we discuss the existing and emergent techniques for observing catalysts using S/TEM. Challenges and opportunities highlighted aim to inspire and accelerate the use of electron microscopy to further investigate the complex interplay of catalytic systems.

Converting Nanotoxicity Data to Information Using Artificial Intelligence and Simulation

Decades of nanotoxicology research have generated extensive and diverse data sets. However, data is not equal to information. The question is how to extract critical information buried in vast data streams. Here we show that artificial intelligence (AI) and molecular simulation play key roles in transforming nanotoxicity data into critical information, i.e., constructing the quantitative nanostructure (physicochemical properties)-toxicity relationships, and elucidating the toxicity-related molecular mechanisms. For AI and molecular simulation to realize their full impacts in this mission, several obstacles must be overcome. These include the paucity of high-quality nanomaterials (NMs) and standardized nanotoxicity data, the lack of model-friendly databases, the scarcity of specific and universal nanodescriptors, and the inability to simulate NMs at realistic spatial and temporal scales. This review provides a comprehensive and representative, but not exhaustive, summary of the current capability gaps and tools required to fill these formidable gaps. Specifically, we discuss the applications of AI and molecular simulation, which can address the large-scale data challenge for nanotoxicology research. The need for model-friendly nanotoxicity databases, powerful nanodescriptors, new modeling approaches, molecular mechanism analysis, and design of the next-generation NMs are also critically discussed. Finally, we provide a perspective on future trends and challenges.

Imaging the Surface/Interface Morphologies Evolution of Silicon Anodes Using in Situ/Operando Electron Microscopy

Si-based rechargeable lithium-ion batteries (LIBs) have generated interest as silicon has remarkably high theoretical specific capacity. It is projected that LIBs will meet the increasing need for extensive energy storage systems, electric vehicles, and portable electronics with high energy densities. However, the Si-based LIB has a substantial problem due to the volume cycle variations brought on by Si, which result in severe capacity loss. Making Si-based anodes-enabled high-performance LIBs that are easy to utilize requires an understanding of the fading mechanism. Due to its distinct advantage in morphological changes from microscale to nanoscale, even approaching atomic resolution, electron microscopy is one of the most popular methods. Based on operando electron microscopy characterization, the general comprehension of the fading mechanism and the morphology evolution of Si-based LIBs are debated in this review. The current advancements in compositional and structural interpretation for Si-based LIBs using advanced electron microscopy characterization methods are outlined. The future development trends in pertinent silicon materials characterization methods are also highlighted, along with numerous potential research avenues for Si-based LIBs design and characterization.

Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning

Metallic nanoparticles were analysed with respect to size and shape by a machine learning approach. This involved a separation of particles from the background (segmentation), a separation of overlapping particles, and the identification of individual particles. An algorithm to separate overlapping particles, based on ultimate erosion of convex shapes (UECS), was implemented. Finally, particle properties like size, circularity, equivalent diameter, and Feret diameter were computed for each particle of the whole particle population. Thus, particle size distributions can be easily created based on the various parameters. However, strongly overlapping particles are difficult and sometimes impossible to separate because of an a priori unknown shape of a particle that is partially lying in the shadow of another particle. The program is able to extract information from a sequence of images of the same sample, thereby increasing the number of analysed nanoparticles to several thousands. The machine learning approach is well-suited to identify particles at only limited particle-to-background contrast as is demonstrated for ultrasmall gold nanoparticles (2 nm).

Nanoparticle Assembly and Oriented Attachment: Correlating Controlling Factors to the Resulting Structures

Nanoparticle assembly and attachment are common pathways of crystal growth by which particles organize into larger scale materials with hierarchical structure and long-range order. In particular, oriented attachment (OA), which is a special type of particle assembly, has attracted great attention in recent years because of the wide range of material structures that result from this process, such as one-dimensional (1D) nanowires, two-dimensional (2D) sheets, three-dimensional (3D) branched structures, twinned crystals, defects, etc. Utilizing in situ transmission electron microscopy techniques, researchers observed orientation-specific forces that act over short distances (∼1 nm) from the particle surfaces and drive the OA process. Integrating recently developed 3D fast force mapping via atomic force microscopy with theories and simulations, researchers have resolved the near-surface solution structure, the molecular details of charge states at particle/fluid interfaces, inhomogeneity of surface charges, and dielectric/magnetic properties of particles that influence short- and long-range forces, such as electrostatic, van der Waals, hydration, and dipole-dipole forces. In this review, we discuss the fundamental principles for understanding particle assembly and attachment processes, and the controlling factors and resulting structures. We review recent progress in the field via examples of both experiments and modeling, and discuss current developments and the future outlook.

Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders

The application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.

Direct Imaging of the Kinetic Crystallization Pathway: Simulation and Liquid-Phase Transmission Electron Microscopy Observations

The crystallization of materials from a suspension determines the structure and function of the final product, and numerous pieces of evidence have pointed out that the classical crystallization pathway may not capture the whole picture of the crystallization pathways. However, visualizing the initial nucleation and further growth of a crystal at the nanoscale has been challenging due to the difficulties of imaging individual atoms or nanoparticles during the crystallization process in solution. Recent progress in nanoscale microscopy had tackled this problem by monitoring the dynamic structural evolution of crystallization in a liquid environment. In this review, we summarized several crystallization pathways captured by the liquid-phase transmission electron microscopy technique and compared the observations with computer simulation. Apart from the classical nucleation pathway, we highlight three nonclassical pathways that are both observed in experiments and computer simulations: formation of an amorphous cluster below the critical nucleus size, nucleation of the crystalline phase from an amorphous intermediate, and transition between multiple crystalline structures before achieving the final product. Among these pathways, we also highlight the similarities and differences between the experimental results of the crystallization of single nanocrystals from atoms and the assembly of a colloidal superlattice from a large number of colloidal nanoparticles. By comparing the experimental results with computer simulations, we point out the importance of theory and simulation in developing a mechanistic approach to facilitate the understanding of the crystallization pathway in experimental systems. We also discuss the challenges and future perspectives for investigating the crystallization pathways at the nanoscale with the development of in situ nanoscale imaging techniques and potential applications to the understanding of biomineralization and protein self-assembly.

Deep learning-assisted analysis of single molecule dynamics from liquid-phase electron microscopy

We apply U-Net and UNet++ to analyze single-molecule movies obtained from liquid-phase electron microscopy. Neural networks allow full automation, and high throughput analysis of these low signal-to-noise ratio images, while achieving higher segmentation accuracy, and avoiding subjective errors as compared to the conventional threshold methods. The analysis enables the quantification of transient dynamics in chemical systems and the capture of rare intermediate states by resolving local conformational changes within a single molecule.

How machine learning can accelerate electrocatalysis discovery and optimization

Advances in machine learning (ML) provide the means to bypass bottlenecks in the discovery of new electrocatalysts using traditional approaches. In this review, we highlight the currently achieved work in ML-accelerated discovery and optimization of electrocatalysts via a tight collaboration between computational models and experiments. First, the applicability of available methods for constructing machine-learned potentials (MLPs), which provide accurate energies and forces for atomistic simulations, are discussed. Meanwhile, the current challenges for MLPs in the context of electrocatalysis are highlighted. Then, we review the recent progress in predicting catalytic activities using surrogate models, including microkinetic simulations and more global proxies thereof. Several typical applications of using ML to rationalize thermodynamic proxies and predict the adsorption and activation energies are also discussed. Next, recent developments of ML-assisted experiments for catalyst characterization, synthesis optimization and reaction condition optimization are illustrated. In particular, the applications in ML-enhanced spectra analysis and the use of ML to interpret experimental kinetic data are highlighted. Additionally, we also show how robotics are applied to high-throughput synthesis, characterization and testing of electrocatalysts to accelerate the materials exploration process and how this equipment can be assembled into self-driven laboratories.

Deep Learning to Reveal the Distribution and Diffusion of Water Molecules in Fuel Cell Catalyst Layers

Water management in the catalyst layers (CLs) of proton-exchange membrane fuel cells is crucial for its commercialization and popularization. However, the high experimental or computational cost in obtaining water distribution and diffusion remains a bottleneck in the existing experimental methods and simulation algorithms, and further mechanistic exploration at the nanoscale is necessary. Herein, we integrate, for the first time, molecular dynamics simulation with our customized analysis framework based on a multiattribute point cloud dataset and an advanced deep learning network. This was achieved through our workflow that generates simulated transport data of water molecules in the CLs as the training and test dataset. Deep learning framework models the multibody solid-liquid system of CLs on a molecular scale and completes the mapping from the Pt/C substrate structure and Nafion aggregates to the density distribution and diffusion coefficient of water molecules. The prediction results are comprehensively analyzed and error evaluated, which reveals the highly anisotropic interaction landscape between 50,000 pairs of interacting nanoparticles and explains the structure and water transport property relationship in the hydrated Nafion film on the molecular scale. Compared to the conventional methods, the proposed deep learning framework shows computational cost efficiency, accuracy, and good visual display. Further, it has a generality potential to model macro- and microscopic mass transport in different components of fuel cells. Our framework is expected to make real-time predictions of the distribution and diffusion of water molecules in CLs as well as establish statistical significance in the structural optimization and design of CLs and other components of fuel cells.

Machine learning for nanoplasmonics

Plasmonic nanomaterials have outstanding optoelectronic properties potentially enabling the next generation of catalysts, sensors, lasers and photothermal devices. Owing to optical and electron techniques, modern nanoplasmonics research generates large datasets characterizing features across length scales. Furthermore, optimizing syntheses leading to specific nanostructures requires time-consuming multiparametric approaches. These complex datasets and trial-and-error practices make nanoplasmonics research ripe for the application of machine learning (ML) and advanced data processing methods. ML algorithms capture relationships between synthesis, structure and performance in a way that far exceeds conventional simulation and theory approaches, enabling effective performance optimization. For example, neural networks can tailor the nanostructure morphology to target desired properties, identify synthetic conditions and extract quantitative information from complex data. Here we discuss the nascent field of ML for nanoplasmonics, describe the opportunities and limitations of ML in nanoplasmonic research, and conclude that ML is potentially transformative, especially if the community curates and shares its big data.

Electron Tomography and Machine Learning for Understanding the Highly Ordered Structure of Leafhopper Brochosomes

Insects known as leafhoppers (Hemiptera: Cicadellidae) produce hierarchically structured nanoparticles known as brochosomes that are exuded and applied to the insect cuticle, thereby providing camouflage and anti-wetting properties to aid insect survival. Although the physical properties of brochosomes are thought to depend on the leafhopper species, the structure-function relationships governing brochosome behavior are not fully understood. Brochosomes have complex hierarchical structures and morphological heterogeneity across species, due to which a multimodal characterization approach is required to effectively elucidate their nanoscale structure and properties. In this work, we study the structural and mechanical properties of brochosomes using a combination of atomic force microscopy (AFM), electron microscopy (EM), electron tomography, and machine learning (ML)-based quantification of large and complex scanning electron microscopy (SEM) image data sets. This suite of techniques allows for the characterization of internal and external brochosome structures, and ML-based image analysis methods of large data sets reveal correlations in the structure across several leafhopper species. Our results show that brochosomes are relatively rigid hollow spheres with characteristic dimensions and morphologies that depend on leafhopper species. Nanomechanical mapping AFM is used to determine a characteristic compression modulus for brochosomes on the order of 1-3 GPa, which is consistent with crystalline proteins. Overall, this work provides an improved understanding of the structural and mechanical properties of leafhopper brochosomes using a new set of ML-based image classification tools that can be broadly applied to nanostructured biological materials.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

Seeking regularity from irregularity: unveiling the synthesis-nanomorphology relationships of heterogeneous nanomaterials using unsupervised machine learning

Nanoscale morphology of functional materials determines their chemical and physical properties. However, despite increasing use of transmission electron microscopy (TEM) to directly image nanomorphology, it remains challenging to quantify the information embedded in TEM data sets, and to use nanomorphology to link synthesis and processing conditions to properties. We develop an automated, descriptor-free analysis workflow for TEM data that utilizes convolutional neural networks and unsupervised learning to quantify and classify nanomorphology, and thereby reveal synthesis-nanomorphology relationships in three different systems. While TEM records nanomorphology readily in two-dimensional (2D) images or three-dimensional (3D) tomograms, we advance the analysis of these images by identifying and applying a universal shape fingerprint function to characterize nanomorphology. After dimensionality reduction through principal component analysis, this function then serves as the input for morphology grouping through unsupervised learning. We demonstrate the wide applicability of our workflow to both 2D and 3D TEM data sets, and to both inorganic and organic nanomaterials, including tetrahedral gold nanoparticles mixed with irregularly shaped impurities, hybrid polymer-patched gold nanoprisms, and polyamide membranes with irregular and heterogeneous 3D crumple structures. In each of these systems, unsupervised nanomorphology grouping identifies both the diversity and the similarity of the nanomaterial across different synthesis conditions, revealing how synthetic parameters guide nanomorphology development. Our work opens possibilities for enhancing synthesis of nanomaterials through artificial intelligence and for understanding and controlling complex nanomorphology, both for 2D systems and in the far less explored case of 3D structures, such as those with embedded voids or hidden interfaces.

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42 650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. A classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr₃ nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.

Visualizing Dynamic Environmental Processes in Liquid at Nanoscale via Liquid-Phase Electron Microscopy

Visualizing the structure and processes in liquids at the nanoscale is essential for understanding the fundamental mechanisms and underlying processes of environmental research. Cutting-edge progress of in situ liquid-phase (scanning) transmission electron microscopy (LP-S/TEM) and inferred possible applications are highlighted as a more and more indispensable tool for visualization of dynamic environmental processes in this Perspective. Advancements in nanofabrication technology, high-speed imaging, comprehensive detectors, and spectroscopy analysis have made it increasingly convenient to use LP S/TEM, thus providing an approach for visualization of direct and insightful scientific information with the exciting possibility of solving an increasing number of tricky environmental problems. This includes evaluating the transformation fate and path of contamination, assessing toxicology of nanomaterials, simulating solid surface corrosion processes in the environment, and observing water pollution control processes. Distinct nanoscale or even atomic understanding of the reaction would provide dependable and precise identification and quantification of contaminants in dynamic processes, thus facilitating trouble-tracing of environmental problems with amplifying complexity.

Machine Learning-Assisted Identification of Copolymer Microstructures Based on Microscopic Images

The microstructure of polymer materials is an important bridge between their molecular structure and macroproperties, which is of great significance to be effectively identified. With the increasing refinement of polymer material design, the microstructure of different polymer materials gradually converges, which is difficult to distinguish. In this study, the machine learning method is applied to recognize the microstructure. A highly accurate and interpretable model based on small experimental data sets has been completed by the methods of transfer learning and feature visualization, making the result of the model that can be explained from the perspective of physical chemistry. This work provides an idea for identifying microstructure and will help further promote intelligent polymer research and development.

Reversible assembly of nanoparticles: theory, strategies and computational simulations

The significant advances in synthesis and functionalization have enabled the preparation of high-quality nanoparticles that have found a plethora of successful applications. The unique physicochemical properties of nanoparticles can be manipulated through the control of size, shape, composition, and surface chemistry, but their technological application possibilities can be further expanded by exploiting the properties that emerge from their assembly. The ability to control the assembly of nanoparticles not only is required for many real technological applications, but allows the combination of the intrinsic properties of nanoparticles and opens the way to the exploitation of their complex interplay, giving access to collective properties. Significant advances and knowledge gained over the past few decades on nanoparticle assembly have made it possible to implement a growing number of strategies for reversible assembly of nanoparticles. In addition to being of interest for basic studies, such advances further broaden the range of applications and the possibility of developing innovative devices using nanoparticles. This review focuses on the reversible assembly of nanoparticles and includes the theoretical aspects related to the concept of reversibility, an up-to-date assessment of the experimental approaches applied to this field and the advanced computational schemes that offer key insights into the assembly mechanisms. We aim to provide readers with a comprehensive guide to address the challenges in assembling reversible nanoparticles and promote their applications.

Machine learning, artificial intelligence, and chemistry: how smart algorithms are reshaping simulation and the laboratory

Machine learning and artificial intelligence are increasingly gaining in prominence through image analysis, language processing, and automation, to name a few applications. Machine learning is also making profound changes in chemistry. From revisiting decades-old analytical techniques for the purpose of creating better calibration curves, to assisting and accelerating traditional in silico simulations, to automating entire scientific workflows, to being used as an approach to deduce underlying physics of unexplained chemical phenomena, machine learning and artificial intelligence are reshaping chemistry, accelerating scientific discovery, and yielding new insights. This review provides an overview of machine learning and artificial intelligence from a chemist’s perspective and focuses on a number of examples of the use of these approaches in computational chemistry and in the laboratory.

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

In-situ irradiation transmission electron microscopy (TEM) offers unique insights into the millisecond-timescale post-cascade process, such as the lifetime and thermal stability of defect clusters, vital to the mechanistic understanding of irradiation damage in nuclear materials. Converting in-situ irradiation TEM video data into meaningful information on defect cluster dynamic properties (e.g., lifetime) has become the major technical bottleneck. Here, we present a solution called the DefectTrack, the first dedicated deep learning-based one-shot multi-object tracking (MOT) model capable of tracking cascade-induced defect clusters in in-situ TEM videos in real-time. DefectTrack has achieved a Multi-Object Tracking Accuracy (MOTA) of 66.43% and a Mostly Tracked (MT) of 67.81% on the test set, which are comparable to state-of-the-art MOT algorithms. We discuss the MOT framework, model selection, training, and evaluation strategies for in-situ TEM applications. Further, we compare the DefectTrack with four human experts in quantifying defect cluster lifetime distributions using statistical tests and discuss the relationship between the material science domain metrics and MOT metrics. Our statistical evaluations on the defect lifetime distribution suggest that the DefectTrack outperforms human experts in accuracy and speed.

Understanding the Influence of Receptive Field and Network Complexity in Neural Network-Guided TEM Image Analysis

Trained neural networks are promising tools to analyze the ever-increasing amount of scientific image data, but it is unclear how to best customize these networks for the unique features in transmission electron micrographs. Here, we systematically examine how neural network architecture choices affect how neural networks segment, or pixel-wise separate, crystalline nanoparticles from amorphous background in transmission electron microscopy (TEM) images. We focus on decoupling the influence of receptive field, or the area of the input image that contributes to the output decision, from network complexity, which dictates the number of trainable parameters. For low-resolution TEM images which rely on amplitude contrast to distinguish nanoparticles from background, we find that the receptive field does not significantly influence segmentation performance. On the other hand, for high-resolution TEM images which rely on both amplitude and phase-contrast changes to identify nanoparticles, receptive field is an important parameter for increased performance, especially in images with minimal amplitude contrast. Rather than depending on atom or nanoparticle size, the ideal receptive field seems to be inversely correlated to the degree of nanoparticle contrast in the image. Our results provide insight and guidance as to how to adapt neural networks for applications with TEM datasets.

Multistep Crystallization of Dynamic Nanoparticle Superlattices in Nonaqueous Solutions

Crystallization is a universal phenomenon underpinning many industrial and natural processes and is fundamental to chemistry and materials science. However, microscopic crystallization pathways of nanoparticle superlattices have been seldom studied mainly owing to the difficulty of real-time observation of individual self-assembling nanoparticles in solution. Here, using in situ electron microscopy, we directly image the full self-assembly pathway from dispersed nanoparticles into ordered superlattices in nonaqueous solution. We show that electron-beam irradiation controls nanoparticle mobility, and the solvent composition largely dictates interparticle interactions and assembly behaviors. We uncover a multistep crystallization pathway consisting of four distinct stages through multi-order-parameter analysis and visualize the formation, migration, and annihilation of multiple types of defects in nanoparticle superlattices. These findings open the door for achieving independent control over imaging conditions and nanoparticle assembly conditions and will enable further study of the microscopic kinetics of assembly and phase transition in nanocolloidal systems.

Molecular Mechanism of Organic Crystal Nucleation: A Perspective of Solution Chemistry and Polymorphism

Crystal nucleation determining the formation and assembly pathway of first organic materials is the central science of various scientific disciplines such as chemical, geochemical, biological, and synthetic materials. However, our current understanding of the molecular mechanisms of nucleation remains limited. Over the past decades, the advancements of new experimental and computational techniques have renewed numerous interests in detailed molecular mechanisms of crystal nucleation, especially structure evolution and solution chemistry. These efforts bifurcate into two categories: (modified) classical nucleation theory (CNT) and non-classical nucleation mechanisms. In this review, we briefly introduce the two nucleation mechanisms and summarize current molecular understandings of crystal nucleation that are specifically applied in polymorphic crystallization systems of small organic molecules. Many important aspects of crystal nucleation including molecular association, solvation, aromatic interactions, and hierarchy in intermolecular interactions were examined and discussed for a series of organic molecular systems. The new understandings relating to molecular self-assembly in nucleating systems have suggested more complex multiple nucleation pathways that are associated with the formation and evolution of molecular aggregates in solution.

Liquid-Phase Transmission Electron Microscopy for Reliable In Situ Imaging of Nanomaterials

Liquid-phase transmission electron microscopy (LPTEM) is a powerful in situ visualization technique for directly characterizing nanomaterials in the liquid state. Despite its successful application in many fields, several challenges remain in achieving more accurate and reliable observations. We present LPTEM in chemical and biological applications, including studies for the morphological transformation and dynamics of nanoparticles, battery systems, catalysis, biomolecules, and organic systems. We describe the possible interactions and effects of the electron beam on specimens during observation and present sample-specific approaches to mitigate and control these electron-beam effects. We provide recent advances in achieving atomic-level resolution for liquid-phase investigation of structures anddynamics. Moreover, we discuss the development of liquid cell platforms and the introduction of machine-learning data processing for quantitative and objective LPTEM analysis.

Automated Image Analysis for Single-Atom Detection in Catalytic Materials by Transmission Electron Microscopy

Single-atom catalytic sites may have existed in all supported transition metal catalysts since their first application. Yet, interest in the design of single-atom heterogeneous catalysts (SACs) only really grew when advances in transmission electron microscopy (TEM) permitted direct confirmation of metal site isolation. While atomic-resolution imaging remains a central characterization tool, poor statistical significance, reproducibility, and interoperability limit its scope for deriving robust characteristics about these frontier catalytic materials. Here, we introduce a customized deep-learning method for automated atom detection in image analysis, a rate-limiting step toward high-throughput TEM. Platinum atoms stabilized on a functionalized carbon support with a challenging irregular three-dimensional morphology serve as a practically relevant test system with promising scope in thermo- and electrochemical applications. The model detects over 20,000 atomic positions for the statistical analysis of important properties for establishing structure-performance relations over nanostructured catalysts, like the surface density, proximity, clustering extent, and dispersion uniformity of supported metal species. Good performance obtained on direct application of the model to an iron SAC based on carbon nitride demonstrates its generalizability for single-atom detection on carbon-related materials. The approach establishes a route to integrate artificial intelligence into routine TEM workflows. It accelerates image processing times by orders of magnitude and reduces human bias by providing an uncertainty analysis that is not readily quantifiable in manual atom identification, improving standardization and scalability.

Toward Totally Defined Nanocatalysis: Deep Learning Reveals the Extraordinary Activity of Single Pd/C Particles

Homogeneous catalysis is typically considered "well-defined" from the standpoint of catalyst structure unambiguity. In contrast, heterogeneous nanocatalysis often falls into the realm of "poorly defined" systems. Supported catalysts are difficult to characterize due to their heterogeneity, variety of morphologies, and large size at the nanoscale. Furthermore, an assortment of active metal nanoparticles examined on the support are negligible compared to those in the bulk catalyst used. To solve these challenges, we studied individual particles of the supported catalyst. We made a significant step forward to fully characterize individual catalyst particles. Combining a nanomanipulation technique inside a field-emission scanning electron microscope with neural network analysis of selected individual particles unexpectedly revealed important aspects of activity for widespread and commercially important Pd/C catalysts. The proposed approach unleashed an unprecedented turnover number of 10⁹ attributed to individual palladium on a nanoglobular carbon particle. Offered in the present study is the Totally Defined Catalysis concept that has tremendous potential for the mechanistic research and development of high-performance catalysts.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.