Big-deep-smart data in imaging for guiding materials design

Scientific Discovery Framework Accelerating Advanced Polymeric Materials Design

Organic polymer materials, as the most abundantly produced materials, possess a flammable nature, making them potential hazards to human casualties and property losses. Target polymer design is still hindered due to the lack of a scientific foundation. Herein, we present a robust, generalizable, yet intelligent polymer discovery framework, which synergizes diverse capabilities, including the in situ burning analyzer, virtual reaction generator, and material genomic model, to achieve results that surpass the sum of individual parts. Notably, the high-throughput analyzer created for the first time, grounded in multiple spectroscopic principles, enables in situ capturing of massive combustion intermediates; then, the created realistic apparatus transforming to the virtual reaction generator acquires exponentially more intermediate information; further, the proposed feature engineering tool, which embedded both polymer hierarchical structures and massive intermediate data, develops the generalizable genomic model with excellent universality (adapting over 20 kinds of polymers) and high accuracy (88.8%), succeeding discovering series of novel polymers. This emerging approach addresses the target polymer design for flame-retardant application and underscores a pivotal role in accelerating polymeric materials discovery.

Machine learning interatomic potential: Bridge the gap between small-scale models and realistic device-scale simulations

Machine learning interatomic potential (MLIP) overcomes the challenges of high computational costs in density-functional theory and the relatively low accuracy in classical large-scale molecular dynamics, facilitating more efficient and precise simulations in materials research and design. In this review, the current state of the four essential stages of MLIP is discussed, including data generation methods, material structure descriptors, six unique machine learning algorithms, and available software. Furthermore, the applications of MLIP in various fields are investigated, notably in phase-change memory materials, structure searching, material properties predicting, and the pre-trained universal models. Eventually, the future perspectives, consisting of standard datasets, transferability, generalization, and trade-off between accuracy and complexity in MLIPs, are reported.

Semantic regularization of electromagnetic inverse problems

Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.

On machine learning analysis of atomic force microscopy images for image classification, sample surface recognition

Atomic force microscopy (AFM or SPM) imaging is one of the best matches with machine learning (ML) analysis among microscopy techniques. The digital format of AFM images allows for direct utilization in ML algorithms without the need for additional processing. Additionally, AFM enables the simultaneous imaging of distributions of over a dozen different physicochemical properties of sample surfaces, a process known as multidimensional imaging. While this wealth of information can be challenging to analyze using traditional methods, ML provides a seamless approach to this task. However, the relatively slow speed of AFM imaging poses a challenge in applying deep learning methods broadly used in image recognition. This prospective is focused on ML recognition/classification when using a relatively small number of AFM images, aka small database. We discuss ML methods other than popular deep-learning neural networks. The described approach has already been successfully used to analyze and classify the surfaces of biological cells. It can be applied to recognize medical images, specific material processing, in forensic studies, even to identify the authenticity of arts. A general template for ML analysis specific to AFM is suggested, with a specific example of the identification of cell phenotype. Special attention is given to the analysis of the statistical significance of the obtained results, an important feature that is often overlooked in papers dealing with machine learning. A simple method for finding statistical significance is also described.

Perspective and prospects of in situ transmission/scanning transmission electron microscopy

In situ transmission/scanning transmission electron microscopy (TEM/STEM) measurements have taken a central stage for establishing structure-chemistry-property relationship over the past couple of decades. The challenges for realizing 'a lab-in-gap', i.e. gap between the objective lens pole pieces, or 'a lab-on-chip', to be used to carry out experiments are being met through continuous instrumental developments. Commercially available TEM columns and sample holder, that have been modified for in situ experimentation, have contributed to uncover structural and chemical changes occurring in the sample when subjected to external stimulus such as temperature, pressure, radiation (photon, ions and electrons), environment (gas, liquid and magnetic or electrical field) or a combination thereof. Whereas atomic resolution images and spectroscopy data are being collected routinely using TEM/STEM, temporal resolution is limited to millisecond. On the other hand, better than femtosecond temporal resolution can be achieved using an ultrafast electron microscopy or dynamic TEM, but the spatial resolution is limited to sub-nanometers. In either case, in situ experiments generate large datasets that need to be transferred, stored and analyzed. The advent of artificial intelligence, especially machine learning platforms, is proving crucial to deal with this big data problem. Further developments are still needed in order to fully exploit our capability to understand, measure and control chemical and/or physical processes. We present the current state of instrumental and computational capabilities and discuss future possibilities.

Cell Surface Parameters for Accessing Neutrophil Activation Level with Atomic Force Microscopy

In this study, we examine the topography and adhesion images of the cell surface of neutrophils during the activation process. Our analysis of cell surface parameters indicates that the most significant changes in neutrophils occur within the first 30 min of activation, suggesting that reactive oxygen species may require approximately this amount of time to activate the cells. Interestingly, we observed surface granular structure as early as 10 min after neutrophil activation when examining atomic force microscopy images. This finding aligns with the reorganization observed within the cells under confocal laser scanning microscopy. By analyzing the cell surface images of adhesion, we identified three spatial surface parameters that correlate with the activation time. This finding enables us to estimate the degree of activation by using atomic force microscopy maps of the cell surface.

EXSCLAIM!: Harnessing materials science literature for self-labeled microscopy datasets

This work introduces the EXSCLAIM! toolkit for the automatic extraction, separation, and caption-based natural language annotation of images from scientific literature. EXSCLAIM! is used to show how rule-based natural language processing and image recognition can be leveraged to construct an electron microscopy dataset containing thousands of keyword-annotated nanostructure images. Moreover, it is demonstrated how a combination of statistical topic modeling and semantic word similarity comparisons can be used to increase the number and variety of keyword annotations on top of the standard annotations from EXSCLAIM! With large-scale imaging datasets constructed from scientific literature, users are well positioned to train neural networks for classification and recognition tasks specific to microscopy-tasks often otherwise inhibited by a lack of sufficient annotated training data.

Deep learning for tumor margin identification in electromagnetic imaging

In this work, a novel method for tumor margin identification in electromagnetic imaging is proposed to optimize the tumor removal surgery. This capability will enable the visualization of the border of the cancerous tissue for the surgeon prior or during the excision surgery. To this end, the border between the normal and tumor parts needs to be identified. Therefore, the images need to be segmented into tumor and normal areas. We propose a deep learning technique which divides the electromagnetic images into two regions: tumor and normal, with high accuracy. We formulate deep learning from a perspective relevant to electromagnetic image reconstruction. A recurrent auto-encoder network architecture (termed here DeepTMI) is presented. The effectiveness of the algorithm is demonstrated by segmenting the reconstructed images of an experimental tissue-mimicking phantom. The structure similarity measure (SSIM) and mean-square-error (MSE) average of normalized reconstructed results by the DeepTMI method are about 0.94 and 0.04 respectively, while that average obtained from the conventional backpropagation (BP) method can hardly overcome 0.35 and 0.41 respectively.

Influence of the Precursor Structure on the Formation of Tungsten Oxide Polymorphs

Understanding material nucleation processes is crucial for the development of synthesis pathways for tailormade materials. However, we currently have little knowledge of the influence of the precursor solution structure on the formation pathway of materials. We here use in situ total scattering to show how the precursor solution structure influences which crystal structure is formed during the hydrothermal synthesis of tungsten oxides. We investigate the synthesis of tungsten oxide from the two polyoxometalate salts, ammonium metatungstate, and ammonium paratungstate. In both cases, a hexagonal ammonium tungsten bronze (NH4)0.25WO3 is formed as the final product. If the precursor solution contains metatungstate clusters, this phase forms directly in the hydrothermal synthesis. However, if the paratungstate B cluster is present at the time of crystallization, a metastable intermediate phase in the form of a pyrochlore-type tungsten oxide, WO3·0.5H2O, initially forms. The pyrochlore structure then undergoes a phase transformation into the tungsten bronze phase. Our studies thus experimentally show that the precursor cluster structure present at the moment of crystallization directly influences the formed crystalline phase and suggests that the precursor structure just prior to crystallization can be used as a tool for targeting specific crystalline phases of interest.

Learning heterogeneous reaction kinetics from X-ray videos pixel by pixel

Reaction rates at spatially heterogeneous, unstable interfaces are notoriously difficult to quantify, yet are essential in engineering many chemical systems, such as batteries1 and electrocatalysts2. Experimental characterizations of such materials by operando microscopy produce rich image datasets3-6, but data-driven methods to learn physics from these images are still lacking because of the complex coupling of reaction kinetics, surface chemistry and phase separation7. Here we show that heterogeneous reaction kinetics can be learned from in situ scanning transmission X-ray microscopy (STXM) images of carbon-coated lithium iron phosphate (LFP) nanoparticles. Combining a large dataset of STXM images with a thermodynamically consistent electrochemical phase-field model, partial differential equation (PDE)-constrained optimization and uncertainty quantification, we extract the free-energy landscape and reaction kinetics and verify their consistency with theoretical models. We also simultaneously learn the spatial heterogeneity of the reaction rate, which closely matches the carbon-coating thickness profiles obtained through Auger electron microscopy (AEM). Across 180,000 image pixels, the mean discrepancy with the learned model is remarkably small (<7%) and comparable with experimental noise. Our results open the possibility of learning nonequilibrium material properties beyond the reach of traditional experimental methods and offer a new non-destructive technique for characterizing and optimizing heterogeneous reactive surfaces.

Universal ion-transport descriptors and classes of inorganic solid-state electrolytes

Solid-state electrolytes (SSEs) with high ion conductivity are pivotal for the development and large-scale adoption of green-energy conversion and storage technologies such as fuel cells, electrocatalysts and solid-state batteries. Yet, SSEs are extremely complex materials for which general rational design principles remain indeterminate. Here, we combine first-principles materials modelling, computational power and modern data analysis techniques to advance towards the solution of such a fundamental and technologically pressing problem. Our data-driven survey reveals that the correlations between ion diffusivity and other materials descriptors in general are monotonic, although not necessarily linear, and largest when the latter are of vibrational nature and explicitly incorporate anharmonic effects. Surprisingly, principal component and k-means clustering analyses show that elastic and vibrational descriptors, rather than the usual ones related to chemical composition and ion mobility, are best suited for reducing the high complexity of SSEs and classifying them into universal classes. Our findings highlight the need for considering databases that incorporate temperature effects to improve our understanding of SSEs and point towards a generalized approach to the design of energy materials.

A bibliometric analysis of inflammatory bowel disease and COVID-19 researches

Background

Patients with inflammatory bowel disease (IBD) often require immunosuppressive therapy and are hence susceptible to various opportunistic viral and bacterial infections. In this regard, many studies on IBD and COVID-19 have been conducted. However, no bibliometric analysis has been performed. This study provides a general overview of IBD and COVID-19.

Methods

Publications about IBD and COVID-19 from 2020 to 2022 were retrieved from the Web of Science Core Collection (WoSCC) database. Bibliometric analysis was performed using VOSviewer, CiteSpace, and HistCite.

Results

A total of 396 publications were retrieved and considered in this study. The maximum number of publications were from the United States, Italy, and England, and the contributions of these countries were significant. Kappelman ranked first in article citations. The Icahn School of Medicine at Mount Sinai and Inflammatory Bowel Diseases were the most prolific affiliation and journal, respectively. The most influential research topics were "management", "impact", "vaccination", and "receptor". The following keywords represented research frontiers: "depression", "the quality of life of IBD patients", "infliximab", "COVID-19 vaccine", and "second vaccination".

Conclusions

Over the past 3 years, most studies on IBD and COVID-19 have focused on clinical research. In particular, topics such as "depression", "the quality of life of IBD patients", "infliximab", "COVID-19 vaccine", and "second vaccination" were noted to have received much attention recently. Future research should focus on our understanding of the immune response to COVID-19 vaccination in biologically treated patients, the psychological impact of COVID-19, IBD management guidelines, and the long-term impact of COVID-19 in IBD patients. This study will provide researchers with a better understanding of research trends on IBD during COVID-19.

Presence and Absence of Barren Plateaus in Tensor-Network Based Machine Learning

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

Bayesian modeling of pattern formation from one snapshot of pattern

Partial differential equations (PDEs) have been widely used to reproduce patterns in nature and to give insight into the mechanism underlying pattern formation. Although many PDE models have been proposed, they rely on the pre-request knowledge of physical laws and symmetries, and developing a model to reproduce a given desired pattern remains difficult. We propose a method, referred to as Bayesian modeling of PDEs (BM-PDEs), to estimate the best dynamical PDE for one snapshot of a objective pattern under the stationary state without ground truth. We apply BM-PDEs to nontrivial patterns, such as quasicrystals (QCs), a double gyroid, and Frank-Kasper structures. We also generate three-dimensional dodecagonal QCs from a PDE model. This is done by using the estimated parameters for the Frank-Kasper A15 structure, which closely approximates the local structures of QCs. Our method works for noisy patterns and the pattern synthesized without the ground-truth parameters, which are required for the application toward experimental data.

Dynamic Modeling of Intrinsic Self-Healing Polymers Using Deep Learning

The properties of self-healing polymers are traditionally identified through destructive testing. This means that the mechanics are explored in hindsight with either theoretical derivations and/or simulations. Here, a self-healing property evolution using energy functional dynamical (SPEED) model is proposed to predict and understand the mechanics of self-healing of polymers using images of cuts dynamically healing over time. Using machine learning, an energy functional minimization (EFM) model extracted an effective underlying dynamical system from a time series of two-dimensional cut images on a self-healing polymer of constant thickness. This model can be used to capture the physics behind the self-healing dynamics in terms of potential and interface energies. When combined with a static property prediction model, the SPEED model can predict the macroscopic evolution of material properties after training only on a small set of experimental measurements. Such temporal evolutions are usually inaccessible from pure experiments or computational modeling due to the need for destructive testing. As an example, we validate this approach on toughness measurements of an intrinsic self-healing conductive polymer by capturing over 100 000 image frames of cuts to build the machine learning (ML) model. The results show that the SPEED model can be applied to predict the temporal evolution of macroscopic properties using few measurements as training data.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

A Robust Neural Network for Extracting Dynamics from Electrostatic Force Microscopy Data

Advances in scanning probe microscopy (SPM) methods such as time-resolved electrostatic force microscopy (trEFM) now permit the mapping of fast local dynamic processes with high resolution in both space and time, but such methods can be time-consuming to analyze and calibrate. Here, we design and train a regression neural network (NN) that accelerates and simplifies the extraction of local dynamics from SPM data directly in a cantilever-independent manner, allowing the network to process data taken with different cantilevers. We validate the NN's ability to recover local dynamics with a fidelity equal to or surpassing conventional, more time-consuming, calibrations using both simulated and real microscopy data. We apply this method to extract accurate photoinduced carrier dynamics on n = 1 butylammonium lead iodide, a halide perovskite semiconductor film that is of interest for applications in both solar photovoltaics and quantum light sources. Finally, we use SHapley Additive exPlanations to evaluate the robustness of the trained model, confirm its cantilever-independence, and explore which parts of the trEFM signal are important to the network.

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Artificial intelligence (AI) promises to reshape scientific inquiry and enable breakthrough discoveries in areas such as energy storage, quantum computing, and biomedicine. Scanning transmission electron microscopy (STEM), a cornerstone of the study of chemical and materials systems, stands to benefit greatly from AI-driven automation. However, present barriers to low-level instrument control, as well as generalizable and interpretable feature detection, make truly automated microscopy impractical. Here, we discuss the design of a closed-loop instrument control platform guided by emerging sparse data analytics. We hypothesize that a centralized controller, informed by machine learning combining limited a priori knowledge and task-based discrimination, could drive on-the-fly experimental decision-making. This platform may unlock practical, automated analysis of a variety of material features, enabling new high-throughput and statistical studies.

Enhanced Cellular Materials through Multiscale, Variable-Section Inner Designs: Mechanical Attributes and Neural Network Modeling

In the current work, the mechanical response of multiscale cellular materials with hollow variable-section inner elements is analyzed, combining experimental, numerical and machine learning techniques. At first, the effect of multiscale designs on the macroscale material attributes is quantified as a function of their inner structure. To that scope, analytical, closed-form expressions for the axial and bending inner element-scale stiffness are elaborated. The multiscale metamaterial performance is numerically probed for variable-section, multiscale honeycomb, square and re-entrant star-shaped lattice architectures. It is observed that a substantial normal, bulk and shear specific stiffness increase can be achieved, which differs depending on the upper-scale lattice pattern. Subsequently, extended mechanical datasets are created for the training of machine learning models of the metamaterial performance. Thereupon, neural network (NN) architectures and modeling parameters that can robustly capture the multiscale material response are identified. It is demonstrated that rather low-numerical-cost NN models can assess the complete set of elastic properties with substantial accuracy, providing a direct link between the underlying design parameters and the macroscale metamaterial performance. Moreover, inverse, multi-objective engineering tasks become feasible. It is shown that unified machine-learning-based representation allows for the inverse identification of the inner multiscale structural topology and base material parameters that optimally meet multiple macroscale performance objectives, coupling the NN metamaterial models with genetic algorithm-based optimization schemes.

Correlative image learning of chemo-mechanics in phase-transforming solids

Constitutive laws underlie most physical processes in nature. However, learning such equations in heterogeneous solids (for example, due to phase separation) is challenging. One such relationship is between composition and eigenstrain, which governs the chemo-mechanical expansion in solids. Here we developed a generalizable, physically constrained image-learning framework to algorithmically learn the chemo-mechanical constitutive law at the nanoscale from correlative four-dimensional scanning transmission electron microscopy and X-ray spectro-ptychography images. We demonstrated this approach on Li_XFePO₄, a technologically relevant battery positive electrode material. We uncovered the functional form of the composition-eigenstrain relation in this two-phase binary solid across the entire composition range (0 ≤ X ≤ 1), including inside the thermodynamically unstable miscibility gap. The learned relation directly validates Vegard's law of linear response at the nanoscale. Our physics-constrained data-driven approach directly visualizes the residual strain field (by removing the compositional and coherency strain), which is otherwise impossible to quantify. Heterogeneities in the residual strain arise from misfit dislocations and were independently verified by X-ray diffraction line profile analysis. Our work provides the means to simultaneously quantify chemical expansion, coherency strain and dislocations in battery electrodes, which has implications on rate capabilities and lifetime. Broadly, this work also highlights the potential of integrating correlative microscopy and image learning for extracting material properties and physics.

Automated Image Analysis for Single-Atom Detection in Catalytic Materials by Transmission Electron Microscopy

Single-atom catalytic sites may have existed in all supported transition metal catalysts since their first application. Yet, interest in the design of single-atom heterogeneous catalysts (SACs) only really grew when advances in transmission electron microscopy (TEM) permitted direct confirmation of metal site isolation. While atomic-resolution imaging remains a central characterization tool, poor statistical significance, reproducibility, and interoperability limit its scope for deriving robust characteristics about these frontier catalytic materials. Here, we introduce a customized deep-learning method for automated atom detection in image analysis, a rate-limiting step toward high-throughput TEM. Platinum atoms stabilized on a functionalized carbon support with a challenging irregular three-dimensional morphology serve as a practically relevant test system with promising scope in thermo- and electrochemical applications. The model detects over 20,000 atomic positions for the statistical analysis of important properties for establishing structure-performance relations over nanostructured catalysts, like the surface density, proximity, clustering extent, and dispersion uniformity of supported metal species. Good performance obtained on direct application of the model to an iron SAC based on carbon nitride demonstrates its generalizability for single-atom detection on carbon-related materials. The approach establishes a route to integrate artificial intelligence into routine TEM workflows. It accelerates image processing times by orders of magnitude and reduces human bias by providing an uncertainty analysis that is not readily quantifiable in manual atom identification, improving standardization and scalability.

Study of crystal properties based on attention mechanism and crystal graph convolutional neural network

The prediction of crystal properties has always been limited by huge computational costs. In recent years, the rise of machine learning methods has gradually made it possible to study crystal properties on a large scale. We propose an attention mechanism-based crystal graph convolutional neural network, which builds a machine learning model by inputting crystallographic information files and target properties. In our research, the attention mechanism is introduced in the crystal graph convolutional neural network (CGCNN) to learn the local chemical environment, and node normalization is added to reduce the risk of overfitting. We collect structural information and calculation data of about 36 000 crystals and examine the prediction performance of the models for the formation energy, total energy, bandgap, and Fermi energy of crystals in our research. Compared with the CGCNN, it is found that the accuracy (ACCU) of the predicted properties can be further improved to varying degrees by the introduction of the attention mechanism. Moreover, the total magnetization and bandgap can be classified under the same neural network framework. The classification ACCU of wide bandgap semiconductor crystals with a bandgap threshold of 2.3 eV reaches 93.2%, and the classification ACCU of crystals with a total magnetization threshold of 0.5 μ_Breaches 88.8%. The work is helpful to realize large-scale prediction and classification of crystal properties, accelerating the discovery of new functional crystal materials.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

Simple Setup Miniaturization with Multiple Benefits for Green Chemistry in Nanoparticle Synthesis

The development of nanomaterials often relies on wet-chemical synthesis performed in reflux setups using round-bottom flasks. Here, an alternative approach to synthesize nanomaterials is presented that uses glass tubes designed for NMR analysis as reactors. This approach uses less solvent and energy, generates less waste, provides safer conditions, is less prone to contamination, and is compatible with high-throughput screening. The benefits of this approach are illustrated by an in breadth study with the synthesis of gold, iridium, osmium, and copper sulfide nanoparticles.

Electrostatic Discovery Atomic Force Microscopy

While offering high resolution atomic and electronic structure, scanning probe microscopy techniques have found greater challenges in providing reliable electrostatic characterization on the same scale. In this work, we offer electrostatic discovery atomic force microscopy, a machine learning based method which provides immediate maps of the electrostatic potential directly from atomic force microscopy images with functionalized tips. We apply this to characterize the electrostatic properties of a variety of molecular systems and compare directly to reference simulations, demonstrating good agreement. This approach offers reliable atomic scale electrostatic maps on any system with minimal computational overhead.

Data-Driven Methods for Accelerating Polymer Design

Optimal design of polymers is a challenging task due to their enormous chemical and configurational space. Recent advances in computations, machine learning, and increasing trends in data and software availability can potentially address this problem and accelerate the molecular-scale design of polymers. Here, the central problem of polymer design is reviewed, and the general ideas of data-driven methods and their working principles in the context of polymer design are discussed. This Review provides a historical perspective and a summary of current trends and outlines future scopes of data-driven methods for polymer research. A few representative case studies on the use of such data-driven methods for discovering new polymers with exceptional properties are presented. Moreover, attempts are made to highlight how data-driven strategies aid in establishing new correlations and advancing the fundamental understanding of polymers. This Review posits that the combination of machine learning, rapid computational characterization of polymers, and availability of large open-sourced homogeneous data will transform polymer research and development over the coming decades. It is hoped that this Review will serve as a useful reference to researchers who wish to develop and deploy data-driven methods for polymer research and education.

Machine-Learning Microstructure for Inverse Material Design

Metallurgy and material design have thousands of years' history and have played a critical role in the civilization process of humankind. The traditional trial-and-error method has been unprecedentedly challenged in the modern era when the number of components and phases in novel alloys keeps increasing, with high-entropy alloys as the representative. New opportunities emerge for alloy design in the artificial intelligence era. Here, a successful machine-learning (ML) method has been developed to identify the microstructure images with eye-challenging morphology for a number of martensitic and ferritic steels. Assisted by it, a new neural-network method is proposed for the inverse design of alloys with 20 components, which can accelerate the design process based on microstructure. The method is also readily applied to other material systems given sufficient microstructure images. This work lays the foundation for inverse alloy design based on microstructure images with extremely similar features.

Optical Fingerprint Classification of Single Upconversion Nanoparticles by Deep Learning

Highly controlled synthesis of upconversion nanoparticles (UCNPs) can be achieved in the heterogeneous design, so that a library of optical properties can be arbitrarily produced by depositing multiple lanthanide ions. Such a control offers the potential in creating nanoscale barcodes carrying high-capacity information. With the increasing creation of optical information, it poses more challenges in decoding them in an accurate, high-throughput, and speedy fashion. Here, we reported that the deep-learning approach can recognize the complexity of the optical fingerprints from different UCNPs. Under a wide-field microscope, the lifetime profiles of hundreds of single nanoparticles can be collected at once, which offers a sufficient amount of data to develop deep-learning algorithms. We demonstrated that high accuracies of over 90% can be achieved in classifying 14 kinds of UCNPs. This work suggests new opportunities in handling the diverse properties of nanoscale optical barcodes toward the establishment of vast luminescent information carriers.

Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics to self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiments (AE) in imaging. Here, we aim to analyze the major pathways toward AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment and consider the latencies, biases, and prior knowledge of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities, and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning. Overall, we argue that ML/AI can dramatically alter the (S)TEM and SPM fields; however, this process is likely to be highly nontrivial and initiated by combined human-ML workflows and will bring challenges both from the microscope and ML/AI sides. At the same time, these methods will enable opportunities and paradigms for scientific discovery and nanostructure fabrication.

Advances and Applications of Atomic-Resolution Scanning Transmission Electron Microscopy

Although scanning transmission electron microscopy (STEM) images of individual heavy atoms were reported 50 years ago, the applications of atomic-resolution STEM imaging became wide spread only after the practical realization of aberration correctors on field-emission STEM/TEM instruments to form sub-Ångstrom electron probes. The innovative designs and advances of electron optical systems, the fundamental understanding of electron–specimen interaction processes, and the advances in detector technology all played a major role in achieving the goal of atomic-resolution STEM imaging of practical materials. It is clear that tremendous advances in computer technology and electronics, image acquisition and processing algorithms, image simulations, and precision machining synergistically made atomic-resolution STEM imaging routinely accessible. It is anticipated that further hardware/software development is needed to achieve three-dimensional atomic-resolution STEM imaging with single-atom chemical sensitivity, even for electron-beam-sensitive materials. Artificial intelligence, machine learning, and big-data science are expected to significantly enhance the impact of STEM and associated techniques on many research fields such as materials science and engineering, quantum and nanoscale science, physics and chemistry, and biology and medicine. This review focuses on advances of STEM imaging from the invention of the field-emission electron gun to the realization of aberration-corrected and monochromated atomic-resolution STEM and its broad applications.

Phase Transition Effect on Ferroelectric Domain Surface Charge Dynamics in BaTiO3 Single Crystal

The ferroelectric domain surface charge dynamics after a cubic-to-tetragonal phase transition on the BaTiO₃ single crystal (001) surface was directly measured through scanning probe microscopy. The captured surface potential distribution shows significant changes: the domain structures formed rapidly, but the surface potential on polarized c domain was unstable and reversed its sign after lengthy lapse; the high broad potential barrier burst at the corrugated a-c domain wall and continued to dissipate thereafter. The generation of polarization charges and the migration of surface screening charges in the surrounding environment take the main responsibility in the experiment. Furthermore, the a-c domain wall suffers large topological defects and polarity variation, resulting in domain wall broadening and stress changes. Thus, the a-c domain wall has excess energy and polarization change is inclined to assemble on it. The potential barrier decay with time after exposing to the surrounding environment also gave proof of the surface screening charge migration at surface. Thus, both domain and domain wall characteristics should be taken into account in ferroelectric application.

AI in Measurement Science

Measurement of biological systems containing biomolecules and bioparticles is a key task in the fields of analytical chemistry, biology, and medicine. Driven by the complex nature of biological systems and unprecedented amounts of measurement data, artificial intelligence (AI) in measurement science has rapidly advanced from the use of silicon-based machine learning (ML) for data mining to the development of molecular computing with improved sensitivity and accuracy. This review presents an overview of fundamental ML methodologies and discusses their applications in disease diagnostics, biomarker discovery, and imaging analysis. We next provide the working principles of molecular computing using logic gates and arithmetical devices, which can be employed for in situ detection, computation, and signal transduction for biological systems. This review concludes by summarizing the strengths and limitations of AI-involved biological measurement in fundamental and applied research.

Deep Learning Method to Accelerate Discovery of Hybrid Polymer-Graphene Composites

Interfacial encoded properties of polymer adlayers adsorbed on the graphene (GE) and silicon dioxide (SiO2) have been constituted a scaffold for the creation of new materials. The holistic understanding of nanoscale intermolecular interaction of 1D/2D polymer assemblies on substrate is the key to bottom-up design of molecular devices. We develop an integrated multidisciplinary approach based on electronic structure computation [density functional theory (DFT)] and big data mining [machine learning (ML)] in parallel with neural network (NN) and statistical analysis (SA) to design hybrid polymers from assembly on substrate. Here we demonstrate that interfacial pressure and structural deformation of polymer network adsorbed on GE and SiO2 offer unique directions for the fabrication of 1D/2D polymers using only a small number of simple molecular building blocks. Our findings serve as the platform for designing a wide range of typical inorganic heterostructures, involving noncovalent intermolecular interaction observed in many nanoscale electronic devices.

Fast Label-Free Nanoscale Composition Mapping of Eukaryotic Cells Via Scanning Dielectric Force Volume Microscopy and Machine Learning

Mapping the biochemical composition of eukaryotic cells without the use of exogenous labels is a long-sought objective in cell biology. Recently, it has been shown that composition maps on dry single bacterial cells with nanoscale spatial resolution can be inferred from quantitative nanoscale dielectric constant maps obtained with the scanning dielectric microscope. Here, it is shown that this approach can also be applied to the much more challenging case of fixed and dry eukaryotic cells, which are highly heterogeneous and show micrometric topographic variations. More importantly, it is demonstrated that the main bottleneck of the technique (the long computation times required to extract the nanoscale dielectric constant maps) can be shortcut by using supervised neural networks, decreasing them from weeks to seconds in a wokstation computer. This easy-to-use data-driven approach opens the door for in situ and on-the-fly label free nanoscale composition mapping of eukaryotic cells with scanning dielectric microscopy.

Hystorian: A processing tool for scanning probe microscopy and other n-dimensional datasets

Research in materials science increasingly depends on the correlation of information from multiple characterisation techniques, acquired in ever larger datasets. Efficient methods of processing and storing these complex datasets are therefore crucial. Reliably keeping track of data processing is also essential to conform with the goals of open science. Here, we introduce Hystorian, a generic materials science data analysis Python package built at its core to improve the traceability, reproducibility, and archival ability of data processing. Proprietary data formats are converted into open hierarchical data format (HDF5) files, with both datasets and subsequent workflows automatically stored into a single location, thus allowing easy management of multiple data types. At present, Hystorian provides a basic scanning probe microscopy and x-ray diffraction analysis toolkit, and is readily extensible to suit user needs. It is also able to wrap over any existing processing functions, making it easy to append in an extant workflow.

Unsupervised Learning of Non-Hermitian Topological Phases

Non-Hermitian topological phases bear a number of exotic properties, such as the non-Hermitian skin effect and the breakdown of conventional bulk-boundary correspondence. In this Letter, we introduce an unsupervised machine learning approach to classify non-Hermitian topological phases based on diffusion maps, which are widely used in manifold learning. We find that the non-Hermitian skin effect will pose a notable obstacle, rendering the straightforward extension of unsupervised learning approaches to topological phases for Hermitian systems ineffective in clustering non-Hermitian topological phases. Through theoretical analysis and numerical simulations of two prototypical models, we show that this difficulty can be circumvented by choosing the "on-site" elements of the projective matrix as the input data. Our results provide a valuable guidance for future studies on learning non-Hermitian topological phases in an unsupervised fashion, both in theory and experiment.

Predicting hydration layers on surfaces using deep learning

Characterisation of the nanoscale interface formed between minerals and water is essential to the understanding of natural processes, such as biomineralization, and to develop new technologies where function is dominated by the mineral-water interface. Atomic force microscopy offers the potential to characterize solid-liquid interfaces in high-resolution, with several experimental and theoretical studies offering molecular scale resolution by linking measurements directly to water density on the surface. However, the theoretical techniques used to interpret such results are computationally intensive and development of the approach has been limited by interpretation challenges. In this work, we develop a deep learning architecture to learn the solid-liquid interface of polymorphs of calcium carbonate, allowing for the rapid predictions of density profiles with reasonable accuracy.

Progress and Roadmap for Intelligent Self-Healing Materials in Autonomous Robotics

Robots are increasingly assisting humans in performing various tasks. Like special agents with elite skills, they can venture to distant locations and adverse environments, such as the deep sea and outer space. Micro/nanobots can also act as intrabody agents for healthcare applications. Self-healing materials that can autonomously perform repair functions are useful to address the unpredictability of the environment and the increasing drive toward the autonomous operation. Having self-healable robotic materials can potentially reduce costs, electronic wastes, and improve a robot endowed with such materials longevity. This review aims to serve as a roadmap driven by past advances and inspire future cross-disciplinary research in robotic materials and electronics. By first charting the history of self-healing materials, new avenues are provided to classify the various self-healing materials proposed over several decades. The materials and strategies for self-healing in robotics and stretchable electronics are also reviewed and discussed. It is believed that this article encourages further innovation in this exciting and emerging branch in robotics interfacing with material science and electronics.

Reducing Time to Discovery: Materials and Molecular Modeling, Imaging, Informatics, and Integration

Multiscale and multimodal imaging of material structures and properties provides solid ground on which materials theory and design can flourish. Recently, KAIST announced 10 flagship research fields, which include KAIST Materials Revolution: Materials and Molecular Modeling, Imaging, Informatics and Integration (M3I3). The M3I3 initiative aims to reduce the time for the discovery, design and development of materials based on elucidating multiscale processing-structure-property relationship and materials hierarchy, which are to be quantified and understood through a combination of machine learning and scientific insights. In this review, we begin by introducing recent progress on related initiatives around the globe, such as the Materials Genome Initiative (U.S.), Materials Informatics (U.S.), the Materials Project (U.S.), the Open Quantum Materials Database (U.S.), Materials Research by Information Integration Initiative (Japan), Novel Materials Discovery (E.U.), the NOMAD repository (E.U.), Materials Scientific Data Sharing Network (China), Vom Materials Zur Innovation (Germany), and Creative Materials Discovery (Korea), and discuss the role of multiscale materials and molecular imaging combined with machine learning in realizing the vision of M3I3. Specifically, microscopies using photons, electrons, and physical probes will be revisited with a focus on the multiscale structural hierarchy, as well as structure-property relationships. Additionally, data mining from the literature combined with machine learning will be shown to be more efficient in finding the future direction of materials structures with improved properties than the classical approach. Examples of materials for applications in energy and information will be reviewed and discussed. A case study on the development of a Ni-Co-Mn cathode materials illustrates M3I3's approach to creating libraries of multiscale structure-property-processing relationships. We end with a future outlook toward recent developments in the field of M3I3.

Complex-Valued Pix2pix—Deep Neural Network for Nonlinear Electromagnetic Inverse Scattering

Nonlinear electromagnetic inverse scattering is an imaging technique with quantitative reconstruction and high resolution. Compared with conventional tomography, it takes into account the more realistic interaction between the internal structure of the scene and the electromagnetic waves. However, there are still open issues and challenges due to its inherent strong non-linearity, ill-posedness and computational cost. To overcome these shortcomings, we apply an image translation network, named as Complex-Valued Pix2pix, on the inverse scattering problem of electromagnetic field. Complex-Valued Pix2pix includes two parts of Generator and Discriminator. The Generator employs a multi-layer complex valued convolutional neural network, while the Discriminator computes the maximum likelihoods between the original value and the reconstructed value from the aspects of the two parts of the complex: real part and imaginary part, respectively. The results show that the Complex-Valued Pix2pix can learn the mapping from the initial contrast to the real contrast in microwave imaging models. Moreover, due to the introduction of discriminator, Complex-Valued Pix2pix can capture more features of nonlinearity than traditional Convolutional Neural Network (CNN) by confrontation training. Therefore, without considering the time cost of training, Complex-Valued Pix2pix may be a more effective way to solve inverse scattering problems than other deep learning methods. The main improvement of this work lies in the realization of a Generative Adversarial Network (GAN) in the electromagnetic inverse scattering problem, adding a discriminator to the traditional Convolutional Neural Network (CNN) method to optimize network training. It has the prospect of outperforming conventional methods in terms of both the image quality and computational efficiency.

Big data and machine learning for materials science

Herein, we review aspects of leading-edge research and innovation in materials science that exploit big data and machine learning (ML), two computer science concepts that combine to yield computational intelligence. ML can accelerate the solution of intricate chemical problems and even solve problems that otherwise would not be tractable. However, the potential benefits of ML come at the cost of big data production; that is, the algorithms demand large volumes of data of various natures and from different sources, from material properties to sensor data. In the survey, we propose a roadmap for future developments with emphasis on computer-aided discovery of new materials and analysis of chemical sensing compounds, both prominent research fields for ML in the context of materials science. In addition to providing an overview of recent advances, we elaborate upon the conceptual and practical limitations of big data and ML applied to materials science, outlining processes, discussing pitfalls, and reviewing cases of success and failure.

Non-negative matrix factorization for mining big data obtained using four-dimensional scanning transmission electron microscopy

Scientific instruments for material characterization have recently been improved to yield big data. For instance, scanning transmission electron microscopy (STEM) allows us to acquire many diffraction patterns from a scanning area, which is referred to as four-dimensional (4D) STEM. Here we study a combination of 4D-STEM and a statistical technique called non-negative matrix factorization (NMF) to deduce sparse diffraction patterns from a 4D-STEM data consisting of 10,000 diffraction patterns. Titanium oxide nanosheets are analyzed using this combined technique, and we discriminate the two diffraction patterns from pristine TiO₂ and reduced Ti₂O₃ areas, where the latter is due to topotactic reduction induced by electron irradiation. The combination of NMF and 4D-STEM is expected to become a standard characterization technique for a wide range materials.

Photoexcitation Induced Ultrafast Nonthermal Amorphization in Sb2Te3

Phase-change materials are of great interest for low-power high-throughput storage devices in next-generation neuromorphic computing technologies. Their operation is based on the contrasting properties of their amorphous and crystalline phases, which can be switched on the nanosecond time scale. Among the archetypal phase change materials based on Ge-Sb-Te alloys, Sb₂Te₃ displays a fast and energy-efficient crystallization-amorphization cycle due to its growth-dominated crystallization and low melting point. This growth-dominated crystallization contrasts with the nucleation-dominated crystallization of Ge₂Sb₂Te₅. Here, we show that the energy required for and the time associated with the amorphization process can be further reduced by using a photoexcitation-based nonthermal path. We employ nonadiabatic quantum molecular dynamics simulations to investigate the time evolution of Sb₂Te₃ with 2.6, 5.2, 7.5, 10.3, and 12.5% photoexcited valence electron-hole carriers. Results reveal that the degree of amorphization increases with excitation, saturating at 10.3% excitation. The rapid amorphization originates from an instantaneous charge transfer from Te-p orbitals to Sb-p orbitals upon photoexcitation. Subsequent evolution of the excited state, within the picosecond time scale, indicates an Sb-Te bonding to antibonding transition. Concurrently, Sb-Sb and Te-Te antibonding decreases, leading to formation of wrong bonds. For photoexcitation of 7.5% valence electrons or larger, the electronic changes destabilize the crystal structure, leading to large atomic diffusion and irreversible loss of long-range order. These results highlight an ultrafast energy-efficient amorphization pathway that could be used to enhance the performance of phase change material-based optoelectronic devices.

A deep-learning approach to realizing functionality in nanoelectronic devices

Many nanoscale devices require precise optimization to function. Tuning them to the desired operation regime becomes increasingly difficult and time-consuming when the number of terminals and couplings grows. Imperfections and device-to-device variations hinder optimization that uses physics-based models. Deep neural networks (DNNs) can model various complex physical phenomena but, so far, are mainly used as predictive tools. Here, we propose a generic deep-learning approach to efficiently optimize complex, multi-terminal nanoelectronic devices for desired functionality. We demonstrate our approach for realizing functionality in a disordered network of dopant atoms in silicon. We model the input-output characteristics of the device with a DNN, and subsequently optimize control parameters in the DNN model through gradient descent to realize various classification tasks. When the corresponding control settings are applied to the physical device, the resulting functionality is as predicted by the DNN model. We expect our approach to contribute to fast, in situ optimization of complex (quantum) nanoelectronic devices.

Machine learning approach for elucidating and predicting the role of synthesis parameters on the shape and size of TiO2 nanoparticles

In the present work a series of design rules are developed in order to tune the morphology of TiO₂ nanoparticles through hydrothermal process. Through a careful experimental design, the influence of relevant process parameters on the synthesis outcome are studied, reaching to the develop predictive models by using Machine Learning methods. The models, after the validation and training, are able to predict with high accuracy the synthesis outcome in terms of nanoparticle size, polydispersity and aspect ratio. Furthermore, they are implemented by reverse engineering approach to do the inverse process, i.e. obtain the optimal synthesis parameters given a specific product characteristic. For the first time, it is presented a synthesis method that allows continuous and precise control of NPs morphology with the possibility to tune the aspect ratio over a large range from 1.4 (perfect truncated bipyramids) to 6 (elongated nanoparticles) and the length from 20 to 140 nm.

K-means-driven Gaussian Process data collection for angle-resolved photoemission spectroscopy

We propose the combination of k-means clustering with Gaussian Process (GP) regression in the analysis and exploration of 4D angle-resolved photoemission spectroscopy (ARPES) data. Using cluster labels as the driving metric on which the GP is trained, this method allows us to reconstruct the experimental phase diagram from as low as 12% of the original dataset size. In addition to the phase diagram, the GP is able to reconstruct spectra in energy-momentum space from this minimal set of data points. These findings suggest that this methodology can be used to improve the efficiency of ARPES data collection strategies for unknown samples. The practical feasibility of implementing this technology at a synchrotron beamline and the overall efficiency implications of this method are discussed with a view on enabling the collection of more samples or rapid identification of regions of interest.