Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

AEcroscopy: A Software-Hardware Framework Empowering Microscopy Toward Automated and Autonomous Experimentation

Microscopy has been pivotal in improving the understanding of structure-function relationships at the nanoscale and is by now ubiquitous in most characterization labs. However, traditional microscopy operations are still limited largely by a human-centric click-and-go paradigm utilizing vendor-provided software, which limits the scope, utility, efficiency, effectiveness, and at times reproducibility of microscopy experiments. Here, a coupled software-hardware platform is developed that consists of a software package termed AEcroscopy (short for Automated Experiments in Microscopy), along with a field-programmable-gate-array device with LabView-built customized acquisition scripts, which overcome these limitations and provide the necessary abstractions toward full automation of microscopy platforms. The platform works across multiple vendor devices on scanning probe microscopes and electron microscopes. It enables customized scan trajectories, processing functions that can be triggered locally or remotely on processing servers, user-defined excitation waveforms, standardization of data models, and completely seamless operation through simple Python commands to enable a plethora of microscopy experiments to be performed in a reproducible, automated manner. This platform can be readily coupled with existing machine-learning libraries and simulations, to provide automated decision-making and active theory-experiment optimization to turn microscopes from characterization tools to instruments capable of autonomous model refinement and physics discovery.

Bayesian Conavigation: Dynamic Designing of the Material Digital Twins via Active Learning

Scientific advancement is universally based on the dynamic interplay between theoretical insights, modeling, and experimental discoveries. However, this feedback loop is often slow, including delayed community interactions and the gradual integration of experimental data into theoretical frameworks. This challenge is particularly exacerbated in domains dealing with high-dimensional object spaces, such as molecules and complex microstructures. Hence, the integration of theory within automated and autonomous experimental setups, or theory in the loop-automated experiment, is emerging as a crucial objective for accelerating scientific research. The critical aspect is to use not only theory but also on-the-fly theory updates during the experiment. Here, we introduce a method for integrating theory into the loop through Bayesian conavigation of theoretical model space and experimentation. Our approach leverages the concurrent development of surrogate models for both simulation and experimental domains at the rates determined by latencies and costs of experiments and computation, alongside the adjustment of control parameters within theoretical models to minimize epistemic uncertainty over the experimental object spaces. This methodology facilitates the creation of digital twins of material structures, encompassing both the surrogate model of behavior that includes the correlative part and the theoretical model itself. While being demonstrated here within the context of functional responses in ferroelectric materials, our approach holds promise for broader applications, such as the exploration of optical properties in nanoclusters, microstructure-dependent properties in complex materials, and properties of molecular systems.

Ion adsorption-enrichment effect and its driving mechanism for nano-dots lithography with SPM probe on water-soluble crystal surfaces

HYPOTHESIS

Water-soluble KDP (KH2PO4) crystals possess excellent optical properties and are employed as frequency converters in clean fusion energy. To improve their performances, there is an immediate necessity to lithograph surface nano-patterns on them. Although the Scanning Probe Microscope (SPM) provides a promising way to achieve this purpose through the water menisci, the driving mechanisms of the lithographic behaviors have not yet been revealed.

SIMULATIONS AND EXPERIMENTS

Multi-scale investigations are constructed to explore the underlying driving mechanisms. The SPM probe-induced ion diffusion-transport behaviors are investigated by molecular dynamics. The ion adsorption-enrichment mechanisms are revealed by 18 adsorption models via the ab initio. The SPM probe-induced self-assembly experiments are performed to prove the local heavy concentration. A comprehensive model is developed to describe the lithography mechanisms of the probe-induced self-assembly nano-dots on water-soluble substrates.

FINDINGS

It is interestingly found that the KDP growth units (H2PO4-) exhibit obvious adsorption-enrichment effect at 3.16 Å from the probe surface, causing local heavy concentration. The H2PO4- would spontaneously adsorb onto the probe surface, which is dominated by the Si-O bonding reactions. The nano-dots with the height of 27 ∼ 48 nm and diameter of 2.0 ∼ 2.7 μm are lithographed on the KDP substrate. The proposed model further confirms that the lithography processes are driven by the solution supersaturation, solute diffusion, and surface free energy.

Self-Driving Laboratories for Chemistry and Materials Science

Self-driving laboratories (SDLs) promise an accelerated application of the scientific method. Through the automation of experimental workflows, along with autonomous experimental planning, SDLs hold the potential to greatly accelerate research in chemistry and materials discovery. This review provides an in-depth analysis of the state-of-the-art in SDL technology, its applications across various scientific disciplines, and the potential implications for research and industry. This review additionally provides an overview of the enabling technologies for SDLs, including their hardware, software, and integration with laboratory infrastructure. Most importantly, this review explores the diverse range of scientific domains where SDLs have made significant contributions, from drug discovery and materials science to genomics and chemistry. We provide a comprehensive review of existing real-world examples of SDLs, their different levels of automation, and the challenges and limitations associated with each domain.

Nanoscale core-shell structure and recrystallization of swift heavy ion tracks in SrTiO3

It is widely accepted that the interaction of swift heavy ions with many complex oxides is predominantly governed by the electronic energy loss that gives rise to nanoscale amorphous ion tracks along the penetration direction. The question of how electronic excitation and electron-phonon coupling affect the atomic system through defect production, recrystallization, and strain effects has not yet been fully clarified. To advance the knowledge of the atomic structure of ion tracks, we irradiated single crystalline SrTiO3 with 629 MeV Xe ions and performed comprehensive electron microscopy investigations complemented by molecular dynamics simulations. This study shows discontinuous ion-track formation along the ion penetration path, comprising an amorphous core and a surrounding few monolayer thick shell of strained/defective crystalline SrTiO3. Using machine-learning-aided analysis of atomic-scale images, we demonstrate the presence of 4-8% strain in the disordered region interfacing with the amorphous core in the initially formed ion tracks. Under constant exposure of the electron beam during imaging, the amorphous part of the ion tracks readily recrystallizes radially inwards from the crystalline-amorphous interface under the constant electron-beam irradiation during the imaging. Cation strain in the amorphous region is observed to be significantly recovered, while the oxygen sublattice remains strained even under the electron irradiation due to the present oxygen vacancies. The molecular dynamics simulations support this observation and suggest that local transient heating and annealing facilitate recrystallization process of the amorphous phase and drive Sr and Ti sublattices to rearrange. In contrast, the annealing of O atoms is difficult, thus leaving a remnant of oxygen vacancies and strain even after recrystallization. This work provides insights for creating and transforming novel interfaces and nanostructures for future functional applications.

Evaluating Stage Motion for Automated Electron Microscopy

Precise control is an essential and elusive quality of emerging self-driving transmission electron microscopes (TEMs). It is widely understood these instruments must be capable of performing rapid, high-volume, and arbitrary movements for practical self-driving operation. However, stage movements are difficult to automate at scale, owing to mechanical instability, hysteresis, and thermal drift. Such difficulties pose major barriers to artificial intelligence-directed microscope designs that require repeatable, precise movements. To guide design of emerging instruments, it is necessary to understand the behavior of existing mechanisms to identify rate limiting steps for full autonomy. Here, we describe a general framework to evaluate stage motion in any TEM. We define metrics to evaluate stage degrees of freedom, propose solutions to improve performance, and comment on fundamental limits to automated experimentation using present hardware.

Accelerating the Design of Multishell Upconverting Nanoparticles through Bayesian Optimization

The photon upconverting properties of lanthanide-doped nanoparticles drive their applications in imaging, optoelectronics, and additive manufacturing. To maximize their brightness, these upconverting nanoparticles (UCNPs) are often synthesized as core/shell heterostructures. However, the large numbers of compositional and structural parameters in multishell heterostructures make optimizing optical properties challenging. Here, we demonstrate the use of Bayesian optimization (BO) to learn the structure and design rules for multishell UCNPs with bright ultraviolet and violet emission. We leverage an automated workflow that iteratively recommends candidate UCNP structures and then simulates their emission spectra using kinetic Monte Carlo. Yb3+/Er3+- and Yb3+/Er3+/Tm3+-codoped UCNP nanostructures optimized with this BO workflow achieve 10- and 110-fold brighter emission within 22 and 40 iterations, respectively. This workflow can be expanded to structures with higher compositional and structural complexity, accelerating the discovery of novel UCNPs while domain-specific knowledge is being developed.

Explainability and human intervention in autonomous scanning probe microscopy

The broad adoption of machine learning (ML)-based autonomous experiments (AEs) in material characterization and synthesis requires strategies development for understanding and intervention in the experimental workflow. Here, we introduce and realize a post-experimental analysis strategy for deep kernel learning-based autonomous scanning probe microscopy. This approach yields real-time and post-experimental indicators for the progression of an active learning process interacting with an experimental system. We further illustrate how this approach can be applied to human-in-the-loop AEs, where human operators make high-level decisions at high latencies setting the policies for AEs, and the ML algorithm performs low-level, fast decisions. The proposed approach is universal and can be extended to other techniques and applications such as combinatorial library analysis.

Autonomous Electron Tomography Reconstruction with Machine Learning

Modern electron tomography has progressed to higher resolution at lower doses by leveraging compressed sensing (CS) methods that minimize total variation (TV). However, these sparsity-emphasized reconstruction algorithms introduce tunable parameters that greatly influence the reconstruction quality. Here, Pareto front analysis shows that high-quality tomograms are reproducibly achieved when TV minimization is heavily weighted. However, in excess, CS tomography creates overly smoothed three-dimensional (3D) reconstructions. Adding momentum to the gradient descent during reconstruction reduces the risk of over-smoothing and better ensures that CS is well behaved. For simulated data, the tedious process of tomography parameter selection is efficiently solved using Bayesian optimization with Gaussian processes. In combination, Bayesian optimization with momentum-based CS greatly reduces the required compute time-an 80% reduction was observed for the 3D reconstruction of SrTiO3 nanocubes. Automated parameter selection is necessary for large-scale tomographic simulations that enable the 3D characterization of a wider range of inorganic and biological materials.

Electrochemical Imaging of Interfaces in Energy Storage via Scanning Probe Methods: Techniques, Applications, and Prospects

Developing a deeper understanding of dynamic chemical, electronic, and morphological changes at interfaces is key to solving practical issues in electrochemical energy storage systems (EESSs). To unravel this complexity, an assortment of tools with distinct capabilities and spatiotemporal resolutions have been used to creatively visualize interfacial processes as they occur. This review highlights how electrochemical scanning probe techniques (ESPTs) such as electrochemical atomic force microscopy, scanning electrochemical microscopy, scanning ion conductance microscopy, and scanning electrochemical cell microscopy are uniquely positioned to address these challenges in EESSs. We describe the operating principles of ESPTs, focusing on the inspection of interfacial structure and chemical processes involved in Li-ion batteries and beyond. We discuss current examples, performance limitations, and complementary ESPTs. Finally, we discuss prospects for imaging improvements and deep learning for automation. We foresee that ESPTs will play an enabling role in advancing EESSs as we transition to renewable energies.

Validating the Use of Gaussian Process Regression for Adaptive Mapping of Residual Stress Fields

Probing the stress state using a high density of measurement points is time intensive and presents a limitation for what is experimentally feasible. Alternatively, individual strain fields used for determining stresses can be reconstructed from a subset of points using a Gaussian process regression (GPR). Results presented in this paper evidence that determining stresses from reconstructed strain fields is a viable approach for reducing the number of measurements needed to fully sample a component's stress state. The approach was demonstrated by reconstructing the stress fields in wire-arc additively manufactured walls fabricated using either a mild steel or low-temperature transition feedstock. Effects of errors in individual GP reconstructed strain maps and how these errors propagate to the final stress maps were assessed. Implications of the initial sampling approach and how localized strains affect convergence are explored to give guidance on how best to implement a dynamic sampling experiment.