Active Learning A Neural Network Model For Gold Clusters & Bulk From Sparse First Principles Training Data

Accelerating Global Search of Large-Sized Silver Clusters Using Cluster Graph Attention Network

Great efforts have been devoted to understanding the stability and reactivity of silver clusters, which usually depend on geometric structures, electronic configuration, and cluster size. Despite the fact that the jellium model and Wulff construction rule have successfully rationalized the stable clusters with "magic number" behavior, some experiments imply that silver clusters with 48 valence electrons also possess puzzling enhanced stability. In this work, using a recently developed deep learning technology, i.e., cluster graph attention network (CGANet), combined with a homemade comprehensive genetic algorithm (CGA) program, we searched the global minimum (GM) structures of Agn (n = 30-60) clusters with graphics processing unit acceleration, whose efficiency is about 2 orders of magnitude higher than that of the conventional density functional theory (DFT) calculations. GM structures and some representative isomers are reported at each size, revealing the competitive structural patterns based on truncated octahedra and icosahedra as well as the icosahedra-based layer-by-layer growth mode of large-sized Ag clusters. Most importantly, the size-dependent evolution behavior of structural and electronic properties of Agn (n = 30-60) clusters can successfully explain the observed stability at Ag48. Therefore, CGANet provides a powerful tool for rapidly exploring the potential energy surface of atoms with an accuracy comparable to that of DFT.

Proton Conducting Neuromorphic Materials and Devices

Neuromorphic computing and artificial intelligence hardware generally aims to emulate features found in biological neural circuit components and to enable the development of energy-efficient machines. In the biological brain, ionic currents and temporal concentration gradients control information flow and storage. It is therefore of interest to examine materials and devices for neuromorphic computing wherein ionic and electronic currents can propagate. Protons being mobile under an external electric field offers a compelling avenue for facilitating biological functionalities in artificial synapses and neurons. In this review, we first highlight the interesting biological analog of protons as neurotransmitters in various animals. We then discuss the experimental approaches and mechanisms of proton doping in various classes of inorganic and organic proton-conducting materials for the advancement of neuromorphic architectures. Since hydrogen is among the lightest of elements, characterization in a solid matrix requires advanced techniques. We review powerful synchrotron-based spectroscopic techniques for characterizing hydrogen doping in various materials as well as complementary scattering techniques to detect hydrogen. First-principles calculations are then discussed as they help provide an understanding of proton migration and electronic structure modification. Outstanding scientific challenges to further our understanding of proton doping and its use in emerging neuromorphic electronics are pointed out.

Active and Transfer Learning of High-Dimensional Neural Network Potentials for Transition Metals

Classical molecular dynamics (MD) simulations represent a very popular and powerful tool for materials modeling and design. The predictive power of MD hinges on the ability of the interatomic potential to capture the underlying physics and chemistry. There have been decades of seminal work on developing interatomic potentials, albeit with a focus predominantly on capturing the properties of bulk materials. Such physics-based models, while extensively deployed for predicting the dynamics and properties of nanoscale systems over the past two decades, tend to perform poorly in predicting nanoscale potential energy surfaces (PESs) when compared to high-fidelity first-principles calculations. These limitations stem from the lack of flexibility in such models, which rely on a predefined functional form. Machine learning (ML) models and approaches have emerged as a viable alternative to capture the diverse size-dependent cluster geometries, nanoscale dynamics, and the complex nanoscale PESs, without sacrificing the bulk properties. Here, we introduce an ML workflow that combines transfer and active learning strategies to develop high-dimensional neural networks (NNs) for capturing the cluster and bulk properties for several different transition metals with applications in catalysis, microelectronics, and energy storage, to name a few. Our NN first learns the bulk PES from the high-quality physics-based models in literature and subsequently augments this learning via retraining with a higher-fidelity first-principles training data set to concurrently capture both the nanoscale and bulk PES. Our workflow departs from status-quo in its ability to learn from a sparsely sampled data set that nonetheless covers a diverse range of cluster configurations from near-equilibrium to highly nonequilibrium as well as learning strategies that iteratively improve the fingerprinting depending on model fidelity. All the developed models are rigorously tested against an extensive first-principles data set of energies and forces of cluster configurations as well as several properties of bulk configurations for 10 different transition metals. Our approach is material agnostic and provides a methodology to transfer and build upon the learnings from decades of seminal work in molecular simulations on to a new generation of ML-trained potentials to accelerate materials discovery and design.

Developing efficient deep learning model for predicting copolymer properties

Deep learning models are gaining popularity and potency in predicting polymer properties. These models can be built using pre-existing data and are useful for the rapid prediction of polymer properties. However, the performance of a deep learning model is intricately connected to its topology and the volume of training data. There is no facile protocol available to select a deep learning architecture, and there is a lack of a large volume of homogeneous sequence-property data of polymers. These two factors are the primary bottleneck for the efficient development of deep learning models for polymers. Here we assess the severity of these factors and propose strategies to address them. We show that a linear layer-by-layer expansion of a neural network can help in identifying the best neural network topology for a given problem. Moreover, we map the discrete sequence space of a polymer to a continuous one-dimensional latent space using a feature extraction technique to identify minimal data points for training a deep learning model. We implement these approaches for two representative cases of building sequence-property surrogate models, viz., the single-molecule radius of gyration of a copolymer and copolymer compatibilizer. This work demonstrates efficient methods for building deep learning models with minimal data and hyperparameters for predicting sequence-defined properties of polymers.

Complex Oxides for Brain-Inspired Computing: A Review

The fields of brain-inspired computing, robotics, and, more broadly, artificial intelligence (AI) seek to implement knowledge gleaned from the natural world into human-designed electronics and machines. In this review, the opportunities presented by complex oxides, a class of electronic ceramic materials whose properties can be elegantly tuned by doping, electron interactions, and a variety of external stimuli near room temperature, are discussed. The review begins with a discussion of natural intelligence at the elementary level in the nervous system, followed by collective intelligence and learning at the animal colony level mediated by social interactions. An important aspect highlighted is the vast spatial and temporal scales involved in learning and memory. The focus then turns to collective phenomena, such as metal-to-insulator transitions (MITs), ferroelectricity, and related examples, to highlight recent demonstrations of artificial neurons, synapses, and circuits and their learning. First-principles theoretical treatments of the electronic structure, and in situ synchrotron spectroscopy of operating devices are then discussed. The implementation of the experimental characteristics into neural networks and algorithm design is then revewed. Finally, outstanding materials challenges that require a microscopic understanding of the physical mechanisms, which will be essential for advancing the frontiers of neuromorphic computing, are highlighted.

A database of low-energy atomically precise nanoclusters

The chemical and structural properties of atomically precise nanoclusters are of great interest in numerous applications, but the structures of the clusters can be computationally expensive to predict. In this work, we present the largest database of cluster structures and properties determined using ab-initio methods to date. We report the methodologies used to discover low-energy clusters as well as the energies, relaxed structures, and physical properties (such as relative stability, HOMO-LUMO gap among others) for 63,015 clusters across 55 elements. We have identified clusters for 593 out of 1595 cluster systems (element-size pairs) explored by literature that have energies lower than those reported in literature by at least 1 meV/atom. We have also identified clusters for 1320 systems for which we were unable to find previous low-energy structures in the literature. Patterns in the data reveal insights into the chemical and structural relationships among the elements at the nanoscale. We describe how the database can be accessed for future studies and the development of nanocluster-based technologies.

Active sampling for neural network potentials: Accelerated simulations of shear-induced deformation in Cu-Ni multilayers

Neural network potentials (NNPs) can greatly accelerate atomistic simulations relative to ab initio methods, allowing one to sample a broader range of structural outcomes and transformation pathways. In this work, we demonstrate an active sampling algorithm that trains an NNP that is able to produce microstructural evolutions with accuracy comparable to those obtained by density functional theory, exemplified during structure optimizations for a model Cu-Ni multilayer system. We then use the NNP, in conjunction with a perturbation scheme, to stochastically sample structural and energetic changes caused by shear-induced deformation, demonstrating the range of possible intermixing and vacancy migration pathways that can be obtained as a result of the speedups provided by the NNP. The code to implement our active learning strategy and NNP-driven stochastic shear simulations is openly available at https://github.com/pnnl/Active-Sampling-for-Atomistic-Potentials.

Big Data in a Nano World: A Review on Computational, Data-Driven Design of Nanomaterials Structures, Properties, and Synthesis

The recent rise of computational, data-driven research has significant potential to accelerate materials discovery. Automated workflows and materials databases are being rapidly developed, contributing to high-throughput data of bulk materials that are growing in quantity and complexity, allowing for correlation between structural-chemical features and functional properties. In contrast, computational data-driven approaches are still relatively rare for nanomaterials discovery due to the rapid scaling of computational cost for finite systems. However, the distinct behaviors at the nanoscale as compared to the parent bulk materials and the vast tunability space with respect to dimensionality and morphology motivate the development of data sets for nanometric materials. In this review, we discuss the recent progress in data-driven research in two aspects: functional materials design and guided synthesis, including commonly used metrics and approaches for designing materials properties and predicting synthesis routes. More importantly, we discuss the distinct behaviors of materials as a result of nanosizing and the implications for data-driven research. Finally, we share our perspectives on future directions for extending the current data-driven research into the nano realm.

Machine learning the metastable phase diagram of covalently bonded carbon

Conventional phase diagram generation involves experimentation to provide an initial estimate of the set of thermodynamically accessible phases and their boundaries, followed by use of phenomenological models to interpolate between the available experimental data points and extrapolate to experimentally inaccessible regions. Such an approach, combined with high throughput first-principles calculations and data-mining techniques, has led to exhaustive thermodynamic databases (e.g. compatible with the CALPHAD method), albeit focused on the reduced set of phases observed at distinct thermodynamic equilibria. In contrast, materials during their synthesis, operation, or processing, may not reach their thermodynamic equilibrium state but, instead, remain trapped in a local (metastable) free energy minimum, which may exhibit desirable properties. Here, we introduce an automated workflow that integrates first-principles physics and atomistic simulations with machine learning (ML), and high-performance computing to allow rapid exploration of the metastable phases to construct "metastable" phase diagrams for materials far-from-equilibrium. Using carbon as a prototypical system, we demonstrate automated metastable phase diagram construction to map hundreds of metastable states ranging from near equilibrium to far-from-equilibrium (400 meV/atom). We incorporate the free energy calculations into a neural-network-based learning of the equations of state that allows for efficient construction of metastable phase diagrams. We use the metastable phase diagram and identify domains of relative stability and synthesizability of metastable materials. High temperature high pressure experiments using a diamond anvil cell on graphite sample coupled with high-resolution transmission electron microscopy (HRTEM) confirm our metastable phase predictions. In particular, we identify the previously ambiguous structure of n-diamond as a cubic-analog of diaphite-like lonsdaelite phase.

Multi-reward Reinforcement Learning Based Bond-Order Potential to Study Strain-Assisted Phase Transitions in Phosphorene

We introduce a multi-reward reinforcement learning (RL) approach to train a flexible bond-order potential (BOP) for 2D phosphorene based on ab initio training data sets. Our approach is based on a continuous action space Monte Carlo tree search algorithm that is general and scalable and presents an efficient multiobjective optimization scheme for high-dimensional materials design problems. As a proof-of-concept, we deploy this scheme to parametrize multiple structural and dynamical properties of 2D phosphorene polymorphs. Our RL-trained BOP model adequately captures the structure, energetics, transformation barriers, equation of state, elastic constants, and phonon dispersions of various 2D P polymorphs. We use this model to probe the impact of temperature and strain rate on the phase transition from black (α-P) to blue phosphorene (β-P) through molecular dynamics simulations. A decrease in critical strain for this phase transition with increase in temperature is observed, and the underlying atomistic mechanisms are discussed.

Learning in continuous action space for developing high dimensional potential energy models

Reinforcement learning (RL) approaches that combine a tree search with deep learning have found remarkable success in searching exorbitantly large, albeit discrete action spaces, as in chess, Shogi and Go. Many real-world materials discovery and design applications, however, involve multi-dimensional search problems and learning domains that have continuous action spaces. Exploring high-dimensional potential energy models of materials is an example. Traditionally, these searches are time consuming (often several years for a single bulk system) and driven by human intuition and/or expertise and more recently by global/local optimization searches that have issues with convergence and/or do not scale well with the search dimensionality. Here, in a departure from discrete action and other gradient-based approaches, we introduce a RL strategy based on decision trees that incorporates modified rewards for improved exploration, efficient sampling during playouts and a "window scaling scheme" for enhanced exploitation, to enable efficient and scalable search for continuous action space problems. Using high-dimensional artificial landscapes and control RL problems, we successfully benchmark our approach against popular global optimization schemes and state of the art policy gradient methods, respectively. We demonstrate its efficacy to parameterize potential models (physics based and high-dimensional neural networks) for 54 different elemental systems across the periodic table as well as alloys. We analyze error trends across different elements in the latent space and trace their origin to elemental structural diversity and the smoothness of the element energy surface. Broadly, our RL strategy will be applicable to many other physical science problems involving search over continuous action spaces.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Data-driven simulation and characterisation of gold nanoparticle melting

The simulation and analysis of the thermal stability of nanoparticles, a stepping stone towards their application in technological devices, require fast and accurate force fields, in conjunction with effective characterisation methods. In this work, we develop efficient, transferable, and interpretable machine learning force fields for gold nanoparticles based on data gathered from Density Functional Theory calculations. We use them to investigate the thermodynamic stability of gold nanoparticles of different sizes (1 to 6 nm), containing up to 6266 atoms, concerning a solid-liquid phase change through molecular dynamics simulations. We predict nanoparticle melting temperatures in good agreement with available experimental data. Furthermore, we characterize the solid-liquid phase change mechanism employing an unsupervised learning scheme to categorize local atomic environments. We thus provide a data-driven definition of liquid atomic arrangements in the inner and surface regions of a nanoparticle and employ it to show that melting initiates at the outer layers.

Development of Multimodal Machine Learning Potentials: Toward a Physics-Aware Artificial Intelligence

Machine learning interatomic potentials (MLIPs) are widely used for describing molecular energy and continue bridging the speed and accuracy gap between quantum mechanical (QM) and classical approaches like force fields. In this Account, we focus on the out-of-the-box approaches to developing transferable MLIPs for diverse chemical tasks. First, we introduce the "Accurate Neural Network engine for Molecular Energies," ANAKIN-ME, method (or ANI for short). The ANI model utilizes Justin Smith Symmetry Functions (JSSFs) and realizes training for vast data sets. The training data set of several orders of magnitude larger than before has become the key factor of the knowledge transferability and flexibility of MLIPs. As the quantity, quality, and types of interactions included in the training data set will dictate the accuracy of MLIPs, the task of proper data selection and model training could be assisted with advanced methods like active learning (AL), transfer learning (TL), and multitask learning (MTL).Next, we describe the AIMNet "Atoms-in-Molecules Network" that was inspired by the quantum theory of atoms in molecules. The AIMNet architecture lifts multiple limitations in MLIPs. It encodes long-range interactions and learnable representations of chemical elements. We also discuss the AIMNet-ME model that expands the applicability domain of AIMNet from neutral molecules toward open-shell systems. The AIMNet-ME encompasses a dependence of the potential on molecular charge and spin. It brings ML and physical models one step closer, ensuring the correct molecular energy behavior over the total molecular charge.We finally describe perhaps the simplest possible physics-aware model, which combines ML and the extended Hückel method. In ML-EHM, "Hierarchically Interacting Particle Neural Network," HIP-NN generates the set of a molecule- and environment-dependent Hamiltonian elements α_μμ and K^‡. As a test example, we show how in contrast to traditional Hückel theory, ML-EHM correctly describes orbital crossing with bond rotations. Hence it learns the underlying physics, highlighting that the inclusion of proper physical constraints and symmetries could significantly improve ML model generalization.

Electrocatalytic Hydrogenation of Biomass-Derived Organics: A Review

Sustainable energy generation calls for a shift away from centralized, high-temperature, energy-intensive processes to decentralized, low-temperature conversions that can be powered by electricity produced from renewable sources. Electrocatalytic conversion of biomass-derived feedstocks would allow carbon recycling of distributed, energy-poor resources in the absence of sinks and sources of high-grade heat. Selective, efficient electrocatalysts that operate at low temperatures are needed for electrocatalytic hydrogenation (ECH) to upgrade the feedstocks. For effective generation of energy-dense chemicals and fuels, two design criteria must be met: (i) a high H:C ratio via ECH to allow for high-quality fuels and blends and (ii) a lower O:C ratio in the target molecules via electrochemical decarboxylation/deoxygenation to improve the stability of fuels and chemicals. The goal of this review is to determine whether the following questions have been sufficiently answered in the open literature, and if not, what additional information is required:(1)What organic functionalities are accessible for electrocatalytic hydrogenation under a set of reaction conditions? How do substitutions and functionalities impact the activity and selectivity of ECH?(2)What material properties cause an electrocatalyst to be active for ECH? Can general trends in ECH be formulated based on the type of electrocatalyst?(3)What are the impacts of reaction conditions (electrolyte concentration, pH, operating potential) and reactor types?