Parallel Multistream Training of High-Dimensional Neural Network Potentials

Machine learning the electric field response of condensed phase systems using perturbed neural network potentials

The interaction of condensed phase systems with external electric fields is of major importance in a myriad of processes in nature and technology, ranging from the field-directed motion of cells (galvanotaxis), to geochemistry and the formation of ice phases on planets, to field-directed chemical catalysis and energy storage and conversion systems including supercapacitors, batteries and solar cells. Molecular simulation in the presence of electric fields would give important atomistic insight into these processes but applications of the most accurate methods such as ab-initio molecular dynamics (AIMD) are limited in scope by their computational expense. Here we introduce Perturbed Neural Network Potential Molecular Dynamics (PNNP MD) to push back the accessible time and length scales of such simulations. We demonstrate that important dielectric properties of liquid water including the field-induced relaxation dynamics, the dielectric constant and the field-dependent IR spectrum can be machine learned up to surprisingly high field strengths of about 0.2 V Å-1 without loss in accuracy when compared to ab-initio molecular dynamics. This is remarkable because, in contrast to most previous approaches, the two neural networks on which PNNP MD is based are exclusively trained on molecular configurations sampled from zero-field MD simulations, demonstrating that the networks not only interpolate but also reliably extrapolate the field response. PNNP MD is based on rigorous theory yet it is simple, general, modular, and systematically improvable allowing us to obtain atomistic insight into the interaction of a wide range of condensed phase systems with external electric fields.

Five-Site Water Models for Ice and Liquid Water Generated by a Series-Parallel Machine Learning Strategy

Icing, a common natural phenomenon, always originates from a molecule. Molecular simulation is crucial for understanding the relevant process but still faces a great challenge in obtaining a uniform and accurate description of ice and liquid water with limited model parameters. Here, we propose a series-parallel machine learning (ML) approach consisting of a classification back-propagation neural network (BPNN), parallel regression BPNNs, and a genetic algorithm to establish conventional TIP5P-BG and temperature-dependent TIP5P-BGT models. The established water models exhibit a comprehensive balance among the crucial physical properties (melting point, density, vaporization enthalpy, self-diffusion coefficient, and viscosity) with mean absolute percentage errors of 2.65 and 2.40%, respectively, and excellent predictive performance on the related properties of liquid water. For ice, the simulation results on the critical nucleus size and growth rate are in good accordance with experiments. This work offers a powerful molecular model for phase transition and icing in nanoconfinement and a construction strategy for a complex molecular model in the extreme case.

Disorder-Dependent Li Diffusion in Li6PS5Cl Investigated by Machine-Learning Potential

Solid-state electrolytes with argyrodite structures, such as Li6PS5Cl, have attracted considerable attention due to their superior safety compared to liquid electrolytes and higher ionic conductivity than other solid electrolytes. Although experimental efforts have been made to enhance conductivity by controlling the degree of disorder, the underlying diffusion mechanism is not yet fully understood. Moreover, existing theoretical analyses based on ab initio molecular dynamics (MD) simulations have limitations in addressing various types of disorder at room temperature. In this study, we directly investigate Li-ion diffusion in Li6PS5Cl at 300 K using large-scale, long-term MD simulations empowered by machine-learning potentials (MLPs). To ensure the convergence of conductivity values within an error range of 10%, we employ a 25 ns simulation using a 5 × 5 × 5 supercell containing 6500 atoms. The computed Li-ion conductivity, activation energies, and equilibrium site occupancies align well with experimental observations. Notably, Li-ion conductivity peaks when Cl ions occupy 25% of the 4c sites rather than at 50% where the disorder is maximized. In addition, Li-ion diffusion shows non-Arrhenius behavior, leading to different activation energies at high temperatures (>400 K). These phenomena are explained by the interplay between inter- and intracage jumps. By elucidation of the key factors affecting Li-ion diffusion in Li6PS5Cl, this work paves the way for optimizing ionic conductivity in the argyrodite family.

The wetting of H2O by CO2

Biphasic interfaces are complex but fascinating regimes that display a number of properties distinct from those of the bulk. The CO2-H2O interface, in particular, has been the subject of a number of studies on account of its importance for the carbon life cycle as well as carbon capture and sequestration schemes. Despite this attention, there remain a number of open questions on the nature of the CO2-H2O interface, particularly concerning the interfacial tension and phase behavior of CO2 at the interface. In this paper, we seek to address these ambiguities using ab initio-quality simulations. Harnessing the benefits of machine-learned potentials and enhanced statistical sampling methods, we present an ab initio-level description of the CO2-H2O interface. Interfacial tensions are predicted from 1 to 500 bars and found to be in close agreement with experiment at pressures for which experimental data are available. Structural analyses indicate the buildup of an adsorbed, saturated CO2 film forming at a low pressure (20 bars) with properties similar to those of the bulk liquid, but preferential perpendicular alignment with respect to the interface. The CO2 monolayer buildup coincides with a reduced structuring of water molecules close to the interface. This study highlights the predictive nature of machine-learned potentials for complex macroscopic properties of biphasic interfaces, and the mechanistic insight obtained into carbon dioxide aggregation at the water interface is of high relevance for geoscience, climate research, and materials science.

Superdiffusive Rotation of Interfacial Water on Noble Metal Surface

Interfacial water on a metal surface acts as an active layer through the reorientation of water, thereby facilitating the energy transfer and chemical reaction across the metal surface in various physicochemical and industrial processes. However, how this active interfacial water collectively behaves on flat noble metal substrates remains largely unknown due to the experimental limitation in capturing librational vibrational motion of interfacial water and prohibitive computational costs at the first-principles level. Herein, by implementing a machine-learning approach to train neural network potentials, we enable performing advanced molecular dynamics simulations with ab initio accuracy at a nanosecond scale to map the distinct rotational motion of water molecules on a metal surface at room temperature. The vibrational density of states of the interfacial water with two-layer profiles reveals that the rotation and vibration of water within the strong adsorption layer on the metal surface behave as if the water molecules in the bulk ice, wherein the O-H stretching frequency is well consistent with the experimental results. Unexpectedly, the water molecules within the adjacent weak adsorption layer exhibit superdiffusive rotation, contrary to the conventional diffusive rotation of bulk water, while the vibrational motion maintains the characteristic of bulk water. The mechanism underlying this abnormal superdiffusive rotation is attributed to the translation-rotation decoupling of water, in which the translation is restrained by the strong hydrogen bonding within the bilayer interfacial water, whereas the rotation is accelerated freely by the asymmetric water environment. This superdiffusive rotation dynamics may elucidate the experimentally observed large fluctuation of the potential of zero charge on Pt and thereby the conventional Helmholtz layer model revised by including the contribution of interfacial water orientation. The surprising superdiffusive rotation of vicinal water next to noble metals will shed new light on the physicochemical processes and the activity of water molecules near metal electrodes or catalysts.

Exploring the stable structures of cerium oxide nanoclusters using high-dimensional neural network potential

Cerium clusters have been extensively applied in industry owing to their extraordinary properties for oxygen storage and redox catalytic activities. However, their atomically precise structures have not been studied because of the lack of a reliable method to efficiently sample their complex structures. Herein, we combined a neural network algorithm with density functional theory calculations to establish a high-dimensional potential to search for the global minimums of cerium oxide clusters. Using Ce14O28 as well as its reduced state Ce14O27 and oxidized state Ce14O29 with ultra-small dimensions of ∼1.0 nm as examples, we found that these three clusters adopt pyramid-like structures with the lowest energies, which was obtained by exploring 100 000 configurations in large feasible spaces. Further the neural network potential-enhanced molecular dynamics calculations indicated that these cluster structures are stable at high temperature. The electronic structure analysis suggested that these clusters are highly active and easily lose oxygen. This work demonstrated that neural network potentials can be useful for exploring the stable structures of metal oxide nanoclusters in practical applications.

Exploring the frontiers of condensed-phase chemistry with a general reactive machine learning potential

Atomistic simulation has a broad range of applications from drug design to materials discovery. Machine learning interatomic potentials (MLIPs) have become an efficient alternative to computationally expensive ab initio simulations. For this reason, chemistry and materials science would greatly benefit from a general reactive MLIP, that is, an MLIP that is applicable to a broad range of reactive chemistry without the need for refitting. Here we develop a general reactive MLIP (ANI-1xnr) through automated sampling of condensed-phase reactions. ANI-1xnr is then applied to study five distinct systems: carbon solid-phase nucleation, graphene ring formation from acetylene, biofuel additives, combustion of methane and the spontaneous formation of glycine from early earth small molecules. In all studies, ANI-1xnr closely matches experiment (when available) and/or previous studies using traditional model chemistry methods. As such, ANI-1xnr proves to be a highly general reactive MLIP for C, H, N and O elements in the condensed phase, enabling high-throughput in silico reactive chemistry experimentation.

Surface stratification determines the interfacial water structure of simple electrolyte solutions

The distribution of ions at the air/water interface plays a decisive role in many natural processes. Several studies have reported that larger ions tend to be surface-active, implying ions are located on top of the water surface, thereby inducing electric fields that determine the interfacial water structure. Here we challenge this view by combining surface-specific heterodyne-detected vibrational sum-frequency generation with neural network-assisted ab initio molecular dynamics simulations. Our results show that ions in typical electrolyte solutions are, in fact, located in a subsurface region, leading to a stratification of such interfaces into two distinctive water layers. The outermost surface is ion-depleted, and the subsurface layer is ion-enriched. This surface stratification is a key element in explaining the ion-induced water reorganization at the outermost air/water interface.

Comparing machine learning potentials for water: Kernel-based regression and Behler-Parrinello neural networks

In this paper, we investigate the performance of different machine learning potentials (MLPs) in predicting key thermodynamic properties of water using RPBE + D3. Specifically, we scrutinize kernel-based regression and high-dimensional neural networks trained on a highly accurate dataset consisting of about 1500 structures, as well as a smaller dataset, about half the size, obtained using only on-the-fly learning. This study reveals that despite minor differences between the MLPs, their agreement on observables such as the diffusion constant and pair-correlation functions is excellent, especially for the large training dataset. Variations in the predicted density isobars, albeit somewhat larger, are also acceptable, particularly given the errors inherent to approximate density functional theory. Overall, this study emphasizes the relevance of the database over the fitting method. Finally, this study underscores the limitations of root mean square errors and the need for comprehensive testing, advocating the use of multiple MLPs for enhanced certainty, particularly when simulating complex thermodynamic properties that may not be fully captured by simpler tests.

Evaluation of Rate Coefficients in the Gas Phase Using Machine-Learned Potentials

We assess the capability of machine-learned potentials to compute rate coefficients by training a neural network (NN) model and applying it to describe the chemical landscape on the C5H5 potential energy surface, which is relevant to molecular weight growth in combustion and interstellar media. We coupled the resulting NN with an automated kinetics workflow code, KinBot, to perform all necessary calculations to compute the rate coefficients. The NN is benchmarked exhaustively by evaluating its performance at the various stages of the kinetics calculations: from the electronic energy through the computation of zero point energy, barrier heights, entropic contributions, the portion of the PES explored, and finally the overall rate coefficients as formulated by transition state theory.

Machine-Learned Potentials by Active Learning from Organic Crystal Structure Prediction Landscapes

A primary challenge in organic molecular crystal structure prediction (CSP) is accurately ranking the energies of potential structures. While high-level solid-state density functional theory (DFT) methods allow for mostly reliable discrimination of the low-energy structures, their high computational cost is problematic because of the need to evaluate tens to hundreds of thousands of trial crystal structures to fully explore typical crystal energy landscapes. Consequently, lower-cost but less accurate empirical force fields are often used, sometimes as the first stage of a hierarchical scheme involving multiple stages of increasingly accurate energy calculations. Machine-learned interatomic potentials (MLIPs), trained to reproduce the results of ab initio methods with computational costs close to those of force fields, can improve the efficiency of the CSP by reducing or eliminating the need for costly DFT calculations. Here, we investigate active learning methods for training MLIPs with CSP datasets. The combination of active learning with the well-developed sampling methods from CSP yields potentials in a highly automated workflow that are relevant over a wide range of the crystal packing space. To demonstrate these potentials, we illustrate efficiently reranking large, diverse crystal structure landscapes to near-DFT accuracy from force field-based CSP, improving the reliability of the final energy ranking. Furthermore, we demonstrate how these potentials can be extended to more accurately model structures far from lattice energy minima through additional on-the-fly training within Monte Carlo simulations.

Hydrogen atom scattering at the Al2O3(0001) surface: a combined experimental and theoretical study

Investigating atom-surface interactions is the key to an in-depth understanding of chemical processes at interfaces, which are of central importance in many fields - from heterogeneous catalysis to corrosion. In this work, we present a joint experimental and theoretical effort to gain insights into the atomistic details of hydrogen atom scattering at the α-Al2O3(0001) surface. Surprisingly, this system has been hardly studied to date, although hydrogen atoms as well as α-Al2O3 are omnipresent in catalysis as reactive species and support oxide, respectively. We address this system by performing hydrogen atom beam scattering experiments and molecular dynamics (MD) simulations based on a high-dimensional machine learning potential trained to density functional theory data. Using this combination of methods we are able to probe the properties of the multidimensional potential energy surface governing the scattering process. Specifically, we compare the angular distribution and the kinetic energy loss of the scattered atoms obtained in experiment with a large number of MD trajectories, which, moreover, allow to identify the underlying impact sites at the surface.

Neural Network Approach for a Rapid Prediction of Metal-Supported Borophene Properties

We developed a high-dimensional neural network potential (NNP) to describe the structural and energetic properties of borophene deposited on silver. This NNP has the accuracy of density functional theory (DFT) calculations while achieving computational speedups of several orders of magnitude, allowing the study of extensive structures that may reveal intriguing moiré patterns or surface corrugations. We describe an efficient approach to constructing the training data set using an iterative technique known as the "adaptive learning approach". The developed NNP is able to produce, with excellent agreement, the structure, energy, and forces obtained at the DFT level. Finally, the calculated stability of various borophene polymorphs, including those not initially included in the training data set, shows better stabilization for ν ∼ 0.1 hole density, and in particular for the allotrope α ( ν = 1 / 9 ) . The stability of borophene on the metal surface is shown to depend on its orientation, implying structural corrugation patterns that can be observed only from long-time simulations on extended systems. The NNP also demonstrates its ability to simulate vibrational densities of states and produce realistic structures with simulated STM images closely matching the experimental ones.

High-Dimensional Neural Network Potentials for Accurate Prediction of Equation of State: A Case Study of Methane

Machine learning-based interatomic potentials, such as those provided by neural networks, are increasingly important in molecular dynamics simulations. In the present work, we consider the applicability and robustness of machine learning molecular dynamics to predict the equation of state properties of methane by using high-dimensional neural network potentials (HDNNPs). We investigate two different strategies for generating training data: one strategy based upon bulk representations using periodic cells and another strategy based upon clusters of molecules. We assess the accuracy of the trained potentials by predicting the equilibrium mass density for a wide range of thermodynamic conditions to characterize the liquid phase, supercritical fluid, and gas phase, as well as the liquid-vapor coexistence curve. Our results show an excellent agreement with reference phase diagrams, with an average error below ∼2% for all studied phases. Moreover, we confirm the applicability of models trained on cluster data sets for producing accurate and reliable results.

Reducing the Cost of Neural Network Potential Generation for Reactive Molecular Systems

Although machine learning potentials have recently had a substantial impact on molecular simulations, the construction of a robust training set can still become a limiting factor, especially due to the requirement of a reference ab initio simulation that covers all the relevant geometries of the system. Recognizing that this can be prohibitive for certain systems, we develop the method of transition tube sampling that mitigates the computational cost of training set and model generation. In this approach, we generate classical or quantum thermal geometries around a transition path describing a conformational change or a chemical reaction using only a sparse set of local normal mode expansions along this path and select from these geometries by an active learning protocol. This yields a training set with geometries that characterize the whole transition without the need for a costly reference trajectory. The performance of the method is evaluated on different molecular systems with the complexity of the potential energy landscape increasing from a single minimum to a double proton-transfer reaction with high barriers. Our results show that the method leads to training sets that give rise to models applicable in classical and path integral simulations alike that are on par with those based directly on ab initio calculations while providing the computational speedup we have come to expect from machine learning potentials.

Mechanistic insight on water dissociation on pristine low-index TiO2 surfaces from machine learning molecular dynamics simulations

Water adsorption and dissociation processes on pristine low-index TiO2 interfaces are important but poorly understood outside the well-studied anatase (101) and rutile (110). To understand these, we construct three sets of machine learning potentials that are simultaneously applicable to various TiO2 surfaces, based on three density-functional-theory approximations. Here we show the water dissociation free energies on seven pristine TiO2 surfaces, and predict that anatase (100), anatase (110), rutile (001), and rutile (011) favor water dissociation, anatase (101) and rutile (100) have mostly molecular adsorption, while the simulations of rutile (110) sensitively depend on the slab thickness and molecular adsorption is preferred with thick slabs. Moreover, using an automated algorithm, we reveal that these surfaces follow different types of atomistic mechanisms for proton transfer and water dissociation: one-step, two-step, or both. These mechanisms can be rationalized based on the arrangements of water molecules on the different surfaces. Our finding thus demonstrates that the different pristine TiO2 surfaces react with water in distinct ways, and cannot be represented using just the low-energy anatase (101) and rutile (110) surfaces.

How to train a neural network potential

The introduction of modern Machine Learning Potentials (MLPs) has led to a paradigm change in the development of potential energy surfaces for atomistic simulations. By providing efficient access to energies and forces, they allow us to perform large-scale simulations of extended systems, which are not directly accessible by demanding first-principles methods. In these simulations, MLPs can reach the accuracy of electronic structure calculations, provided that they have been properly trained and validated using a suitable set of reference data. Due to their highly flexible functional form, the construction of MLPs has to be done with great care. In this Tutorial, we describe the necessary key steps for training reliable MLPs, from data generation via training to final validation. The procedure, which is illustrated for the example of a high-dimensional neural network potential, is general and applicable to many types of MLPs.

Optimizing the architecture of Behler-Parrinello neural network potentials

The architecture of neural network potentials is typically optimized at the beginning of the training process and remains unchanged throughout. Here, we investigate the accuracy of Behler-Parrinello neural network potentials for varying training set sizes. Using the QM9 and 3BPA datasets, we show that adjusting the network architecture according to the training set size improves the accuracy significantly. We demonstrate that both an insufficient and an excessive number of fitting parameters can have a detrimental impact on the accuracy of the neural network potential. Furthermore, we investigate the influences of descriptor complexity, neural network depth, and activation function on the model's performance. We find that for the neural network potentials studied here, two hidden layers yield the best accuracy and that unbounded activation functions outperform bounded ones.

Data-Efficient Machine Learning Potentials from Transfer Learning of Periodic Correlated Electronic Structure Methods: Liquid Water at AFQMC, CCSD, and CCSD(T) Accuracy

Obtaining the atomistic structure and dynamics of disordered condensed-phase systems from first-principles remains one of the forefront challenges of chemical theory. Here we exploit recent advances in periodic electronic structure and provide a data-efficient approach to obtain machine-learned condensed-phase potential energy surfaces using AFQMC, CCSD, and CCSD(T) from a very small number (≤200) of energies by leveraging a transfer learning scheme starting from lower-tier electronic structure methods. We demonstrate the effectiveness of this approach for liquid water by performing both classical and path integral molecular dynamics simulations on these machine-learned potential energy surfaces. By doing this, we uncover the interplay of dynamical electron correlation and nuclear quantum effects across the entire liquid range of water while providing a general strategy for efficiently utilizing periodic correlated electronic structure methods to explore disordered condensed-phase systems.

A fully quantum-mechanical treatment for kaolinite

Neural network potentials for kaolinite minerals have been fitted to data extracted from density functional theory calculations that were performed using the revPBE + D3 and revPBE + vdW functionals. These potentials have then been used to calculate the static and dynamic properties of the mineral. We show that revPBE + vdW is better at reproducing the static properties. However, revPBE + D3 does a better job of reproducing the experimental IR spectrum. We also consider what happens to these properties when a fully quantum treatment of the nuclei is employed. We find that nuclear quantum effects (NQEs) do not make a substantial difference to the static properties. However, when NQEs are included, the dynamic properties of the material change substantially.

Are Neural Network Potentials Trained on Liquid States Transferable to Crystal Nucleation? A Test on Ice Nucleation in the mW Water Model

Neural network potentials (NNPs) are increasingly being used to study processes that happen on long time scales. A typical example is crystal nucleation, which rate is controlled by the occurrence of a rare fluctuation, i.e., the appearance of the critical nucleus. Because the properties of this nucleus are far from those of the bulk crystal, it is yet unclear whether NN potentials trained on equilibrium liquid states can accurately describe nucleation processes. So far, nucleation studies on NNPs have been limited to ab initio models whose nucleation properties are unknown, preventing an accurate comparison. Here we train a NN potential on the mW model of water─a classical three-body potential whose nucleation time scale is accessible in standard simulations. We show that a NNP trained only on a small number of liquid state points can reproduce with great accuracy the nucleation rates and free energy barriers of the original model, computed from both spontaneous and biased trajectories, strongly supporting the use of NNPs to study nucleation events.

A neural network potential with self-trained atomic fingerprints: A test with the mW water potential

We present a neural network (NN) potential based on a new set of atomic fingerprints built upon two- and three-body contributions that probe distances and local orientational order, respectively. Compared with the existing NN potentials, the atomic fingerprints depend on a small set of tunable parameters that are trained together with the NN weights. In addition to simplifying the selection of the atomic fingerprints, this strategy can also considerably increase the overall accuracy of the network representation. To tackle the simultaneous training of the atomic fingerprint parameters and NN weights, we adopt an annealing protocol that progressively cycles the learning rate, significantly improving the accuracy of the NN potential. We test the performance of the network potential against the mW model of water, which is a classical three-body potential that well captures the anomalies of the liquid phase. Trained on just three state points, the NN potential is able to reproduce the mW model in a very wide range of densities and temperatures, from negative pressures to several GPa, capturing the transition from an open random tetrahedral network to a dense interpenetrated network. The NN potential also reproduces very well properties for which it was not explicitly trained, such as dynamical properties and the structure of the stable crystalline phases of mW.

Thermodynamics of diamond formation from hydrocarbon mixtures in planets

Hydrocarbon mixtures are extremely abundant in the Universe, and diamond formation from them can play a crucial role in shaping the interior structure and evolution of planets. With first-principles accuracy, we first estimate the melting line of diamond, and then reveal the nature of chemical bonding in hydrocarbons at extreme conditions. We finally establish the pressure-temperature phase boundary where it is thermodynamically possible for diamond to form from hydrocarbon mixtures with different atomic fractions of carbon. Notably, here we show a depletion zone at pressures above 200 GPa and temperatures below 3000 K-3500 K where diamond formation is thermodynamically favorable regardless of the carbon atomic fraction, due to a phase separation mechanism. The cooler condition of the interior of Neptune compared to Uranus means that the former is much more likely to contain the depletion zone. Our findings can help explain the dichotomy of the two ice giants manifested by the low luminosity of Uranus, and lead to a better understanding of (exo-)planetary formation and evolution.

Oxidation and de-alloying of PtMn particle models: a computational investigation

We present a computational study of the energetics and mechanisms of oxidation of Pt-Mn systems. We use slab models and simulate the oxidation process over the most stable (111) facet at a given Pt₂Mn composition to make the problem computationally affordable, and combine Density-Functional Theory (DFT) with neural network potentials and metadynamics simulations to accelerate the mechanistic search. We find, first, that Mn has a strong tendency to alloy with Pt. This tendency is optimally realized when Pt and Mn are mixed in the bulk, but, at a composition in which the Mn content is high enough such as for Pt₂Mn, Mn atoms will also be found in the surface outmost layer. These surface Mn atoms can dissociate O₂ and generate MnO_x species, transforming the surface-alloyed Mn atoms into MnO_x surface oxide structures supported on a metallic framework in which one or more vacancy sites are simultaneously created. The thus-formed vacancies promote the successive steps of the oxidation process: the vacancy sites can be filled by surface oxygen atoms, which can then interact with Mn atoms in deeper layers, or subsurface Mn atoms can intercalate into interstitial sites. Both these steps facilitate the extraction of further bulk Mn atoms into MnO_x oxide surface structures, and thus the progress of the oxidation process, with typical rate-determining energy barriers in the range 0.9-1.0 eV.

Atomic-Scale Mechanism of Platinum Catalyst Restructuring under a Pressure of Reactant Gas

Heterogeneous catalysis is key for chemical transformations. Understanding how catalysts' active sites dynamically evolve at the atomic scale under reaction conditions is a prerequisite for accurately determining catalytic mechanisms and predictably developing catalysts. We combine in situ time-dependent scanning tunneling microscopy observations and machine-learning-accelerated first-principles atomistic simulations to uncover the mechanism of restructuring of Pt catalysts under a pressure of carbon monoxide (CO). We show that a high CO coverage at a Pt step edge triggers the formation of atomic protrusions of low-coordination Pt atoms, which then detach from the step edge to create sub-nano-islands on the terraces, where under-coordinated sites are stabilized by the CO adsorbates. The fast and accurate machine-learning potential is key to enabling the exploration of tens of thousands of configurations for the CO-covered restructuring catalyst. These studies open an avenue to achieve an atomic-scale understanding of the structural dynamics of more complex metal nanoparticle catalysts under reaction conditions.

Plasma Oxidation of Copper: Molecular Dynamics Study with Neural Network Potentials

The formation of thin oxide films is of significant scientific and practical interest. In particular, the semiconductor industry is interested in developing a plasma atomic layer etching process to pattern copper, replacing the dual Damascene process. Using a nonthermal oxygen plasma to convert the metallic copper into copper oxide, followed by a formic acid organometallic reaction to etch the copper oxide, this process has shown great promise. However, the current process is not optimal because the plasma oxidation step is not self-limiting, hampering the degree of thickness control. In the present study, a neural network potential for the binary interaction between copper and oxygen is developed and validated against first-principles calculations. This potential covers the entire range of potential energy surfaces of metallic copper, copper oxides, atomic oxygen, and molecular oxygen. The usable kinetic energy ranges from 0 to 20 eV. Using this potential, the plasma oxidation of copper surfaces was studied with large-scale molecular dynamics at atomic resolution, with an accuracy approaching that of the first principle calculations. An amorphous layer of CuO is formed on Cu, with thicknesses reaching 2.5 nm. Plasma is found to create an intense local heating effect that rapidly dissipates across the thickness of the film. The range of this heating effect depends on the kinetic energy of the ions. A higher ion energy leads to a longer range, which sustains faster-than-thermal rates for longer periods of time for the oxide growth. Beyond the range of this agitation, the growth is expected to be limited to the thermally activated rate. High-frequency, repeated ion impacts result in a microannealing effect that leads to a quasicrystalline oxide beneath the amorphized layer. The crystalline layer slows down oxide growth. Growth rate is fitted to the temperature gradient due to ion-induced thermal agitations, to obtain an apparent activation energy of 1.0 eV. A strategy of lowering the substrate temperature and increasing plasma power is proposed as being favorable for more self-limited oxidation.

Machine learning interatomic potentials for aluminium: application to solidification phenomena

In studying solidification process by simulations on the atomic scale, the modeling of crystal nucleation or amorphization requires the construction of interatomic interactions that are able to reproduce the properties of both the solid and the liquid states. Taking into account rare nucleation events or structural relaxation under deep undercooling conditions requires much larger length scales and longer time scales than those achievable byab initiomolecular dynamics (AIMD). This problem is addressed by means of classical molecular dynamics simulations using a well established high dimensional neural network potential trained on a set of configurations generated by AIMD relevant for solidification phenomena. Our dataset contains various crystalline structures and liquid states at different pressures, including their time fluctuations in a wide range of temperatures. Applied to elemental aluminium, the resulting potential is shown to be efficient to reproduce the basic structural, dynamics and thermodynamic quantities in the liquid and undercooled states. Early stages of crystallization are further investigated on a much larger scale with one million atoms, allowing us to unravel features of the homogeneous nucleation mechanisms in the fcc phase at ambient pressure as well as in the bcc phase at high pressure with unprecedented accuracy close to theab initioone. In both cases, a single step nucleation process is observed.

Quantum neural networks force fields generation

Accurate molecular force fields are of paramount importance for the efficient implementation of molecular dynamics techniques at large scales. In the last decade, machine learning (ML) methods have demonstrated impressive performances in predicting accurate values for energy and forces when trained on finite size ensembles generated with ab initio techniques. At the same time, quantum computers have recently started to offer new viable computational paradigms to tackle such problems. On the one hand, quantum algorithms may notably be used to extend the reach of electronic structure calculations. On the other hand, quantum ML is also emerging as an alternative and promising path to quantum advantage. Here we follow this second route and establish a direct connection between classical and quantum solutions for learning neural network (NN) potentials. To this end, we design a quantum NN architecture and apply it successfully to different molecules of growing complexity. The quantum models exhibit larger effective dimension with respect to classical counterparts and can reach competitive performances, thus pointing towards potential quantum advantages in natural science applications via quantum ML.

Water Flow in Single-Wall Nanotubes: Oxygen Makes It Slip, Hydrogen Makes It Stick

Experimental measurements have reported ultrafast and radius-dependent water transport in carbon nanotubes which are absent in boron nitride nanotubes. Despite considerable effort, the origin of this contrasting (and fascinating) behavior is not understood. Here, with the aid of machine learning-based molecular dynamics simulations that deliver first-principles accuracy, we investigate water transport in single-wall carbon and boron nitride nanotubes. Our simulations reveal a large, radius-dependent hydrodynamic slippage on both materials, with water experiencing indeed a ≈5 times lower friction on carbon surfaces compared to boron nitride. Analysis of the diffusion mechanisms across the two materials reveals that the fast water transport on carbon is governed by facile oxygen motion, whereas the higher friction on boron nitride arises from specific hydrogen-nitrogen interactions. This work not only delivers a clear reference of quantum mechanical accuracy for water flow in single-wall nanotubes but also provides detailed mechanistic insight into its radius and material dependence for future technological application.

Nanosecond solvation dynamics of the hematite/liquid water interface at hybrid DFT accuracy using committee neural network potentials

Metal oxide/water interfaces play an important role in biology, catalysis, energy storage and photocatalytic water splitting. The atomistic structure at these interfaces is often difficult to characterize by experimental techniques, whilst results from ab initio molecular dynamics simulations tend to be uncertain due to the limited length and time scales accessible. In this work, we train a committee neural network potential to simulate the hematite/water interface at the hybrid DFT level of theory to reach the nanosecond timescale and systems containing more than 3000 atoms. The NNP enables us to converge dynamical properties, not possible with brute-force ab initio molecular dynamics. Our simulations uncover a rich solvation dynamics at the hematite/water interface spanning three different time scales: picosecond H-bond dynamics between surface hydroxyls and the first water layer, in-plane/out-of-plane tilt motion of surface hydroxyls on the 10 ps time scale, and diffusion of water molecules from the oxide surface characterized by a mean residence lifetime of about 60 ps. Calculation of vibrational spectra confirm that H-bonds between surface hydroxyls and first layer water molecules are stronger than H-bonds in bulk water. Our study showcases how state of the art machine learning approaches can routinely be utilized to explore the structural dynamics at transition metal oxide interfaces with complex electronic structure. It foreshadows that c-NNPs are a promising tool to tackle the sampling problem in ab initio electrochemistry with explicit solvent molecules.

Flexible machine-learning interatomic potential for simulating structural disordering behavior of Li7La3Zr2O12 solid electrolytes

Batteries based on solid-state electrolytes, including Li₇La₃Zr₂O₁₂ (LLZO), promise improved safety and increased energy density; however, atomic disorder at grain boundaries and phase boundaries can severely deteriorate their performance. Machine-learning (ML) interatomic potentials offer a uniquely compelling solution for simulating chemical processes, rare events, and phase transitions associated with these complex interfaces by mixing high scalability with quantum-level accuracy, provided that they can be trained to properly address atomic disorder. To this end, we report the construction and validation of an ML potential that is specifically designed to simulate crystalline, disordered, and amorphous LLZO systems across a wide range of conditions. The ML model is based on a neural network algorithm and is trained using ab initio data. Performance tests prove that the developed ML potential can predict accurate structural and vibrational characteristics, elastic properties, and Li diffusivity of LLZO comparable to ab initio simulations. As a demonstration of its applicability to larger systems, we show that the potential can correctly capture grain boundary effects on diffusivity, as well as the thermal transition behavior of LLZO. These examples show that the ML potential enables simulations of transitions between well-defined and disordered structures with quantum-level accuracy at speeds thousands of times faster than ab initio methods.

Machine learning molecular dynamics simulations toward exploration of high-temperature properties of nuclear fuel materials: case study of thorium dioxide

Predicting materials properties of nuclear fuel compounds is a challenging task in materials science. Their thermodynamical behaviors around and above the operational temperature are essential for the design of nuclear reactors. However, they are not easy to measure, because the target temperature range is too high to perform various standard experiments safely and accurately. Moreover, theoretical methods such as first-principles calculations also suffer from the computational limitations in calculating thermodynamical properties due to their high calculation-costs and complicated electronic structures stemming from f-orbital occupations of valence electrons in actinide elements. Here, we demonstrate, for the first time, machine-learning molecular-dynamics to theoretically explore high-temperature thermodynamical properties of a nuclear fuel material, thorium dioxide. The target compound satisfies first-principles calculation accuracy because f-electron occupation coincidentally diminishes and the scheme meets sampling sufficiency because it works at the computational cost of classical molecular-dynamics levels. We prepare a set of training data using first-principles molecular dynamics with small number of atoms, which cannot directly evaluate thermodynamical properties but captures essential atomistic dynamics at the high temperature range. Then, we construct a machine-learning molecular-dynamics potential and carry out large-scale molecular-dynamics calculations. Consequently, we successfully access two kinds of thermodynamic phase transitions, namely the melting and the anomalous \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda$$\end{document}λ transition induced by large diffusions of oxygen atoms. Furthermore, we quantitatively reproduce various experimental data in the best agreement manner by selecting a density functional scheme known as SCAN. Our results suggest that the present scale-up simulation-scheme using machine-learning techniques opens up a new pathway on theoretical studies of not only nuclear fuel compounds, but also a variety of similar materials that contain both heavy and light elements, like thorium dioxide.

Machine learning accelerated random structure searching: Application to yttrium superhydrides

The search for new superhydrides, promising materials for both hydrogen storage and high temperature superconductivity, made great progress, thanks to atomistic simulations and Crystal Structure Prediction (CSP) algorithms. When they are combined with Density Functional Theory (DFT), these methods are highly reliable and often match a great part of the experimental results. However, systems of increasing complexity (number of atoms and chemical species) become rapidly challenging as the number of minima to explore grows exponentially with the number of degrees of freedom in the simulation cell. An efficient sampling strategy preserving a sustainable computational cost then remains to be found. We propose such a strategy based on an active-learning process where machine learning potentials and DFT simulations are jointly used, opening the way to the discovery of complex structures. As a proof of concept, this method is applied to the exploration of tin crystal structures under various pressures. We showed that the α phase, not included in the learning process, is correctly retrieved, despite its singular nature of bonding. Moreover, all the expected phases are correctly predicted under pressure (20 and 100 GPa), suggesting the high transferability of our approach. The method has then been applied to the search of yttrium superhydrides (YH x) crystal structures under pressure. The YH6 structure of space group Im-3m is successfully retrieved. However, the exploration of more complex systems leads to the appearance of a large number of structures. The selection of the relevant ones to be included in the active learning process is performed through the analysis of atomic environments and the clustering algorithm. Finally, a metric involving a distance based on x-ray spectra is introduced, which guides the structural search toward experimentally relevant structures. The global process (active-learning and new selection methods) is finally considered to explore more complex and unknown YH x phases, unreachable by former CSP algorithms. New complex phases are found, demonstrating the ability of our approach to push back the exponential wall of complexity related to CSP.

Assessing the persistence of chalcogen bonds in solution with neural network potentials

Non-covalent bonding patterns are commonly harvested as a design principle in the field of catalysis, supramolecular chemistry and functional materials to name a few. Yet, their computational description generally neglects finite temperature and environment effects, which promote competing interactions and alter their static gas-phase properties. Recently, neural network potentials (NNPs) trained on Density Functional Theory (DFT) data have become increasingly popular to simulate molecular phenomena in condensed phase with an accuracy comparable to ab initio methods. To date, most applications have centered on solid-state materials or fairly simple molecules made of a limited number of elements. Herein, we focus on the persistence and strength of chalcogen bonds involving a benzotelluradiazole in condensed phase. While the tellurium-containing heteroaromatic molecules are known to exhibit pronounced interactions with anions and lone pairs of different atoms, the relevance of competing intermolecular interactions, notably with the solvent, is complicated to monitor experimentally but also challenging to model at an accurate electronic structure level. Here, we train direct and baselined NNPs to reproduce hybrid DFT energies and forces in order to identify what are the most prevalent non-covalent interactions occurring in a solute-Cl$^-$-THF mixture. The simulations in explicit solvent highlight competition with chalcogen bonds formed with the solvent and the short-range directionality of the interaction with direct consequences for the molecular properties in the solution. The comparison with other potentials (e.g., AMOEBA, direct NNP and continuum solvent model) also demonstrates that baselined NNPs offer a reliable picture of the non-covalent interaction interplay occurring in solution.

REANN: A PyTorch-based end-to-end multi-functional deep neural network package for molecular, reactive, and periodic systems

In this work, we present a general purpose deep neural network package for representing energies, forces, dipole moments, and polarizabilities of atomistic systems. This so-called recursively embedded atom neural network model takes advantages of both the physically inspired atomic descriptor based neural networks and the message-passing based neural networks. Implemented in the PyTorch framework, the training process is parallelized on both the central processing unit and the graphics processing unit with high efficiency and low memory in which all hyperparameters can be optimized automatically. We demonstrate the state-of-the-art accuracy, high efficiency, scalability, and universality of this package by learning not only energies (with or without forces) but also dipole moment vectors and polarizability tensors in various molecular, reactive, and periodic systems. An interface between a trained model and LAMMPs is provided for large scale molecular dynamics simulations. We hope that this open-source toolbox will allow for future method development and applications of machine learned potential energy surfaces and quantum-chemical properties of molecules, reactions, and materials.

A Differentiable Neural-Network Force Field for Ionic Liquids

We present NeuralIL, a model for the potential energy of an ionic liquid that accurately reproduces first-principles results with orders-of-magnitude savings in computational cost. Built on the basis of a multilayer perceptron and spherical Bessel descriptors of the atomic environments, NeuralIL is implemented in such a way as to be fully automatically differentiable. It can thus be trained on ab initio forces instead of just energies, to make the most out of the available data, and can efficiently predict arbitrary derivatives of the potential energy. Using ethylammonium nitrate as the test system, we obtain out-of-sample accuracies better than 2 meV atom-1 (<0.05 kcal mol-1) in the energies and 70 meV Å-1 in the forces. We show that encoding the element-specific density in the spherical Bessel descriptors is key to achieving this. Harnessing the information provided by the forces drastically reduces the amount of atomic configurations required to train a neural network force field based on atom-centered descriptors. We choose the Swish-1 activation function and discuss the role of this choice in keeping the neural network differentiable. Furthermore, the possibility of training on small data sets allows for an ensemble-learning approach to the detection of extrapolation. Finally, we find that a separate treatment of long-range interactions is not required to achieve a high-quality representation of the potential energy surface of these dense ionic systems.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

CO organization at ambient pressure on stepped Pt surfaces: first principles modeling accelerated by neural networks

Step and kink sites at Pt surfaces have crucial importance in catalysis. We employ a high dimensional neural network potential (HDNNP) trained using first-principles calculations to determine the adsorption structure of CO under ambient conditions (T = 300 K and P = 1 atm) on these surfaces. To thoroughly explore the potential energy surface (PES), we use a modified basin hopping method. We utilize the explored PES to identify the adsorbate structures and show that under the considered conditions several low free energy structures exist. Under the considered temperature and pressure conditions, the step edge (or kink) is totally occupied by on-top CO molecules. We show that the step structure and the structure of CO molecules on the step dictate the arrangement of CO molecules on the lower terrace. On surfaces with (111) steps, like Pt(553), CO forms quasi-hexagonal structures on the terrace with the top site preferred, with on average two top site CO for one multiply bonded CO, while in contrast surfaces with (100) steps, like Pt(557), present a majority of multiply bonded CO on their terrace. Short terraced surfaces, like Pt(643), with square (100) steps that are broken by kink sites constrain the CO arrangement parallel to the step edge. Overall, this effort provides detailed analysis on the influence of the step edge structure, kink sites, and terrace width on the organization of CO molecules on non-reconstructed stepped surfaces, yielding initial structures for understanding restructuring events driven by CO at high coverages and ambient pressure.

Training algorithm matters for the performance of neural network potential: A case study of Adam and the Kalman filter optimizers

One hidden yet important issue for developing neural network potentials (NNPs) is the choice of training algorithm. In this article, we compare the performance of two popular training algorithms, the adaptive moment estimation algorithm (Adam) and the extended Kalman filter algorithm (EKF), using the Behler-Parrinello neural network and two publicly accessible datasets of liquid water [Morawietz et al., Proc. Natl. Acad. Sci. U. S. A. 113, 8368-8373, (2016) and Cheng et al., Proc. Natl. Acad. Sci. U. S. A. 116, 1110-1115, (2019)]. This is achieved by implementing EKF in TensorFlow. It is found that NNPs trained with EKF are more transferable and less sensitive to the value of the learning rate, as compared to Adam. In both cases, error metrics of the validation set do not always serve as a good indicator for the actual performance of NNPs. Instead, we show that their performance correlates well with a Fisher information based similarity measure.

Automated Construction of Neural Network Potential Energy Surface: The Enhanced Self-Organizing Incremental Neural Network Deep Potential Method

In recent years, the use of deep learning (neural network) potential energy surface (NNPES) in molecular dynamics simulation has experienced explosive growth as it can be as accurate as quantum chemistry methods while being as efficient as classical mechanic methods. However, the development of NNPES is highly nontrivial. In particular, it has been troubling to construct a dataset that is as small as possible yet can cover the target chemical space. In this work, an ESOINN-DP method is developed, which has the enhanced self-organizing incremental neural network (ESOINN) and a newly proposed error indicator at its core. With ESOINN-DP, one can construct the NNPES with little human intervention, and this method ensures that the constructed reference dataset covers the target chemical space with minimum redundancy. The performance of the ESOINN-DP method has been well validated by developing neural network potential energy surfaces for water clusters, tripeptides, and by de-redundancy of a sub-dataset of the ANI-1 database. We believe that the ESOINN-DP method provides a novel idea for the construction of NNPES and, especially, the reference datasets, and it can be used for molecular dynamics (MD) simulations of various gas-phase and condensed-phase chemical systems.

Subsystem Density Functional Theory Augmented by a Delta Learning Approach to Achieve Kohn-Sham Accuracy

Simulations based on electronic structure theory naturally include polarization and have no transferability problems. In particular, Kohn-Sham density functional theory (KS-DFT) has become the method of reference for ab initio molecular dynamics simulations of condensed matter systems. However, the high computational cost often poses strict limits on the affordable system size as well as on the extension of sampling (number of configurations). In this work, we propose an improvement to the subsystem density functional theory approach, known as the Kim-Gordon (KG) scheme, thus enabling the sampling of configurations for condensed molecular systems keeping the KS-DFT level accuracy at a fraction of computer time. Our scheme compensates the known KG shortcomings of the electronic kinetic energy term by adding a simple correction and can match KS-DFT accuracy in energies and forces. The computationally cheap correction is determined by means of a machine learning procedure. The proposed KG scheme is applied within a linear scaling self-consistent field formalism and is assessed by a series of molecular dynamics simulations of liquid water under different conditions. Although system-dependent, the correction is transferable between system sizes and temperatures.

Machine learning potentials for complex aqueous systems made simple

Understanding complex materials, in particular those with solid–liquid interfaces, such as water on surfaces or under confinement, is a key challenge for technological and scientific progress. Although established simulation approaches have been able to provide important atomistic insight, ab initio techniques struggle with the required time and length scales, while force field methods can often be limited in terms of their accuracy. Here we show how these limitations can be overcome in a simple and automated machine learning procedure to provide accurate models of interactions at the ab initio level, as illustrated for a variety of complex aqueous systems. These developments open up the prospect of the straightforward exploration of many technologically relevant systems by molecular simulations.

Simulation techniques based on accurate and efficient representations of potential energy surfaces are urgently needed for the understanding of complex systems such as solid–liquid interfaces. Here we present a machine learning framework that enables the efficient development and validation of models for complex aqueous systems. Instead of trying to deliver a globally optimal machine learning potential, we propose to develop models applicable to specific thermodynamic state points in a simple and user-friendly process. After an initial ab initio simulation, a machine learning potential is constructed with minimum human effort through a data-driven active learning protocol. Such models can afterward be applied in exhaustive simulations to provide reliable answers for the scientific question at hand or to systematically explore the thermal performance of ab initio methods. We showcase this methodology on a diverse set of aqueous systems comprising bulk water with different ions in solution, water on a titanium dioxide surface, and water confined in nanotubes and between molybdenum disulfide sheets. Highlighting the accuracy of our approach with respect to the underlying ab initio reference, the resulting models are evaluated in detail with an automated validation protocol that includes structural and dynamical properties and the precision of the force prediction of the models. Finally, we demonstrate the capabilities of our approach for the description of water on the rutile titanium dioxide (110) surface to analyze the structure and mobility of water on this surface. Such machine learning models provide a straightforward and uncomplicated but accurate extension of simulation time and length scales for complex systems.

Improved description of atomic environments using low-cost polynomial functions with compact support

The prediction of chemical properties using machine learning techniques calls for a set of appropriate descriptors that accurately describe atomic and, on a larger scale, molecular environments. A mapping of conformational information on a space spanned by atom-centred symmetry functions (SF) has become a standard technique for energy and force predictions using high-dimensional neural network potentials (HDNNP). An appropriate choice of SFs is particularly crucial for accurate force predictions. Established atom-centred SFs, however, are limited in their flexibility, since their functional form restricts the angular domain that can be sampled without introducing problematic derivative discontinuities. Here, we introduce a class of atom-centred SFs based on polynomials with compact support called polynomial symmetry functions (PSF), which enable a free choice of both, the angular and the radial domain covered. We demonstrate that the accuracy of PSFs is either on par or considerably better than that of conventional, atom-centred SFs. In particular, a generic set of PSFs with an intuitive choice of the angular domain inspired by organic chemistry considerably improves prediction accuracy for organic molecules in the gaseous and liquid phase, with reductions in force prediction errors over a test set approaching 50% for certain systems. Contrary to established atom-centred SFs, computation of PSF does not involve any exponentials, and their intrinsic compact support supersedes use of separate cutoff functions, facilitating the choice of their free parameters. Most importantly, the number of floating point operations required to compute polynomial SFs introduced here is considerably lower than that of other state-of-the-art SFs, enabling their efficient implementation without the need of highly optimised code structures or caching, with speedups with respect to other state-of-the-art SFs reaching a factor of 4.5 to 5. This low-effort performance benefit substantially simplifies their use in new programs and emerging platforms such as graphical processing units. Overall, polynomial SFs with compact support improve accuracy of both, energy and force predictions with HDNNPs while enabling significant speedups compared to their well-established counterparts.

Machine Learning for Electronically Excited States of Molecules

Electronically excited states of molecules are at the heart of photochemistry, photophysics, as well as photobiology and also play a role in material science. Their theoretical description requires highly accurate quantum chemical calculations, which are computationally expensive. In this review, we focus on not only how machine learning is employed to speed up such excited-state simulations but also how this branch of artificial intelligence can be used to advance this exciting research field in all its aspects. Discussed applications of machine learning for excited states include excited-state dynamics simulations, static calculations of absorption spectra, as well as many others. In order to put these studies into context, we discuss the promises and pitfalls of the involved machine learning techniques. Since the latter are mostly based on quantum chemistry calculations, we also provide a short introduction into excited-state electronic structure methods and approaches for nonadiabatic dynamics simulations and describe tricks and problems when using them in machine learning for excited states of molecules.