A look inside the black box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters

The role of charge in microdroplet redox chemistry

In charged water microdroplets, which occur in nature or in the lab upon ultrasonication or in electrospray processes, the thermodynamics for reactive chemistry can be dramatically altered relative to the bulk phase. Here, we provide a theoretical basis for the observation of accelerated chemistry by simulating water droplets of increasing charge imbalance to create redox agents such as hydroxyl and hydrogen radicals and solvated electrons. We compute the hydration enthalpy of OH- and H+ that controls the electron transfer process, and the corresponding changes in vertical ionization energy and vertical electron affinity of the ions, to create OH• and H• reactive species. We find that at ~ 20 - 50% of the Rayleigh limit of droplet charge the hydration enthalpy of both OH- and H+ have decreased by >50 kcal/mol such that electron transfer becomes thermodynamically favorable, in correspondence with the more favorable vertical electron affinity of H+ and the lowered vertical ionization energy of OH-. We provide scaling arguments that show that the nanoscale calculations and conclusions extend to the experimental microdroplet length scale. The relevance of the droplet charge for chemical reactivity is illustrated for the formation of H2O2, and has clear implications for other redox reactions observed to occur with enhanced rates in microdroplets.

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges, such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs, such as social networks. We demonstrate a novel hardware-software codesign approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations, including a planner and vectorization for the gather-scatter operations targeted for Graphcore's Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed codesign approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes, and sparsity. We demonstrate that such a codesign approach can reduce the training time of atomistic GNNs and can improve their performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

Descriptors of water aggregation

We rely on a total of 23 (cluster size, 8 structural, and 14 connectivity) descriptors to investigate structural patterns and connectivity motifs associated with water cluster aggregation. In addition to the cluster size n (number of molecules), the 8 structural descriptors can be further categorized into (i) one-body (intramolecular): covalent OH bond length (rOH) and HOH bond angle (θHOH), (ii) two-body: OO distance (rOO), OHO angle (θOHO), and HOOX dihedral angle (ϕHOOX), where X lies on the bisector of the HOH angle, (iii) three-body: OOO angle (θOOO), and (iv) many-body: modified tetrahedral order parameter (q) to account for two-, three-, four-, five-coordinated molecules (qm, m = 2, 3, 4, 5) and radius of gyration (Rg). The 14 connectivity descriptors are all many-body in nature and consist of the AD, AAD, ADD, AADD, AAAD, AAADD adjacencies [number of hydrogen bonds accepted (A) and donated (D) by each water molecule], Wiener index, Average Shortest Path Length, hydrogen bond saturation (% HB), and number of non-short-circuited three-membered cycles, four-membered cycles, five-membered cycles, six-membered cycles, and seven-membered cycles. We mined a previously reported database of 4 948 959 water cluster minima for (H2O)n, n = 3-25 to analyze the evolution and correlation of these descriptors for the clusters within 5 kcal/mol of the putative minima. It was found that rOH and % HB correlated strongly with cluster size n, which was identified as the strongest predictor of energetic stability. Marked changes in the adjacencies and cycle count were observed, lending insight into changes in the hydrogen bond network upon aggregation. A Principal Component Analysis (PCA) was employed to identify descriptor dependencies and group clusters into specific structural patterns across different cluster sizes. The results of this study inform our understanding of how water clusters evolve in size and what appropriate descriptors of their structural and connectivity patterns are with respect to system size, stability, and similarity. The approach described in this study is general and can be easily extended to other hydrogen-bonded systems.

A convolutional neural network to characterize mouse hindlimb foot strikes during voluntary wheel running

Voluntary wheel running (VWR) is widely used to study how exercise impacts a variety of physiologies and pathologies in rodents. The primary activity readout of VWR is aggregated wheel turns over a given time interval (most often, days). Given the typical running frequency of mice (∼4 Hz) and the intermittency of voluntary running, aggregate wheel turn counts, therefore, provide minimal insight into the heterogeneity of voluntary activity. To overcome this limitation, we developed a six-layer convolutional neural network (CNN) to determine the hindlimb foot strike frequency of mice exposed to VWR. Aged female C57BL/6 mice (22 months, n = 6) were first exposed to wireless angled running wheels for 2 h/d, 5 days/wk for 3 weeks with all VWR activities recorded at 30 frames/s. To validate the CNN, we manually classified foot strikes within 4800 1-s videos (800 randomly chosen for each mouse) and converted those values to frequency. Upon iterative optimization of model architecture and training on a subset of classified videos (4400), the CNN model achieved an overall training set accuracy of 94%. Once trained, the CNN was validated on the remaining 400 videos (accuracy: 81%). We then applied transfer learning to the CNN to predict the foot strike frequency of young adult female C57BL6 mice (4 months, n = 6) whose activity and gait differed from old mice during VWR (accuracy: 68%). In summary, we have developed a novel quantitative tool that non-invasively characterizes VWR activity at a much greater resolution than was previously accessible. This enhanced resolution holds potential to overcome a primary barrier to relating intermittent and heterogeneous VWR activity to induced physiological responses.

Appropriate clusterset selection for the prediction of thermodynamic properties of liquid water with QCE theory

Evident in many physical and chemical phenomena, thermodynamics is the study of how energy is stored, transformed and transferred in a molecule or material. However, prediction of these properties with simulation techniques is a non-trivial task as several factors such as composition and intermolecular interactions come into play. While molecular dynamics and ab initio molecular dynamics are the most common techniques for the prediction of thermodynamic properties, there exists many shortcomings associated with their use. Therefore, in this work we instead apply QCE theory to predict the thermodynamic properties of liquid water. This theory assumes that a condensed phase system can be represented as a 'mixture' of varying sized clusters rather than as a continuum. As QCE theory relies on first-principle simulations and statistical thermodynamics to determine the thermodynamic behavior of a system, appropriate selection of clusters is a crucial step towards achieving accurate predictions. In this study, we use molecular dynamics and ab initio calculations to obtain initial configurations of 400 water clusters, Wn where n = 3 to 10 and contrast their stability using two different criteria. The role of entropy towards cluster stabilization is investigated by comparing the binding (ΔE_BIND/mol) and Gibbs free binding energy per molecule (ΔG_BIND/mol) of various Wn at 298.15 K. Initial clustersets are constructed by exploring two-, three-, four and five-combinations of clustersets using the minimum ΔG_BIND/mol structures of Wn. We also expand the ΔG_BIND/mol criteria for Wn of sizes 3 to 7 to include values larger than 0.0 kJ mol^-1 and smaller than 3.0 kJ mol^-1 as a means of improving thermodynamic predictions. 37 of the 459 resulting clustersets predicted the correct boiling point of water and its thermodynamic properties with an accuracy of 95%. A scaled population-weighted infrared spectrum was compared to experimental results to validate the composition of the top 5 clustersets. The predicted spectra showed an exact match within the low frequency range (<1000 cm^-1) with some discrepancy at the high frequency range (>3400 cm^-1). This work highlights that ΔG_BIND/mol is so far the best criteria to apply when determining an appropriate clusterset for QCE theory.

Active sampling for neural network potentials: Accelerated simulations of shear-induced deformation in Cu-Ni multilayers

Neural network potentials (NNPs) can greatly accelerate atomistic simulations relative to ab initio methods, allowing one to sample a broader range of structural outcomes and transformation pathways. In this work, we demonstrate an active sampling algorithm that trains an NNP that is able to produce microstructural evolutions with accuracy comparable to those obtained by density functional theory, exemplified during structure optimizations for a model Cu-Ni multilayer system. We then use the NNP, in conjunction with a perturbation scheme, to stochastically sample structural and energetic changes caused by shear-induced deformation, demonstrating the range of possible intermixing and vacancy migration pathways that can be obtained as a result of the speedups provided by the NNP. The code to implement our active learning strategy and NNP-driven stochastic shear simulations is openly available at https://github.com/pnnl/Active-Sampling-for-Atomistic-Potentials.

Do Machine-Learning Atomic Descriptors and Order Parameters Tell the Same Story? The Case of Liquid Water

Machine-learning (ML) has become a key workhorse in molecular simulations. Building an ML model in this context involves encoding the information on chemical environments using local atomic descriptors. In this work, we focus on the Smooth Overlap of Atomic Positions (SOAP) and their application in studying the properties of liquid water both in the bulk and at the hydrophobic air-water interface. By using a statistical test aimed at assessing the relative information content of different distance measures defined on the same data space, we investigate if these descriptors provide the same information as some of the common order parameters that are used to characterize local water structure such as hydrogen bonding, density, or tetrahedrality to name a few. Our analysis suggests that the ML description and the standard order parameters of the local water structure are not equivalent. In particular, a combination of these order parameters probing local water environments can predict SOAP similarity only approximately, and vice versa, the environments that are similar according to SOAP are not necessarily similar according to the standard order parameters. We also elucidate the role of some of the metaparameters in the SOAP definition in encoding chemical information.

Chimney Identification Tool for Automated Detection of Hydrothermal Chimneys from High-Resolution Bathymetry Using Machine Learning

Identifying the locations of hydrothermal chimneys across mapped areas of seafloor spreading ridges unlocks the ability to research questions about their correlations to geology, the cooling of the lithosphere, and deep-sea biogeography. We developed a Chimney Identification Tool (CIT) that utilizes a Convolutional Neural Network (CNN) to classify 1 m gridded AUV bathymetry and identify the locations of hydrothermal vent chimneys. A CNN is a type of Machine-Learning model that is able to classify raster data based on the shapes and textures in the input, making it ideal for this task. The criteria that have been used in previous manual classifications of chimneys have focused on the round base and spire shape of the features, and are not easily quantifiable. Machine-Learning techniques have previously been implemented with sonar data to classify seafloor geology, but this is the first application of these methods to hydrothermal systems. In developing the CIT, we compiled the bathymetry data from two rasters from the Endeavor Ridge—each gridded at a 1 m resolution—containing 34 locations of known hydrothermal chimneys, and from the 92° W segment of the Galapagos Spreading Center (GSC) containing 14. The CIT produced a primary group of outputs with 96% agreement with the manual classification; moreover, it correctly caught 29 of the 34 known chimneys from Endeavor and 10 of the 14 from the GSC. The CIT is trained to identify features with the characteristic shape of a hydrothermal vent chimney; therefore, it is susceptible to the misclassification of unusually shaped cases, given the limited training data. As a result, to provide the option of having a more inclusive application, the CIT also produced a secondary group of output locations with 61% agreement with the manual classification; moreover, it caught three of the four additional known chimneys from the GSC and four of the five from Endeavor. The CIT will be used in future investigations where an inventory of individual chimneys is important, such as the cataloguing of off-axis hydrothermal venting and the investigation of chimney distribution in connection to seafloor eruptions.

A Benchmark Protocol for DFT Approaches and Data-Driven Models for Halide-Water Clusters

Dissolved ions in aqueous media are ubiquitous in many physicochemical processes, with a direct impact on research fields, such as chemistry, climate, biology, and industry. Ions play a crucial role in the structure of the surrounding network of water molecules as they can either weaken or strengthen it. Gaining a thorough understanding of the underlying forces from small clusters to bulk solutions is still challenging, which motivates further investigations. Through a systematic analysis of the interaction energies obtained from high-level electronic structure methodologies, we assessed various dispersion-corrected density functional approaches, as well as ab initio-based data-driven potential models for halide ion-water clusters. We introduced an active learning scheme to automate the generation of optimally weighted datasets, required for the development of efficient bottom-up anion-water models. Using an evolutionary programming procedure, we determined optimized and reference configurations for such polarizable and first-principles-based representation of the potentials, and we analyzed their structural characteristics and energetics in comparison with estimates from DF-MP2 and DFT+D quantum chemistry computations. Moreover, we presented new benchmark datasets, considering both equilibrium and non-equilibrium configurations of higher-order species with an increasing number of water molecules up to 54 for each F, Cl, Br, and I anions, and we proposed a validation protocol to cross-check methods and approaches. In this way, we aim to improve the predictive ability of future molecular computer simulations for determining the ongoing conflicting distribution of different ions in aqueous environments, as well as the transition from nanoscale clusters to macroscopic condensed phases.

Manifolds of low energy structures for a magic number of hydrated sulfate: SO42-(H2O)24

Low energy structures of SO₄^2-(H₂O)₂₄ have been obtained using a combination of classical molecular dynamics simulations and refinement of structures and energies by quantum chemical calculations. Extensive exploration of the potential energy surface led to a number of low-energy structures, confirmed by accurate calibration calculations. An overall analysis of this large set was made after devising appropriate structural descriptors such as the numbers of cycles and their combinations. Low energy structures bear common motifs, the most prominent being fused cycles involving alternatively four and six water molecules. The latter adopt specific conformations which ensure the appropriate surface curvature to form a closed cage without dangling O-H bonds and at the same time provide 12-coordination of the sulfate ion. A prominent feature to take into account is isomerism via inversion of hydrogen bond orientations along cycles. This generates large families of ca. 100 isomers for this cluster size, spanning energy windows of 10-30 kJ mol^-1. This relatively ignored isomerism must be taken into account to identify reliably the lowest energy minima. The overall picture is that the magic number cluster SO₄^2-(H₂O)₂₄ does not correspond to formation of a single, remarkable structure, but rather to a manifold of structural families with similar stabilities. Extensive calculations on isomerization mechanisms within a family indicate that large barriers are associated to direct inversion of hydrogen bond networks. Possible implications of these results for magic number clusters of other anions are discussed.

A Machine Learning Model to Classify Dynamic Processes in Liquid Water*

The dynamics of water molecules plays a vital role in understanding water. We combined computer simulation and deep learning to study the dynamics of H-bonds between water molecules. Based on ab initio molecular dynamics simulations and a newly defined directed Hydrogen (H-) bond population operator, we studied a typical dynamic process in bulk water: interchange, in which the H-bond donor reverses roles with the acceptor. By designing a recurrent neural network-based model, we have successfully classified the interchange and breakage processes in water. We have found that the ratio between them is approximately 1 : 4, and it hardly depends on temperatures from 280 to 360 K. This work implies that deep learning has the great potential to help distinguish complex dynamic processes containing H-bonds in other systems.

Machine learning meets chemical physics

Over recent years, the use of statistical learning techniques applied to chemical problems has gained substantial momentum. This is particularly apparent in the realm of physical chemistry, where the balance between empiricism and physics-based theory has traditionally been rather in favor of the latter. In this guest Editorial for the special topic issue on "Machine Learning Meets Chemical Physics," a brief rationale is provided, followed by an overview of the topics covered. We conclude by making some general remarks.