A generalized deep learning approach for local structure identification in molecular simulations

Identifying local structure in molecular simulations is of utmost importance. The most common existing approach to identify local structure is to calculate some geometrical quantity referred to as an order parameter. In simple cases order parameters are physically intuitive and trivial to develop (e.g., ion-pair distance), however in most cases, order parameter development becomes a much more difficult endeavor (e.g., crystal structure identification). Using ideas from computer vision, we adapt a specific type of neural network called a PointNet to identify local structural environments in molecular simulations. A primary challenge in applying machine learning techniques to simulation is selecting the appropriate input features. This challenge is system-specific and requires significant human input and intuition. In contrast, our approach is a generic framework that requires no system-specific feature engineering and operates on the raw output of the simulations, i.e., atomic positions. We demonstrate the method on crystal structure identification in Lennard-Jones (four different phases), water (eight different phases), and mesophase (six different phases) systems. The method achieves as high as 99.5% accuracy in crystal structure identification. The method is applicable to heterogeneous nucleation and it can even predict the crystal phases of atoms near external interfaces. We demonstrate the versatility of our approach by using our method to identify surface hydrophobicity based solely upon positions and orientations of surrounding water molecules. Our results suggest the approach will be broadly applicable to many types of local structure in simulations.

Deep Learning in Chemistry

Machine learning enables computers to address problems by learning from data. Deep learning is a type of machine learning that uses a hierarchical recombination of features to extract pertinent information and then learn the patterns represented in the data. Over the last eight years, its abilities have increasingly been applied to a wide variety of chemical challenges, from improving computational chemistry to drug and materials design and even synthesis planning. This review aims to explain the concepts of deep learning to chemists from any background and follows this with an overview of the diverse applications demonstrated in the literature. We hope that this will empower the broader chemical community to engage with this burgeoning field and foster the growing movement of deep learning accelerated chemistry.

From Molecular Fragments to the Bulk: Development of a Neural Network Potential for MOF-5

The development of first-principles-quality reactive atomistic potentials for organic-inorganic hybrid materials is still a substantial challenge because of the very different physics of the atomic interactions-from covalent via ionic bonding to dispersion-that have to be described in an accurate and balanced way. In this work we used a prototypical metal-organic framework, MOF-5, as a benchmark case to investigate the applicability of high-dimensional neural network potentials (HDNNPs) to this class of materials. In HDNNPs, which belong to the class of machine learning potentials, the energy is constructed as a sum of environment-dependent atomic energy contributions. We demonstrate that by the use of this approach it is possible to obtain a high-quality potential for the periodic MOF-5 crystal using density functional theory (DFT) reference calculations of small molecular fragments only. The resulting HDNNP, which has a root-mean-square error (RMSE) of 1.6 meV/atom for the energies of molecular fragments not included in the training set, is able to provide the equilibrium lattice constant of the bulk MOF-5 structure with an error of about 0.1% relative to DFT, and also, the negative thermal expansion behavior is accurately predicted. The total energy RMSE of periodic structures that are completely absent in the training set is about 6.5 meV/atom, with errors on the order of 2 meV/atom for energy differences. We show that in contrast to energy differences, achieving a high accuracy for total energies requires careful variation of the stoichiometries of the training structures to avoid energy offsets, as atomic energies are not physical observables. The forces, which have RMSEs of about 94 meV/ a₀ for the molecular fragments and 130 meV/ a₀ for bulk structures not included in the training set, are insensitive to such offsets. Therefore, forces, which are the relevant properties for molecular dynamics simulations, provide a realistic estimate of the accuracy of atomistic potentials.

Physically informed artificial neural networks for atomistic modeling of materials

Large-scale atomistic computer simulations of materials heavily rely on interatomic potentials predicting the energy and Newtonian forces on atoms. Traditional interatomic potentials are based on physical intuition but contain few adjustable parameters and are usually not accurate. The emerging machine-learning (ML) potentials achieve highly accurate interpolation within a large DFT database but, being purely mathematical constructions, suffer from poor transferability to unknown structures. We propose a new approach that can drastically improve the transferability of ML potentials by informing them of the physical nature of interatomic bonding. This is achieved by combining a rather general physics-based model (analytical bond-order potential) with a neural-network regression. This approach, called the physically informed neural network (PINN) potential, is demonstrated by developing a general-purpose PINN potential for Al. We suggest that the development of physics-based ML potentials is the most effective way forward in the field of atomistic simulations.

Neural Networks Are Promising Tools for the Prediction of the Viscosity of Unsaturated Polyester Resins

Unsaturated polyester resins are widely used for the preparation of composite materials and fulfill the majority of practical requirements for industrial and domestic applications at low cost. These resins consist of a highly viscous polyester oligomer and a reactive diluent, which allows its process ability and its crosslinking. The viscosity of the initial polyester and the reactive diluent mixture is critical for practical applications. So far, these viscosities were determined by trial and error which implies a time-consuming succession of manipulations, to achieve the targeted viscosities. In this work, we developed a strategy for predicting the viscosities of unsaturated polyesters formulation based on neural networks. In a first step 15 unsaturated polyesters have been synthesized through high-temperature polycondensation using usual monomers. Experimental Hansen solubility parameters (HSP) were determined from solubility experiment with HSPiP software and glass transition temperatures (T_g) were measured by Differential Scanning Calorimetry (DSC). Quantitative Structure—Property Relationship (QSPR) coupled to multiple linear regressions have been used to get a prediction of Hansen solubility parameters δ_d, δ_p, and δ_h from structural composition. A second QSPR regression has been done on glass transition temperature (prediction vs. experimental coefficient of determination R² = 0.93) of these unsaturated polyesters. These unsaturated polyesters were next diluted in several solvents with different natures (ethers, esters, alcohol, aromatics for example) at different concentrations. Viscosities at room temperature of these polyesters in solution were finally measured in order to create a database of 220 entries with 7 descriptors (polyester molecular weight, T_g, dispersity index Ð, polyester-solvent HSP RED, molar volume of the solvent, δ_h of the solvent, concentration of polyester in solvent). The QSPR method for predicting the viscosity from these 6 descriptors proved to be ineffective (R² = 0.56) as viscosities exhibit non-linear phenomena. A Neural Network with an optimized number of 12 hidden neurons has been trained with 179 entries to predict the viscosity. A correlation between experimental and predicted viscosities based on 41 testing instances gave a correlation coefficient R² of 0.88 and a predicted vs. measured slope of 0.98. Thanks to Neural Networks, new developments with eco-friendly reactive diluents can be accelerated.

A novel approach to describe chemical environments in high-dimensional neural network potentials

A central concern of molecular dynamics simulations is the potential energy surfaces that govern atomic interactions. These hypersurfaces define the potential energy of the system and have generally been calculated using either predefined analytical formulas (classical) or quantum mechanical simulations (ab initio). The former can accurately reproduce only a selection of material properties, whereas the latter is restricted to short simulation times and small systems. Machine learning potentials have recently emerged as a third approach to model atomic interactions, and are purported to offer the accuracy of ab initio simulations with the speed of classical potentials. However, the performance of machine learning potentials depends crucially on the description of a local atomic environment. A set of invariant, orthogonal, and differentiable descriptors for an atomic environment is proposed, implemented in a neural network potential for solid-state silicon, and tested in molecular dynamics simulations. Neural networks using the proposed descriptors are found to outperform ones using the Behler-Parinello and smooth overlap of atomic position descriptors in the literature.

Unsupervised machine learning in atomistic simulations, between predictions and understanding

Automated analyses of the outcome of a simulation have been an important part of atomistic modeling since the early days, addressing the need of linking the behavior of individual atoms and the collective properties that are usually the final quantity of interest. Methods such as clustering and dimensionality reduction have been used to provide a simplified, coarse-grained representation of the structure and dynamics of complex systems from proteins to nanoparticles. In recent years, the rise of machine learning has led to an even more widespread use of these algorithms in atomistic modeling and to consider different classification and inference techniques as part of a coherent toolbox of data-driven approaches. This perspective briefly reviews some of the unsupervised machine-learning methods-that are geared toward classification and coarse-graining of molecular simulations-seen in relation to the fundamental mathematical concepts that underlie all machine-learning techniques. It discusses the importance of using concise yet complete representations of atomic structures as the starting point of the analyses and highlights the risk of introducing preconceived biases when using machine learning to rationalize and understand structure-property relations. Supervised machine-learning techniques that explicitly attempt to predict the properties of a material given its structure are less susceptible to such biases. Current developments in the field suggest that using these two classes of approaches side-by-side and in a fully integrated mode, while keeping in mind the relations between the data analysis framework and the fundamental physical principles, will be key to realizing the full potential of machine learning to help understand the behavior of complex molecules and materials.

Supervised machine learning of ultracold atoms with speckle disorder

We analyze how accurately supervised machine learning techniques can predict the lowest energy levels of one-dimensional noninteracting ultracold atoms subject to the correlated disorder due to an optical speckle field. Deep neural networks with different numbers of hidden layers and neurons per layer are trained on large sets of instances of the speckle field, whose energy levels have been preventively determined via a high-order finite difference technique. The Fourier components of the speckle field are used as the feature vector to represent the speckle-field instances. A comprehensive analysis of the details that determine the possible success of supervised machine learning tasks, namely the depth and the width of the neural network, the size of the training set, and the magnitude of the regularization parameter, is presented. It is found that ground state energies of previously unseen instances can be predicted with an essentially negligible error given a computationally feasible number of training instances. First and second excited state energies can be predicted too, albeit with slightly lower accuracy and using more layers of hidden neurons. We also find that a three-layer neural network is remarkably resilient to Gaussian noise added to the training-set data (up to 10% noise level), suggesting that cold-atom quantum simulators could be used to train artificial neural networks.

A critical comparison of neural network potentials for molecular reaction dynamics with exact permutation symmetry

Several symmetry strategies have been compared in fitting full dimensional accurate potentials for reactive systems based on a neural network approach.

Representation of coupled adiabatic potential energy surfaces using neural network based quasi-diabatic Hamiltonians: 1,2 2A′ states of LiFH

A general algorithm for determining diabatic representations from adiabatic energies, energy gradients and derivative couplings using neural networks is introduced.

Bayesian machine learning for quantum molecular dynamics

This article discusses applications of Bayesian machine learning for quantum molecular dynamics.

The role of molecular modelling and simulation in the discovery and deployment of metal-organic frameworks for gas storage and separation

Metal-organic frameworks (MOFs) are highly tuneable, extended-network, crystalline, nanoporous materials with applications in gas storage, separations, and sensing. We review how molecular models and simulations of gas adsorption in MOFs have informed the discovery of performant MOFs for methane, hydrogen, and oxygen storage, xenon, carbon dioxide, and chemical warfare agent capture, and xylene enrichment. Particularly, we highlight how large, open databases of MOF crystal structures, post-processed to enable molecular simulations, are a platform for computational materials discovery. We discuss how to orient research efforts to routinise the computational discovery of MOFs for adsorption-based engineering applications.

Representing local atomic environment using descriptors based on local correlations

Statistical learning of material properties is an emerging topic of research and has been tremendously successful in areas such as representing complex energy landscapes as well as in technologically relevant areas, like identification of better catalysts and electronic materials. However, analysis of large data sets to efficiently learn characteristic features of a complex energy landscape, for example, depends on the ability of descriptors to effectively screen different local atomic environments. Thus, discovering appropriate descriptors of bulk or defect properties and the functional dependence of such properties on these descriptors remains a difficult and tedious process. To this end, we develop a framework to generate descriptors based on many-body correlations that can effectively capture intrinsic geometric features of the local environment of an atom. These descriptors are based on the spectrum of two-body, three-body, four-body, and higher order correlations between an atom and its neighbors and are evaluated by calculating the corresponding two-body, three-body, and four-body overlap integrals. They are invariant to global translation, global rotation, reflection, and permutations of atomic indices. By systematically testing the ability to capture the local atomic environment, it is shown that the local correlation descriptors are able to successfully reconstruct structures containing 10-25 atoms which was previously not possible.

Deep Learning for Nonadiabatic Excited-State Dynamics

In this work we show that deep learning (DL) can be used for exploring complex and highly nonlinear multistate potential energy surfaces of polyatomic molecules and related nonadiabatic dynamics. Our DL is based on deep neural networks (DNNs), which are used as accurate representations of the CASSCF ground- and excited-state potential energy surfaces (PESs) of CH₂NH. After geometries near conical intersection are included in the training set, the DNN models accurately reproduce excited-state topological structures; photoisomerization paths; and, importantly, conical intersections. We have also demonstrated that the results from nonadiabatic dynamics run with the DNN models are very close to those from the dynamics run with the pure ab initio method. The present work should encourage further studies of using machine learning methods to explore excited-state potential energy surfaces and nonadiabatic dynamics of polyatomic molecules.

Insight into the Solvation Structure of Tetraglyme-Based Electrolytes via First-Principles Molecular Dynamics Simulation

Glyme-lithium salt equimolar mixtures, as solvate ionic liquid electrolytes for rechargeable lithium secondary batteries, are of great interest, due to the desirable properties such as high oxidative stability, low vapor pressure, and nonflammability. However, the fundamental understanding of the solvation shell structure in glyme electrolytes has not been clearly established. Herein, we employ first-principles molecular dynamics (FPMD) simulation to study the lithium bis(trifluoromethylsulfonyl)-amide (LiTFSA) and tetraglyme (G4) electrolyte system. For the case of equimolar ratio, a positive correlation between the total coordination number of Li⁺ ions and the phase stability is clearly established. At the ground state of equimolar LiTFSA-G4 electrolyte, most of the Li⁺ ions are coordinated to four O atoms of a curled G4 molecule and one O atom of a TFSA^- anion, equivalent to the second most stable contact ion pair in gas-phase cluster calculations. By contrast, Li⁺ ions prefer to be coordinated by two G4 molecules and not in direct contact with TFSA^- anions at a low concentration of Li salt. The significantly increased probability of pairing between the Li-G4 complexes and TFSA^- anions at the equimolar ratio could be highly relevant to its ionic-liquid-like properties.

Activation of Reactions in the Complex Region Using Microwave Irradiation

Many mode-specific behaviors in the gas phase and at the gas-surface interface have been reported in the past decades. Infrared activation of a reagent vibrational mode is often used to study these reactions. In this work, an inexpensive and easily applied scheme using microwave irradiation is proposed for activating complex-forming reactions by transferring populations between closely spaced resonances. The important combustion reaction of H + O₂ ↔ O + OH is used as a model system to demonstrate the feasibility of the proposed approach. The existence of a nonzero transition dipole moment matrix element between two highly excited resonance states separated by a small energy gap in the model system may allow one to use microwave irradiation to intervene and control the model reaction. The high energy resonance states of the model reaction can also release their energy by photon emission, which is in agreement with the experimentally observed chemiluminescence process.

Machine learning prediction of interaction energies in rigid water clusters

Classical force fields form a computationally efficient avenue for calculating the energetics of large systems. However, due to the constraints of the underlying analytical form, it is sometimes not accurate enough. Quantum mechanical (QM) methods, although accurate, are computationally prohibitive for large systems. In order to circumvent the bottle-neck of interaction energy estimation of large systems, data driven approaches based on machine learning (ML) have been employed in recent years. In most of these studies, the method of choice is artificial neural networks (ANN). In this work, we have shown an alternative ML method, support vector regression (SVR), that provides comparable accuracy with better computational efficiency. We have further used many body expansion (MBE) along with SVR to predict interaction energies in water clusters (decamers). In the case of dimer and trimer interaction energies, the root mean square errors (RMSEs) of the SVR based scheme are 0.12 kcal mol-1 and 0.34 kcal mol-1, respectively. We show that the SVR and MBE based scheme has a RMSE of 2.78% in the estimation of decamer interaction energy against the parent QM method in a computationally efficient way.

Computation of the S1 ← S0 Vibronic Absorption Spectrum of Formaldehyde by Variational Gaussian Wavepacket and Semiclassical IVR Methods

The vibronic absorption spectrum of the electric dipole forbidden and vibronically allowed S₁(¹ A₂) ← S₀(¹ A₁) transition of formaldehyde is calculated by Gaussian wavepacket and semiclassical methods, along with numerically exact reference calculations, using the potential energy surface of Fu, Shepler, and Bowman ( J. Am. Chem. Soc. 2011, 133, 7957). Specifically, the variational multiconfigurational Gaussian (vMCG) approach and the Herman-Kluk semiclassical initial value representation (HK-SCIVR) are compared to assess the accuracy and convergence of these methods, benchmarked against numerically exact time-dependent wavepacket propagation (TDWP) on the reference potential energy surface. The vMCG calculation is shown to converge quite well with about 100 variationally evolving Gaussian functions and using a local cubic expansion instead of the conventional local harmonic approximation. By contrast, the HK-SCIVR approach with ∼10⁵ trajectories reproduces the vibrationally structured spectral envelope correctly but yields a strongly broadened spectrum. The comparison of the computed absorption spectrum with experiment shows that the relevant vibronic progressions are reasonably reproduced by all computations, but deviations of the order of 10-100 cm^-1 occur, underscoring that both electronic structure calculations and dynamical approaches remain challenging in the calculation of typical small-molecule excited-state spectra by trajectory-based methods.

Predicted infrared spectra in the HF stretching band of the H2-HF complex

The infrared spectra with hydrogen fluoride (HF) and deuterium fluoride (DF) (v₂ = 1 ← 0) for eight isotropic species of H₂-HF complex are predicted, based on our newly constructed high-accuracy ab initio potential energy surface [D. Yang et al., J. Chem. Phys. 148, 184301 (2018)]. The radial discrete variable representation/angular finite basis representation method and Lanczos algorithm were used to determine the ro-vibrational energy levels and wave functions for eight species of H₂-HF complex (para-H₂-HF, ortho-H₂-HF, para-D₂-HF, ortho-D₂-HF, para-H₂-DF, ortho-H₂-DF, para-D₂-DF, and ortho-D₂-DF) with separating the inter- and intra-molecular vibrations. Bound states properties including their dissociation energies and rotational constants were presented. The calculated band origins are all red shifted to the isolated HF molecule and in good agreement with available experimental values. The frequencies and line intensities of ro-vibrational transitions in the HF stretching band were further calculated, and the predicted infrared spectra are consistent with available observed spectra. Among them, the spectra for three isotopic species of H₂-HF (para-H₂-DF, para-D₂-DF, and ortho-D₂-DF) were predicted for the first time.

Machine Learning Adaptive Basis Sets for Efficient Large Scale Density Functional Theory Simulation

It is chemically intuitive that an optimal atom centered basis set must adapt to its atomic environment, for example by polarizing toward nearby atoms. Adaptive basis sets of small size can be significantly more accurate than traditional atom centered basis sets of the same size. The small size and well conditioned nature of these basis sets leads to large saving in computational cost, in particular in a linear scaling framework. Here, it is shown that machine learning can be used to predict such adaptive basis sets using local geometrical information only. As a result, various properties of standard DFT calculations can be easily obtained at much lower costs, including nuclear gradients. In our approach, a rotationally invariant parametrization of the basis is obtained by employing a potential anchored on neighboring atoms to ultimately construct a rotation matrix that turns a traditional atom centered basis set into a suitable adaptive basis set. The method is demonstrated using MD simulations of liquid water, where it is shown that minimal basis sets yield structural properties in fair agreement with basis set converged results, while reducing the computational cost in the best case by a factor of 200 and the required flops by 4 orders of magnitude. Already a very small training set yields satisfactory results as the variational nature of the method provides robustness.

Metadynamics for training neural network model chemistries: A competitive assessment

Neural network model chemistries (NNMCs) promise to facilitate the accurate exploration of chemical space and simulation of large reactive systems. One important path to improving these models is to add layers of physical detail, especially long-range forces. At short range, however, these models are data driven and data limited. Little is systematically known about how data should be sampled, and "test data" chosen randomly from some sampling techniques can provide poor information about generality. If the sampling method is narrow, "test error" can appear encouragingly tiny while the model fails catastrophically elsewhere. In this manuscript, we competitively evaluate two common sampling methods: molecular dynamics (MD), normal-mode sampling, and one uncommon alternative, Metadynamics (MetaMD), for preparing training geometries. We show that MD is an inefficient sampling method in the sense that additional samples do not improve generality. We also show that MetaMD is easily implemented in any NNMC software package with cost that scales linearly with the number of atoms in a sample molecule. MetaMD is a black-box way to ensure samples always reach out to new regions of chemical space, while remaining relevant to chemistry near k_bT. It is a cheap tool to address the issue of generalization.

A reactive, scalable, and transferable model for molecular energies from a neural network approach based on local information

Despite the ever-increasing computer power, accurate ab initio calculations for large systems (thousands to millions of atoms) remain infeasible. Instead, approximate empirical energy functions are used. Most current approaches are either transferable between different chemical systems, but not particularly accurate, or they are fine-tuned to a specific application. In this work, a data-driven method to construct a potential energy surface based on neural networks is presented. Since the total energy is decomposed into local atomic contributions, the evaluation is easily parallelizable and scales linearly with system size. With prediction errors below 0.5 kcal mol^-1 for both unknown molecules and configurations, the method is accurate across chemical and configurational space, which is demonstrated by applying it to datasets from nonreactive and reactive molecular dynamics simulations and a diverse database of equilibrium structures. The possibility to use small molecules as reference data to predict larger structures is also explored. Since the descriptor only uses local information, high-level ab initio methods, which are computationally too expensive for large molecules, become feasible for generating the necessary reference data used to train the neural network.