wACSF—Weighted atom-centered symmetry functions as descriptors in machine learning potentials

Neural Network Potential Energy Surfaces for Small Molecules and Reactions

We review progress in neural network (NN)-based methods for the construction of interatomic potentials from discrete samples (such as ab initio energies) for applications in classical and quantum dynamics including reaction dynamics and computational spectroscopy. The main focus is on methods for building molecular potential energy surfaces (PES) in internal coordinates that explicitly include all many-body contributions, even though some of the methods we review limit the degree of coupling, due either to a desire to limit computational cost or to limited data. Explicit and direct treatment of all many-body contributions is only practical for sufficiently small molecules, which are therefore our primary focus. This includes small molecules on surfaces. We consider direct, single NN PES fitting as well as more complex methods that impose structure (such as a multibody representation) on the PES function, either through the architecture of one NN or by using multiple NNs. We show how NNs are effective in building representations with low-dimensional functions including dimensionality reduction. We consider NN-based approaches to build PESs in the sums-of-product form important for quantum dynamics, ways to treat symmetry, and issues related to sampling data distributions and the relation between PES errors and errors in observables. We highlight combinations of NNs with other ideas such as permutationally invariant polynomials or sums of environment-dependent atomic contributions, which have recently emerged as powerful tools for building highly accurate PESs for relatively large molecular and reactive systems.

Recursive evaluation and iterative contraction of N-body equivariant features

Mapping an atomistic configuration to a symmetrized N-point correlation of a field associated with the atomic positions (e.g., an atomic density) has emerged as an elegant and effective solution to represent structures as the input of machine-learning algorithms. While it has become clear that low-order density correlations do not provide a complete representation of an atomic environment, the exponential increase in the number of possible N-body invariants makes it difficult to design a concise and effective representation. We discuss how to exploit recursion relations between equivariant features of different order (generalizations of N-body invariants that provide a complete representation of the symmetries of improper rotations) to compute high-order terms efficiently. In combination with the automatic selection of the most expressive combination of features at each order, this approach provides a conceptual and practical framework to generate systematically improvable, symmetry adapted representations for atomistic machine learning.

Exploiting Machine Learning to Efficiently Predict Multidimensional Optical Spectra in Complex Environments

The excited-state dynamics of chromophores in complex environments determine a range of vital biological and energy capture processes. Time-resolved, multidimensional optical spectroscopies provide a key tool to investigate these processes. Although theory has the potential to decode these spectra in terms of the electronic and atomistic dynamics, the need for large numbers of excited-state electronic structure calculations severely limits first-principles predictions of multidimensional optical spectra for chromophores in the condensed phase. Here, we leverage the locality of chromophore excitations to develop machine learning models to predict the excited-state energy gap of chromophores in complex environments for efficiently constructing linear and multidimensional optical spectra. By analyzing the performance of these models, which span a hierarchy of physical approximations, across a range of chromophore-environment interaction strengths, we provide strategies for the construction of machine learning models that greatly accelerate the calculation of multidimensional optical spectra from first principles.

Neural network potential from bispectrum components: A case study on crystalline silicon

In this article, we present a systematic study on developing machine learning force fields (MLFFs) for crystalline silicon. While the main-stream approach of fitting a MLFF is to use a small and localized training set from molecular dynamics simulations, it is unlikely to cover the global features of the potential energy surface. To remedy this issue, we used randomly generated symmetrical crystal structures to train a more general Si-MLFF. Furthermore, we performed substantial benchmarks among different choices of material descriptors and regression techniques on two different sets of silicon data. Our results show that neural network potential fitting with bispectrum coefficients as descriptors is a feasible method for obtaining accurate and transferable MLFFs.

Surface-Related Features Responsible for Cytotoxic Behavior of MXenes Layered Materials Predicted with Machine Learning Approach

To speed up the implementation of the two-dimensional materials in the development of potential biomedical applications, the toxicological aspects toward human health need to be addressed. Due to time-consuming and expensive analysis, only part of the continuously expanding family of 2D materials can be tested in vitro. The machine learning methods can be used-by extracting new insights from available biological data sets, and provide further guidance for experimental studies. This study identifies the most relevant highly surface-specific features that might be responsible for cytotoxic behavior of 2D materials, especially MXenes. In particular, two factors, namely, the presence of transition metal oxides and lithium atoms on the surface, are identified as cytotoxicity-generating features. The developed machine learning model succeeds in predicting toxicity for other 2D MXenes, previously not tested in vitro, and hence, is able to complement the existing knowledge coming from in vitro studies. Thus, we claim that it might be one of the solutions for reducing the number of toxicological studies needed, and allows for minimizing failures in future biological applications.

High-Precision Atomic Charge Prediction for Protein Systems Using Fragment Molecular Orbital Calculation and Machine Learning

Here, we have constructed neural network-based models that predict atomic partial charges with high accuracy at low computational cost. The models were trained using high-quality data acquired from quantum mechanics calculations using the fragment molecular orbital method. We have succeeded in obtaining highly accurate atomic partial charges for three representative molecular systems of proteins, including one large biomolecule (approx. 2000 atoms). The novelty of our approach is the ability to take into account the electronic polarization in the system, which is a system-dependent phenomenon, being important in the field of drug design. Our high-precision models are useful for the prediction of atomic partial charges and expected to be widely applicable in structure-based drug designs such as structural optimization, high-speed and high-precision docking, and molecular dynamics calculations.

Extending the Applicability of the ANI Deep Learning Molecular Potential to Sulfur and Halogens

Machine learning (ML) methods have become powerful, predictive tools in a wide range of applications, such as facial recognition and autonomous vehicles. In the sciences, computational chemists and physicists have been using ML for the prediction of physical phenomena, such as atomistic potential energy surfaces and reaction pathways. Transferable ML potentials, such as ANI-1x, have been developed with the goal of accurately simulating organic molecules containing the chemical elements H, C, N, and O. Here, we provide an extension of the ANI-1x model. The new model, dubbed ANI-2x, is trained to three additional chemical elements: S, F, and Cl. Additionally, ANI-2x underwent torsional refinement training to better predict molecular torsion profiles. These new features open a wide range of new applications within organic chemistry and drug development. These seven elements (H, C, N, O, F, Cl, and S) make up ∼90% of drug-like molecules. To show that these additions do not sacrifice accuracy, we have tested this model across a range of organic molecules and applications, including the COMP6 benchmark, dihedral rotations, conformer scoring, and nonbonded interactions. ANI-2x is shown to accurately predict molecular energies compared to density functional theory with a ∼10⁶ factor speedup and a negligible slowdown compared to ANI-1x and shows subchemical accuracy across most of the COMP6 benchmark. The resulting model is a valuable tool for drug development which can potentially replace both quantum calculations and classical force fields for a myriad of applications.

QSAR without borders

Prediction of chemical bioactivity and physical properties has been one of the most important applications of statistical and more recently, machine learning and artificial intelligence methods in chemical sciences. This field of research, broadly known as quantitative structure-activity relationships (QSAR) modeling, has developed many important algorithms and has found a broad range of applications in physical organic and medicinal chemistry in the past 55+ years. This Perspective summarizes recent technological advances in QSAR modeling but it also highlights the applicability of algorithms, modeling methods, and validation practices developed in QSAR to a wide range of research areas outside of traditional QSAR boundaries including synthesis planning, nanotechnology, materials science, biomaterials, and clinical informatics. As modern research methods generate rapidly increasing amounts of data, the knowledge of robust data-driven modelling methods professed within the QSAR field can become essential for scientists working both within and outside of chemical research. We hope that this contribution highlighting the generalizable components of QSAR modeling will serve to address this challenge.

Quantum Chemistry in the Age of Machine Learning

As the quantum chemistry (QC) community embraces machine learning (ML), the number of new methods and applications based on the combination of QC and ML is surging. In this Perspective, a view of the current state of affairs in this new and exciting research field is offered, challenges of using machine learning in quantum chemistry applications are described, and potential future developments are outlined. Specifically, examples of how machine learning is used to improve the accuracy and accelerate quantum chemical research are shown. Generalization and classification of existing techniques are provided to ease the navigation in the sea of literature and to guide researchers entering the field. The emphasis of this Perspective is on supervised machine learning.

Machine-Learning-Assisted De Novo Design of Organic Molecules and Polymers: Opportunities and Challenges

Organic molecules and polymers have a broad range of applications in biomedical, chemical, and materials science fields. Traditional design approaches for organic molecules and polymers are mainly experimentally-driven, guided by experience, intuition, and conceptual insights. Though they have been successfully applied to discover many important materials, these methods are facing significant challenges due to the tremendous demand of new materials and vast design space of organic molecules and polymers. Accelerated and inverse materials design is an ideal solution to these challenges. With advancements in high-throughput computation, artificial intelligence (especially machining learning, ML), and the growth of materials databases, ML-assisted materials design is emerging as a promising tool to flourish breakthroughs in many areas of materials science and engineering. To date, using ML-assisted approaches, the quantitative structure property/activity relation for material property prediction can be established more accurately and efficiently. In addition, materials design can be revolutionized and accelerated much faster than ever, through ML-enabled molecular generation and inverse molecular design. In this perspective, we review the recent progresses in ML-guided design of organic molecules and polymers, highlight several successful examples, and examine future opportunities in biomedical, chemical, and materials science fields. We further discuss the relevant challenges to solve in order to fully realize the potential of ML-assisted materials design for organic molecules and polymers. In particular, this study summarizes publicly available materials databases, feature representations for organic molecules, open-source tools for feature generation, methods for molecular generation, and ML models for prediction of material properties, which serve as a tutorial for researchers who have little experience with ML before and want to apply ML for various applications. Last but not least, it draws insights into the current limitations of ML-guided design of organic molecules and polymers. We anticipate that ML-assisted materials design for organic molecules and polymers will be the driving force in the near future, to meet the tremendous demand of new materials with tailored properties in different fields.

Atomic partial charge predictions for furanoses by random forest regression with atom type symmetry function

Furanoses that are components for many important biomolecules have complicated conformational spaces due to the flexible ring and exo-cyclic moieties. Machine learning algorithms, which require descriptors as structural inputs, can be used to efficiently compute conformational adaptive (CA) charges to capture the electrostatic potential variations caused by the conformational changes in the molecular mechanics (MM) calculations. In the present study, we introduced atom type symmetry function (ATSF) developed based on atom centered symmetry function (ACSF) for describing conformations for furanoses, in which atoms were categorized by atom types defined by their properties or connectivity in classic molecular mechanics (MM) force field parameters to generate a suitable coordinate size. Random forest regression (RFR) models with ATSF showed improvements for predicting CA charges and dipole moments for furanoses compared to those with ACSF and atom name symmetry functions where atoms were categorized by their unique atom names. The CA charges predicted by RFR models with ATSF showed more comparable reproductions of the carbohydrate-water and carbohydrate-protein interactions computed with RESP charges individually derived from QM calculations than the ensemble-averaged atomic charge sets commonly employed in molecular mechanics force fields, suggesting that the predicted CA charges were capable of including electrostatic variations in their dynamic charge values. Improvements by ATSF showed that categorizing atoms by atom types introduced chemical structural perceptions to descriptors and produced a suitable coordinate size in ATSF to capture key structural features for furanoses. This categorizing scheme also allows ATSF to be readily adopted by other biomolecules thanks to the broad implementations of MM force fields.

Halide-induced Step Faceting and Dissolution Energetics from Atomistic Machine Learned Potentials on Cu(100)

Adsorbates impact the surface stability and reactivity of metallic electrodes, affecting the corrosion, dissolution, and deposition behavior. Here, we use density functional theory (DFT) and DFT-based Behler-Parrinello neural networks (BPNN) to investigate the geometries, surface formation energies, and atom removal energies of stepped and kinked surfaces vicinal to Cu(100) with a c(2×2) Cl adlayer. DFT calculations indicate that the stable structures for the adsorbate-free vicinal surfaces favor steps with <110> orientation, while the addition of the c(2×2) Cl adlayer leads to <100> step facets, in agreement with scanning tunneling microscopy (STM) observations. The BPNN calculations produce energies in good agreement with DFT results (root mean square error of 1.3 meV/atom for a randomly chosen set of structures excluded from the training set). We draw three conclusions from the BPNN calculations. First, Cl on the upper <100> step edges occupies the three fold hollow sites (as opposed to the four-fold sites on the terraces), congruent with deviations of the STM height profile for the adsorbate at the upper step edge. Second, disruptions in the continuity of the halide overlayer at the steps result in significant long-range step-step interactions. Third, anisotropic metal dissolution and deposition energetics arise from phase shifts of the c(2×2) adlayer at orthogonal <100> steps. This DFT-BPNN approach offers an effective strategy for tackling large-scale surface structure challenges with atomic-level accuracy.

A shared-weight neural network architecture for predicting molecular properties

Quantum chemical methods scale poorly with increasing molecular size and machine learning models have emerged as a promising, computationally-efficient alternative. We present a shared-weight neural network architecture based on modified atom-centered symmetry functions (ACSFs) and show that it performs similarly to the more computationally expensive per-element neural networks of previous work with ACSFs. The model achieves chemically accurate predictions, with a mean absolute error as low as 0.63 kcal mol-1 on energy predictions in the QM9 data set. Additionally, we show that it can reliably predict atomic forces.

Data-Driven Materials Science: Status, Challenges, and Perspectives

Data-driven science is heralded as a new paradigm in materials science. In this field, data is the new resource, and knowledge is extracted from materials datasets that are too big or complex for traditional human reasoning-typically with the intent to discover new or improved materials or materials phenomena. Multiple factors, including the open science movement, national funding, and progress in information technology, have fueled its development. Such related tools as materials databases, machine learning, and high-throughput methods are now established as parts of the materials research toolset. However, there are a variety of challenges that impede progress in data-driven materials science: data veracity, integration of experimental and computational data, data longevity, standardization, and the gap between industrial interests and academic efforts. In this perspective article, the historical development and current state of data-driven materials science, building from the early evolution of open science to the rapid expansion of materials data infrastructures are discussed. Key successes and challenges so far are also reviewed, providing a perspective on the future development of the field.

Embedded Atom Neural Network Potentials: Efficient and Accurate Machine Learning with a Physically Inspired Representation

We propose a simple, but efficient and accurate, machine learning (ML) model for developing a high-dimensional potential energy surface. This so-called embedded atom neural network (EANN) approach is inspired by the well-known empirical embedded atom method (EAM) model used in the condensed phase. It simply replaces the scalar embedded atom density in EAM with a Gaussian-type orbital based density vector and represents the complex relationship between the embedded density vector and atomic energy by neural networks. We demonstrate that the EANN approach is equally accurate as several established ML models in representing both big molecular and extended periodic systems, yet with much fewer parameters and configurations. It is highly efficient as it implicitly contains the three-body information without an explicit sum of the conventional costly angular descriptors. With high accuracy and efficiency, EANN potentials can vastly accelerate molecular dynamics and spectroscopic simulations in complex systems at ab initio level.

Parallel Multistream Training of High-Dimensional Neural Network Potentials

Over the past years high-dimensional neural network potentials (HDNNPs), fitted to accurately reproduce ab initio potential energy surfaces, have become a powerful tool in chemistry, physics and materials science. Here, we focus on the training of the neural networks that lies at the heart of the HDNNP method. We present an efficient approach for optimizing the weight parameters of the neural network via multistream Kalman filtering, using potential energies and forces as reference data. In this procedure, the choice of the free parameters of the Kalman filter can have a significant impact on the fit quality. Carrying out a large parameter study, we determine optimal settings and demonstrate how to optimize training results of HDNNPs. Moreover, we illustrate our HDNNP training approach by revisiting previously presented fits for water and developing a new potential for copper sulfide. This material, accessible in computer simulations so far only via first-principles methods, forms a particularly complex solid structure at low temperatures and undergoes a phase transition to a superionic state upon heating. Analyzing MD simulations carried out with the Cu₂S HDNNP, we confirm that the underlying ab initio reference method indeed reproduces this behavior.

Representing local atomic environment using descriptors based on local correlations

Statistical learning of material properties is an emerging topic of research and has been tremendously successful in areas such as representing complex energy landscapes as well as in technologically relevant areas, like identification of better catalysts and electronic materials. However, analysis of large data sets to efficiently learn characteristic features of a complex energy landscape, for example, depends on the ability of descriptors to effectively screen different local atomic environments. Thus, discovering appropriate descriptors of bulk or defect properties and the functional dependence of such properties on these descriptors remains a difficult and tedious process. To this end, we develop a framework to generate descriptors based on many-body correlations that can effectively capture intrinsic geometric features of the local environment of an atom. These descriptors are based on the spectrum of two-body, three-body, four-body, and higher order correlations between an atom and its neighbors and are evaluated by calculating the corresponding two-body, three-body, and four-body overlap integrals. They are invariant to global translation, global rotation, reflection, and permutations of atomic indices. By systematically testing the ability to capture the local atomic environment, it is shown that the local correlation descriptors are able to successfully reconstruct structures containing 10-25 atoms which was previously not possible.

SchNetPack: A Deep Learning Toolbox For Atomistic Systems

SchNetPack is a toolbox for the development and application of deep neural networks that predict potential energy surfaces and other quantum-chemical properties of molecules and materials. It contains basic building blocks of atomistic neural networks, manages their training, and provides simple access to common benchmark datasets. This allows for an easy implementation and evaluation of new models. For now, SchNetPack includes implementations of (weighted) atom-centered symmetry functions and the deep tensor neural network SchNet, as well as ready-to-use scripts that allow one to train these models on molecule and material datasets. Based on the PyTorch deep learning framework, SchNetPack allows one to efficiently apply the neural networks to large datasets with millions of reference calculations, as well as parallelize the model across multiple GPUs. Finally, SchNetPack provides an interface to the Atomic Simulation Environment in order to make trained models easily accessible to researchers that are not yet familiar with neural networks.

Bond Type Restricted Property Weighted Radial Distribution Functions for Accurate Machine Learning Prediction of Atomization Energies

Understanding the performance of machine learning algorithms is essential for designing more accurate and efficient statistical models. It is not always possible to unravel the reasoning of neural networks. Here, we propose a method for calculating machine learning kernels in closed and analytic form by combining atomic property weighted radial distribution function (AP-RDF) descriptor with a Gaussian kernel. This allowed us to analyze and improve the performance of the Bag-of-Bonds descriptor when the bond type restriction is included in AP-RDF. The improvement is achieved for the prediction of molecular atomization energies (MAE = 1.7 kcal/mol for QM7 data set) and is due to the incorporation of a tensor product into the kernel, which captures the multidimensional representation of the AP-RDF. On the other hand, the numerical version of the AP-RDF is a constant size descriptor, making it more computationally efficient than Bag-of-Bonds. We have also discussed a connection between molecular quantum similarity and machine learning kernels with first-principles kinds of descriptors.

Feature optimization for atomistic machine learning yields a data-driven construction of the periodic table of the elements

By representing elements as points in a low-dimensional chemical space it is possible to improve the performance of a machine-learning model for a chemically-diverse dataset. The resulting coordinates are reminiscent of the main groups of the periodic table.