Deep-neural-network solution of the electronic Schrödinger equation

Solving the nuclear pairing model with neural network quantum states

We present a variational Monte Carlo method that solves the nuclear many-body problem in the occupation number formalism exploiting an artificial neural network representation of the ground-state wave function. A memory-efficient version of the stochastic reconfiguration algorithm is developed to train the network by minimizing the expectation value of the Hamiltonian. We benchmark this approach against widely used nuclear many-body methods by solving a model used to describe pairing in nuclei for different types of interaction and different values of the interaction strength. Despite its polynomial computational cost, our method outperforms coupled-cluster and provides energies that are in excellent agreement with the numerically exact full configuration-interaction values.

Discovering Quantum Phase Transitions with Fermionic Neural Networks

Deep neural networks have been very successful as highly accurate wave function Ansätze for variational Monte Carlo calculations of molecular ground states. We present an extension of one such Ansatz, FermiNet, to calculations of the ground states of periodic Hamiltonians, and study the homogeneous electron gas. FermiNet calculations of the ground-state energies of small electron gas systems are in excellent agreement with previous initiator full configuration interaction quantum Monte Carlo and diffusion Monte Carlo calculations. We investigate the spin-polarized homogeneous electron gas and demonstrate that the same neural network architecture is capable of accurately representing both the delocalized Fermi liquid state and the localized Wigner crystal state. The network converges on the translationally invariant ground state at high density and spontaneously breaks the symmetry to produce the crystalline ground state at low density, despite being given no a priori knowledge that a phase transition exists.

Electronic excited states in deep variational Monte Carlo

Obtaining accurate ground and low-lying excited states of electronic systems is crucial in a multitude of important applications. One ab initio method for solving the Schrödinger equation that scales favorably for large systems is variational quantum Monte Carlo (QMC). The recently introduced deep QMC approach uses ansatzes represented by deep neural networks and generates nearly exact ground-state solutions for molecules containing up to a few dozen electrons, with the potential to scale to much larger systems where other highly accurate methods are not feasible. In this paper, we extend one such ansatz (PauliNet) to compute electronic excited states. We demonstrate our method on various small atoms and molecules and consistently achieve high accuracy for low-lying states. To highlight the method's potential, we compute the first excited state of the much larger benzene molecule, as well as the conical intersection of ethylene, with PauliNet matching results of more expensive high-level methods.

A transferable recommender approach for selecting the best density functional approximations in chemical discovery

Approximate density functional theory has become indispensable owing to its balanced cost-accuracy trade-off, including in large-scale screening. To date, however, no density functional approximation (DFA) with universal accuracy has been identified, leading to uncertainty in the quality of data generated from density functional theory. With electron density fitting and Δ-learning, we build a DFA recommender that selects the DFA with the lowest expected error with respect to the gold standard (but cost-prohibitive) coupled cluster theory in a system-specific manner. We demonstrate this recommender approach on the evaluation of vertical spin splitting energies of transition metal complexes. Our recommender predicts top-performing DFAs and yields excellent accuracy (about 2 kcal mol-1) for chemical discovery, outperforming both individual Δ-learning models and the best conventional single-functional approach from a set of 48 DFAs. By demonstrating transferability to diverse synthesized compounds, our recommender potentially addresses the accuracy versus scope dilemma broadly encountered in computational chemistry.

Organic reaction mechanism classification using machine learning

A mechanistic understanding of catalytic organic reactions is crucial for the design of new catalysts, modes of reactivity and the development of greener and more sustainable chemical processes^1-13. Kinetic analysis lies at the core of mechanistic elucidation by facilitating direct testing of mechanistic hypotheses from experimental data. Traditionally, kinetic analysis has relied on the use of initial rates¹⁴, logarithmic plots and, more recently, visual kinetic methods^15-18, in combination with mathematical rate law derivations. However, the derivation of rate laws and their interpretation require numerous mathematical approximations and, as a result, they are prone to human error and are limited to reaction networks with only a few steps operating under steady state. Here we show that a deep neural network model can be trained to analyse ordinary kinetic data and automatically elucidate the corresponding mechanism class, without any additional user input. The model identifies a wide variety of classes of mechanism with outstanding accuracy, including mechanisms out of steady state such as those involving catalyst activation and deactivation steps, and performs excellently even when the kinetic data contain substantial error or only a few time points. Our results demonstrate that artificial-intelligence-guided mechanism classification is a powerful new tool that can streamline and automate mechanistic elucidation. We are making this model freely available to the community and we anticipate that this work will lead to further advances in the development of fully automated organic reaction discovery and development.

Ab initio calculation of real solids via neural network ansatz

Neural networks have been applied to tackle many-body electron correlations for small molecules and physical models in recent years. Here we propose an architecture that extends molecular neural networks with the inclusion of periodic boundary conditions to enable ab initio calculation of real solids. The accuracy of our approach is demonstrated in four different types of systems, namely the one-dimensional periodic hydrogen chain, the two-dimensional graphene, the three-dimensional lithium hydride crystal, and the homogeneous electron gas, where the obtained results, e.g. total energies, dissociation curves, and cohesive energies, reach a competitive level with many traditional ab initio methods. Moreover, electron densities of typical systems are also calculated to provide physical intuition of various solids. Our method of extending a molecular neural network to periodic systems can be easily integrated into other neural network structures, highlighting a promising future of ab initio solution of more complex solid systems using neural network ansatz, and more generally endorsing the application of machine learning in materials simulation and condensed matter physics.

Perspective on the Current State-of-the-Art of Quantum Computing for Drug Discovery Applications

Computational chemistry is an essential tool in the pharmaceutical industry. Quantum computing is a fast evolving technology that promises to completely shift the computational capabilities in many areas of chemical research by bringing into reach currently impossible calculations. This perspective illustrates the near-future applicability of quantum computation of molecules to pharmaceutical problems. We briefly summarize and compare the scaling properties of state-of-the-art quantum algorithms and provide novel estimates of the quantum computational cost of simulating progressively larger embedding regions of a pharmaceutically relevant covalent protein-drug complex involving the drug Ibrutinib. Carrying out these calculations requires an error-corrected quantum architecture that we describe. Our estimates showcase that recent developments on quantum phase estimation algorithms have dramatically reduced the quantum resources needed to run fully quantum calculations in active spaces of around 50 orbitals and electrons, from estimated over 1000 years using the Trotterization approach to just a few days with sparse qubitization, painting a picture of fast and exciting progress in this nascent field.

Electronic-Structure Properties from Atom-Centered Predictions of the Electron Density

The electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. A natural choice to construct a model that yields transferable and linear-scaling predictions is to represent the scalar field using a multicentered atomic basis analogous to that routinely used in density fitting approximations. However, the nonorthogonality of the basis poses challenges for the learning exercise, as it requires accounting for all the atomic density components at once. We devise a gradient-based approach to directly minimize the loss function of the regression problem in an optimized and highly sparse feature space. In so doing, we overcome the limitations associated with adopting an atom-centered model to learn the electron density over arbitrarily complex data sets, obtaining very accurate predictions using a comparatively small training set. The enhanced framework is tested on 32-molecule periodic cells of liquid water, presenting enough complexity to require an optimal balance between accuracy and computational efficiency. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom with respect to the reference density functional calculations. Finally, we test our method on the highly heterogeneous QM9 benchmark data set, showing that a small fraction of the training data is enough to derive ground-state total energies within chemical accuracy.

GeoPacker: A novel deep learning framework for protein side-chain modeling

Atomic interactions play essential roles in protein folding, structure stabilization, and function performance. Recent advances in deep learning-based methods have achieved impressive success not only in protein structure prediction, but also in protein sequence design. However, highly efficient and accurate protein side-chain prediction methods that can give detailed atomic interactions are still lacking. In the present study, we developed a deep learning based method, GeoPacker, that uses geometric deep learning coupled ResNet for protein side-chain modeling. GeoPacker explicitly represents atomic interactions with rotational and translational invariance for information extraction of relative locations. GeoPacker outperformed the state-of-the-art energy function-based methods in side-chain structure prediction accuracy and runs about 10 and 700 times faster than the deep learning-based method DLPacker and OPUS-rota4 with comparable prediction accuracy, respectively. The performance of GeoPacker does not depend on the secondary structures that the residues belong to. GeoPacker gives highly accurate predictions for buried residues in the protein core as well as protein-protein interface, making it a useful tool for protein structure modeling, protein, and interaction design.

Obtaining Electronic Properties of Molecules through Combining Density Functional Tight Binding with Machine Learning

We have introduced a machine learning workflow that allows for optimizing electronic properties in the density functional tight binding method (DFTB). The workflow allows for the optimization of electronic properties by generating two-center integrals, either by training basis function parameters directly or by training a spline model for the diatomic integrals, which are then used to build the Hamiltonian and the overlap matrices. Using our workflow, we have managed to obtain improved electronic properties, such as charge distributions, dipole moments, and approximated polarizabilities. While both machine learning approaches enabled us to improve on the electronic properties of the molecules as compared with existing DFTB parametrizations, only by training on the basis function parameters we were able to obtain consistent Hamiltonians and overlap matrices in the physically reasonable ranges or to improve on multiple electronic properties simultaneously.

Molecular-orbital-based machine learning for open-shell and multi-reference systems with kernel addition Gaussian process regression

We introduce a novel machine learning strategy, kernel addition Gaussian process regression (KA-GPR), in molecular-orbital-based machine learning (MOB-ML) to learn the total correlation energies of general electronic structure theories for closed- and open-shell systems by introducing a machine learning strategy. The learning efficiency of MOB-ML(KA-GPR) is the same as the original MOB-ML method for the smallest criegee molecule, which is a closed-shell molecule with multi-reference characters. In addition, the prediction accuracies of different small free radicals could reach the chemical accuracy of 1 kcal/mol by training on one example structure. Accurate potential energy surfaces for the H₁₀ chain (closed-shell) and water OH bond dissociation (open-shell) could also be generated by MOB-ML(KA-GPR). To explore the breadth of chemical systems that KA-GPR can describe, we further apply MOB-ML to accurately predict the large benchmark datasets for closed- (QM9, QM7b-T, and GDB-13-T) and open-shell (QMSpin) molecules.

Computational design of magnetic molecules and their environment using quantum chemistry, machine learning and multiscale simulations

Having served as a playground for fundamental studies on the physics of d and f electrons for almost a century, magnetic molecules are now becoming increasingly important for technological applications, such as magnetic resonance, data storage, spintronics and quantum information. All of these applications require the preservation and control of spins in time, an ability hampered by the interaction with the environment, namely with other spins, conduction electrons, molecular vibrations and electromagnetic fields. Thus, the design of a novel magnetic molecule with tailored properties is a formidable task, which does not only concern its electronic structures but also calls for a deep understanding of the interaction among all the degrees of freedom at play. This Review describes how state-of-the-art ab initio computational methods, combined with data-driven approaches to materials modelling, can be integrated into a fully multiscale strategy capable of defining design rules for magnetic molecules.

Machine learning, artificial intelligence, and chemistry: how smart algorithms are reshaping simulation and the laboratory

Machine learning and artificial intelligence are increasingly gaining in prominence through image analysis, language processing, and automation, to name a few applications. Machine learning is also making profound changes in chemistry. From revisiting decades-old analytical techniques for the purpose of creating better calibration curves, to assisting and accelerating traditional in silico simulations, to automating entire scientific workflows, to being used as an approach to deduce underlying physics of unexplained chemical phenomena, machine learning and artificial intelligence are reshaping chemistry, accelerating scientific discovery, and yielding new insights. This review provides an overview of machine learning and artificial intelligence from a chemist’s perspective and focuses on a number of examples of the use of these approaches in computational chemistry and in the laboratory.

Orbital Mixer: Using Atomic Orbital Features for Basis-Dependent Prediction of Molecular Wavefunctions

Leveraging ab initio data at scale has enabled the development of machine learning models capable of extremely accurate and fast molecular property prediction. A central paradigm of many previous studies focuses on generating predictions for only a fixed set of properties. Recent lines of research instead aim to explicitly learn the electronic structure via molecular wavefunctions, from which other quantum chemical properties can be directly derived. While previous methods generate predictions as a function of only the atomic configuration, in this work we present an alternate approach that directly purposes basis-dependent information to predict molecular electronic structure. Our model, Orbital Mixer, is composed entirely of multi-layer perceptrons (MLPs) using MLP-Mixer layers within a simple, intuitive, and scalable architecture that achieves competitive Hamiltonian and molecular orbital energy and coefficient prediction accuracies compared to the state-of-the-art.

Molecular Dipole Moment Learning via Rotationally Equivariant Gaussian Process Regression with Derivatives in Molecular-orbital-based Machine Learning

This study extends the accurate and transferable molecular-orbital-based machine learning (MOB-ML) approach to modeling the contribution of electron correlation to dipole moments at the cost of Hartree--Fock computations. A molecular-orbital-based (MOB) pairwise decomposition of the correlation part of the dipole moment is applied, and these pair dipole moments could be further regressed as a universal function of molecular orbitals (MOs).The dipole MOB features consist of the energy MOB features and their responses to electric fields. An interpretable and rotationally equivariant Gaussian process regression (GPR) with derivatives algorithm is introduced to learn the dipole moment more efficiently. The proposed problem setup, feature design, and ML algorithm are shown to provide highly-accurate models for both dipole moment and energies on water and fourteen small molecules. To demonstrate the ability of MOB-ML to function as generalized density-matrix functionals for molecular dipole moments and energies of organic molecules, we further apply the proposed MOB-ML approach to train and test the molecules from the QM9 dataset. The application of local scalable GPR with Gaussian mixture model unsupervised clustering (GMM/GPR) scales up MOB-ML to a large-data regime while retaining the prediction accuracy. In addition, compared with literature results, MOB-ML provides the best test MAEs of 4.21 mDebye and 0.045 kcal/mol for dipole moment and energy models, respectively, when training on 110000 QM9 molecules. The excellent transferability of the resulting QM9 models is also illustrated by the accurate predictions for four different series of peptides.

Fermionic wave functions from neural-network constrained hidden states

Large systems of interacting quantum particles present a notorious computational challenge, since they require to solve an eigenvalue problem in an exponentially large dimensional space. The problem can be approached variationally: A trial wave function is proposed, depending on a set of parameters that are determined by an optimization procedure. Neural networks are a great candidate for the task as they provide an extremely flexible family of trial states. We introduce a neural-network-based trial wave function formalism for the study of systems of interacting fermions. This formalism is based on the addition of extra hidden fermions that, aided by neural networks, “mediate” the correlations between the particles in the wave function when projected back on the physical space.

We introduce a systematically improvable family of variational wave functions for the simulation of strongly correlated fermionic systems. This family consists of Slater determinants in an augmented Hilbert space involving “hidden” additional fermionic degrees of freedom. These determinants are projected onto the physical Hilbert space through a constraint that is optimized, together with the single-particle orbitals, using a neural network parameterization. This construction draws inspiration from the success of hidden-particle representations but overcomes the limitations associated with the mean-field treatment of the constraint often used in this context. Our construction provides an extremely expressive family of wave functions, which is proved to be universal. We apply this construction to the ground-state properties of the Hubbard model on the square lattice, achieving levels of accuracy that are competitive with those of state-of-the-art variational methods.

Construction of a Deep Neural Network Energy Function for Protein Physics

The traditional approach of computational biology consists of calculating molecule properties by using approximate classical potentials. Interactions between atoms are described by an energy function derived from physical principles or fitted to experimental data. Their functional form is usually limited to pairwise interactions between atoms and does not consider complex multibody effects. More recently, neural networks have emerged as an alternative way of describing the interactions between biomolecules. In this approach, the energy function does not have an explicit functional form and is learned bottom-up from simulations at the atomistic or quantum level. In this study, we attempt a top-down approach and use deep learning methods to obtain an energy function by exploiting the large amount of experimental data acquired with years in the field of structural biology. The energy function is represented by a probability density model learned from a large repertoire of building blocks representing local clusters of amino acids paired with their sequence signature. We demonstrated the feasibility of this approach by generating a neural network energy function and testing its validity on several applications such as discriminating decoys, assessing qualities of structural models, sampling structural conformations, and designing new protein sequences. We foresee that, in the future, our methodology could exploit the continuously increasing availability of experimental data and simulations and provide a new method for the parametrization of protein energy functions.

Quantum machine learning for chemistry and physics

Machine learning (ML) has emerged as a formidable force for identifying hidden but pertinent patterns within a given data set with the objective of subsequent generation of automated predictive behavior. In recent years, it is safe to conclude that ML and its close cousin, deep learning (DL), have ushered in unprecedented developments in all areas of physical sciences, especially chemistry. Not only classical variants of ML, even those trainable on near-term quantum hardwares have been developed with promising outcomes. Such algorithms have revolutionized materials design and performance of photovoltaics, electronic structure calculations of ground and excited states of correlated matter, computation of force-fields and potential energy surfaces informing chemical reaction dynamics, reactivity inspired rational strategies of drug designing and even classification of phases of matter with accurate identification of emergent criticality. In this review we shall explicate a subset of such topics and delineate the contributions made by both classical and quantum computing enhanced machine learning algorithms over the past few years. We shall not only present a brief overview of the well-known techniques but also highlight their learning strategies using statistical physical insight. The objective of the review is not only to foster exposition of the aforesaid techniques but also to empower and promote cross-pollination among future research in all areas of chemistry which can benefit from ML and in turn can potentially accelerate the growth of such algorithms.

A Look Inside the Black Box of Machine Learning Photodynamics Simulations

ConspectusPhotochemical reactions are of great importance in chemistry, biology, and materials science because they take advantage of a renewable energy source, mild reaction conditions, and high atom economy. Light absorption can excite molecules to a higher energy electronic state of the same spin multiplicity. The following nonadiabatic processes induce molecular transformations that afford exotic molecular architectures and high-energy-isomers that are inaccessible by thermal means. Computational simulations now complement time-resolved instrumentation to reveal ultrafast excited-state mechanistic information for photochemical reactions that is essential in disentangling elusive spectroscopic features, excited-state lifetimes, and excited-state mechanistic critical points. Nonadiabatic molecular dynamics (NAMD), powered by surface hopping techniques, is among the most widely applied techniques to model the photochemical reactions of medium-sized molecules. However, the computational efficiency is limited because of the requisite thousands of multiconfigurational quantum-chemical calculations multiplied by hundreds of trajectories. Machine learning (ML) has emerged as a revolutionary force in computational chemistry to predict the outcome of the resource-intensive multiconfigurational calculations on the fly. An ML potential trained with a substantial set of quantum-chemical calculations can predict the energies and forces with errors under chemical accuracy at a negligible cost. The integration of ML potentials in NAMD dramatically extends the maximum simulation time scale by ∼10 000-fold to the nanosecond regime.In this Account, we present a comprehensive demonstration of ML photodynamics simulations and summarize our most recent applications in resolving complex photochemical reactions. First, we address three fundamental components of ML techniques for photodynamics simulations: the quantum-chemical data set, the ML potential, and NAMD. Second, we describe best practices in building training data and our procedure toward training the ML photodynamics model with our recent literature contributions. We introduce a convenient training data generation scheme combining Wigner sampling and geometrical interpolation. It trains reliable and effective ML potentials suitable for subsequent active learning to detect undersampled data. We demonstrate how active learning automatically discovers new mechanistic pathways and reproduces experimental results. We point out that atomic permutation is an essential data augmentation approach to improve the learnability of distance-based molecular descriptors for highly symmetric molecules. Third, we demonstrate the utility of ML-photodynamics by showing the results of ML photodynamics simulations of (1) photo-torquoselective 4π disrotatory electrocyclic ring closing of norbornyl cyclohexadiene, which reveals a thermal conversion from experimentally unobserved intermediates to the reactant in 1 ns; (2) [2 + 2] photocycloaddition of substituted [3]-syn-ladderdienes in competition with 4π and 6π electrocyclic ring-opening reactions, uncovering substituent effects to explain the reported increased quantum yield of substituted cubane precursors; and (3) photochemical 4π disrotatory electrocyclic reactions of fluorobenzenes in nanoseconds with XMS-CASPT2-level training data. We expect this Account to broaden understanding of ML photodynamics and inspire future developments and applications to increasingly large molecules within complex environments on long time scales.

A Study on the Relationship between Painter’s Psychology and Anime Creation Style Based on a Deep Neural Network

We conduct an in-depth study on the model construction of deep neural networks and design a model of painter's psychology and anime creation style to realize the study of the relationship between painter's psychology and anime creation style based on deep neural networks. This paper proposes an animation creation psychology classification algorithm that integrates human cognitive deep network structure optimization. The algorithm analyzes the connection between different convolutional layer features and animation characteristics through animation creation style CNN feature visualization. It interactively uses the knowledge of animation creation psychology expression techniques to optimize the network structure. This paper proposes a scene animation network based on spectral difference perception style. By analyzing the characteristics and differences in the spectrum between realistic and anime domain images, the generator is guided to learn the mapping relationships better to fit the style distribution of anime domain images. This paper uses a fully convolutional structure; the network is more lightweight and supports image inputs of arbitrary size, which can keep the semantic system of the background unchanged while highly deforming the five facial features, moving toward the goal of human-scene fusion for the animation task.

Fast Near Ab Initio Potential Energy Surfaces Using Machine Learning

A machine-learning based approach for evaluating potential energies for quantum mechanical studies of properties of the ground and excited vibrational states of small molecules is developed. This approach uses the molecular-orbital-based machine learning (MOB-ML) method to generate electronic energies with the accuracy of CCSD(T) calculations at the same cost as a Hartree-Fock calculation. To further reduce the computational cost of the potential energy evaluations without sacrificing the CCSD(T) level accuracy, GPU-accelerated Neural Network Potential Energy Surfaces (NN-PES) are trained to geometries and energies that are collected from small-scale Diffusion Monte Carlo (DMC) simulations, which are run using energies evaluated using the MOB-ML model. The combined NN+(MOB-ML) approach is used in variational calculations of the ground and low-lying vibrational excited states of water and in DMC calculations of the ground states of water, CH₅⁺, and its deuterated analogues. For both of these molecules, comparisons are made to the results obtained using potentials that were fit to much larger sets of electronic energies than were required to train the MOB-ML models. The NN+(MOB-ML) approach is also used to obtain a potential surface for C₂H₅⁺, which is a carbocation with a nonclassical equilibrium structure for which there is currently no available potential surface. This potential is used to explore the CH stretching vibrations, focusing on those of the bridging hydrogen atom. For both CH₅⁺ and C₂H₅⁺ the MOB-ML model is trained using geometries that were sampled from an AIMD trajectory, which was run at 350 K. By comparison, the structures sampled in the ground state calculations can have energies that are as much as ten times larger than those used to train the MOB-ML model. For water a higher temperature AIMD trajectory is needed to obtain accurate results due to the smaller thermal energy. A second MOB-ML model for C₂H₅⁺ was developed with additional higher energy structures in the training set. The two models are found to provide nearly identical descriptions of the ground state of C₂H₅⁺.

Artificial intelligence guided conformational mining of intrinsically disordered proteins

Artificial intelligence recently achieved the breakthrough of predicting the three-dimensional structures of proteins. The next frontier is presented by intrinsically disordered proteins (IDPs), which, representing 30% to 50% of proteomes, readily access vast conformational space. Molecular dynamics (MD) simulations are promising in sampling IDP conformations, but only at extremely high computational cost. Here, we developed generative autoencoders that learn from short MD simulations and generate full conformational ensembles. An encoder represents IDP conformations as vectors in a reduced-dimensional latent space. The mean vector and covariance matrix of the training dataset are calculated to define a multivariate Gaussian distribution, from which vectors are sampled and fed to a decoder to generate new conformations. The ensembles of generated conformations cover those sampled by long MD simulations and are validated by small-angle X-ray scattering profile and NMR chemical shifts. This work illustrates the vast potential of artificial intelligence in conformational mining of IDPs.

Adapting reservoir computing to solve the Schrödinger equation

Reservoir computing is a machine learning algorithm that excels at predicting the evolution of time series, in particular, dynamical systems. Moreover, it has also shown superb performance at solving partial differential equations. In this work, we adapt this methodology to integrate the time-dependent Schrödinger equation, propagating an initial wavefunction in time. Since such wavefunctions are complex-valued high-dimensional arrays, the reservoir computing formalism needs to be extended to cope with complex-valued data. Furthermore, we propose a multi-step learning strategy that avoids overfitting the training data. We illustrate the performance of our adapted reservoir computing method by application to four standard problems in molecular vibrational dynamics.

Neural Network-Based Dynamic Segmentation and Weighted Integrated Matching of Cross-Media Piano Performance Audio Recognition and Retrieval Algorithm

This paper presents a dynamic segmentation and weighted comprehensive matching algorithm based on neural networks for cross-media piano performance audio recognition and retrieval. The 3D convolutional neural network process is separated to compress the network parameters and improve the computational speed. Skip connection and layer-wise learning rate solve the problem that the separated network is challenging to train. The piano performance audio recognition is facilitated by shuffle operation. In pattern recognition, music retrieval algorithms are gaining more and more attention due to their ease of implementation and efficiency. However, the problems of imprecise dynamic note segmentation and inconsistent matching templates directly affect the accuracy of the MIR algorithm. We propose a dynamic threshold-based segmentation and weighted comprehensive matching algorithm to solve these problems. The amplitude difference step is dynamically set, and the notes are segmented according to the changing threshold to improve the accuracy of note segmentation. A standard score frequency is used to transform the pitch template to achieve input normalization to enhance the accuracy of matching. Direct matching and DTW matching are fused to improve the adaptability and robustness of the algorithm. Finally, the effectiveness of the method is experimentally demonstrated. This paper implements the data collection and processing, audio recognition, and retrieval algorithm for cross-media piano performance big data through three main modules: the collection, processing, and storage module of cross-media piano performance big data, the building module of audio recognition of cross-media piano performance big data, and the dynamic precision module of cross-media piano performance big data.

Solving the electronic Schrödinger equation for multiple nuclear geometries with weight-sharing deep neural networks

The Schrödinger equation describes the quantum-mechanical behaviour of particles, making it the most fundamental equation in chemistry. A solution for a given molecule allows computation of any of its properties. Finding accurate solutions for many different molecules and geometries is thus crucial to the discovery of new materials such as drugs or catalysts. Despite its importance, the Schrödinger equation is notoriously difficult to solve even for single molecules, as established methods scale exponentially with the number of particles. Combining Monte Carlo techniques with unsupervised optimization of neural networks was recently discovered as a promising approach to overcome this curse of dimensionality, but the corresponding methods do not exploit synergies that arise when considering multiple geometries. Here we show that sharing the vast majority of weights across neural network models for different geometries substantially accelerates optimization. Furthermore, weight-sharing yields pretrained models that require only a small number of additional optimization steps to obtain high-accuracy solutions for new geometries.

Beyond Woodward-Fieser Rules: Design Principles of Property-Oriented Chromophores Based on Explainable Deep Learning Optical Spectroscopy

An adequate understanding of molecular structure-property relationships is important for developing new molecules with desired properties. Although deep learning optical spectroscopy (DLOS) has been successfully applied to predict the optical and photophysical properties of organic chromophores, how specific functional groups and solvents affect the optical properties is not clearly understood. Here, we employed an explainable DLOS method by applying the integrated gradients method to DLOS. The integrated gradients method allows us to obtain attributions, indicating how much the functional group contributes to the optical properties including the absorption wavelength and bandwidth, extinction coefficients, emission wavelength and bandwidth, photoluminescence quantum yield, and lifetime. The attributions of 54 functional groups and 9 solvent molecules to seven optical properties are quantified and can be used to estimate the optical properties of chromophores as in the Woodward-Fieser rule. Unlike the Woodward-Fieser rule for only the absorption wavelength, the attributions obtained in this work can be applied to estimate all seven optical properties, which makes a significant extension of the Woodward-Fieser rules. In addition, we demonstrated a strategy for utilizing the attributions in the design of molecules and in tuning the optical properties of the molecules. The design of molecular structures using attributions can revolutionize the development of optimal molecules.

Learning aerodynamics with neural network

We propose a neural network (NN) architecture, the Element Spatial Convolution Neural Network (ESCNN), towards the airfoil lift coefficient prediction task. The ESCNN outperforms existing state-of-the-art NNs in terms of prediction accuracy, with two orders of less parameters. We further investigate and explain how the ESCNN succeeds in making accurate predictions with standard convolution layers. We discover that the ESCNN has the ability to extract physical patterns that emerge from aerodynamics, and such patterns are clearly reflected within a layer of the network. We show that the ESCNN is capable of learning the physical laws and equation of aerodynamics from simulation data.

Stability Analysis of Substituted Cobaltocenium [Bis(cyclopentadienyl)cobalt(III)] Employing Chemistry-Informed Neural Networks

Cationic cobaltocenium derivatives are promising components of the anion exchange membranes because of their excellent thermal and alkaline stability under the operating conditions of a fuel cell. Here, we present an efficient modeling approach to assessing the chemical stability of substituted cobaltocenium CoCp₂⁺, based on the computed electronic structure enhanced by machine learning techniques. Within the aqueous environment, the positive charge of the metal cation is balanced by the hydroxide anion through formation of the CoCp₂⁺OH^- complexes, whose dissociation is studied within the implicit solvent employing the density functional theory. The data set of about 118 the CoCp₂⁺OH^- complexes based on 42 substituent groups characterized by a range of electron-donating (ED) and electron-withdrawing (EW) properties is constructed and analyzed. Given 12 carefully chosen chemistry-informed descriptors of the complexes and relevant fragments, the stability of the complexes is found to strongly correlate with the energies of the highest occupied and lowest unoccupied molecular orbitals, modulated by a switching function of the Hirshfeld charge. The latter is used as a measure of the electron-withdrawing-donating character of the substituents. On the basis of this observation from the conventional regression analysis, two fully connected, feed-forward neural network (FNN) models with different unit structures, called the chemistry-informed (CINN) and the quadratic (QNN) neural networks, together with a support vector regression (SVR) model are developed. Both deep neural network models predict the bond dissociation energies of the cobaltocenium complexes with mean relative errors less than 10.56% and average absolute errors less than 1.63 kcal/mol, superior to the conventional regression and the SVR model prediction. The results show the potential of QNN to efficiently capture more complex relationships. The concept of incorporating the domain (chemical) knowledge/insight into the neural network structure paves the way to applications of machine learning techniques with small data sets, ultimately leading to better predictive models compared to the classical machine learning method SVR and conventional regression analysis.

A New Approach to Modeling Focused Infrared Heating Based on Quantum Mechanical Formulations

The focused infrared (IR) heating method is an energy-efficient heating technology for engineering applications. Numerical models of focused IR heating technology have been introduced based on the theory of ray optics. The ray optics-based IR models have provided good simulation results; however, they are mathematically complex because the ray optics models need to account for the complex paths of the IR rays and the geometrical information of the heating devices. This paper presents a new approach for modeling the focused IR heating method using quantum mechanical formulations. Even though the IR heating condition is not a pure quantum phenomenon, it is efficient to employ the concept of the superposition principle of wave functions in IR distribution modeling. The proposed model makes an abstraction by replacing the distributed IR rays with an energy particle with independent wave functions at different eigenstates, based on the Schrödinger equation. The new approach results in a simpler equation for modeling the focused IR heating method. An electrical-thermal simulation of the focused IR heating with the new model provides results in good agreement with the experimental data.

Transferable Neural Network Potential Energy Surfaces for Closed-Shell Organic Molecules: Extension to Ions

Transferable high dimensional neural network potentials (HDNNPs) have shown great promise as an avenue to increase the accuracy and domain of applicability of existing atomistic force fields for organic systems relevant to life science. We have previously reported such a potential (Schrödinger-ANI) that has broad coverage of druglike molecules. We extend that work here to cover ionic and zwitterionic druglike molecules expected to be relevant to drug discovery research activities. We report a novel HDNNP architecture, which we call QRNN, that predicts atomic charges and uses these charges as descriptors in an energy model that delivers conformational energies within chemical accuracy when measured against the reference theory it is trained to. Further, we find that delta learning based on a semiempirical level of theory approximately halves the errors. We test the models on torsion energy profiles, relative conformational energies, geometric parameters, and relative tautomer errors.

Potential Application of Machine-Learning-Based Quantum Chemical Methods in Environmental Chemistry

It is an important topic in environmental sciences to understand the behavior and toxicology of chemical pollutants. Quantum chemical methodologies have served as useful tools for probing behavior and toxicology of chemical pollutants in recent decades. In recent years, machine learning (ML) techniques have brought revolutionary developments to the field of quantum chemistry, which may be beneficial for investigating environmental behavior and toxicology of chemical pollutants. However, the ML-based quantum chemical methods (ML-QCMs) have only scarcely been used in environmental chemical studies so far. To promote applications of the promising methods, this Perspective summarizes recent progress in the ML-QCMs and focuses on their potential applications in environmental chemical studies that could hardly be achieved by the conventional quantum chemical methods. Potential applications and challenges of the ML-QCMs in predicting degradation networks of chemical pollutants, searching global minima for atmospheric nanoclusters, discovering heterogeneous or photochemical transformation pathways of pollutants, as well as predicting environmentally relevant end points with wave functions as descriptors are introduced and discussed.

Neural network approaches for solving Schrödinger equation in arbitrary quantum wells

In this work we approach the Schrödinger equation in quantum wells with arbitrary potentials, using the machine learning technique. Two neural networks with different architectures are proposed and trained using a set of potentials, energies, and wave functions previously generated with the classical finite element method. Three accuracy indicators have been proposed for testing the estimates given by the neural networks. The networks are trained by the gradient descent method and the training validation is done with respect to a large training data set. The two networks are then tested for two different potential data sets and the results are compared. Several cases with analytical potential have also been solved.

Natural quantum reservoir computing for temporal information processing

Reservoir computing is a temporal information processing system that exploits artificial or physical dissipative dynamics to learn a dynamical system and generate the target time-series. This paper proposes the use of real superconducting quantum computing devices as the reservoir, where the dissipative property is served by the natural noise added to the quantum bits. The performance of this natural quantum reservoir is demonstrated in a benchmark time-series regression problem and a practical problem classifying different objects based on temporal sensor data. In both cases the proposed reservoir computer shows a higher performance than a linear regression or classification model. The results indicate that a noisy quantum device potentially functions as a reservoir computer, and notably, the quantum noise, which is undesirable in the conventional quantum computation, can be used as a rich computation resource.

Quantum Mechanics and Machine Learning Synergies: Graph Attention Neural Networks to Predict Chemical Reactivity

There is a lack of scalable quantitative measures of reactivity that cover the full range of functional groups in organic chemistry, ranging from highly unreactive C-C bonds to highly reactive naked ions. Measuring reactivity experimentally is costly and time-consuming, and no single method has sufficient dynamic range to cover the astronomical size of chemical reactivity space. In previous quantum chemistry studies, we have introduced Methyl Cation Affinities (MCA*) and Methyl Anion Affinities (MAA*), using a solvation model, as quantitative measures of reactivity for organic functional groups over the broadest range. Although MCA* and MAA* offer good estimates of reactivity parameters, their calculation through Density Functional Theory (DFT) simulations is time-consuming. To circumvent this problem, we first use DFT to calculate MCA* and MAA* for more than 2,400 organic molecules thereby establishing a large data set of chemical reactivity scores. We then design deep learning methods to predict the reactivity of molecular structures and train them using this curated data set in combination with different representations of molecular structures. Using 10-fold cross-validation, we show that graph attention neural networks applied to a relational model of molecular structures produce the most accurate estimates of reactivity, achieving over 91% test accuracy for predicting the MCA* ± 3.0 or MAA* ± 3.0, over 50 orders of magnitude. Finally, we demonstrate the application of these reactivity scores to two tasks: (1) chemical reaction prediction and (2) combinatorial generation of reaction mechanisms. The curated data sets of MCA* and MAA* scores is available through the ChemDB chemoinformatics web portal at cdb.ics.uci.edu under Chemical Reactivities data sets.

Neural Network Potentials: A Concise Overview of Methods

In the past two decades, machine learning potentials (MLPs) have reached a level of maturity that now enables applications to large-scale atomistic simulations of a wide range of systems in chemistry, physics, and materials science. Different machine learning algorithms have been used with great success in the construction of these MLPs. In this review, we discuss an important group of MLPs relying on artificial neural networks to establish a mapping from the atomic structure to the potential energy. In spite of this common feature, there are important conceptual differences among MLPs, which concern the dimensionality of the systems, the inclusion of long-range electrostatic interactions, global phenomena like nonlocal charge transfer, and the type of descriptor used to represent the atomic structure, which can be either predefined or learnable. A concise overview is given along with a discussion of the open challenges in the field.

Expected final online publication date for the Annual Review of Physical Chemistry, Volume 73 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

From vibrational spectroscopy and quantum tunnelling to periodic band structures – a self-supervised, all-purpose neural network approach to general quantum problems

A general, feedforward neural network strategy for the treatment of a broad range of quantum problems including rotational and vibrational spectroscopy, tunnelling and band structure calculations is presented in this study.

nablaDFT: Large-Scale Conformational Energy and Hamiltonian Prediction benchmark and dataset

In this work we present nablaDFT, the new dataset and benchmark for the Density Functional Theory Hamiltonian and energy prediction. We provide data for over 1 million different molecules and over 5 million conformations and baseline models for both tasks.

Gauge Equivariant Neural Networks for Quantum Lattice Gauge Theories

Gauge symmetries play a key role in physics appearing in areas such as quantum field theories of the fundamental particles and emergent degrees of freedom in quantum materials. Motivated by the desire to efficiently simulate many-body quantum systems with exact local gauge invariance, gauge equivariant neural-network quantum states are introduced, which exactly satisfy the local Hilbert space constraints necessary for the description of quantum lattice gauge theory with Z_{d} gauge group and non-Abelian Kitaev D(G) models on different geometries. Focusing on the special case of Z_{2} gauge group on a periodically identified square lattice, the equivariant architecture is analytically shown to contain the loop-gas solution as a special case. Gauge equivariant neural-network quantum states are used in combination with variational quantum Monte Carlo to obtain compact descriptions of the ground state wave function for the Z_{2} theory away from the exactly solvable limit, and to demonstrate the confining or deconfining phase transition of the Wilson loop order parameter.

DeepNCI: DFT Noncovalent Interaction Correction with Transferable Multimodal Three-Dimensional Convolutional Neural Networks

A multimodal deep learning model, DeepNCI, is proposed for improving noncovalent interactions (NCIs) calculated via density functional theory (DFT). DeepNCI is composed of a three-dimensional convolutional neural network (3D CNN) for abstracting critical and comprehensive features from 3D electron density, and a neural network for modeling one-dimensional quantum chemical properties. By merging features from two networks, DeepNCI is able to reduce the root-mean-square error of DFT-calculated NCI from 1.19 kcal/mol to ∼0.2 kcal/mol for a NCI molecular database (>1000 molecules). The representativeness of the joint features can be visualized by t-distributed stochastic neighbor embedding (t-SNE), where they can distinguish categorized NCI systems quite well. Therefore, the fused model performs better than its component networks. In addition, the 3D CNN takes electron density as inputs that are in the same range, despite the size of molecular systems, so it can promote model applicability and transferability. To clarify the applicability of DeepNCI, an application domain (AD) has been defined with merged features using the K-nearest-neighbor method. The calculations for external test sets are shown that AD can properly monitor the reliability for a prediction. The model transferability is tested with a small database of homolysis bond dissociation energy including only dozens of samples. With NCI database pretrained parameters, the same or better performance than the reported results is achieved by transfer learning. This suggests that the DeepNCI model is transferable and it may transfer to other relative tasks, which possibly can resolve some small sampling problems. The source code of DeepNCI can be freely accessed at https://github.com/wenzelee/DeepNCI.

Artificial intelligence: machine learning for chemical sciences

Research in molecular sciences witnessed the rise and fall of Artificial Intelligence (AI)/ Machine Learning (ML) methods, especially artificial neural networks, few decades ago. However, we see a major resurgence in the use of modern ML methods in scientific research during the last few years. These methods have had phenomenal success in the areas of computer vision, speech recognition, natural language processing (NLP), etc. This has inspired chemists and biologists to apply these algorithms to problems in natural sciences. Availability of high performance Graphics Processing Unit (GPU) accelerators, large datasets, new algorithms, and libraries has enabled this surge. ML algorithms have successfully been applied to various domains in molecular sciences by providing much faster and sometimes more accurate solutions compared to traditional methods like Quantum Mechanical (QM) calculations, Density Functional Theory (DFT) or Molecular Mechanics (MM) based methods, etc. Some of the areas where the potential of ML methods are shown to be effective are in drug design, prediction of high-level quantum mechanical energies, molecular design, molecular dynamics materials, and retrosynthesis of organic compounds, etc. This article intends to conceptually introduce various modern ML methods and their relevance and applications in computational natural sciences. Synopsis Recent surge in the application of machine learning (ML) methods in fundamental sciences has led to a perspective that these methods may become important tools in chemical science. This perspective provides an overview of the modern ML methods and their successful applications in chemistry during the last few years.

SpookyNet: Learning force fields with electronic degrees of freedom and nonlocal effects

Machine-learned force fields combine the accuracy of ab initio methods with the efficiency of conventional force fields. However, current machine-learned force fields typically ignore electronic degrees of freedom, such as the total charge or spin state, and assume chemical locality, which is problematic when molecules have inconsistent electronic states, or when nonlocal effects play a significant role. This work introduces SpookyNet, a deep neural network for constructing machine-learned force fields with explicit treatment of electronic degrees of freedom and nonlocality, modeled via self-attention in a transformer architecture. Chemically meaningful inductive biases and analytical corrections built into the network architecture allow it to properly model physical limits. SpookyNet improves upon the current state-of-the-art (or achieves similar performance) on popular quantum chemistry data sets. Notably, it is able to generalize across chemical and conformational space and can leverage the learned chemical insights, e.g. by predicting unknown spin states, thus helping to close a further important remaining gap for today's machine learning models in quantum chemistry.

Symmetric and antisymmetric kernels for machine learning problems in quantum physics and chemistry

We derive symmetric and antisymmetric kernels by symmetrizing and antisymmetrizing conventional kernels and analyze their properties. In particular, we compute the feature space dimensions of the resulting polynomial kernels, prove that the reproducing kernel Hilbert spaces induced by symmetric and antisymmetric Gaussian kernels are dense in the space of symmetric and antisymmetric functions, and propose a Slater determinant representation of the antisymmetric Gaussian kernel, which allows for an efficient evaluation even if the state space is high-dimensional. Furthermore, we show that by exploiting symmetries or antisymmetries the size of the training data set can be significantly reduced. The results are illustrated with guiding examples and simple quantum physics and chemistry applications.

DeepCME: A deep learning framework for computing solution statistics of the chemical master equation

Stochastic models of biomolecular reaction networks are commonly employed in systems and synthetic biology to study the effects of stochastic fluctuations emanating from reactions involving species with low copy-numbers. For such models, the Kolmogorov's forward equation is called the chemical master equation (CME), and it is a fundamental system of linear ordinary differential equations (ODEs) that describes the evolution of the probability distribution of the random state-vector representing the copy-numbers of all the reacting species. The size of this system is given by the number of states that are accessible by the chemical system, and for most examples of interest this number is either very large or infinite. Moreover, approximations that reduce the size of the system by retaining only a finite number of important chemical states (e.g. those with non-negligible probability) result in high-dimensional ODE systems, even when the number of reacting species is small. Consequently, accurate numerical solution of the CME is very challenging, despite the linear nature of the underlying ODEs. One often resorts to estimating the solutions via computationally intensive stochastic simulations. The goal of the present paper is to develop a novel deep-learning approach for computing solution statistics of high-dimensional CMEs by reformulating the stochastic dynamics using Kolmogorov's backward equation. The proposed method leverages superior approximation properties of Deep Neural Networks (DNNs) to reliably estimate expectations under the CME solution for several user-defined functions of the state-vector. This method is algorithmically based on reinforcement learning and it only requires a moderate number of stochastic simulations (in comparison to typical simulation-based approaches) to train the "policy function". This allows not just the numerical approximation of various expectations for the CME solution but also of its sensitivities with respect to all the reaction network parameters (e.g. rate constants). We provide four examples to illustrate our methodology and provide several directions for future research.