Learning neural network potentials from experimental data via Differentiable Trajectory Reweighting

Structural Coarse-Graining via Multiobjective Optimization with Differentiable Simulation

In the realm of multiscale molecular simulations, structure-based coarse-graining is a prominent approach for creating efficient coarse-grained (CG) representations of soft matter systems, such as polymers. This involves optimizing CG interactions by matching static correlation functions of the corresponding degrees of freedom in all-atom (AA) models. Here, we present a versatile method, namely, differentiable coarse-graining (DiffCG), which combines multiobjective optimization and differentiable simulation. The DiffCG approach is capable of constructing robust CG models by iteratively optimizing the effective potentials to simultaneously match multiple target properties. We demonstrate our approach by concurrently optimizing bonded and nonbonded potentials of a CG model of polystyrene (PS) melts. The resulting CG-PS model effectively reproduces both the structural characteristics, such as the equilibrium probability distribution of microscopic degrees of freedom and the thermodynamic pressure of the AA counterpart. More importantly, leveraging the multiobjective optimization capability, we develop a precise and efficient CG model for PS melts that is transferable across a wide range of temperatures, i.e., from 400 to 600 K. It is achieved via optimizing a pairwise potential with nonlinear temperature dependence in the CG model to simultaneously match target data from AA-MD simulations at multiple thermodynamic states. The temperature transferable CG-PS model demonstrates its ability to accurately predict the radial distribution functions and density at different temperatures, including those that are not included in the target thermodynamic states. Our work opens up a promising route for developing accurate and transferable CG models of complex soft-matter systems through multiobjective optimization with differentiable simulation.

Improving potential energy surfaces using measured Feshbach resonance states

The structure and dynamics of a molecular system is governed by its potential energy surface (PES), representing the total energy as a function of the nuclear coordinates. Obtaining accurate potential energy surfaces is limited by the exponential scaling of Hilbert space, restricting quantitative predictions of experimental observables from first principles to small molecules with just a few electrons. Here, we present an explicitly physics-informed approach for improving and assessing the quality of families of PESs by modifying them through linear coordinate transformations based on experimental data. We demonstrate this "morphing" of the PES for the He - H2+ complex using recent comprehensive Feshbach resonance (FR) measurements for reference PESs at three different levels of quantum chemistry. In all cases, the positions and intensities of peaks in the energy distributions are improved. We find these observables to be mainly sensitive to the long-range part of the PES.

Machine Learning Potentials with the Iterative Boltzmann Inversion: Training to Experiment

Methodologies for training machine learning potentials (MLPs) with quantum-mechanical simulation data have recently seen tremendous progress. Experimental data have a very different character than simulated data, and most MLP training procedures cannot be easily adapted to incorporate both types of data into the training process. We investigate a training procedure based on iterative Boltzmann inversion that produces a pair potential correction to an existing MLP using equilibrium radial distribution function data. By applying these corrections to an MLP for pure aluminum based on density functional theory, we observe that the resulting model largely addresses previous overstructuring in the melt phase. Interestingly, the corrected MLP also exhibits improved performance in predicting experimental diffusion constants, which are not included in the training procedure. The presented method does not require autodifferentiating through a molecular dynamics solver and does not make assumptions about the MLP architecture. Our results suggest a practical framework for incorporating experimental data into machine learning models to improve the accuracy of molecular dynamics simulations.

A Multiple-Fidelity Method for Accurate Simulation of MoS2 Properties Using JAX-ReaxFF and Neural Network Potentials

Reactive force field (ReaxFF) is a commonly used force field for modeling chemical reactions at the atomic level. Recently, JAX-ReaxFF, combined with automatic differentiation, has been used to efficiently parametrize ReaxFF. However, its analytical formula may lead to inaccurate predictions. While neural network-based potentials (NNPs) trained on density functional theory-labeled data offer a more accurate method, it requires a large amount of training data to be trained from scratch. To overcome these issues, we present a multiple-fidelity method that combines JAX-ReaxFF and NNP and apply the method on MoS2, a promising two-dimensional semiconductor for flexible electronics. By incorporating implicit prior physical information, ReaxFF can serve as a cost-effective way to generate pretraining data, facilitating more accurate simulations of MoS2. Moreover, in the Mo-S-H system, the pretraining strategy can reduce root-mean-square errors of energy by 20%. This approach can be extended to a wide variety of material systems, accelerating their computational research.

Developing an Implicit Solvation Machine Learning Model for Molecular Simulations of Ionic Media

Molecular dynamics (MD) simulations of biophysical systems require accurate modeling of their native environment, i.e., aqueous ionic solution, as it critically impacts the structure and function of biomolecules. On the other hand, the models should be computationally efficient to enable simulations of large spatiotemporal scales. Here, we present the deep implicit solvation model for sodium chloride solutions that satisfies both requirements. Owing to the use of the neural network potential, the model can capture the many-body potential of mean force, while the implicit water treatment renders the model inexpensive. We demonstrate our approach first for pure ionic solutions with concentrations ranging from physiological to 2 M. We then extend the model to capture the effective ion interactions in the vicinity and far away from a DNA molecule. In both cases, the structural properties are in good agreement with all-atom MD, showcasing a general methodology for the efficient and accurate modeling of ionic media.

Lessons Learned from Multiobjective Automatic Optimizations of Classical Three-Site Rigid Water Models Using Microscopic and Macroscopic Target Experimental Observables

The development of accurate water models is of primary importance for molecular simulations. Despite their intrinsic approximations, three-site rigid water models are still ubiquitously used to simulate a variety of molecular systems. Automatic optimization approaches have been recently used to iteratively refine three-site water models to fit macroscopic (average) thermodynamic properties, providing state-of-the-art three-site models that still present some deviations from the liquid water properties. Here, we show the results obtained by automatically optimizing three-site rigid water models to fit a combination of microscopic and macroscopic experimental observables. We use Swarm-CG, a multiobjective particle-swarm-optimization algorithm, for training the models to reproduce the experimental radial distribution functions of liquid water at various temperatures (rich in microscopic-level information on, e.g., the local orientation and interactions of the water molecules). We systematically analyze the agreement of these models with experimental observables and the effect of adding macroscopic information to the training set. Our results demonstrate how adding microscopic-rich information in the training of water models allows one to achieve state-of-the-art accuracy in an efficient way. Limitations in the approach and in the approximated description of water in these three-site models are also discussed, providing a demonstrative case useful for the optimization of approximated molecular models, in general.

Top-Down Machine Learning of Coarse-Grained Protein Force Fields

Developing accurate and efficient coarse-grained representations of proteins is crucial for understanding their folding, function, and interactions over extended time scales. Our methodology involves simulating proteins with molecular dynamics and utilizing the resulting trajectories to train a neural network potential through differentiable trajectory reweighting. Remarkably, this method requires only the native conformation of proteins, eliminating the need for labeled data derived from extensive simulations or memory-intensive end-to-end differentiable simulations. Once trained, the model can be employed to run parallel molecular dynamics simulations and sample folding events for proteins both within and beyond the training distribution, showcasing its extrapolation capabilities. By applying Markov state models, native-like conformations of the simulated proteins can be predicted from the coarse-grained simulations. Owing to its theoretical transferability and ability to use solely experimental static structures as training data, we anticipate that this approach will prove advantageous for developing new protein force fields and further advancing the study of protein dynamics, folding, and interactions.

Data-Driven Approach to Coarse-Graining Simple Liquids in Confinement

We propose a data-driven framework for identifying coarse-grained (CG) Lennard-Jones (LJ) potential parameters in confined systems for simple liquids. Our approach involves the use of a Deep Neural Network (DNN) that is trained to approximate the solution of the Inverse Liquid State (ILST) problem for confined systems. The DNN model inherently incorporates essential physical characteristics specific to confined fluids, enabling an accurate prediction of inhomogeneity effects. By utilizing transfer learning, we predict single-site LJ potentials of simple multiatomic liquids confined in a slit-like channel, which effectively replicate both the fluid structure and molecular force of the target All-Atom (AA) system when the electrostatic interactions are not dominant. In addition, we showcase the synergy between the data-driven approach and the well-known Bottom-Up coarse-graining method utilizing Relative-Entropy (RE) Minimization. Through the sequential utilization of these two methods, the robustness of the iterative RE method is significantly augmented, leading to a remarkable enhancement in convergence.

Machine learning coarse-grained potentials of protein thermodynamics

A generalized understanding of protein dynamics is an unsolved scientific problem, the solution of which is critical to the interpretation of the structure-function relationships that govern essential biological processes. Here, we approach this problem by constructing coarse-grained molecular potentials based on artificial neural networks and grounded in statistical mechanics. For training, we build a unique dataset of unbiased all-atom molecular dynamics simulations of approximately 9 ms for twelve different proteins with multiple secondary structure arrangements. The coarse-grained models are capable of accelerating the dynamics by more than three orders of magnitude while preserving the thermodynamics of the systems. Coarse-grained simulations identify relevant structural states in the ensemble with comparable energetics to the all-atom systems. Furthermore, we show that a single coarse-grained potential can integrate all twelve proteins and can capture experimental structural features of mutated proteins. These results indicate that machine learning coarse-grained potentials could provide a feasible approach to simulate and understand protein dynamics.

DMFF: An Open-Source Automatic Differentiable Platform for Molecular Force Field Development and Molecular Dynamics Simulation

In the simulation of molecular systems, the underlying force field (FF) model plays an extremely important role in determining the reliability of the simulation. However, the quality of the state-of-the-art molecular force fields is still unsatisfactory in many cases, and the FF parameterization process largely relies on human experience, which is not scalable. To address this issue, we introduce DMFF, an open-source molecular FF development platform based on an automatic differentiation technique. DMFF serves as a powerful tool for both top-down and bottom-up FF development. Using DMFF, both energies/forces and thermodynamic quantities such as ensemble averages and free energies can be evaluated in a differentiable way, realizing an automatic, yet highly flexible FF optimization workflow. DMFF also eases the evaluation of forces and virial tensors for complicated advanced FFs, helping the fast validation of new models in molecular dynamics simulation. DMFF has been released as an open-source package under the LGPL-3.0 license and is available at https://github.com/deepmodeling/DMFF.

Neural potentials of proteins extrapolate beyond training data

We evaluate neural network (NN) coarse-grained (CG) force fields compared to traditional CG molecular mechanics force fields. We conclude that NN force fields are able to extrapolate and sample from unseen regions of the free energy surface when trained with limited data. Our results come from 88 NN force fields trained on different combinations of clustered free energy surfaces from four protein mapped trajectories. We used a statistical measure named total variation similarity to assess the agreement between reference free energy surfaces from mapped atomistic simulations and CG simulations from trained NN force fields. Our conclusions support the hypothesis that NN CG force fields trained with samples from one region of the proteins' free energy surface can, indeed, extrapolate to unseen regions. Additionally, the force matching error was found to only be weakly correlated with a force field's ability to reconstruct the correct free energy surface.

Neural network atomistic potentials for global energy minima search in carbon clusters

The global energy optimization problem is an acute and important problem in chemistry. It is crucial to know the geometry of the lowest energy isomer (global minimum, GM) of a given compound for the evaluation of its chemical and physical properties. This problem is especially relevant for atomic clusters. Due to the exponential growth of the number of local minima geometries with the increase of the number of atoms in the cluster, it is important to find a computationally efficient and reliable method to navigate the energy landscape and locate a true global minima structure. Newly developed neural network (NN) atomistic potentials offer a numerically efficient and relatively accurate approach for molecular structure optimization. An important question that needs to be answered is "Can NN potentials, trained on a given set, represent the potential energy surface (PES) of a neighboring domain?". In this work, we tested the applicability of ANI-1ccx and ANI-nr NN atomistic potentials for the global minima optimization of carbon clusters Cn (n = 3-10). We showed that with the introduction of the cluster connectivity restriction and consequent DFT or ab initio calculations, ANI-1ccx and ANI-nr can be considered as robust PES pre-samplers that can capture the GM structure even for large clusters such as C20.

Multibody Terms in Protein Coarse-Grained Models: A Top-Down Perspective

Coarse-grained models allow computational investigation of biomolecular processes occurring on long time and length scales, intractable with atomistic simulation. Traditionally, many coarse-grained models rely mostly on pairwise interaction potentials. However, the decimation of degrees of freedom should, in principle, lead to a complex many-body effective interaction potential. In this work, we use experimental data on mutant stability to parametrize coarse-grained models for two proteins with and without many-body terms. We demonstrate that many-body terms are necessary to reproduce quantitatively the effects of point mutations on protein stability, particularly to implicitly take into account the effect of the solvent.

The confluence of machine learning and multiscale simulations

Multiscale modeling has a long history of use in structural biology, as computational biologists strive to overcome the time- and length-scale limits of atomistic molecular dynamics. Contemporary machine learning techniques, such as deep learning, have promoted advances in virtually every field of science and engineering and are revitalizing the traditional notions of multiscale modeling. Deep learning has found success in various approaches for distilling information from fine-scale models, such as building surrogate models and guiding the development of coarse-grained potentials. However, perhaps its most powerful use in multiscale modeling is in defining latent spaces that enable efficient exploration of conformational space. This confluence of machine learning and multiscale simulation with modern high-performance computing promises a new era of discovery and innovation in structural biology.

Statistically Optimal Force Aggregation for Coarse-Graining Molecular Dynamics

Machine-learned coarse-grained (CG) models have the potential for simulating large molecular complexes beyond what is possible with atomistic molecular dynamics. However, training accurate CG models remains a challenge. A widely used methodology for learning bottom-up CG force fields maps forces from all-atom molecular dynamics to the CG representation and matches them with a CG force field on average. We show that there is flexibility in how to map all-atom forces to the CG representation and that the most commonly used mapping methods are statistically inefficient and potentially even incorrect in the presence of constraints in the all-atom simulation. We define an optimization statement for force mappings and demonstrate that substantially improved CG force fields can be learned from the same simulation data when using optimized force maps. The method is demonstrated on the miniproteins chignolin and tryptophan cage and published as open-source code.

Machine learned coarse-grained protein force-fields: Are we there yet?

The successful recent application of machine learning methods to scientific problems includes the learning of flexible and accurate atomic-level force-fields for materials and biomolecules from quantum chemical data. In parallel, the machine learning of force-fields at coarser resolutions is rapidly gaining relevance as an efficient way to represent the higher-body interactions needed in coarse-grained force-fields to compensate for the omitted degrees of freedom. Coarse-grained models are important for the study of systems at time and length scales exceeding those of atomistic simulations. However, the development of transferable coarse-grained models via machine learning still presents significant challenges. Here, we discuss recent developments in this field and current efforts to address the remaining challenges.

Learning pair potentials using differentiable simulations

Learning pair interactions from experimental or simulation data is of great interest for molecular simulations. We propose a general stochastic method for learning pair interactions from data using differentiable simulations (DiffSim). DiffSim defines a loss function based on structural observables, such as the radial distribution function, through molecular dynamics (MD) simulations. The interaction potentials are then learned directly by stochastic gradient descent, using backpropagation to calculate the gradient of the structural loss metric with respect to the interaction potential through the MD simulation. This gradient-based method is flexible and can be configured to simulate and optimize multiple systems simultaneously. For example, it is possible to simultaneously learn potentials for different temperatures or for different compositions. We demonstrate the approach by recovering simple pair potentials, such as Lennard-Jones systems, from radial distribution functions. We find that DiffSim can be used to probe a wider functional space of pair potentials compared with traditional methods like iterative Boltzmann inversion. We show that our methods can be used to simultaneously fit potentials for simulations at different compositions and temperatures to improve the transferability of the learned potentials.

Deep coarse-grained potentials via relative entropy minimization

Neural network (NN) potentials are a natural choice for coarse-grained (CG) models. Their many-body capacity allows highly accurate approximations of the potential of mean force, promising CG simulations of unprecedented accuracy. CG NN potentials trained bottom-up via force matching (FM), however, suffer from finite data effects: They rely on prior potentials for physically sound predictions outside the training data domain, and the corresponding free energy surface is sensitive to errors in the transition regions. The standard alternative to FM for classical potentials is relative entropy (RE) minimization, which has not yet been applied to NN potentials. In this work, we demonstrate, for benchmark problems of liquid water and alanine dipeptide, that RE training is more data efficient, due to accessing the CG distribution during training, resulting in improved free energy surfaces and reduced sensitivity to prior potentials. In addition, RE learns to correct time integration errors, allowing larger time steps in CG molecular dynamics simulation, while maintaining accuracy. Thus, our findings support the use of training objectives beyond FM, as a promising direction for improving CG NN potential's accuracy and reliability.

Machine Learning and Optoelectronic Materials Discovery: A Growing Synergy

Novel optoelectronic materials have the potential to revolutionize the ongoing green transition by both providing more efficient photovoltaic (PV) devices and lowering energy consumption of devices like LEDs and sensors. The lead candidate materials for these applications are both organic semiconductors and more recently perovskites. This Perspective illustrates how novel machine learning techniques can help explore these materials, from speeding up ab initio calculations toward experimental guidance. Furthermore, based on existing work, perspectives around machine-learned molecular dynamics potentials, physically informed neural networks, and generative methods are outlined.