A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems

Process optimization of line patterns in extreme ultraviolet lithography using machine learning and a simulated annealing algorithm

Resolution, line edge/width roughness, and sensitivity (RLS) are critical indicators for evaluating the imaging performance of resists. As the technology node gradually shrinks, stricter indicator control is required for high-resolution imaging. However, current research can improve only part of the RLS indicators of resists for line patterns, and it is difficult to improve the overall imaging performance of resists in extreme ultraviolet lithography. Here, we report a lithographic process optimization system of line patterns, where RLS models are first established by adopting a machine learning method, and then these models are optimized using a simulated annealing algorithm. Finally, the process parameter combination with optimal imaging quality of line patterns can be obtained. This system can control resist RLS indicators, and it exhibits high optimization accuracy, which facilitates the reduction of process optimization time and cost and accelerates the development of the lithography process.

Learning local equivariant representations for large-scale atomistic dynamics

A simultaneously accurate and computationally efficient parametrization of the potential energy surface of molecules and materials is a long-standing goal in the natural sciences. While atom-centered message passing neural networks (MPNNs) have shown remarkable accuracy, their information propagation has limited the accessible length-scales. Local methods, conversely, scale to large simulations but have suffered from inferior accuracy. This work introduces Allegro, a strictly local equivariant deep neural network interatomic potential architecture that simultaneously exhibits excellent accuracy and scalability. Allegro represents a many-body potential using iterated tensor products of learned equivariant representations without atom-centered message passing. Allegro obtains improvements over state-of-the-art methods on QM9 and revMD17. A single tensor product layer outperforms existing deep MPNNs and transformers on QM9. Furthermore, Allegro displays remarkable generalization to out-of-distribution data. Molecular simulations using Allegro recover structural and kinetic properties of an amorphous electrolyte in excellent agreement with ab-initio simulations. Finally, we demonstrate parallelization with a simulation of 100 million atoms.

Orbital Mixer: Using Atomic Orbital Features for Basis-Dependent Prediction of Molecular Wavefunctions

Leveraging ab initio data at scale has enabled the development of machine learning models capable of extremely accurate and fast molecular property prediction. A central paradigm of many previous studies focuses on generating predictions for only a fixed set of properties. Recent lines of research instead aim to explicitly learn the electronic structure via molecular wavefunctions, from which other quantum chemical properties can be directly derived. While previous methods generate predictions as a function of only the atomic configuration, in this work we present an alternate approach that directly purposes basis-dependent information to predict molecular electronic structure. Our model, Orbital Mixer, is composed entirely of multi-layer perceptrons (MLPs) using MLP-Mixer layers within a simple, intuitive, and scalable architecture that achieves competitive Hamiltonian and molecular orbital energy and coefficient prediction accuracies compared to the state-of-the-art.

Active learning of reactive Bayesian force fields applied to heterogeneous catalysis dynamics of H/Pt

Atomistic modeling of chemically reactive systems has so far relied on either expensive ab initio methods or bond-order force fields requiring arduous parametrization. Here, we describe a Bayesian active learning framework for autonomous "on-the-fly" training of fast and accurate reactive many-body force fields during molecular dynamics simulations. At each time-step, predictive uncertainties of a sparse Gaussian process are evaluated to automatically determine whether additional ab initio training data are needed. We introduce a general method for mapping trained kernel models onto equivalent polynomial models whose prediction cost is much lower and independent of the training set size. As a demonstration, we perform direct two-phase simulations of heterogeneous H2 turnover on the Pt(111) catalyst surface at chemical accuracy. The model trains itself in three days and performs at twice the speed of a ReaxFF model, while maintaining much higher fidelity to DFT and excellent agreement with experiment.

E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials

This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations. While most contemporary symmetry-aware models use invariant convolutions and only act on scalars, NequIP employs E(3)-equivariant convolutions for interactions of geometric tensors, resulting in a more information-rich and faithful representation of atomic environments. The method achieves state-of-the-art accuracy on a challenging and diverse set of molecules and materials while exhibiting remarkable data efficiency. NequIP outperforms existing models with up to three orders of magnitude fewer training data, challenging the widely held belief that deep neural networks require massive training sets. The high data efficiency of the method allows for the construction of accurate potentials using high-order quantum chemical level of theory as reference and enables high-fidelity molecular dynamics simulations over long time scales.

Graph neural networks accelerated molecular dynamics

Molecular Dynamics (MD) simulation is a powerful tool for understanding the dynamics and structure of matter. Since the resolution of MD is atomic-scale, achieving long timescale simulations with femtosecond integration is very expensive. In each MD step, numerous iterative computations are performed to calculate energy based on different types of interaction and their corresponding spatial gradients. These repetitive computations can be learned and surrogated by a deep learning model, such as a Graph Neural Network (GNN). In this work, we developed a GNN Accelerated MD (GAMD) model that directly predicts forces, given the state of the system (atom positions, atom types), bypassing the evaluation of potential energy. By training the GNN on a variety of data sources (simulation data derived from classical MD and density functional theory), we show that GAMD can predict the dynamics of two typical molecular systems, Lennard-Jones system and water system, in the NVT ensemble with velocities regulated by a thermostat. We further show that GAMD’s learning and inference are agnostic to the scale, where it can scale to much larger systems at test time. We also perform a comprehensive benchmark test comparing our implementation of GAMD to production-level MD software, showing GAMD’s competitive performance on the large-scale simulation.

Quality of uncertainty estimates from neural network potential ensembles

Neural network potentials (NNPs) combine the computational efficiency of classical interatomic potentials with the high accuracy and flexibility of the ab initio methods used to create the training set, but can also result in unphysical predictions when employed outside their training set distribution. Estimating the epistemic uncertainty of a NNP is required in active learning or on-the-fly generation of potentials. Inspired from their use in other machine-learning applications, NNP ensembles have been used for uncertainty prediction in several studies, with the caveat that ensembles do not provide a rigorous Bayesian estimate of the uncertainty. To test whether NNP ensembles provide accurate uncertainty estimates, we train such ensembles in four different case studies and compare the predicted uncertainty with the errors on out-of-distribution validation sets. Our results indicate that NNP ensembles are often overconfident, underestimating the uncertainty of the model, and require to be calibrated for each system and architecture. We also provide evidence that Bayesian NNPs, obtained by sampling the posterior distribution of the model parameters using Monte Carlo techniques, can provide better uncertainty estimates.

First-principles materials simulation and design for alkali and alkaline metal ion batteries accelerated by machine learning

The challenge of regeneration of batteries requires a performance improvement in the alkali/alkaline metal ion battery (AMIB) materials, whereas the traditional research paradigm fully based on experiments and theoretical simulations needs massive research and development investment. During the last decade, machine learning (ML) has made breakthroughs in many complex disciplines, which testifies to their high processing speed and ability to capture relationships. Inspired by these achievements, ML has also been introduced to bring a new paradigm for shortening the development of AMIB materials. In this Perspective, the focus will be on how this new ML technology solves the key problems of redox potentials, ionic conductivity and stability parameters in first-principles materials' simulation and design for AMIBs. It is found that ML not only accelerates the property prediction, but also gives physicochemical insights into AMIB materials' design. In addition, the final part of this paper summarizes current achievements and looks forward to the progress of a novel paradigm in direct/inverse design with the increasing number of databases, skills, and ML technologies for AMIBs.

Leveraging Transfer Learning and Chemical Principles toward Interpretable Materials Properties

Machine learning is emerging as a new paradigm to rationalize chemical properties for deepening our understanding of chemistry and providing instructive clues on better materials performance. While the complex architecture of machine learning contributes to unprecedented capability in this task, it prevents easy interpretation, leading to extensive criticisms on the lack of physical foundations for the black-box like models. Here, we demonstrate a transfer learning strategy that leverages fundamental principles of chemistry to offer adequate physical insights for the interpretation. Through interpreting the models for the formation energies of inorganic compounds, the proposed strategy revealed the deficiency of deep neural network in handling interelemental patterns and proved the more proper abstraction of recurrent neural network with attention mechanism, which led to predicting the elegant form of periodic table with high precision. The success demonstrates a new solution toward models with full transparency in materials informatics.