An implementation of the Levenberg–Marquardt algorithm for simultaneous-energy-gradient fitting using two-layer feed-forward neural networks

Computational Investigation of the Potential and Limitations of Machine Learning with Neural Network Circuits Based on Synaptic Transistors

Synaptic transistors have been proposed to implement neuron activation functions of neural networks (NNs). While promising to enable compact, fast, inexpensive, and energy-efficient dedicated NN circuits, they also have limitations compared to digital NNs (realized as codes for digital processors), including shape choices of the activation function using particular types of transistor implementation, and instabilities due to noise and other factors present in analog circuits. We present a computational study of the effects of these factors on NN performance and find that, while accuracy competitive with traditional NNs can be realized for many applications, there is high sensitivity to the instability in the shape of the activation function, suggesting that, when highly accurate NNs are required, high-precision circuitry should be developed beyond what has been reported for synaptic transistors to date.

Evaluation of Rate Coefficients in the Gas Phase Using Machine-Learned Potentials

We assess the capability of machine-learned potentials to compute rate coefficients by training a neural network (NN) model and applying it to describe the chemical landscape on the C5H5 potential energy surface, which is relevant to molecular weight growth in combustion and interstellar media. We coupled the resulting NN with an automated kinetics workflow code, KinBot, to perform all necessary calculations to compute the rate coefficients. The NN is benchmarked exhaustively by evaluating its performance at the various stages of the kinetics calculations: from the electronic energy through the computation of zero point energy, barrier heights, entropic contributions, the portion of the PES explored, and finally the overall rate coefficients as formulated by transition state theory.

Heat Transfer Analysis of Nanofluid Flow in a Rotating System with Magnetic Field Using an Intelligent Strength Stochastic-Driven Approach

This paper investigates the heat transfer of two-phase nanofluid flow between horizontal plates in a rotating system with a magnetic field and external forces. The basic continuity and momentum equations are considered to formulate the governing mathematical model of the problem. Furthermore, certain similarity transformations are used to reduce a governing system of non-linear partial differential equations (PDEs) into a non-linear system of ordinary differential equations. Moreover, an efficient stochastic technique based on feed-forward neural networks (FFNNs) with a back-propagated Levenberg–Marquardt (BLM) algorithm is developed to examine the effect of variations in various parameters on velocity, gravitational acceleration, temperature, and concentration profiles of the nanofluid. To validate the accuracy, efficiency, and computational complexity of the FFNN–BLM algorithm, different performance functions are defined based on mean absolute deviations (MAD), error in Nash–Sutcliffe efficiency (ENSE), and Theil’s inequality coefficient (TIC). The approximate solutions achieved by the proposed technique are validated by comparing with the least square method (LSM), machine learning algorithms such as NARX-LM, and numerical solutions by the Runge–Kutta–Fehlberg method (RKFM). The results demonstrate that the mean percentage error in our solutions and values of ENSE, TIC, and MAD is almost zero, showing the design algorithm’s robustness and correctness.

Mechanism and Kinetics of Methane Combustion. Part II: Potential Energy Surface for Hydrogen-Abstraction Reaction of CH4 + O(3P)

Methane combustion plays an important role in various fields such as combustion chemistry and atmospheric chemistry of the stratosphere. Highly accurate study of its initial reaction remains a key challenge. Here, through extensive studies with a state-of-the-art ab initio and neural network method, we present a potential energy surface of the O(³P) + CH₄ → OH + CH₃ reaction on the ground state 1³A and the first excited state 2³A. In this work, the energies of 10 167 points covering all important regions are obtained with state-averaged complete active space self-consistent field calculations and then fitted using the Levenberg-Marquardt algorithm with a root-mean-square error of 0.391 and 0.442 kcal/mol for the 1³A and 2³A states, respectively. This study explores the characteristics of the radical van der Waals (VdW) complex and reveals a detailed mechanism of the methane combustion initial reaction. Within the scope of this mechanism, this surface gives a fairly accurate description of the regions around the saddle point, conical intersection, and vdW wells in the entrance for efficient computational simulations. As a theoretical study on a prototypical polyatomic reaction, it is hopeful that this work will modify our understanding of the primary process in hydrocarbon combustion.

Machine learning to predict effective reaction rates in 3D porous media from pore structural features

Large discrepancies between well-mixed reaction rates and effective reactions rates estimated under fluid flow conditions have been a major issue for predicting reactive transport in porous media systems. In this study, we introduce a framework that accurately predicts effective reaction rates directly from pore structural features by combining 3D pore-scale numerical simulations with machine learning (ML). We first perform pore-scale reactive transport simulations with fluid–solid reactions in hundreds of porous media and calculate effective reaction rates from pore-scale concentration fields. We then train a Random Forests model with 11 pore structural features and effective reaction rates to quantify the importance of structural features in determining effective reaction rates. Based on the importance information, we train artificial neural networks with varying number of features and demonstrate that effective reaction rates can be accurately predicted with only three pore structural features, which are specific surface, pore sphericity, and coordination number. Finally, global sensitivity analyses using the ML model elucidates how the three structural features affect effective reaction rates. The proposed framework enables accurate predictions of effective reaction rates directly from a few measurable pore structural features, and the framework is readily applicable to a wide range of applications involving porous media flows.

Energy- and Local-Gradient-Based Neural Network Method for Accurately Describing Long-Range Interaction: Application to the H2 + CO+ Reaction

The long-range interaction plays an important role in theoretically describing ion-molecule reaction. However, most energy-based neural network fitting methods usually introduce spurious long-range interactions. In this work, we propose an energy- and local-gradient-based neural network (ELGNN) method to fit potential energy surfaces (PESs). K-means clustering is employed to divide the whole configuration space into three regions: reactant asymptotic region, interaction region, and product asymptotic region. In the interaction region, only the energies of sampled points are computed, while in the asymptotic regions, the gradients of partially sampled configurations are calculated as well, and both the energies and energy gradients (if necessary) are used to fit long-range interactions. These regions are joined together by switching functions. The ELGNN method is first applied to fit the PES of the H₂ + CO⁺ reaction, which has significant long-range interactions. It is found that the ELGNN method works better than the energy-based NN method in describing long-range interactions. The dynamics and kinetics of the reaction are then investigated on the new PES.

Soft Sensors for State of Charge, State of Energy, and Power Loss in Formula Student Electric Vehicle

The proliferation of electric vehicle (EV) technology is an important step towards a more sustainable future. In the current work, two-layer feed-forward artificial neural-network-based machine learning is applied to design soft sensors to estimate the state of charge (SOC), state of energy (SOE), and power loss (PL) of a formula student electric vehicle (FSEV) battery-pack system. The proposed soft sensors were designed to predict the SOC, SOE, and PL of the EV battery pack on the basis of the input current profile. The input current profile was derived on the basis of the designed vehicle parameters, and formula Bharat track features and guidelines. All developed soft sensors were tested for mean squared error (MSE) and R-squared metrics of the dataset partitions; equations relating the derived and predicted outputs; error histograms of the training, validation, and testing datasets; training state indicators such as gradient, mu, and validation fails; validation performance over successive epochs; and predicted versus derived plots over one lap time. Moreover, the prediction accuracy of the proposed soft sensors was compared against linear or nonlinear regression models and parametric structure models used for system identification such as autoregressive with exogenous variables (ARX), autoregressive moving average with exogenous variables (ARMAX), output error (OE) and Box Jenkins (BJ). The testing dataset accuracy of the proposed FSEV SOC, SOE, PL soft sensors was 99.96%, 99.96%, and 99.99%, respectively. The proposed soft sensors attained higher prediction accuracy than that of the modelling structures mentioned above. FSEV results also indicated that the SOC and SOE dropped from 97% to 93.5% and 93.8%, respectively, during the running time of 118 s (one lap time). Thus, two-layer feed-forward neural-network-based soft sensors can be applied for the effective monitoring and prediction of SOC, SOE, and PL during the operation of EVs.

A fundamental invariant-neural network representation of quasi-diabatic Hamiltonians for the two lowest states of H3

The FI-NN approach is capable of representing highly accurate diabatic PESs with particular and complicated symmetry problems.

Toward Social Media Content Recommendation Integrated with Data Science and Machine Learning Approach for E-Learners

Electronic Learning (e-learning) has made a great success and recently been estimated as a billion-dollar industry. The users of e-learning acquire knowledge of diversified content available in an application using innovative means. There is much e-learning software available—for example, LMS (Learning Management System) and Moodle. The functionalities of this software were reviewed and we recognized that learners have particular problems in getting relevant recommendations. For example, there might be essential discussions about a particular topic on social networks, such as Twitter, but that discussion is not linked up and recommended to the learners for getting the latest updates on technology-updated news related to their learning context. This has been set as the focus of the current project based on symmetry between user project specification. The developed project recommends relevant symmetric articles to e-learners from the social network of Twitter and the academic platform of DBLP. For recommendations, a Reinforcement learning model with optimization is employed, which utilizes the learners’ local context, learners’ profile available in the e-learning system, and the learners’ historical views. The recommendations by the system are relevant tweets, popular relevant Twitter users, and research papers from DBLP. For matching the local context, profile, and history with the tweet text, we recognized that terms in the e-learning system need to be expanded to cover a wide range of concepts. However, this diversification should not include such terms which are irrelevant. To expand terms of the local context, profile and history, the software used the dataset of Grow-bag, which builds concept graphs of large-scale Computer Science topics based on the co-occurrence scores of Computer Science terms. This application demonstrated the need and success of e-learning software that is linked with social media and sends recommendations for the content being learned by the e-Learners in the e-learning environment. However, the current application only focuses on the Computer Science domain. There is a need for generalizing such applications to other domains in the future.

Neural Network Potential Energy Surfaces for Small Molecules and Reactions

We review progress in neural network (NN)-based methods for the construction of interatomic potentials from discrete samples (such as ab initio energies) for applications in classical and quantum dynamics including reaction dynamics and computational spectroscopy. The main focus is on methods for building molecular potential energy surfaces (PES) in internal coordinates that explicitly include all many-body contributions, even though some of the methods we review limit the degree of coupling, due either to a desire to limit computational cost or to limited data. Explicit and direct treatment of all many-body contributions is only practical for sufficiently small molecules, which are therefore our primary focus. This includes small molecules on surfaces. We consider direct, single NN PES fitting as well as more complex methods that impose structure (such as a multibody representation) on the PES function, either through the architecture of one NN or by using multiple NNs. We show how NNs are effective in building representations with low-dimensional functions including dimensionality reduction. We consider NN-based approaches to build PESs in the sums-of-product form important for quantum dynamics, ways to treat symmetry, and issues related to sampling data distributions and the relation between PES errors and errors in observables. We highlight combinations of NNs with other ideas such as permutationally invariant polynomials or sums of environment-dependent atomic contributions, which have recently emerged as powerful tools for building highly accurate PESs for relatively large molecular and reactive systems.

Exclusive Neural Network Representation of the Quasi-Diabatic Hamiltonians Including Conical Intersections

We propose a numerically simple and straightforward, yet accurate and efficient neural networks-based fitting strategy to construct coupled potential energy surfaces (PESs) in a quasi-diabatic representation. The fundamental invariants are incorporated to account for the complete nuclear permutation inversion symmetry. Instead of derivative couplings or interstate couplings, a so-called modified derivative coupling term is fitted by neural networks, resulting in accurate description of near degeneracy points, such as the conical intersections. The adiabatic energies, energy gradients, and derivative couplings are well reproduced, and the vanishing of derivative couplings as well as the isotropic topography of adiabatic and diabatic energies in asymptotic regions are automatically satisfied. All of these features of the coupled global PESs are requisite for accurate dynamics simulations. Our approach is expected to be very useful in developing highly accurate coupled PESs in a quasi-diabatic representation in an efficient machine learning-based way.

Representation of coupled adiabatic potential energy surfaces using neural network based quasi-diabatic Hamiltonians: 1,2 2A′ states of LiFH

A general algorithm for determining diabatic representations from adiabatic energies, energy gradients and derivative couplings using neural networks is introduced.

Two-state diabatic potential energy surfaces of ClH2 based on nonadiabatic couplings with neural networks

A neural network-fitting procedure based on nonadiabatic couplings is proposed to generate two-state diabatic PESs with conical intersections.

Observation of the geometric phase effect in the H + HD → H2+ D reaction

Theory has established the importance of geometric phase (GP) effects in the adiabatic dynamics of molecular systems with a conical intersection connecting the ground- and excited-state potential energy surfaces, but direct observation of their manifestation in chemical reactions remains a major challenge. Here, we report a high-resolution crossed molecular beams study of the H + HD → H2+ D reaction at a collision energy slightly above the conical intersection. Velocity map ion imaging revealed fast angular oscillations in product quantum state–resolved differential cross sections in the forward scattering direction for H2products at specific rovibrational levels. The experimental results agree with adiabatic quantum dynamical calculations only when the GP effect is included.

Inclusion of Machine Learning Kernel Ridge Regression Potential Energy Surfaces in On-the-Fly Nonadiabatic Molecular Dynamics Simulation

We discuss a theoretical approach that employs machine learning potential energy surfaces (ML-PESs) in the nonadiabatic dynamics simulation of polyatomic systems by taking 6-aminopyrimidine as a typical example. The Zhu-Nakamura theory is employed in the surface hopping dynamics, which does not require the calculation of the nonadiabatic coupling vectors. The kernel ridge regression is used in the construction of the adiabatic PESs. In the nonadiabatic dynamics simulation, we use ML-PESs for most geometries and switch back to the electronic structure calculations for a few geometries either near the S₁/S₀ conical intersections or in the out-of-confidence regions. The dynamics results based on ML-PESs are consistent with those based on CASSCF PESs. The ML-PESs are further used to achieve the highly efficient massive dynamics simulations with a large number of trajectories. This work displays the powerful role of ML methods in the nonadiabatic dynamics simulation of polyatomic systems.

A neural network potential energy surface for the F + CH4reaction including multiple channels based on coupled cluster theory

We report here a new global and full dimensional potential energy surface (PES) for the F + CH₄reaction.

Manifold absolute pressure estimation using neural network with hybrid training algorithm

In a modern small gasoline engine fuel injection system, the load of the engine is estimated based on the measurement of the manifold absolute pressure (MAP) sensor, which took place in the intake manifold. This paper present a more economical approach on estimating the MAP by using only the measurements of the throttle position and engine speed, resulting in lower implementation cost. The estimation was done via two-stage multilayer feed-forward neural network by combining Levenberg-Marquardt (LM) algorithm, Bayesian Regularization (BR) algorithm and Particle Swarm Optimization (PSO) algorithm. Based on the results found in 20 runs, the second variant of the hybrid algorithm yields a better network performance than the first variant of hybrid algorithm, LM, LM with BR and PSO by estimating the MAP closely to the simulated MAP values. By using a valid experimental training data, the estimator network that trained with the second variant of the hybrid algorithm showed the best performance among other algorithms when used in an actual retrofit fuel injection system (RFIS). The performance of the estimator was also validated in steady-state and transient condition by showing a closer MAP estimation to the actual value.

Ab Initio Investigation of O-H Dissociation from the Al-OH2 Complex Using Molecular Dynamics and Neural Network Fitting

The dissociation dynamics of the O-H bond in Al-OH2 is investigated on an approximated ab initio potential energy surface (PES). By adopting a dynamic sampling method, we obtain a database of 92 834 configurations. The potential energy for each point is calculated using MP2/6-311G (3df, 2p) calculations; then, a 60-neuron feed-forward neural network is utilized to fit the data to construct an analytic PES. The root-mean-square error (rmse) for the training set is reported as 0.0036 eV, while the rmse for the independent testing set is 0.0034 eV. Such excellent fitting accuracy indeed confirms the reliability of the constructed PES. Subsequently, quasi-classical molecular dynamics (MD) trajectories are performed on the constructed PES at various levels of vibrational excitation in the range of 1.03 to 2.23 eV to investigate the probability of O-H bond dissociation. The results indicate a linear relationship between reaction probability and internal energy, from which we can determine the minimum activation internal energy required for the dissociation as 0.62 eV. Moreover, the O-H bond rupture is shown to be highly correlated with the formation of Al-O bond.