Solving Fokker-Planck equation using deep learning

Deep learning-based state prediction of the Lorenz system with control parameters

Nonlinear dynamical systems with control parameters may not be well modeled by shallow neural networks. In this paper, the stable fixed-point solutions, periodic and chaotic solutions of the parameter-dependent Lorenz system are learned simultaneously via a very deep neural network. The proposed deep learning model consists of a large number of identical linear layers, which provide excellent nonlinear mapping capability. Residual connections are applied to ease the flow of information and a large training dataset is further utilized. Extensive numerical results show that the chaotic solutions can be accurately forecasted for several Lyapunov times and long-term predictions are achieved for periodic solutions. Additionally, the dynamical characteristics such as bifurcation diagrams and largest Lyapunov exponents can be well recovered from the learned solutions. Finally, the principal factors contributing to the high prediction accuracy are discussed.

Solving the non-local Fokker-Planck equations by deep learning

Physics-informed neural networks (PiNNs) recently emerged as a powerful solver for a large class of partial differential equations (PDEs) under various initial and boundary conditions. In this paper, we propose trapz-PiNNs, physics-informed neural networks incorporated with a modified trapezoidal rule recently developed for accurately evaluating fractional Laplacian and solve the space-fractional Fokker-Planck equations in 2D and 3D. We describe the modified trapezoidal rule in detail and verify the second-order accuracy. We demonstrate that trapz-PiNNs have high expressive power through predicting the solution with low L 2 relative error by a variety of numerical examples. We also use local metrics, such as point-wise absolute and relative errors, to analyze where it could be further improved. We present an effective method for improving the performance of trapz-PiNN on local metrics, provided that physical observations or high-fidelity simulation of the true solution are available. The trapz-PiNN is able to solve PDEs with fractional Laplacian with arbitrary α ∈ ( 0 , 2 ) and on rectangular domains. It also has the potential to be generalized into higher dimensions or other bounded domains.

A Free-Space-Based Model for Predicting Peanut Moisture Content during Natural Drying

This study aimed to investigate the water dissipation pattern from peanut pods under natural drying conditions after harvest. The Shandong peanut Luhua 22 was used to examine the effects of varying moisture content, bulk density, and porosity on the relative permittivity of the peanut at a signal frequency of 5.8 GHz. The peanut dielectric constant, porosity, and bulk density were used as inputs and peanut kernel moisture as outputs. Support vector regression (SVR), extreme learning machine (ELM), sparrow search algorithm-support vector regression (SSA-SVR), and sparrow search algorithm-extreme learning machine (SSA-ELM) were used to create a prediction model of peanut kernel moisture content. The results show that the water content of peanut kernels decreased in a fast and then slow manner throughout the drying process and that the water content of kernels was stable at 5–8% at the end of drying. The relative permittivity of peanut kernels increased with an increase in the water content and bulk density but decreased with an increase in porosity. The developed SVR, ELM, SSA-SVR, and SSA-ELM water-content prediction models were validated and analyzed in this study, with the model test set coefficients of determination of 0.936, 0.949,0.984, and 0.994, respectively. In comparison to SVR, ELM, and SSA-SVR, the SSA-ELM root mean square error was reduced by 0.0080, 0.0060, and 0.0012, respectively. According to the findings, the ELM neural network model, which is based on the optimization of the SSA, has an improved prediction accuracy. This prediction model provides a theoretical foundation for the variations in peanut seed moisture content during the natural drying process after harvesting peanuts in Shandong, which will be useful for future peanut storage and transportation.

Complex nonlinear dynamics and vibration suppression of conceptual airfoil models: A state-of-the-art overview

During the past few decades, several significant progresses have been made in exploring complex nonlinear dynamics and vibration suppression of conceptual aeroelastic airfoil models. Additionally, some new challenges have arisen. To the best of the author's knowledge, most studies are concerned with the deterministic case; however, the effects of stochasticity encountered in practical flight environments on the nonlinear dynamical behaviors of the airfoil systems are neglected. Crucially, coupling interaction of the structure nonlinearities and uncertainty fluctuations can lead to some difficulties on the airfoil models, including accurate modeling, response solving, and vibration suppression. At the same time, most of the existing studies depend mainly on a mathematical model established by physical mechanisms. Unfortunately, it is challenging and even impossible to obtain an accurate physical model of the complex wing structure in engineering practice. The emergence of data science and machine learning provides new opportunities for understanding the aeroelastic airfoil systems from the data-driven point of view, such as data-driven modeling, prediction, and control from the recorded data. Nevertheless, relevant data-driven problems of the aeroelastic airfoil systems are not addressed well up to now. This survey contributes to conducting a comprehensive overview of recent developments toward understanding complex dynamical behaviors and vibration suppression, especially for stochastic dynamics, early warning, and data-driven problems, of the conceptual two-dimensional airfoil models with different structural nonlinearities. The results on the airfoil models are summarized and discussed. Besides, several potential development directions that are worth further exploration are also highlighted.

Data driven adaptive Gaussian mixture model for solving Fokker-Planck equation

The Fokker-Planck (FP) equation provides a powerful tool for describing the state transition probability density function of complex dynamical systems governed by stochastic differential equations (SDEs). Unfortunately, the analytical solution of the FP equation can be found in very few special cases. Therefore, it has become an interest to find a numerical approximation method of the FP equation suitable for a wider range of nonlinear systems. In this paper, a machine learning method based on an adaptive Gaussian mixture model (AGMM) is proposed to deal with the general FP equations. Compared with previous numerical discretization methods, the proposed method seamlessly integrates data and mathematical models. The prior knowledge generated by the assumed mathematical model can improve the performance of the learning algorithm. Also, it yields more interpretability for machine learning methods. Numerical examples for one-dimensional and two-dimensional SDEs with one and/or two noises are given. The simulation results show the effectiveness and robustness of the AGMM technique for solving the FP equation. In addition, the computational complexity and the optimization algorithm of the model are also discussed.

Exact Time-Dependent Solutions and Information Geometry of a Rocking Ratchet

The noise-induced transport due to spatial symmetry-breaking is a key mechanism for the generation of a uni-directional motion by a Brownian motor. By utilising an asymmetric sawtooth periodic potential and three different types of periodic forcing G(t) (sinusoidal, square and sawtooth waves) with period T and amplitude A, we investigate the performance (energetics, mean current, Stokes efficiency) of a rocking ratchet in light of thermodynamic quantities (entropy production) and the path-dependent information geometric measures. For each G(t), we calculate exact time-dependent probability density functions under different conditions by varying T, A and the strength of the stochastic noise D in an unprecedentedly wide range. Overall similar behaviours are found for different cases of G(t). In particular, in all cases, the current, Stokes efficiency and the information rate normalised by A and D exhibit one or multiple local maxima and minima as A increases. However, the dependence of the current and Stokes efficiency on A can be quite different, while the behaviour of the information rate normalised by A and D tends to resemble that of the Stokes efficiency. In comparison, the irreversibility measured by a normalised entropy production is independent of A. The results indicate the utility of the information geometry as a proxy of a motor efficiency.

Mitigation of rare events in multistable systems driven by correlated noise

We consider rare transitions induced by colored noise excitation in multistable systems. We show that undesirable transitions can be mitigated by a simple time-delay feedback control if the control parameters are judiciously chosen. We devise a parsimonious method for selecting the optimal control parameters, without requiring any Monte Carlo simulations of the system. This method relies on a new nonlinear Fokker-Planck equation whose stationary response distribution is approximated by a rapidly convergent iterative algorithm. In addition, our framework allows us to accurately predict, and subsequently suppress, the modal drift and tail inflation in the controlled stationary distribution. We demonstrate the efficacy of our method on two examples, including an optical laser model perturbed by multiplicative colored noise.

Acceleration of Global Optimization Algorithm by Detecting Local Extrema Based on Machine Learning

This paper features the study of global optimization problems and numerical methods of their solution. Such problems are computationally expensive since the objective function can be multi-extremal, nondifferentiable, and, as a rule, given in the form of a “black box”. This study used a deterministic algorithm for finding the global extremum. This algorithm is based neither on the concept of multistart, nor nature-inspired algorithms. The article provides computational rules of the one-dimensional algorithm and the nested optimization scheme which could be applied for solving multidimensional problems. Please note that the solution complexity of global optimization problems essentially depends on the presence of multiple local extrema. In this paper, we apply machine learning methods to identify regions of attraction of local minima. The use of local optimization algorithms in the selected regions can significantly accelerate the convergence of global search as it could reduce the number of search trials in the vicinity of local minima. The results of computational experiments carried out on several hundred global optimization problems of different dimensionalities presented in the paper confirm the effect of accelerated convergence (in terms of the number of search trials required to solve a problem with a given accuracy).

Solving the inverse problem of time independent Fokker-Planck equation with a self supervised neural network method

The Fokker–Planck equation (FPE) has been used in many important applications to study stochastic processes with the evolution of the probability density function (pdf). Previous studies on FPE mainly focus on solving the forward problem which is to predict the time-evolution of the pdf from the underlying FPE terms. However, in many applications the FPE terms are usually unknown and roughly estimated, and solving the forward problem becomes more challenging. In this work, we take a different approach of starting with the observed pdfs to recover the FPE terms using a self-supervised machine learning method. This approach, known as the inverse problem, has the advantage of requiring minimal assumptions on the FPE terms and allows data-driven scientific discovery of unknown FPE mechanisms. Specifically, we propose an FPE-based neural network (FPE-NN) which directly incorporates the FPE terms as neural network weights. By training the network on observed pdfs, we recover the FPE terms. Additionally, to account for noise in real-world observations, FPE-NN is able to denoise the observed pdfs by training the pdfs alongside the network weights. Our experimental results on various forms of FPE show that FPE-NN can accurately recover FPE terms and denoising the pdf plays an essential role.

Machine learning framework for computing the most probable paths of stochastic dynamical systems

The emergence of transition phenomena between metastable states induced by noise plays a fundamental role in a broad range of nonlinear systems. The computation of the most probable paths is a key issue to understanding the mechanism of transition behaviors. The shooting method is a common technique for this purpose to solve the Euler-Lagrange equation for the associated action functional, while losing its efficacy in high-dimensional systems. In the present work, we develop a machine learning framework to compute the most probable paths in the sense of Onsager-Machlup action functional theory. Specifically, we reformulate the boundary value problem of a Hamiltonian system and design a neural network to remedy the shortcomings of the shooting method. The successful applications of our algorithms to several prototypical examples demonstrate its efficacy and accuracy for stochastic systems with both (Gaussian) Brownian noise and (non-Gaussian) Lévy noise. This approach is effective in exploring the internal mechanisms of rare events triggered by random fluctuations in various scientific fields.

Neural network representation of the probability density function of diffusion processes

Physics-informed neural networks are developed to characterize the state of dynamical systems in a random environment. The neural network approximates the probability density function (pdf) or the characteristic function (chf) of the state of these systems, which satisfy the Fokker-Planck equation or an integro-differential equation under Gaussian and/or Poisson white noises. We examine analytically and numerically the advantages and disadvantages of solving each type of differential equation to characterize the state. It is also demonstrated how prior information of the dynamical system can be exploited to design and simplify the neural network architecture. Numerical examples show that (1) the neural network solution can approximate the target solution even for partial integro-differential equations and a system of partial differential equations describing the time evolution of the pdf/chf, (2) solving either the Fokker-Planck equation or the chf differential equation using neural networks yields similar pdfs of the state, and (3) the solution to these differential equations can be used to study the behavior of the state for different types of random forcings.