Data-driven identification of interpretable reduced-order models using sparse regression

Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data

The sparse identification of nonlinear dynamics (SINDy) is a regression framework for the discovery of parsimonious dynamic models and governing equations from time-series data. As with all system identification methods, noisy measurements compromise the accuracy and robustness of the model discovery procedure. In this work we develop a variant of the SINDy algorithm that integrates automatic differentiation and recent time-stepping constrained motivated by Rudy et al (2019 J. Computat. Phys. 396 483–506) for simultaneously (1) denoising the data, (2) learning and parametrizing the noise probability distribution, and (3) identifying the underlying parsimonious dynamical system responsible for generating the time-series data. Thus within an integrated optimization framework, noise can be separated from signal, resulting in an architecture that is approximately twice as robust to noise as state-of-the-art methods, handling as much as 40% noise on a given time-series signal and explicitly parametrizing the noise probability distribution. We demonstrate this approach on several numerical examples, from Lotka-Volterra models to the spatio-temporal Lorenz 96 model. Further, we show the method can learn a diversity of probability distributions for the measurement noise, including Gaussian, uniform, Gamma, and Rayleigh distributions.

Sparse nonlinear models of chaotic electroconvection

Convection is a fundamental fluid transport phenomenon, where the large-scale motion of a fluid is driven, for example, by a thermal gradient or an electric potential. Modelling convection has given rise to the development of chaos theory and the reduced-order modelling of multiphysics systems; however, these models have been limited to relatively simple thermal convection phenomena. In this work, we develop a reduced-order model for chaotic electroconvection at high electric Rayleigh number. The chaos in this system is related to the standard Lorenz model obtained from Rayleigh-Benard convection, although our system is driven by a more complex three-way coupling between the fluid, the charge density, and the electric field. Coherent structures are extracted from temporally and spatially resolved charge density fields via proper orthogonal decomposition (POD). A nonlinear model is then developed for the chaotic time evolution of these coherent structures using the sparse identification of nonlinear dynamics (SINDy) algorithm, constrained to preserve the symmetries observed in the original system. The resulting model exhibits the dominant chaotic dynamics of the original high-dimensional system, capturing the essential nonlinear interactions with a simple reduced-order model.

SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics

Accurately modelling the nonlinear dynamics of a system from measurement data is a challenging yet vital topic. The sparse identification of nonlinear dynamics (SINDy) algorithm is one approach to discover dynamical systems models from data. Although extensions have been developed to identify implicit dynamics, or dynamics described by rational functions, these extensions are extremely sensitive to noise. In this work, we develop SINDy-PI (parallel, implicit), a robust variant of the SINDy algorithm to identify implicit dynamics and rational nonlinearities. The SINDy-PI framework includes multiple optimization algorithms and a principled approach to model selection. We demonstrate the ability of this algorithm to learn implicit ordinary and partial differential equations and conservation laws from limited and noisy data. In particular, we show that the proposed approach is several orders of magnitude more noise robust than previous approaches, and may be used to identify a class of ODE and PDE dynamics that were previously unattainable with SINDy, including for the double pendulum dynamics and simplified model for the Belousov-Zhabotinsky (BZ) reaction.

Regression Tree Model for Predicting Game Scores for the Golden State Warriors in the National Basketball Association

Data mining is becoming increasingly used in sports. Sport data analyses help fans to understand games and teams’ results. Information provided by such analyses is useful for game lovers. Specifically, the information can help fans to predict which team will win a game. Many scholars have devoted attention to predicting the results of various sporting events. In addition to predicting wins and losses, scholars have explored team scores. Most studies on score prediction have used linear regression models to predict the scores of ball games; nevertheless, studies have yet to use regression tree models to predict basketball scores. Therefore, the present study analyzed game data of the Golden State Warriors and their opponents in the 2017–2018 season of the National Basketball Association (NBA). Strong and weak symmetry requirements were identified for each team. We developed a regression tree model for score prediction. After predicting the scores of each player on two teams, we summed and compared the predicted total scores to obtain the predicted results (lose or win) of the team of interest. The results of this study revealed that the regression tree model can effectively predict the score of each player and the total score of the team. The model achieved a predictive accuracy of 87.5%.

Responsive Economic Model Predictive Control for Next-Generation Manufacturing

There is an increasing push to make automated systems capable of carrying out tasks which humans perform, such as driving, speech recognition, and anomaly detection. Automated systems, therefore, are increasingly required to respond to unexpected conditions. Two types of unexpected conditions of relevance in the chemical process industries are anomalous conditions and the responses of operators and engineers to controller behavior. Enhancing responsiveness of an advanced control design known as economic model predictive control (EMPC) (which uses predictions of future process behavior to determine an economically optimal manner in which to operate a process) to unexpected conditions of these types would advance the move toward artificial intelligence properties for this controller beyond those which it has today and would provide new thoughts on interpretability and verification for the controller. This work provides theoretical studies which relate nonlinear systems considerations for EMPC to these higher-level concepts using two ideas for EMPC formulations motivated by specific situations related to self-modification of a control design after human perceptions of the process response are received and to controller handling of anomalies.

Data-Driven Model Reduction for Coupled Flow and Geomechanics Based on DMD Methods

Learning reservoir flow dynamics is of primary importance in creating robust predictive models for reservoir management including hydraulic fracturing processes. Physics-based models are to a certain extent exact, but they entail heavy computational infrastructure for simulating a wide variety of parameters and production scenarios. Reduced-order models offer computational advantages without compromising solution accuracy, especially if they can assimilate large volumes of production data without having to reconstruct the original model (data-driven models). Dynamic mode decomposition (DMD) entails the extraction of relevant spatial structure (modes) based on data (snapshots) that can be used to predict the behavior of reservoir fluid flow in porous media. In this paper, we will further enhance the application of the DMD, by introducing sparse DMD and local DMD. The former is particularly useful when there is a limited number of sparse measurements as in the case of reservoir simulation, and the latter can improve the accuracy of developed DMD models when the process dynamics show a moving boundary behavior like hydraulic fracturing. For demonstration purposes, we first show the methodology applied to (flow only) single- and two-phase reservoir models using the SPE10 benchmark. Both online and offline processes will be used for evaluation. We observe that we only require a few DMD modes, which are determined by the sparse DMD structure, to capture the behavior of the reservoir models. Then, we applied the local DMDc for creating a proxy for application in a hydraulic fracturing process. We also assessed the trade-offs between problem size and computational time for each reservoir model. The novelty of our method is the application of sparse DMD and local DMDc, which is a data-driven technique for fast and accurate simulations.

Model selection for hybrid dynamical systems via sparse regression

Hybrid systems are traditionally difficult to identify and analyse using classical dynamical systems theory. Moreover, recently developed model identification methodologies largely focus on identifying a single set of governing equations solely from measurement data. In this article, we develop a new methodology, Hybrid-Sparse Identification of Nonlinear Dynamics, which identifies separate nonlinear dynamical regimes, employs information theory to manage uncertainty and characterizes switching behaviour. Specifically, we use the nonlinear geometry of data collected from a complex system to construct a set of coordinates based on measurement data and augmented variables. Clustering the data in these measurement-based coordinates enables the identification of nonlinear hybrid systems. This methodology broadly empowers nonlinear system identification without constraining the data locally in time and has direct connections to hybrid systems theory. We demonstrate the success of this method on numerical examples including a mass–spring hopping model and an infectious disease model. Characterizing complex systems that switch between dynamic behaviours is integral to overcoming modern challenges such as eradication of infectious diseases, the design of efficient legged robots and the protection of cyber infrastructures.

A Nonlinear Support Vector Machine-Based Feature Selection Approach for Fault Detection and Diagnosis: Application to the Tennessee Eastman Process

In this article, we present (1) a feature selection algorithm based on nonlinear support vector machine (SVM) for fault detection and diagnosis in continuous processes and (2) results for the Tennessee Eastman benchmark process. The presented feature selection algorithm is derived from the sensitivity analysis of the dual C-SVM objective function. This enables simultaneous modeling and feature selection paving the way for simultaneous fault detection and diagnosis, where feature ranking guides fault diagnosis. We train fault-specific two-class SVM models to detect faulty operations, while using the feature selection algorithm to improve the accuracy and perform the fault diagnosis. Our results show that the developed SVM models outperform the available ones in the literature both in terms of detection accuracy and latency. Moreover, it is shown that the loss of information is minimized with the use of feature selection techniques compared to feature extraction techniques such as principal component analysis (PCA). This further facilitates a more accurate interpretation of the results.

Detecting and Handling Cyber-Attacks in Model Predictive Control of Chemical Processes

Since industrial control systems are usually integrated with numerous physical devices, the security of control systems plays an important role in safe operation of industrial chemical processes. However, due to the use of a large number of control actuators and measurement sensors and the increasing use of wireless communication, control systems are becoming increasingly vulnerable to cyber-attacks, which may spread rapidly and may cause severe industrial incidents. To mitigate the impact of cyber-attacks in chemical processes, this work integrates a neural network (NN)-based detection method and a Lyapunov-based model predictive controller for a class of nonlinear systems. A chemical process example is used to illustrate the application of the proposed NN-based detection and LMPC methods to handle cyber-attacks.