Heterogeneous recurrence monitoring and control of nonlinear stochastic processes

Natural Exponential and Three-Dimensional Chaotic System

Existing chaotic system exhibits unpredictability and nonrepeatability in a deterministic nonlinear architecture, presented as a combination of definiteness and stochasticity. However, traditional two-dimensional chaotic systems cannot provide sufficient information in the dynamic motion and usually feature low sensitivity to initial system input, which makes them computationally prohibitive in accurate time series prediction and weak periodic component detection. Here, a natural exponential and three-dimensional chaotic system with higher sensitivity to initial system input conditions showing astonishing extensibility in time series prediction and image processing is proposed. The chaotic performance evaluated theoretically and experimentally by Poincare mapping, bifurcation diagram, phase space reconstruction, Lyapunov exponent, and correlation dimension provides a new perspective of nonlinear physical modeling and validation. The complexity, robustness, and consistency are studied by recursive and entropy analysis and comparison. The method improves the efficiency of time series prediction, nonlinear dynamics-related problem solving and expands the potential scope of multi-dimensional chaotic systems.

Six-Sigma Quality Management of Additive Manufacturing

Quality is a key determinant in deploying new processes, products, or services and influences the adoption of emerging manufacturing technologies. The advent of additive manufacturing (AM) as a manufacturing process has the potential to revolutionize a host of enterprise-related functions from production to the supply chain. The unprecedented level of design flexibility and expanded functionality offered by AM, coupled with greatly reduced lead times, can potentially pave the way for mass customization. However, widespread application of AM is currently hampered by technical challenges in process repeatability and quality management. The breakthrough effect of six sigma (6S) has been demonstrated in traditional manufacturing industries (e.g., semiconductor and automotive industries) in the context of quality planning, control, and improvement through the intensive use of data, statistics, and optimization. 6S entails a data-driven DMAIC methodology of five steps-define, measure, analyze, improve, and control. Notwithstanding the sustained successes of the 6S knowledge body in a variety of established industries ranging from manufacturing, healthcare, logistics, and beyond, there is a dearth of concentrated application of 6S quality management approaches in the context of AM. In this article, we propose to design, develop, and implement the new DMAIC methodology for the 6S quality management of AM. First, we define the specific quality challenges arising from AM layerwise fabrication and mass customization (even one-of-a-kind production). Second, we present a review of AM metrology and sensing techniques, from materials through design, process, and environment, to postbuild inspection. Third, we contextualize a framework for realizing the full potential of data from AM systems and emphasize the need for analytical methods and tools. We propose and delineate the utility of new data-driven analytical methods, including deep learning, machine learning, and network science, to characterize and model the interrelationships between engineering design, machine setting, process variability, and final build quality. Fourth, we present the methodologies of ontology analytics, design of experiments (DOE), and simulation analysis for AM system improvements. In closing, new process control approaches are discussed to optimize the action plans, once an anomaly is detected, with specific consideration of lead time and energy consumption. We posit that this work will catalyze more in-depth investigations and multidisciplinary research efforts to accelerate the application of 6S quality management in AM.

Recurrence network analysis of design-quality interactions in additive manufacturing

Powder bed fusion (PBF) additive manufacturing (AM) provides a great level of flexibility in the design-driven build of metal products. However, the more complex the design, the more difficult it becomes to control the quality of AM builds. The quality challenge persistently hampers the widespread application of AM technology. Advanced imaging (e.g., X-ray computed tomography scans and high-resolution optical images) has been increasingly explored to enhance the visibility of information and improve the AM quality control. Realizing the full potential of imaging data depends on the advent of information processing methodologies for the analysis of design-quality interactions. This paper presents a design of AM experiment to investigate how design parameters (e.g., build orientation, thin-wall width, thin-wall height, and contour space) interact with quality characteristics in thin-wall builds. Note that the build orientation refers to the position of thin-walls in relation to the recoating direction on the plate, and the contour space indicates the width between rectangle hatches. First, we develop a novel generalized recurrence network (GRN) to represent the AM spatial image data. Then, GRN quantifiers, namely degree, betweenness, pagerank, closeness, and eigenvector centralities, are extracted to characterize the quality of layerwise builds. Further, we establish a regression model to predict how the design complexity impacts GRN behaviors in each layer of thin-wall builds. Experimental results show that network features are sensitive to build orientations, width, height, and contour space under the significant level α = 0.05. Thin-walls with the width bigger than 0.1 mm printed under orientation 0° are found to yield better quality compared to 60° and 90°. Also, thin-walls build with orientation 60° are more sensitive to the changes in contour space compare to the other two orientations. As a result, the orientation 60° should be avoided while printing thin-wall structures. The proposed design-quality analysis shows great potential to optimize engineering design and enhance the quality of PBF-AM builds.

Statistical Analysis of Inter-attribute Relationships in Unfractionated Heparin Injection Problems

Unfractionated heparin (UFH) is commonly used in the intensive care unit (ICU) to prevent blood clotting. Recently, many researchers focus on the development of data- driven methods to solve UFH related problems, which usually involves time series analysis. The performance of data-driven methods depends on whether the inter-correlation of attributes (or variables) in the dataset is closely examined and addressed. This study performs attribute selection, optimal time delay and inter-attributes relations on ICU time series data, in order to provide insights of time series data for UFH related problems. Medical records of 3211 patients with 22 attributes extracted from MIMIC (Medical Information Mart for Intensive Care) III database are used for the experiment. Experimental result shows that some of commonly selected attributes in the literature are less sensitive to the variations of UFH injection. Furthermore, some attributes are inter-dependent, which can increase the complexity of data-driven models, implying that the number of attributes could be reduced. There are 9 attributes found highly related and fast responding in 22 commonly used attributes. This study shows strong potential to provide clinicians with information about sensitive attributes that can help determine the UFH injection policy in ICU.

Intrinsic recurrence quantification analysis of nonlinear and nonstationary short-term time series

Recurrence analysis is a powerful tool to appraise the nonlinear dynamics of complex systems and delineate the inherent laminar, divergent, or transient behaviors. Oftentimes, the effectiveness of recurrence quantification hinges upon the accurate reconstruction of the state space from a univariate time series with a uniform sampling rate. Few, if any, existing approaches quantify the recurrence properties from a short-term time series, particularly those sampled at a non-uniform rate, which are fairly ubiquitous in studies of rare or extreme events. This paper presents a novel intrinsic recurrence quantification analysis to portray the recurrence behaviors in complex dynamical systems with only short-term observations. As opposed to the traditional recurrence analysis, the proposed approach represents recurrence dynamics of a short-term time series in an intrinsic state space formed by proper rotations, attained from intrinsic time-scale decomposition (ITD) of the short time series. It is shown that intrinsic recurrence quantification analysis (iRQA), patterns harnessed from the corresponding recurrence plot, captures the underlying nonlinear and nonstationary dynamics of those short time series. In addition, as ITD does not require uniform sampling of the time series, iRQA is also applicable to unevenly spaced temporal data. Our findings are corroborated in two case studies: change detection in the Lorenz time series and early-stage identification of atrial fibrillation using short-term electrocardiogram signals.

Heterogeneous recurrence analysis of spatial data

Nonlinear dynamical systems often generate significant amounts of observational data such as time series, as well as high-dimensional spatial data. To delineate recurrence dynamics in the spatial data, prior efforts either extended the recurrence plot, which is a widely used tool for time series, to a four-dimensional hyperspace or utilized the network approach for recurrence analysis. However, very little has been done to differentiate heterogeneous types of recurrences in the spatial data (e.g., recurrence variations of state transitions in the spatial domain). Therefore, we propose a novel heterogeneous recurrence approach for spatial data analysis. First, spatial data are traversed with the Hilbert Space-Filling Curve to transform the variations of recurrence patterns from the spatial domain to the state-space domain. Second, we design an Iterated Function System to derive the fractal representation for the state-space trajectory of spatial data. Such a fractal representation effectively captures self-similar behaviors of recurrence variations and multi-state transitions in the spatial data. Third, we develop the Heterogeneous Recurrence Quantification Analysis of spatial data. Experimental results in both simulation and real-world case studies show that the proposed approach yields superior performance in the extraction of salient features to characterize and quantify heterogeneous recurrence dynamics in spatial data.

Heterogeneous Recurrence Analysis of Disease-Altered Spatiotemporal Patterns in Multi-Channel Cardiac Signals

Heart diseases alter the rhythmic behaviors of cardiac electrical activity. Recent advances in sensing technology bring the ease to acquire space-time electrical activity of the heart such as vectorcardiogram (VCG) signals. Recurrence analysis of successive heartbeats is conducive to detect the disease-altered cardiac activities. However, conventional recurrence analysis is more concerned about homogeneous recurrences, and overlook heterogeneous types of recurrence variations in VCG signals (i.e., in terms of state properties and transition dynamics). This paper presents a new framework of heterogeneous recurrence analysis for the characterization and modeling of disease-altered spatiotemporal patterns in multi-channel cardiac signals. Experimental results show that the proposed approach yields an accuracy of 96.9%, a sensitivity of 95.0%, and a specificity of 98.7% for the identification of myocardial infarctions. The proposed method of heterogeneous recurrence analysis shows strong potential to be further extended for the analysis of other physiological signals such as electroencephalogram (EEG) and electromyography (EMG) signals towards medical decision making.

Application of symbolic recurrence to experimental data, from firearm prevalence to fish swimming

Recurrence plots and recurrence quantification analysis are powerful tools to study the behavior of dynamical systems. What we learn through these tools is typically determined by the choice of a distance threshold in the phase space, which introduces arbitrariness in the definition of recurrence. Not only does symbolic recurrence overcome this difficulty, but also it offers a richer representation that book-keeps the recurrent portions of the phase space. Using symbolic recurrences, we can construct recurrence plots, perform quantification analysis, and examine causal links between dynamical systems from their time-series. Although previous efforts have demonstrated the feasibility of such a symbolic framework on synthetic data, the study of real time-series remains elusive. Here, we seek to bridge this gap by systematically examining a wide range of experimental datasets, from firearm prevalence and media coverage in the United States to the effect of sex on the interaction of swimming fish. This work offers a compelling demonstration of the potential of symbolic recurrence in the study of real-world applications across different research fields while providing a computer code for researchers to perform their own time-series explorations.

Recurrence network modeling and analysis of spatial data

Nonlinear dynamical systems exhibit complex recurrence behaviors. Recurrence plot is widely used to graphically represent the patterns of recurrence dynamics and further facilitates the quantification of recurrence patterns, namely, recurrence quantification analysis. However, traditional recurrence methods tend to be limited in their ability to handle spatial data due to high dimensionality and geometric characteristics. Prior efforts have been made to generalize the recurrence plot to a four-dimensional space for spatial data analysis, but this framework can only provide graphical visualization of recurrence patterns in the projected reduced-dimension space (i.e., two- or three- dimensions). In this paper, we propose a new weighted recurrence network approach for spatial data analysis. A weighted network model is introduced to represent the recurrence patterns in spatial data, which account for both pixel intensities and spatial distance simultaneously. Note that each network node represents a location in the high-dimensional spatial data. Network edges and weights preserve complex spatial structures and recurrence patterns. Network representation is shown to be an effective means to provide a complete picture of recurrence patterns in the spatial data. Furthermore, we leverage network statistics to characterize and quantify recurrence properties and features in the spatial data. Experimental results in both simulation and real-world case studies show that the generalized recurrence network approach yields superior performance in the visualization of recurrence patterns in spatial data and in the extraction of salient features to characterize recurrence dynamics in spatial systems.

Investigating Echo-State Networks Dynamics by Means of Recurrence Analysis

In this paper, we elaborate over the well-known interpretability issue in echo-state networks (ESNs). The idea is to investigate the dynamics of reservoir neurons with time-series analysis techniques developed in complex systems research. Notably, we analyze time series of neuron activations with recurrence plots (RPs) and recurrence quantification analysis (RQA), which permit to visualize and characterize high-dimensional dynamical systems. We show that this approach is useful in a number of ways. First, the 2-D representation offered by RPs provides a visualization of the high-dimensional reservoir dynamics. Our results suggest that, if the network is stable, reservoir and input generate similar line patterns in the respective RPs. Conversely, as the ESN becomes unstable, the patterns in the RP of the reservoir change. As a second result, we show that an RQA measure, called , is highly correlated with the well-established maximal local Lyapunov exponent. This suggests that complexity measures based on RP diagonal lines distribution can quantify network stability. Finally, our analysis shows that all RQA measures fluctuate on the proximity of the so-called edge of stability, where an ESN typically achieves maximum computational capability. We leverage on this property to determine the edge of stability and show that our criterion is more accurate than two well-known counterparts, both based on the Jacobian matrix of the reservoir. Therefore, we claim that RPs and RQA-based analyses are valuable tools to design an ESN, given a specific problem.

Heterogeneous recurrence analysis of heartbeat dynamics for the identification of sleep apnea events

Obstructive sleep apnea (OSA) is a common sleep disorder that affects 24% of adult men and 9% of adult women. It occurs due to the occlusion of the upper airway during sleep, thereby leading to a decrease of blood oxygen level that triggers arousals and sleep fragmentation. OSA significantly impacts the quality of sleep and it is known to be responsible for a number of health complications, such as high blood pressure and type 2 diabetes. Traditional diagnosis of OSA relies on polysomnography, which is expensive, time-consuming and inaccessible to the general population. Recent advancement of sensing provides an unprecedented opportunity for the screening of OSA events using single-channel electrocardiogram (ECG). However, existing approaches are limited in their ability to characterize nonlinear dynamics underlying ECG signals. As such, hidden patterns of OSA-altered cardiac electrical activity cannot be fully revealed and understood. This paper presents a new heterogeneous recurrence model to characterize the heart rate variability for the identification of OSA. A nonlinear state space is firstly reconstructed from a time series of RR intervals that are extracted from single-channel ECGs. Further, the state space is recursively partitioned into a hierarchical structure of local recurrence regions. A new fractal representation is designed to efficiently characterize state transitions among segmented sub-regions. Statistical measures are then developed to quantify heterogeneous recurrence patterns. In addition, we integrate classification models with heterogeneous recurrence features to differentiate healthy subjects from OSA patients. Experimental results show that the proposed approach captures heterogeneous recurrence patterns in the transformed space and provides an effective tool to detect OSA using one-lead ECG signals.

Complex network based techniques to identify extreme events and (sudden) transitions in spatio-temporal systems

We present here two promising techniques for the application of the complex network approach to continuous spatio-temporal systems that have been developed in the last decade and show large potential for future application and development of complex systems analysis. First, we discuss the transforming of a time series from such systems to a complex network. The natural approach is to calculate the recurrence matrix and interpret such as the adjacency matrix of an associated complex network, called recurrence network. Using complex network measures, such as transitivity coefficient, we demonstrate that this approach is very efficient for identifying qualitative transitions in observational data, e.g., when analyzing paleoclimate regime transitions. Second, we demonstrate the use of directed spatial networks constructed from spatio-temporal measurements of such systems that can be derived from the synchronized-in-time occurrence of extreme events in different spatial regions. Although there are many possibilities to investigate such spatial networks, we present here the new measure of network divergence and how it can be used to develop a prediction scheme of extreme rainfall events.

Dynamics of on-off neural firing patterns and stochastic effects near a sub-critical Hopf bifurcation

On-off firing patterns, in which repetition of clusters of spikes are interspersed with epochs of subthreshold oscillations or quiescent states, have been observed in various nervous systems, but the dynamics of this event remain unclear. Here, we report that on-off firing patterns observed in three experimental models (rat sciatic nerve subject to chronic constrictive injury, rat CA1 pyramidal neuron, and rabbit blood pressure baroreceptor) appeared as an alternation between quiescent state and burst containing multiple period-1 spikes over time. Burst and quiescent state had various durations. The interspike interval (ISI) series of on-off firing pattern was suggested as stochastic using nonlinear prediction and autocorrelation function. The resting state was changed to a period-1 firing pattern via on-off firing pattern as the potassium concentration, static pressure, or depolarization current was changed. During the changing process, the burst duration of on-off firing pattern increased and the duration of the quiescent state decreased. Bistability of a limit cycle corresponding to period-1 firing and a focus corresponding to resting state was simulated near a sub-critical Hopf bifurcation point in the deterministic Morris-Lecar (ML) model. In the stochastic ML model, noise-induced transitions between the coexisting regimes formed an on-off firing pattern, which closely matched that observed in the experiment. In addition, noise-induced exponential change in the escape rate from the focus, and noise-induced coherence resonance were identified. The distinctions between the on-off firing pattern and stochastic firing patterns generated near three other types of bifurcations of equilibrium points, as well as other viewpoints on the dynamics of on-off firing pattern, are discussed. The results not only identify the on-off firing pattern as noise-induced stochastic firing pattern near a sub-critical Hopf bifurcation point, but also offer practical indicators to discriminate bifurcation types and neural excitability types.

Identification and quantification analysis of nonlinear dynamics properties of combustion instability in a diesel engine

The cycling combustion instabilities in a diesel engine have been analyzed based on chaos theory. The objective was to investigate the dynamical characteristics of combustion in diesel engine. In this study, experiments were performed under the entire operating range of a diesel engine (the engine speed was changed from 600 to 1400 rpm and the engine load rate was from 0% to 100%), and acquired real-time series of in-cylinder combustion pressure using a piezoelectric transducer installed on the cylinder head. Several methods were applied to identify and quantitatively analyze the combustion process complexity in the diesel engine including delay-coordinate embedding, recurrence plot (RP), Recurrence Quantification Analysis, correlation dimension (CD), and the largest Lyapunov exponent (LLE) estimation. The results show that the combustion process exhibits some determinism. If LLE is positive, then the combustion system has a fractal dimension and CD is no more than 1.6 and within the diesel engine operating range. We have concluded that the combustion system of diesel engine is a low-dimensional chaotic system and the maximum values of CD and LLE occur at the lowest engine speed and load. This means that combustion system is more complex and sensitive to initial conditions and that poor combustion quality leads to the decrease of fuel economy and the increase of exhaust emissions.