1
|
Yang L, Lin W, Leng S. Conditional cross-map-based technique: From pairwise dynamical causality to causal network reconstruction. CHAOS (WOODBURY, N.Y.) 2023; 33:2894465. [PMID: 37276551 DOI: 10.1063/5.0144310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Accepted: 05/08/2023] [Indexed: 06/07/2023]
Abstract
Causality detection methods based on mutual cross mapping have been fruitfully developed and applied to data originating from nonlinear dynamical systems, where the causes and effects are non-separable. However, these pairwise methods still have shortcomings in discriminating typical network structures, including common drivers, indirect dependencies, and facing the curse of dimensionality, when they are stepping to causal network reconstruction. A few endeavors have been devoted to conquer these shortcomings. Here, we propose a novel method that could be regarded as one of these endeavors. Our method, named conditional cross-map-based technique, can eliminate third-party information and successfully detect direct dynamical causality, where the detection results can exactly be categorized into four standard normal forms by the designed criterion. To demonstrate the practical usefulness of our model-free, data-driven method, data generated from different representative models covering all kinds of network motifs and measured from real-world systems are investigated. Because correct identification of the direct causal links is essential to successful modeling, predicting, and controlling the underlying complex systems, our method does shed light on uncovering the inner working mechanisms of real-world systems only using the data experimentally obtained in a variety of disciplines.
Collapse
Affiliation(s)
- Liufei Yang
- Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
| | - Wei Lin
- Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
- School of Mathematical Sciences and Shanghai Centre for Mathematical Sciences, Fudan University, Shanghai 200433, China
- Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China
| | - Siyang Leng
- Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
- Institute of AI and Robotics, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China
| |
Collapse
|
2
|
Association Measure and Compact Prediction for Chemical Process Data from an Information-Theoretic Perspective. Processes (Basel) 2022. [DOI: 10.3390/pr10122659] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
Mutual information (MI) has been widely used for association mining in complex chemical processes, but how to precisely estimate MI between variables of different numerical types, discriminate their association relationships with targets and finally achieve compact and interpretable prediction has not been discussed in detail, which may limit MI in more complicated industrial applications. Therefore, this paper first reviews the existing information-based association measures and proposes a general framework, GIEF, to consistently detect associations and independence between different types of variables. Then, the study defines four mutually exclusive association relations of variables from an information-theoretic perspective to guide feature selection and compact prediction in high-dimensional processes. Based on GIEF and conditional mutual information maximization (CMIM), a new algorithm, CMIM-GIEF, is proposed and tested on a fluidized catalytic cracking (FCC) process with 217 variables, one which achieves significantly improved accuracies with fewer variables in predicting the yields of four crucial products. The compact variables identified are also consistent with the results of Shapley Additive exPlanations (SHAP) and industrial experience, proving good adaptivity of the method for chemical process data.
Collapse
|
3
|
Zhang Y, Wang G, Li Z, Xie M, Celler B, Su S, Xu P, Yao D. Matlab Open Source Code: Noise-Assisted Multivariate Empirical Mode Decomposition Based Causal Decomposition for Causality Inference of Bivariate Time Series. Front Neuroinform 2022; 16:851645. [PMID: 35784185 PMCID: PMC9243260 DOI: 10.3389/fninf.2022.851645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 05/04/2022] [Indexed: 11/22/2022] Open
Abstract
Causality inference has arrested much attention in academic studies. Currently, multiple methods such as Granger causality, Convergent Cross Mapping (CCM), and Noise-assisted Multivariate Empirical Mode Decomposition (NA-MEMD) are introduced to solve the problem. Motivated by the researchers who uploaded the open-source code for causality inference, we hereby present the Matlab code of NA-MEMD Causal Decomposition to help users implement the algorithm in multiple scenarios. The code is developed on Matlab2020 and is mainly divided into three subfunctions: na_memd, Plseries, and cd_na_memd. na_memd is called in the main function to generate the matrix of Intrinsic Mode Functions (IMFs) and Plseries can display the average frequency and phase difference of IMFs of the same order in a matrix which can be used for the selection of the main Intrinsic Causal Component (ICC) and ICCs set. cd_na_memd is called to perform causal redecomposition after removing the main ICC from the original time series and output the result of NA-MEMD Causal Decomposition. The performance of the code is evaluated from the perspective of executing time, robustness, and validity. With the data amount enlarging, the executing time increases linearly with it and the value of causal strength oscillates in an ideally small interval which represents the relatively high robustness of the code. The validity is verified based on the open-access predator-prey data (wolf-moose bivariate time series from Isle Royale National Park in Michigan, USA) and our result is aligned with that of Causal Decomposition.
Collapse
Affiliation(s)
- Yi Zhang
- School of Aeronautics and Astronautics, University of Electronic Science and Technology of China, Chengdu, China
- Key Laboratory for Neuro Information of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Center for Information in BioMedicine, University of Electronic Science and Technology of China, Chengdu, China
| | - Guan Wang
- School of Aeronautics and Astronautics, University of Electronic Science and Technology of China, Chengdu, China
| | - Ziwen Li
- Glasgow College, University of Electronic Science and Technology of China, Chengdu, China
| | - Mingjun Xie
- School of Aeronautics and Astronautics, University of Electronic Science and Technology of China, Chengdu, China
| | - Branko Celler
- Biomedical Systems Laboratory, University of New South Wales, Sydney, NSW, Australia
| | - Steven Su
- Center for Health Technologies, Faculty of Engineering and Information Technology, University of Technology, Sydney, NSW, Australia
| | - Peng Xu
- Key Laboratory for Neuro Information of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Center for Information in BioMedicine, University of Electronic Science and Technology of China, Chengdu, China
- *Correspondence: Peng Xu
| | - Dezhong Yao
- Key Laboratory for Neuro Information of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Center for Information in BioMedicine, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
4
|
Papana A. Connectivity Analysis for Multivariate Time Series: Correlation vs. Causality. ENTROPY (BASEL, SWITZERLAND) 2021; 23:1570. [PMID: 34945876 PMCID: PMC8700128 DOI: 10.3390/e23121570] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/24/2021] [Revised: 11/17/2021] [Accepted: 11/24/2021] [Indexed: 12/16/2022]
Abstract
The study of the interdependence relationships of the variables of an examined system is of great importance and remains a challenging task. There are two distinct cases of interdependence. In the first case, the variables evolve in synchrony, connections are undirected and the connectivity is examined based on symmetric measures, such as correlation. In the second case, a variable drives another one and they are connected with a causal relationship. Therefore, directed connections entail the determination of the interrelationships based on causality measures. The main open question that arises is the following: can symmetric correlation measures or directional causality measures be applied to infer the connectivity network of an examined system? Using simulations, we demonstrate the performance of different connectivity measures in case of contemporaneous or/and temporal dependencies. Results suggest the sensitivity of correlation measures when temporal dependencies exist in the data. On the other hand, causality measures do not spuriously indicate causal effects when data present only contemporaneous dependencies. Finally, the necessity of introducing effective instantaneous causality measures is highlighted since they are able to handle both contemporaneous and causal effects at the same time. Results based on instantaneous causality measures are promising; however, further investigation is required in order to achieve an overall satisfactory performance.
Collapse
Affiliation(s)
- Angeliki Papana
- Department of Economics, University of Macedonia, 54636 Thessaloniki, Greece
| |
Collapse
|
5
|
A Multiscale Clustering Approach for Non-IID Nominal Data. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2021; 2021:8993543. [PMID: 34671393 PMCID: PMC8523274 DOI: 10.1155/2021/8993543] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/29/2021] [Revised: 09/13/2021] [Accepted: 09/15/2021] [Indexed: 11/30/2022]
Abstract
Multiscale brings great benefits for people to observe objects or problems from different perspectives. Multiscale clustering has been widely studied in various disciplines. However, most of the research studies are only for the numerical dataset, which is a lack of research on the clustering of nominal dataset, especially the data are nonindependent and identically distributed (Non-IID). Aiming at the current research situation, this paper proposes a multiscale clustering framework based on Non-IID nominal data. Firstly, the benchmark-scale dataset is clustered based on coupled metric similarity measure. Secondly, it is proposed to transform the clustering results from benchmark scale to target scale that the two algorithms are named upscaling based on single chain and downscaling based on Lanczos kernel, respectively. Finally, experiments are performed using five public datasets and one real dataset of the Hebei province of China. The results showed that the method can provide us not only competitive performance but also reduce computational cost.
Collapse
|
6
|
Jia Z, Lin Y, Wang J, Ning X, He Y, Zhou R, Zhou Y, Lehman LWH. Multi-View Spatial-Temporal Graph Convolutional Networks With Domain Generalization for Sleep Stage Classification. IEEE Trans Neural Syst Rehabil Eng 2021; 29:1977-1986. [PMID: 34487495 PMCID: PMC8556658 DOI: 10.1109/tnsre.2021.3110665] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
Sleep stage classification is essential for sleep assessment and disease diagnosis. Although previous attempts to classify sleep stages have achieved high classification performance, several challenges remain open: 1) How to effectively utilize time-varying spatial and temporal features from multi-channel brain signals remains challenging. Prior works have not been able to fully utilize the spatial topological information among brain regions. 2) Due to the many differences found in individual biological signals, how to overcome the differences of subjects and improve the generalization of deep neural networks is important. 3) Most deep learning methods ignore the interpretability of the model to the brain. To address the above challenges, we propose a multi-view spatial-temporal graph convolutional networks (MSTGCN) with domain generalization for sleep stage classification. Specifically, we construct two brain view graphs for MSTGCN based on the functional connectivity and physical distance proximity of the brain regions. The MSTGCN consists of graph convolutions for extracting spatial features and temporal convolutions for capturing the transition rules among sleep stages. In addition, attention mechanism is employed for capturing the most relevant spatial-temporal information for sleep stage classification. Finally, domain generalization and MSTGCN are integrated into a unified framework to extract subject-invariant sleep features. Experiments on two public datasets demonstrate that the proposed model outperforms the state-of-the-art baselines.
Collapse
|
7
|
Shahsavari Baboukani P, Graversen C, Alickovic E, Østergaard J. Estimating Conditional Transfer Entropy in Time Series Using Mutual Information and Nonlinear Prediction. ENTROPY (BASEL, SWITZERLAND) 2020; 22:E1124. [PMID: 33286893 PMCID: PMC7597255 DOI: 10.3390/e22101124] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Revised: 09/22/2020] [Accepted: 09/28/2020] [Indexed: 12/31/2022]
Abstract
We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of new information and improvement of the prediction accuracy provided by the variables. Then, using a greedy approach, the most informative subsets are selected in an iterative way. The algorithm terminates, when the highest ranked variable is not able to significantly improve the accuracy of the prediction as compared to that obtained using the existing selected subsets. In a simulation study, we compare our estimator to existing state-of-the-art methods at different data lengths and directed dependencies strengths. It is demonstrated that the proposed estimator has a significantly higher accuracy than that of existing methods, especially for the difficult case, where the data are highly correlated and coupled. Moreover, we show its false detection of directed dependencies due to instantaneous couplings effect is lower than that of existing measures. We also show applicability of the proposed estimator on real intracranial electroencephalography data.
Collapse
Affiliation(s)
| | - Carina Graversen
- Eriksholm Research Centre, Oticon A/S, 3070 Snekkersten, Denmark; (C.G.); (E.A.)
| | - Emina Alickovic
- Eriksholm Research Centre, Oticon A/S, 3070 Snekkersten, Denmark; (C.G.); (E.A.)
- Department of Electrical Engineering, Linköping University, 581 83 Linköping, Sweden
| | - Jan Østergaard
- Department of Electronic Systems, Aalborg University, 9220 Aalborg, Denmark;
| |
Collapse
|
8
|
A new method for multivariable nonlinear coupling relations analysis in complex electromechanical system. Appl Soft Comput 2020. [DOI: 10.1016/j.asoc.2020.106457] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
9
|
Jia Z, Lin Y, Liu Y, Jiao Z, Wang J. Refined nonuniform embedding for coupling detection in multivariate time series. Phys Rev E 2020; 101:062113. [PMID: 32688603 DOI: 10.1103/physreve.101.062113] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Accepted: 05/13/2020] [Indexed: 11/07/2022]
Abstract
State-space reconstruction is essential to analyze the dynamics and internal interactions of complex systems. However, it is difficult to estimate high-dimensional conditional mutual information and select the optimal time delays in most existing nonuniform state-space reconstruction methods. Therefore, we propose a nonuniform embedding method framed in information theory for state-space reconstruction. We use a low-dimensional approximation of conditional mutual information criterion for time delay selection, which is effectively solved by the particle swarm optimization algorithm. The obtained embedded vector has relatively strong independence and low redundancy, which better characterizes multivariable complex systems and detects coupling within complex systems. In addition, the proposed nonuniform embedding method exhibits good performance in coupling detection of linear stochastic, nonlinear stochastic, chaotic systems. In the actual application, the importance of small airports that cause delay propagation has been demonstrated by constructing the delay propagation network.
Collapse
Affiliation(s)
- Ziyu Jia
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China.,Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China.,Key Laboratory of Intelligent Passenger Service of Civil Aviation, CAAC, Beijing 101318, China
| | - Youfang Lin
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China.,Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China.,Key Laboratory of Intelligent Passenger Service of Civil Aviation, CAAC, Beijing 101318, China
| | - Yunxiao Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China.,Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China.,Key Laboratory of Intelligent Passenger Service of Civil Aviation, CAAC, Beijing 101318, China
| | - Zehui Jiao
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
| | - Jing Wang
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China.,Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing 100044, China.,Key Laboratory of Intelligent Passenger Service of Civil Aviation, CAAC, Beijing 101318, China.,Beijing Laboratory of National Economic Security Early-warning Engineering, Beijing Jiaotong University, Beijing 100044, China
| |
Collapse
|
10
|
Papana A. Non-Uniform Embedding Scheme and Low-Dimensional Approximation Methods for Causality Detection. ENTROPY 2020; 22:e22070745. [PMID: 33286517 PMCID: PMC7517293 DOI: 10.3390/e22070745] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 06/30/2020] [Accepted: 07/02/2020] [Indexed: 11/16/2022]
Abstract
Information causality measures have proven to be very effective in uncovering the connectivity patterns of multivariate systems. The non-uniform embedding (NUE) scheme has been developed to address the “curse of dimensionality”, since the estimation relies on high-dimensional conditional mutual information (CMI) terms. Although the NUE scheme is a dimension reduction technique, the estimation of high-dimensional CMIs is still required. A possible solution is the utilization of low-dimensional approximation (LA) methods for the computation of CMIs. In this study, we aim to provide useful insights regarding the effectiveness of causality measures that rely on NUE and/or on LA methods. In a comparative study, three causality detection methods are evaluated, namely partial transfer entropy (PTE) defined using uniform embedding, PTE using the NUE scheme (PTENUE), and PTE utilizing both NUE and an LA method (LATE). Results from simulations on well known coupled systems suggest the superiority of PTENUE over the other two measures in identifying the true causal effects, having also the least computational cost. The effectiveness of PTENUE is also demonstrated in a real application, where insights are presented regarding the leading forces in financial data.
Collapse
Affiliation(s)
- Angeliki Papana
- Department of Economics, University of Macedonia, 54006 Thessaloniki, Greece
| |
Collapse
|