1
|
Chen WJ, Ko IS, Lin CA, Chen CJ, Wu JS, Chan CK. Detection of Anticipatory Dynamics between a Pair of Zebrafish. ENTROPY (BASEL, SWITZERLAND) 2023; 26:13. [PMID: 38275492 PMCID: PMC11154341 DOI: 10.3390/e26010013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 12/15/2023] [Accepted: 12/18/2023] [Indexed: 01/27/2024]
Abstract
Anticipatory dynamics (AD) is unusual in that responses from an information receiver can appear ahead of triggers from the source, and direction of information flow (DIF) is needed to establish causality. Although it is believed that anticipatory dynamics is important for animals' survival, natural examples are rare. Time series (trajectories) from a pair of interacting zebrafish are used to look for the existence of AD in natural systems. In order to obtain the DIF between the two trajectories, we have made use of a special experimental design to designate information source. However, we have also used common statistical tools such as Granger causality and transfer entropy to detect DIF. In our experiments, we found that a majority of the fish pairs do not show any anticipatory behaviors and only a few pairs displayed possible AD. Interestingly, for fish in this latter group, they do not display AD all the time. Our findings suggest that the formation of schooling of fish might not need the help of AD, and new tools are needed in the detection of causality in AD system.
Collapse
Affiliation(s)
| | | | | | | | | | - C. K. Chan
- Institute of Physics, Academia Sinica, Taipei 115, Taiwan; (W.-J.C.); (I.-S.K.); (C.-A.L.); (C.-J.C.); (J.-S.W.)
| |
Collapse
|
2
|
Miwakeichi F, Galka A. Comparison of Bootstrap Methods for Estimating Causality in Linear Dynamic Systems: A Review. ENTROPY (BASEL, SWITZERLAND) 2023; 25:1070. [PMID: 37510017 PMCID: PMC10378223 DOI: 10.3390/e25071070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2023] [Revised: 07/14/2023] [Accepted: 07/16/2023] [Indexed: 07/30/2023]
Abstract
In this study, we present a thorough comparison of the performance of four different bootstrap methods for assessing the significance of causal analysis in time series data. For this purpose, multivariate simulated data are generated by a linear feedback system. The methods investigated are uncorrelated Phase Randomization Bootstrap (uPRB), which generates surrogate data with no cross-correlation between variables by randomizing the phase in the frequency domain; Time Shift Bootstrap (TSB), which generates surrogate data by randomizing the phase in the time domain; Stationary Bootstrap (SB), which calculates standard errors and constructs confidence regions for weakly dependent stationary observations; and AR-Sieve Bootstrap (ARSB), a resampling method based on AutoRegressive (AR) models that approximates the underlying data-generating process. The uPRB method accurately identifies variable interactions but fails to detect self-feedback in some variables. The TSB method, despite performing worse than uPRB, is unable to detect feedback between certain variables. The SB method gives consistent causality results, although its ability to detect self-feedback decreases, as the mean block width increases. The ARSB method shows superior performance, accurately detecting both self-feedback and causality across all variables. Regarding the analysis of the Impulse Response Function (IRF), only the ARSB method succeeds in detecting both self-feedback and causality in all variables, aligning well with the connectivity diagram. Other methods, however, show considerable variations in detection performance, with some detecting false positives and others only detecting self-feedback.
Collapse
Affiliation(s)
- Fumikazu Miwakeichi
- Department of Statistical Modeling, The Institute of Statistical Mathematics, Tokyo 190-8562, Japan
- Statistical Science Program, Graduate Institute for Advanced Studies, SOKENDAI, Tokyo 190-8562, Japan
| | - Andreas Galka
- Clinic for Pediatric and Adolescent Medicine II, University Clinic, University of Kiel, 24105 Kiel, Germany
| |
Collapse
|
3
|
Choong HJ, Kim EJ, He F. Causality Analysis with Information Geometry: A Comparison. ENTROPY (BASEL, SWITZERLAND) 2023; 25:e25050806. [PMID: 37238561 DOI: 10.3390/e25050806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Revised: 05/12/2023] [Accepted: 05/13/2023] [Indexed: 05/28/2023]
Abstract
The quantification of causality is vital for understanding various important phenomena in nature and laboratories, such as brain networks, environmental dynamics, and pathologies. The two most widely used methods for measuring causality are Granger Causality (GC) and Transfer Entropy (TE), which rely on measuring the improvement in the prediction of one process based on the knowledge of another process at an earlier time. However, they have their own limitations, e.g., in applications to nonlinear, non-stationary data, or non-parametric models. In this study, we propose an alternative approach to quantify causality through information geometry that overcomes such limitations. Specifically, based on the information rate that measures the rate of change of the time-dependent distribution, we develop a model-free approach called information rate causality that captures the occurrence of the causality based on the change in the distribution of one process caused by another. This measurement is suitable for analyzing numerically generated non-stationary, nonlinear data. The latter are generated by simulating different types of discrete autoregressive models which contain linear and nonlinear interactions in unidirectional and bidirectional time-series signals. Our results show that information rate causalitycan capture the coupling of both linear and nonlinear data better than GC and TE in the several examples explored in the paper.
Collapse
Affiliation(s)
- Heng Jie Choong
- Centre for Fluid and Complex Systems, Coventry University, Coventry CV1 5FB, UK
| | - Eun-Jin Kim
- Centre for Fluid and Complex Systems, Coventry University, Coventry CV1 5FB, UK
| | - Fei He
- Centre for Computational Science and Mathematical Modelling, Coventry University, Coventry CV1 5FB, UK
| |
Collapse
|
4
|
Varley TF. Flickering Emergences: The Question of Locality in Information-Theoretic Approaches to Emergence. ENTROPY (BASEL, SWITZERLAND) 2022; 25:54. [PMID: 36673195 PMCID: PMC9858457 DOI: 10.3390/e25010054] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 12/08/2022] [Accepted: 12/25/2022] [Indexed: 05/25/2023]
Abstract
"Emergence", the phenomenon where a complex system displays properties, behaviours, or dynamics not trivially reducible to its constituent elements, is one of the defining properties of complex systems. Recently, there has been a concerted effort to formally define emergence using the mathematical framework of information theory, which proposes that emergence can be understood in terms of how the states of wholes and parts collectively disclose information about the system's collective future. In this paper, we show how a common, foundational component of information-theoretic approaches to emergence implies an inherent instability to emergent properties, which we call flickering emergence. A system may, on average, display a meaningful emergent property (be it an informative coarse-graining, or higher-order synergy), but for particular configurations, that emergent property falls apart and becomes misinformative. We show existence proofs that flickering emergence occurs in two different frameworks (one based on coarse-graining and another based on multivariate information decomposition) and argue that any approach based on temporal mutual information will display it. Finally, we argue that flickering emergence should not be a disqualifying property of any model of emergence, but that it should be accounted for when attempting to theorize about how emergence relates to practical models of the natural world.
Collapse
Affiliation(s)
- Thomas F. Varley
- Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN 47405, USA;
- School of Informatics, Computing, & Engineering, Indiana University Bloomington, Bloomington, IN 47405, USA
| |
Collapse
|
5
|
Uthamacumaran A, Zenil H. A Review of Mathematical and Computational Methods in Cancer Dynamics. Front Oncol 2022; 12:850731. [PMID: 35957879 PMCID: PMC9359441 DOI: 10.3389/fonc.2022.850731] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2022] [Accepted: 05/25/2022] [Indexed: 12/16/2022] Open
Abstract
Cancers are complex adaptive diseases regulated by the nonlinear feedback systems between genetic instabilities, environmental signals, cellular protein flows, and gene regulatory networks. Understanding the cybernetics of cancer requires the integration of information dynamics across multidimensional spatiotemporal scales, including genetic, transcriptional, metabolic, proteomic, epigenetic, and multi-cellular networks. However, the time-series analysis of these complex networks remains vastly absent in cancer research. With longitudinal screening and time-series analysis of cellular dynamics, universally observed causal patterns pertaining to dynamical systems, may self-organize in the signaling or gene expression state-space of cancer triggering processes. A class of these patterns, strange attractors, may be mathematical biomarkers of cancer progression. The emergence of intracellular chaos and chaotic cell population dynamics remains a new paradigm in systems medicine. As such, chaotic and complex dynamics are discussed as mathematical hallmarks of cancer cell fate dynamics herein. Given the assumption that time-resolved single-cell datasets are made available, a survey of interdisciplinary tools and algorithms from complexity theory, are hereby reviewed to investigate critical phenomena and chaotic dynamics in cancer ecosystems. To conclude, the perspective cultivates an intuition for computational systems oncology in terms of nonlinear dynamics, information theory, inverse problems, and complexity. We highlight the limitations we see in the area of statistical machine learning but the opportunity at combining it with the symbolic computational power offered by the mathematical tools explored.
Collapse
Affiliation(s)
| | - Hector Zenil
- Machine Learning Group, Department of Chemical Engineering and Biotechnology, The University of Cambridge, Cambridge, United Kingdom
- The Alan Turing Institute, British Library, London, United Kingdom
- Oxford Immune Algorithmics, Reading, United Kingdom
- Algorithmic Dynamics Lab, Karolinska Institute, Stockholm, Sweden
- Algorithmic Nature Group, LABORES, Paris, France
| |
Collapse
|
6
|
Abrahão FS, Zenil H. Emergence and algorithmic information dynamics of systems and observers. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2022; 380:20200429. [PMID: 35599568 PMCID: PMC9125223 DOI: 10.1098/rsta.2020.0429] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
One of the challenges of defining emergence is that one observer's prior knowledge may cause a phenomenon to present itself as emergent that to another observer appears reducible. By formalizing the act of observing as mutual perturbations between dynamical systems, we demonstrate that the emergence of algorithmic information does depend on the observer's formal knowledge, while being robust vis-a-vis other subjective factors, particularly: the choice of programming language and method of measurement; errors or distortions during the observation; and the informational cost of processing. This is called observer-dependent emergence (ODE). In addition, we demonstrate that the unbounded and rapid increase of emergent algorithmic information implies asymptotically observer-independent emergence (AOIE). Unlike ODE, AOIE is a type of emergence for which emergent phenomena will be considered emergent no matter what formal theory an observer might bring to bear. We demonstrate the existence of an evolutionary model that displays the diachronic variant of AOIE and a network model that displays the holistic variant of AOIE. Our results show that, restricted to the context of finite discrete deterministic dynamical systems, computable systems and irreducible information content measures, AOIE is the strongest form of emergence that formal theories can attain. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'.
Collapse
Affiliation(s)
- Felipe S. Abrahão
- National Laboratory for Scientific Computing (LNCC), 25651-075 Petropolis, Rio de Janeiro, Brazil
- Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, 75005 Paris, France
| | - Hector Zenil
- Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, 75005 Paris, France
- Oxford Immune Algorithmics, RG1 3EU Reading, UK
- The Alan Turing Institute, British Library 2QR, 96 Euston Rd, London NW1 2DB, UK
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Karolinska Institutet, 171 77 Stockholm, Sweden
| |
Collapse
|
7
|
Abrahão FS, Zenil H. Emergence and algorithmic information dynamics of systems and observers. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2022. [PMID: 35599568 DOI: 10.6084/m9.figshare.c.5901204] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
One of the challenges of defining emergence is that one observer's prior knowledge may cause a phenomenon to present itself as emergent that to another observer appears reducible. By formalizing the act of observing as mutual perturbations between dynamical systems, we demonstrate that the emergence of algorithmic information does depend on the observer's formal knowledge, while being robust vis-a-vis other subjective factors, particularly: the choice of programming language and method of measurement; errors or distortions during the observation; and the informational cost of processing. This is called observer-dependent emergence (ODE). In addition, we demonstrate that the unbounded and rapid increase of emergent algorithmic information implies asymptotically observer-independent emergence (AOIE). Unlike ODE, AOIE is a type of emergence for which emergent phenomena will be considered emergent no matter what formal theory an observer might bring to bear. We demonstrate the existence of an evolutionary model that displays the diachronic variant of AOIE and a network model that displays the holistic variant of AOIE. Our results show that, restricted to the context of finite discrete deterministic dynamical systems, computable systems and irreducible information content measures, AOIE is the strongest form of emergence that formal theories can attain. This article is part of the theme issue 'Emergent phenomena in complex physical and socio-technical systems: from cells to societies'.
Collapse
Affiliation(s)
- Felipe S Abrahão
- National Laboratory for Scientific Computing (LNCC), 25651-075 Petropolis, Rio de Janeiro, Brazil
- Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, 75005 Paris, France
| | - Hector Zenil
- Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, 75005 Paris, France
- Oxford Immune Algorithmics, RG1 3EU Reading, UK
- The Alan Turing Institute, British Library 2QR, 96 Euston Rd, London NW1 2DB, UK
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Karolinska Institute, 171 77 Stockholm, Sweden
| |
Collapse
|
8
|
Kolmogorov Basic Graphs and Their Application in Network Complexity Analysis. ENTROPY 2021; 23:e23121604. [PMID: 34945910 PMCID: PMC8700381 DOI: 10.3390/e23121604] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 11/25/2021] [Accepted: 11/27/2021] [Indexed: 11/17/2022]
Abstract
Throughout the years, measuring the complexity of networks and graphs has been of great interest to scientists. The Kolmogorov complexity is known as one of the most important tools to measure the complexity of an object. We formalized a method to calculate an upper bound for the Kolmogorov complexity of graphs and networks. Firstly, the most simple graphs possible, those with O(1) Kolmogorov complexity, were identified. These graphs were then used to develop a method to estimate the complexity of a given graph. The proposed method utilizes the simple structures within a graph to capture its non-randomness. This method is able to capture features that make a network closer to the more non-random end of the spectrum. The resulting algorithm takes a graph as an input and outputs an upper bound to its Kolmogorov complexity. This could be applicable in, for example evaluating the performances of graph compression methods.
Collapse
|
9
|
Natal J, Ávila I, Tsukahara VB, Pinheiro M, Maciel CD. Entropy: From Thermodynamics to Information Processing. ENTROPY 2021; 23:e23101340. [PMID: 34682064 PMCID: PMC8534765 DOI: 10.3390/e23101340] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Revised: 09/13/2021] [Accepted: 09/24/2021] [Indexed: 11/18/2022]
Abstract
Entropy is a concept that emerged in the 19th century. It used to be associated with heat harnessed by a thermal machine to perform work during the Industrial Revolution. However, there was an unprecedented scientific revolution in the 20th century due to one of its most essential innovations, i.e., the information theory, which also encompasses the concept of entropy. Therefore, the following question is naturally raised: “what is the difference, if any, between concepts of entropy in each field of knowledge?” There are misconceptions, as there have been multiple attempts to conciliate the entropy of thermodynamics with that of information theory. Entropy is most commonly defined as “disorder”, although it is not a good analogy since “order” is a subjective human concept, and “disorder” cannot always be obtained from entropy. Therefore, this paper presents a historical background on the evolution of the term “entropy”, and provides mathematical evidence and logical arguments regarding its interconnection in various scientific areas, with the objective of providing a theoretical review and reference material for a broad audience.
Collapse
Affiliation(s)
- Jordão Natal
- Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of São Paulo (USP), São Carlos 3566-590, Brazil;
- Correspondence: (J.N.); (C.D.M.)
| | - Ivonete Ávila
- Laboratory of Combustion and Carbon Captur, Department of Energy, School of Engineering, State University of São Paulo (Unesp), São Carlos 3566-590, Brazil;
| | - Victor Batista Tsukahara
- Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of São Paulo (USP), São Carlos 3566-590, Brazil;
| | | | - Carlos Dias Maciel
- Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of São Paulo (USP), São Carlos 3566-590, Brazil;
- Correspondence: (J.N.); (C.D.M.)
| |
Collapse
|
10
|
Algorithmic Information Distortions in Node-Aligned and Node-Unaligned Multidimensional Networks. ENTROPY 2021; 23:e23070835. [PMID: 34210065 PMCID: PMC8304561 DOI: 10.3390/e23070835] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 02/27/2021] [Revised: 05/18/2021] [Accepted: 05/21/2021] [Indexed: 11/18/2022]
Abstract
In this article, we investigate limitations of importing methods based on algorithmic information theory from monoplex networks into multidimensional networks (such as multilayer networks) that have a large number of extra dimensions (i.e., aspects). In the worst-case scenario, it has been previously shown that node-aligned multidimensional networks with non-uniform multidimensional spaces can display exponentially larger algorithmic information (or lossless compressibility) distortions with respect to their isomorphic monoplex networks, so that these distortions grow at least linearly with the number of extra dimensions. In the present article, we demonstrate that node-unaligned multidimensional networks, either with uniform or non-uniform multidimensional spaces, can also display exponentially larger algorithmic information distortions with respect to their isomorphic monoplex networks. However, unlike the node-aligned non-uniform case studied in previous work, these distortions in the node-unaligned case grow at least exponentially with the number of extra dimensions. On the other hand, for node-aligned multidimensional networks with uniform multidimensional spaces, we demonstrate that any distortion can only grow up to a logarithmic order of the number of extra dimensions. Thus, these results establish that isomorphisms between finite multidimensional networks and finite monoplex networks do not preserve algorithmic information in general and highlight that the algorithmic information of the multidimensional space itself needs to be taken into account in multidimensional network complexity analysis.
Collapse
|
11
|
Ayaluri MR, K. SR, Konda SR, Chidirala SR. Efficient steganalysis using convolutional auto encoder network to ensure original image quality. PeerJ Comput Sci 2021; 7:e356. [PMID: 33817006 PMCID: PMC7959617 DOI: 10.7717/peerj-cs.356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Accepted: 12/18/2020] [Indexed: 06/12/2023]
Abstract
Steganalysis is the process of analyzing and predicting the presence of hidden information in images. Steganalysis would be most useful to predict whether the received images contain useful information. However, it is more difficult to predict the hidden information in images which is computationally difficult. In the existing research method, this is resolved by introducing the deep learning approach which attempts to perform steganalysis tasks in effectively. However, this research method does not concentrate the noises present in the images. It might increase the computational overhead where the error cost adjustment would require more iteration. This is resolved in the proposed research technique by introducing the novel research method called Non-Gaussian Noise Aware Auto Encoder Convolutional Neural Network (NGN-AEDNN). Classification technique provides a more flexible way for steganalysis where the multiple features present in the environment would lead to an inaccurate prediction rate. Here, learning accuracy is improved by introducing noise removal techniques before performing a learning task. Non-Gaussian Noise Removal technique is utilized to remove the noises before learning. Also, Gaussian noise removal is applied at every iteration of the neural network to adjust the error rate without the involvement of noisy features. This proposed work can ensure efficient steganalysis by accurate learning task. Matlab has been employed to implement the method by performing simulations from which it is proved that the proposed research technique NGN-AEDNN can ensure the efficient steganalysis outcome with the reduced computational overhead when compared with the existing methods.
Collapse
Affiliation(s)
| | | | - Srinivasa Reddy Konda
- Computer Science and Engineering, BVRIT Hyderabad College of Engineering for Women, Hyderabad, India
| | | |
Collapse
|
12
|
Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S. Multi-label feature selection with constraint regression and adaptive spectral graph. Knowl Based Syst 2021. [DOI: 10.1016/j.knosys.2020.106621] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
13
|
Mutual Information as a General Measure of Structure in Interaction Networks. ENTROPY 2020; 22:e22050528. [PMID: 33286300 PMCID: PMC7517023 DOI: 10.3390/e22050528] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/10/2020] [Revised: 04/03/2020] [Accepted: 04/08/2020] [Indexed: 11/16/2022]
Abstract
Entropy-based indices are long-established measures of biological diversity, nowadays used to gauge partitioning of diversity at different spatial scales. Here, we tackle the measurement of diversity of interactions among two sets of organisms, such as plants and their pollinators. Actual interactions in ecological communities are depicted as bipartite networks or interaction matrices. Recent studies concentrate on distinctive structural patterns, such as nestedness or modularity, found in different modes of interaction. By contrast, we investigate mutual information as a general measure of structure in interactive networks. Mutual information (MI) measures the degree of reciprocal matching or specialization between interacting organisms. To ascertain its usefulness as a general measure, we explore (a) analytical solutions for different models; (b) the response of MI to network parameters, especially size and occupancy; (c) MI in nested, modular, and compound topologies. MI varies with fundamental matrix parameters: dimension and occupancy, for which it can be adjusted or normalized. Apparent differences among topologies are contingent on dimensions and occupancy, rather than on topological patterns themselves. As a general measure of interaction structure, MI is applicable to conceptually and empirically fruitful analyses, such as comparing similar ecological networks along geographical gradients or among interaction modalities in mutualistic or antagonistic networks.
Collapse
|
14
|
Wan P, Chen X, Tu J, Dehmer M, Zhang S, Emmert-Streib F. On graph entropy measures based on the number of independent sets and matchings. Inf Sci (N Y) 2020. [DOI: 10.1016/j.ins.2019.11.020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
15
|
Silva JM, Pinho E, Matos S, Pratas D. Statistical Complexity Analysis of Turing Machine tapes with Fixed Algorithmic Complexity Using the Best-Order Markov Model. ENTROPY (BASEL, SWITZERLAND) 2020; 22:e22010105. [PMID: 33285880 PMCID: PMC7516407 DOI: 10.3390/e22010105] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/11/2019] [Revised: 01/07/2020] [Accepted: 01/13/2020] [Indexed: 06/12/2023]
Abstract
Sources that generate symbolic sequences with algorithmic nature may differ in statistical complexity because they create structures that follow algorithmic schemes, rather than generating symbols from a probabilistic function assuming independence. In the case of Turing machines, this means that machines with the same algorithmic complexity can create tapes with different statistical complexity. In this paper, we use a compression-based approach to measure global and local statistical complexity of specific Turing machine tapes with the same number of states and alphabet. Both measures are estimated using the best-order Markov model. For the global measure, we use the Normalized Compression (NC), while, for the local measures, we define and use normal and dynamic complexity profiles to quantify and localize lower and higher regions of statistical complexity. We assessed the validity of our methodology on synthetic and real genomic data showing that it is tolerant to increasing rates of editions and block permutations. Regarding the analysis of the tapes, we localize patterns of higher statistical complexity in two regions, for a different number of machine states. We show that these patterns are generated by a decrease of the tape's amplitude, given the setting of small rule cycles. Additionally, we performed a comparison with a measure that uses both algorithmic and statistical approaches (BDM) for analysis of the tapes. Naturally, BDM is efficient given the algorithmic nature of the tapes. However, for a higher number of states, BDM is progressively approximated by our methodology. Finally, we provide a simple algorithm to increase the statistical complexity of a Turing machine tape while retaining the same algorithmic complexity. We supply a publicly available implementation of the algorithm in C++ language under the GPLv3 license. All results can be reproduced in full with scripts provided at the repository.
Collapse
Affiliation(s)
- Jorge M. Silva
- Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal; (E.P.); (S.M.); (D.P.)
| | - Eduardo Pinho
- Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal; (E.P.); (S.M.); (D.P.)
| | - Sérgio Matos
- Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal; (E.P.); (S.M.); (D.P.)
- Department of Electronics, Telecommunications and Informatics, University of Aveiro, 3810-193 Aveiro, Portugal
| | - Diogo Pratas
- Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal; (E.P.); (S.M.); (D.P.)
- Department of Electronics, Telecommunications and Informatics, University of Aveiro, 3810-193 Aveiro, Portugal
- Department of Virology, University of Helsinki, 00100 Helsinki, Finland
| |
Collapse
|
16
|
Ponce-Flores M, Frausto-Solís J, Santamaría-Bonfil G, Pérez-Ortega J, González-Barbosa JJ. Time Series Complexities and Their Relationship to Forecasting Performance. ENTROPY 2020; 22:e22010089. [PMID: 33285864 PMCID: PMC7516527 DOI: 10.3390/e22010089] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 01/06/2020] [Accepted: 01/07/2020] [Indexed: 11/16/2022]
Abstract
Entropy is a key concept in the characterization of uncertainty for any given signal, and its extensions such as Spectral Entropy and Permutation Entropy. They have been used to measure the complexity of time series. However, these measures are subject to the discretization employed to study the states of the system, and identifying the relationship between complexity measures and the expected performance of the four selected forecasting methods that participate in the M4 Competition. This relationship allows the decision, in advance, of which algorithm is adequate. Therefore, in this paper, we found the relationships between entropy-based complexity framework and the forecasting error of four selected methods (Smyl, Theta, ARIMA, and ETS). Moreover, we present a framework extension based on the Emergence, Self-Organization, and Complexity paradigm. The experimentation with both synthetic and M4 Competition time series show that the feature space induced by complexities, visually constrains the forecasting method performance to specific regions; where the logarithm of its metric error is poorer, the Complexity based on the emergence and self-organization is maximal.
Collapse
Affiliation(s)
- Mirna Ponce-Flores
- Graduate Program Division, Tecnológico Nacional de México/Instituto Tecnológico de Ciudad Madero, Cd. Madero 89440, Mexico;
- Correspondence: (M.P.-F.); (J.F.-S.); (G.S.-B.)
| | - Juan Frausto-Solís
- Graduate Program Division, Tecnológico Nacional de México/Instituto Tecnológico de Ciudad Madero, Cd. Madero 89440, Mexico;
- Correspondence: (M.P.-F.); (J.F.-S.); (G.S.-B.)
| | - Guillermo Santamaría-Bonfil
- Information Technologies Department, Consejo Nacional de Ciencia y Tecnología—Instituto Nacional de Electricidad y Energías Limpias, Cuernavaca 62490, Mexico
- Correspondence: (M.P.-F.); (J.F.-S.); (G.S.-B.)
| | - Joaquín Pérez-Ortega
- Computing Department, Tecnológico Nacional de México/Centro Nacional de Investigación y Desarrollo Tecnológico, Cuernavaca 62490, Mexico;
| | - Juan J. González-Barbosa
- Graduate Program Division, Tecnológico Nacional de México/Instituto Tecnológico de Ciudad Madero, Cd. Madero 89440, Mexico;
| |
Collapse
|
17
|
Zenil H, Kiani NA, Marabita F, Deng Y, Elias S, Schmidt A, Ball G, Tegnér J. An Algorithmic Information Calculus for Causal Discovery and Reprogramming Systems. iScience 2019; 19:1160-1172. [PMID: 31541920 PMCID: PMC6831824 DOI: 10.1016/j.isci.2019.07.043] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2018] [Revised: 04/27/2019] [Accepted: 07/26/2019] [Indexed: 12/26/2022] Open
Abstract
We introduce and develop a method that demonstrates that the algorithmic information content of a system can be used as a steering handle in the dynamical phase space, thus affording an avenue for controlling and reprogramming systems. The method consists of applying a series of controlled interventions to a networked system while estimating how the algorithmic information content is affected. We demonstrate the method by reconstructing the phase space and their generative rules of some discrete dynamical systems (cellular automata) serving as controlled case studies. Next, the model-based interventional or causal calculus is evaluated and validated using (1) a huge large set of small graphs, (2) a number of larger networks with different topologies, and finally (3) biological networks derived from a widely studied and validated genetic network (E. coli) as well as on a significant number of differentiating (Th17) and differentiated human cells from a curated biological network data.
Collapse
Affiliation(s)
- Hector Zenil
- Algorithmic Dynamics Lab, Center for Molecular Medicine, Karolinska Institutet, Stockholm 171 76, Sweden; Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Oxford Immune Algorithmics, Reading RG1 3EU, UK; Science for Life Laboratory, Solna 171 65, Sweden; Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, Paris 75006, France.
| | - Narsis A Kiani
- Algorithmic Dynamics Lab, Center for Molecular Medicine, Karolinska Institutet, Stockholm 171 76, Sweden; Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden; Algorithmic Nature Group, LABORES for the Natural and Digital Sciences, Paris 75006, France
| | - Francesco Marabita
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden
| | - Yue Deng
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden
| | - Szabolcs Elias
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden
| | - Angelika Schmidt
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden
| | - Gordon Ball
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden
| | - Jesper Tegnér
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine, Karolinska Institutet, Solna, Stockholm 171 76, Sweden; Science for Life Laboratory, Solna 171 65, Sweden; Biological and Environmental Sciences and Engineering Division, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia
| |
Collapse
|
18
|
Zenil H, Kiani NA, Tegnér J. The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy. ENTROPY 2019; 21:e21060560. [PMID: 33267274 PMCID: PMC7515049 DOI: 10.3390/e21060560] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 05/17/2019] [Accepted: 05/20/2019] [Indexed: 12/03/2022]
Abstract
The principle of maximum entropy (Maxent) is often used to obtain prior probability distributions as a method to obtain a Gibbs measure under some restriction giving the probability that a system will be in a certain state compared to the rest of the elements in the distribution. Because classical entropy-based Maxent collapses cases confounding all distinct degrees of randomness and pseudo-randomness, here we take into consideration the generative mechanism of the systems considered in the ensemble to separate objects that may comply with the principle under some restriction and whose entropy is maximal but may be generated recursively from those that are actually algorithmically random offering a refinement to classical Maxent. We take advantage of a causal algorithmic calculus to derive a thermodynamic-like result based on how difficult it is to reprogram a computer code. Using the distinction between computable and algorithmic randomness, we quantify the cost in information loss associated with reprogramming. To illustrate this, we apply the algorithmic refinement to Maxent on graphs and introduce a Maximal Algorithmic Randomness Preferential Attachment (MARPA) Algorithm, a generalisation over previous approaches. We discuss practical implications of evaluation of network randomness. Our analysis provides insight in that the reprogrammability asymmetry appears to originate from a non-monotonic relationship to algorithmic probability. Our analysis motivates further analysis of the origin and consequences of the aforementioned asymmetries, reprogrammability, and computation.
Collapse
Affiliation(s)
- Hector Zenil
- Algorithmic Dynamics Lab, Karolinska Institute, 17177 Stockholm, Sweden
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine Solna, Karolinska Institute, 17177 Stockholm, Sweden
- Algorithmic Nature Group, Laboratory of Scientific Research (LABORES) for the Natural and Digital Sciences, 75006 Paris, France
- Oxford Immune Algorithmics, Oxford University Innovation, Reading RG1 7TT, UK
- Biological and Environmental Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
- Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
- Correspondence:
| | - Narsis A. Kiani
- Algorithmic Dynamics Lab, Karolinska Institute, 17177 Stockholm, Sweden
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine Solna, Karolinska Institute, 17177 Stockholm, Sweden
- Algorithmic Nature Group, Laboratory of Scientific Research (LABORES) for the Natural and Digital Sciences, 75006 Paris, France
| | - Jesper Tegnér
- Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine Solna, Karolinska Institute, 17177 Stockholm, Sweden
- Biological and Environmental Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
- Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
| |
Collapse
|
19
|
Zhang Y, Shang P. The complexity-entropy causality plane based on multiscale power spectrum entropy of financial time series. CHAOS (WOODBURY, N.Y.) 2018; 28:123120. [PMID: 30599536 DOI: 10.1063/1.5054714] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Accepted: 11/26/2018] [Indexed: 06/09/2023]
Abstract
The complexity of financial time series is an important issue for nonlinear dynamic systems. We propose multiscale power spectral entropy. Based on this method, this paper uses the complex entropy causal plane ( C p s e ) to evaluate the complexity of the stock market. Multiscale power spectral entropy takes full advantage of the interrelationships between data in state space and estimates system complexity from different temporal resolutions. Then, we use a complex causal entropy plane to track changes in stock signals. The simulation data are used to test the performance of this method. Finally, we compare the C p s e method with the traditional power spectral entropy method. The results show that the C p s e method is more sensitive to changes in the stock market and can fully extract the intrinsic dynamics of the stock sequence.
Collapse
Affiliation(s)
- Yali Zhang
- Department of Mathematics, School of Science, Beijing Jiaotong University, Beijing 100044, China
| | - Pengjian Shang
- Department of Mathematics, School of Science, Beijing Jiaotong University, Beijing 100044, China
| |
Collapse
|
20
|
Craciunescu T, Murari A, Gelfusa M. Improving Entropy Estimates of Complex Network Topology for the Characterization of Coupling in Dynamical Systems. ENTROPY (BASEL, SWITZERLAND) 2018; 20:e20110891. [PMID: 33266615 PMCID: PMC7512473 DOI: 10.3390/e20110891] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Revised: 11/13/2018] [Accepted: 11/19/2018] [Indexed: 05/25/2023]
Abstract
A new measure for the characterization of interconnected dynamical systems coupling is proposed. The method is based on the representation of time series as weighted cross-visibility networks. The weights are introduced as the metric distance between connected nodes. The structure of the networks, depending on the coupling strength, is quantified via the entropy of the weighted adjacency matrix. The method has been tested on several coupled model systems with different individual properties. The results show that the proposed measure is able to distinguish the degree of coupling of the studied dynamical systems. The original use of the geodesic distance on Gaussian manifolds as a metric distance, which is able to take into account the noise inherently superimposed on the experimental data, provides significantly better results in the calculation of the entropy, improving the reliability of the coupling estimates. The application to the interaction between the El Niño Southern Oscillation (ENSO) and the Indian Ocean Dipole and to the influence of ENSO on influenza pandemic occurrence illustrates the potential of the method for real-life problems.
Collapse
Affiliation(s)
- Teddy Craciunescu
- National Institute for Laser, Plasma and Radiation Physics, RO-077125 Magurele-Bucharest, Romania
| | - Andrea Murari
- Consorzio RFX (CNR, ENEA, INFN, Universita’ di Padova, Acciaierie Venete SpA), 35127 Padova, Italy
- EUROfusion Consortium, JET, Culham Science Centre, Abingdon OX14 3DB, UK
| | - Michela Gelfusa
- Department of Industrial Engineering, University of Rome Tor Vergata, 00133 Rome, Italy
| |
Collapse
|
21
|
Zenil H, Hernández-Orozco S, Kiani NA, Soler-Toscano F, Rueda-Toicen A, Tegnér J. A Decomposition Method for Global Evaluation of Shannon Entropy and Local Estimations of Algorithmic Complexity. ENTROPY 2018; 20:e20080605. [PMID: 33265694 PMCID: PMC7513128 DOI: 10.3390/e20080605] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/28/2018] [Revised: 06/18/2018] [Accepted: 07/31/2018] [Indexed: 12/28/2022]
Abstract
We investigate the properties of a Block Decomposition Method (BDM), which extends the power of a Coding Theorem Method (CTM) that approximates local estimations of algorithmic complexity based on Solomonoff–Levin’s theory of algorithmic probability providing a closer connection to algorithmic complexity than previous attempts based on statistical regularities such as popular lossless compression schemes. The strategy behind BDM is to find small computer programs that produce the components of a larger, decomposed object. The set of short computer programs can then be artfully arranged in sequence so as to produce the original object. We show that the method provides efficient estimations of algorithmic complexity but that it performs like Shannon entropy when it loses accuracy. We estimate errors and study the behaviour of BDM for different boundary conditions, all of which are compared and assessed in detail. The measure may be adapted for use with more multi-dimensional objects than strings, objects such as arrays and tensors. To test the measure we demonstrate the power of CTM on low algorithmic-randomness objects that are assigned maximal entropy (e.g., π) but whose numerical approximations are closer to the theoretical low algorithmic-randomness expectation. We also test the measure on larger objects including dual, isomorphic and cospectral graphs for which we know that algorithmic randomness is low. We also release implementations of the methods in most major programming languages—Wolfram Language (Mathematica), Matlab, R, Perl, Python, Pascal, C++, and Haskell—and an online algorithmic complexity calculator.
Collapse
Affiliation(s)
- Hector Zenil
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institute and SciLifeLab, SE-171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
- Department of Computer Science, University of Oxford, Oxford OX1 3QD, UK
- Correspondence:
| | - Santiago Hernández-Orozco
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institute and SciLifeLab, SE-171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
- Posgrado en Ciencia e Ingeniería de la Computación, Universidad Nacional Autónoma de México (UNAM), Mexico City 04510, Mexico
| | - Narsis A. Kiani
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institute and SciLifeLab, SE-171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
| | | | - Antonio Rueda-Toicen
- Algorithmic Dynamics Lab, Unit of Computational Medicine, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institute and SciLifeLab, SE-171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
- Instituto Nacional de Bioingeniería, Universidad Central de Venezuela, Caracas 1051, Venezuela
| | - Jesper Tegnér
- Unit of Computational Medicine, Department of Medicine Solna, Center for Molecular Medicine, SciLifeLab and Karolinska Institute, Stockholm SE-171 77, Sweden
- Biological and Environmental Sciences and Engineering Division, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
| |
Collapse
|
22
|
Hernández-Orozco S, Kiani NA, Zenil H. Algorithmically probable mutations reproduce aspects of evolution, such as convergence rate, genetic memory and modularity. ROYAL SOCIETY OPEN SCIENCE 2018; 5:180399. [PMID: 30225028 PMCID: PMC6124114 DOI: 10.1098/rsos.180399] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2018] [Accepted: 07/20/2018] [Indexed: 05/07/2023]
Abstract
Natural selection explains how life has evolved over millions of years from more primitive forms. The speed at which this happens, however, has sometimes defied formal explanations when based on random (uniformly distributed) mutations. Here, we investigate the application of a simplicity bias based on a natural but algorithmic distribution of mutations (no recombination) in various examples, particularly binary matrices, in order to compare evolutionary convergence rates. Results both on synthetic and on small biological examples indicate an accelerated rate when mutations are not statistically uniform but algorithmically uniform. We show that algorithmic distributions can evolve modularity and genetic memory by preservation of structures when they first occur sometimes leading to an accelerated production of diversity but also to population extinctions, possibly explaining naturally occurring phenomena such as diversity explosions (e.g. the Cambrian) and massive extinctions (e.g. the End Triassic) whose causes are currently a cause for debate. The natural approach introduced here appears to be a better approximation to biological evolution than models based exclusively upon random uniform mutations, and it also approaches a formal version of open-ended evolution based on previous formal results. These results validate some suggestions in the direction that computation may be an equally important driver of evolution. We also show that inducing the method on problems of optimization, such as genetic algorithms, has the potential to accelerate convergence of artificial evolutionary algorithms.
Collapse
Affiliation(s)
- Santiago Hernández-Orozco
- Posgrado en Ciencia e Ingeniería de la Computación, Universidad Nacional Autónoma de México (UNAM), Mexico
- Algorithmic Dynamics Lab, Unit of Computational Medicine, SciLifeLab, Department of Medicine Solna, Centre for Molecular Medicine, Stockholm, Sweden
- Algorithmic Nature Group, LABORES, Paris, France
| | - Narsis A. Kiani
- Algorithmic Dynamics Lab, Unit of Computational Medicine, SciLifeLab, Department of Medicine Solna, Centre for Molecular Medicine, Stockholm, Sweden
- Algorithmic Nature Group, LABORES, Paris, France
| | - Hector Zenil
- Algorithmic Dynamics Lab, Unit of Computational Medicine, SciLifeLab, Department of Medicine Solna, Centre for Molecular Medicine, Stockholm, Sweden
- Algorithmic Nature Group, LABORES, Paris, France
| |
Collapse
|
23
|
Zenil H, Kiani NA, Tegnér J. A Review of Graph and Network Complexity from an Algorithmic Information Perspective. ENTROPY 2018; 20:e20080551. [PMID: 33265640 PMCID: PMC7513075 DOI: 10.3390/e20080551] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2018] [Revised: 07/18/2018] [Accepted: 07/20/2018] [Indexed: 11/22/2022]
Abstract
Information-theoretic-based measures have been useful in quantifying network complexity. Here we briefly survey and contrast (algorithmic) information-theoretic methods which have been used to characterize graphs and networks. We illustrate the strengths and limitations of Shannon’s entropy, lossless compressibility and algorithmic complexity when used to identify aspects and properties of complex networks. We review the fragility of computable measures on the one hand and the invariant properties of algorithmic measures on the other demonstrating how current approaches to algorithmic complexity are misguided and suffer of similar limitations than traditional statistical approaches such as Shannon entropy. Finally, we review some current definitions of algorithmic complexity which are used in analyzing labelled and unlabelled graphs. This analysis opens up several new opportunities to advance beyond traditional measures.
Collapse
Affiliation(s)
- Hector Zenil
- Algorithmic Dynamics Lab, Centre for Molecular Medicine, Karolinska Institute, 171 77 Stockholm, Sweden
- Unit of Computational Medicine, Department of Medicine, Karolinska Institute, 171 77 Stockholm, Sweden
- Science for Life Laboratory (SciLifeLab), 171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
- Biological and Environmental Sciences and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
- Correspondence: or
| | - Narsis A. Kiani
- Algorithmic Dynamics Lab, Centre for Molecular Medicine, Karolinska Institute, 171 77 Stockholm, Sweden
- Unit of Computational Medicine, Department of Medicine, Karolinska Institute, 171 77 Stockholm, Sweden
- Science for Life Laboratory (SciLifeLab), 171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
| | - Jesper Tegnér
- Unit of Computational Medicine, Department of Medicine, Karolinska Institute, 171 77 Stockholm, Sweden
- Science for Life Laboratory (SciLifeLab), 171 77 Stockholm, Sweden
- Algorithmic Nature Group, Laboratoire de Recherche Scientifique (LABORES) for the Natural and Digital Sciences, 75005 Paris, France
- Biological and Environmental Sciences and Engineering Division (BESE), King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
| |
Collapse
|
24
|
Symmetry and Correspondence of Algorithmic Complexity over Geometric, Spatial and Topological Representations. ENTROPY 2018; 20:e20070534. [PMID: 33265623 PMCID: PMC7513059 DOI: 10.3390/e20070534] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/25/2018] [Revised: 07/12/2018] [Accepted: 07/14/2018] [Indexed: 11/16/2022]
Abstract
We introduce a definition of algorithmic symmetry in the context of geometric and spatial complexity able to capture mathematical aspects of different objects using as a case study polyominoes and polyhedral graphs. We review, study and apply a method for approximating the algorithmic complexity (also known as Kolmogorov–Chaitin complexity) of graphs and networks based on the concept of Algorithmic Probability (AP). AP is a concept (and method) capable of recursively enumerate all properties of computable (causal) nature beyond statistical regularities. We explore the connections of algorithmic complexity—both theoretical and numerical—with geometric properties mainly symmetry and topology from an (algorithmic) information-theoretic perspective. We show that approximations to algorithmic complexity by lossless compression and an Algorithmic Probability-based method can characterize spatial, geometric, symmetric and topological properties of mathematical objects and graphs.
Collapse
|
25
|
Vallverdú J, Castro O, Mayne R, Talanov M, Levin M, Baluška F, Gunji Y, Dussutour A, Zenil H, Adamatzky A. Slime mould: The fundamental mechanisms of biological cognition. Biosystems 2018; 165:57-70. [DOI: 10.1016/j.biosystems.2017.12.011] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2017] [Revised: 12/18/2017] [Accepted: 12/20/2017] [Indexed: 01/27/2023]
|