1
|
Spendlove JC, Neilsen TB, Transtrum MK. Information geometry analysis example for absolute and relative transmission loss in a shallow ocean. JASA EXPRESS LETTERS 2024; 4:076001. [PMID: 38949613 DOI: 10.1121/10.0026449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Accepted: 06/10/2024] [Indexed: 07/02/2024]
Abstract
The model manifold, an information geometry tool, is a geometric representation of a model that can quantify the expected information content of modeling parameters. For a normal-mode sound propagation model in a shallow ocean environment, transmission loss (TL) is calculated for a vertical line array and model manifolds are constructed for both absolute and relative TL. For the example presented in this paper, relative TL yields more compact model manifolds with seabed environments that are less statistically distinguishable than manifolds of absolute TL. This example illustrates how model manifolds can be used to improve experimental design for inverse problems.
Collapse
Affiliation(s)
- Jay C Spendlove
- Department of Physics and Astronomy, Brigham Young University, Provo, Utah 84602, , ,
| | - Tracianne B Neilsen
- Department of Physics and Astronomy, Brigham Young University, Provo, Utah 84602, , ,
| | - Mark K Transtrum
- Department of Physics and Astronomy, Brigham Young University, Provo, Utah 84602, , ,
| |
Collapse
|
2
|
Guel-Cortez AJ, Kim EJ, Mehrez MW. Minimum Information Variability in Linear Langevin Systems via Model Predictive Control. ENTROPY (BASEL, SWITZERLAND) 2024; 26:323. [PMID: 38667877 PMCID: PMC11049317 DOI: 10.3390/e26040323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Revised: 04/06/2024] [Accepted: 04/08/2024] [Indexed: 04/28/2024]
Abstract
Controlling the time evolution of a probability distribution that describes the dynamics of a given complex system is a challenging problem. Achieving success in this endeavour will benefit multiple practical scenarios, e.g., controlling mesoscopic systems. Here, we propose a control approach blending the model predictive control technique with insights from information geometry theory. Focusing on linear Langevin systems, we use model predictive control online optimisation capabilities to determine the system inputs that minimise deviations from the geodesic of the information length over time, ensuring dynamics with minimum "geometric information variability". We validate our methodology through numerical experimentation on the Ornstein-Uhlenbeck process and Kramers equation, demonstrating its feasibility. Furthermore, in the context of the Ornstein-Uhlenbeck process, we analyse the impact on the entropy production and entropy rate, providing a physical understanding of the effects of minimum information variability control.
Collapse
Affiliation(s)
| | - Eun-jin Kim
- Centre for Fluid and Complex Systems, Coventry University, Priory St, Coventry CV1 5FB, UK;
| | - Mohamed W. Mehrez
- Zebra Technologies, 2100 Meadowvale Blvd, Mississauga, ON L5N 7J9, Canada;
| |
Collapse
|
3
|
Kim EJ, Thiruthummal AA. Stochastic Dynamics of Fusion Low-to-High Confinement Mode (L-H) Transition: Correlation and Causal Analyses Using Information Geometry. ENTROPY (BASEL, SWITZERLAND) 2023; 26:17. [PMID: 38248143 DOI: 10.3390/e26010017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 12/17/2023] [Accepted: 12/20/2023] [Indexed: 01/23/2024]
Abstract
We investigate the stochastic dynamics of the prey-predator model of the Low-to-High confinement mode (L-H) transition in magnetically confined fusion plasmas. By considering stochastic noise in the turbulence and zonal flows as well as constant and time-varying input power Q, we perform multiple stochastic simulations of over a million trajectories using GPU computing. Due to stochastic noise, some trajectories undergo the L-H transition while others do not, leading to a mixture of H-mode and dithering at a given time and/or input power. One of the consequences of this is that H-mode characteristics appear at a smaller input power QQc as a second peak. The coexisting H-mode and dithering near Q=Qc leads to a prominent bimodal PDF with a gradual L-H transition rather than a sudden transition at Q=Qc and uncertainty in the input power. Also, a time-dependent input power leads to increased variability (dispersion) in stochastic trajectories and a more prominent bimodal PDF. We provide an interpretation of the results using information geometry to elucidate self-regulation between zonal flows, turbulence, and information causality rate to unravel causal relations involved in the L-H transition.
Collapse
Affiliation(s)
- Eun-Jin Kim
- Centre for Fluids and Complex Systems, Coventry University, Coventry CV1 2TT, UK
| | | |
Collapse
|
4
|
Aoki Y, Kazui H, Pascual-Marqui RD, Bruña R, Yoshiyama K, Wada T, Kanemoto H, Suzuki Y, Suehiro T, Satake Y, Yamakawa M, Hata M, Canuet L, Ishii R, Iwase M, Ikeda M. Normalized Power Variance: A new Field Orthogonal to Power in EEG Analysis. Clin EEG Neurosci 2023; 54:611-619. [PMID: 35345930 DOI: 10.1177/15500594221088736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
To date, electroencephalogram (EEG) has been used in the diagnosis of epilepsy, dementia, and disturbance of consciousness via the inspection of EEG waves and identification of abnormal electrical discharges and slowing of basic waves. In addition, EEG power analysis combined with a source estimation method like exact-low-resolution-brain-electromagnetic-tomography (eLORETA), which calculates the power of cortical electrical activity from EEG data, has been widely used to investigate cortical electrical activity in neuropsychiatric diseases. However, the recently developed field of mathematics "information geometry" indicates that EEG has another dimension orthogonal to power dimension - that of normalized power variance (NPV). In addition, by introducing the idea of information geometry, a significantly faster convergent estimator of NPV was obtained. Research into this NPV coordinate has been limited thus far. In this study, we applied this NPV analysis of eLORETA to idiopathic normal pressure hydrocephalus (iNPH) patients prior to a cerebrospinal fluid (CSF) shunt operation, where traditional power analysis could not detect any difference associated with CSF shunt operation outcome. Our NPV analysis of eLORETA detected significantly higher NPV values at the high convexity area in the beta frequency band between 17 shunt responders and 19 non-responders. Considering our present and past research findings about NPV, we also discuss the advantage of this application of NPV representing a sensitive early warning signal of cortical impairment. Overall, our findings demonstrated that EEG has another dimension - that of NPV, which contains a lot of information about cortical electrical activity that can be useful in clinical practice.
Collapse
Affiliation(s)
- Yasunori Aoki
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
- Department of Psychiatry, Nippon Life Hospital, Osaka, Japan
| | - Hiroaki Kazui
- Department of Neuropsychiatry, Kochi Medical School, Kochi University, Kochi, Japan
| | - Roberto D Pascual-Marqui
- The KEY Institute for Brain-Mind Research, University Hospital of Psychiatry, Zurich, Switzerland
| | - Ricardo Bruña
- UCM-UPM Centre for Biomedical Technology, Laboratory of Cognitive and Computational Neuroscience, Complutense University of Madrid, Madrid, Spain
- Department of Electrical Engineering, La Laguna University, Tenerife, Spain
| | - Kenji Yoshiyama
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Tamiki Wada
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Hideki Kanemoto
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Yukiko Suzuki
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Takashi Suehiro
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Yuto Satake
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Maki Yamakawa
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Masahiro Hata
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Leonides Canuet
- Neurology department, Nuestra Senora del Rosario hospital, Madrid, Spain
| | - Ryouhei Ishii
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
- Graduate School of Comprehensive Rehabilitation, Osaka Prefecture University, Osaka, Japan
| | - Masao Iwase
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| | - Manabu Ikeda
- Department of Psychiatry, Graduate School of Medicine, Osaka University, Osaka, Japan
| |
Collapse
|
5
|
Zhang J. Jeffreys Divergence and Generalized Fisher Information Measures on Fokker-Planck Space-Time Random Field. ENTROPY (BASEL, SWITZERLAND) 2023; 25:1445. [PMID: 37895566 PMCID: PMC10606917 DOI: 10.3390/e25101445] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 10/01/2023] [Accepted: 10/11/2023] [Indexed: 10/29/2023]
Abstract
In this paper, we present the derivation of Jeffreys divergence, generalized Fisher divergence, and the corresponding De Bruijn identities for space-time random field. First, we establish the connection between Jeffreys divergence and generalized Fisher information of a single space-time random field with respect to time and space variables. Furthermore, we obtain the Jeffreys divergence between two space-time random fields obtained by different parameters under the same Fokker-Planck equations. Then, the identities between the partial derivatives of the Jeffreys divergence with respect to space-time variables and the generalized Fisher divergence are found, also known as the De Bruijn identities. Later, at the end of the paper, we present three examples of the Fokker-Planck equations on space-time random fields, identify their density functions, and derive the Jeffreys divergence, generalized Fisher information, generalized Fisher divergence, and their corresponding De Bruijn identities.
Collapse
Affiliation(s)
- Jiaxing Zhang
- School of Mathematics, Tianjin University, Tianjin 300350, China
| |
Collapse
|
6
|
Koltuksuz A, Yucel C, Maazu Kademi A. An information geometrical evaluation of Shannon information metrics on a discrete n-dimensional digital manifold. Heliyon 2023; 9:e16653. [PMID: 37484350 PMCID: PMC10360576 DOI: 10.1016/j.heliyon.2023.e16653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Revised: 05/22/2023] [Accepted: 05/23/2023] [Indexed: 07/25/2023] Open
Abstract
The definition and nature of information have perplexed scientists due to its dual nature in measurements. The information is discrete and continuous when evaluated on a metric scale, and the Laplace-Beltrami operator and Gauss-Bonnet Theorem can map one to another. On the other hand, defining the information as a discrete entity on the surface area of an n-dimensional discrete digital manifold provides a unique way of calculating the entropy of a manifold. The software simulation shows that the surface area of the discrete n-dimensional digital manifold is an effectively computable function. Moreover, it also provides the information-geometrical evaluation of Shannon information metrics.
Collapse
Affiliation(s)
- Ahmet Koltuksuz
- Department of Computer Engineering, School of Engineering of Yasar University, Izmir, Turkey
| | - Cagatay Yucel
- Department of Computing and Informatics, Bournemouth University, Fern Barrow, Poole, BH12 5BB, UK
| | - Anas Maazu Kademi
- Department of Computer Engineering, School of Engineering of Yasar University, Izmir, Turkey
| |
Collapse
|
7
|
Fleig P, Balasubramanian V. Playing it safe: information constrains collective betting strategies. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.18.537333. [PMID: 37131671 PMCID: PMC10153189 DOI: 10.1101/2023.04.18.537333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Every interaction of a living organism with its environment involves the placement of a bet. Armed with partial knowledge about a stochastic world, the organism must decide its next step or near-term strategy, an act that implicitly or explicitly involves the assumption of a model of the world. Better information about environmental statistics can improve the bet quality, but in practice resources for information gathering are always limited. We argue that theories of optimal inference dictate that "complex" models are harder to infer with bounded information and lead to larger prediction errors. Thus, we propose a principle of playing it safe where, given finite information gathering capacity, biological systems should be biased towards simpler models of the world, and thereby to less risky betting strategies. In the framework of Bayesian inference, we show that there is an optimally safe adaptation strategy determined by the Bayesian prior. We then demonstrate that, in the context of stochastic phenotypic switching by bacteria, implementation of our principle of "playing it safe" increases fitness (population growth rate) of the bacterial collective. We suggest that the principle applies broadly to problems of adaptation, learning and evolution, and illuminates the types of environments in which organisms are able to thrive.
Collapse
|
8
|
Pessoa P. Information geometry and Bose-Einstein condensation. CHAOS (WOODBURY, N.Y.) 2023; 33:033101. [PMID: 37003840 DOI: 10.1063/5.0136244] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Accepted: 02/03/2023] [Indexed: 06/19/2023]
Abstract
It is a long held conjecture in the connection between information geometry (IG) and thermodynamics that the curvature endowed by IG diverges at phase transitions. Recent work on the IG of Bose-Einstein (BE) gases challenged this conjecture by saying that in the limit of fugacity approaching unit-where BE condensation is expected-the curvature does not diverge; rather, it converges to zero. However, as the discontinuous behavior that identifies condensation is only observed at the thermodynamic limit, a study of the IG curvature at a finite number of particles, N, is in order from which the thermodynamic behavior can be observed by taking the thermodynamic limit ( N→∞) posteriorly. This article presents such a study. We find that for a trapped gas, as N increases, the values of curvature decrease proportionally to a power of N, while the temperature at which the maximum value of curvature occurs approaches the usually defined critical temperature. This means that, in the thermodynamic limit, the curvature has a limited value where a phase transition is observed, contradicting the forementioned conjecture.
Collapse
Affiliation(s)
- Pedro Pessoa
- Physics Department, University at Albany (SUNY), Albany, New York 12222, USA
| |
Collapse
|
9
|
Quinn KN, Abbott MC, Transtrum MK, Machta BB, Sethna JP. Information geometry for multiparameter models: new perspectives on the origin of simplicity. REPORTS ON PROGRESS IN PHYSICS. PHYSICAL SOCIETY (GREAT BRITAIN) 2022; 86:10.1088/1361-6633/aca6f8. [PMID: 36576176 PMCID: PMC10018491 DOI: 10.1088/1361-6633/aca6f8] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 11/29/2022] [Indexed: 05/20/2023]
Abstract
Complex models in physics, biology, economics, and engineering are oftensloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce themodel manifoldof predictions, whose coordinates are the model parameters. Itshyperribbonstructure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold-emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a 'projected maximum likelihood' prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed 'intensive' embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.
Collapse
Affiliation(s)
- Katherine N Quinn
- Center for the Physics of Biological Function, Princeton University, Princeton, NJ, United States of America
| | - Michael C Abbott
- Department of Physics, Yale University, New Haven, CT, United States of America
| | - Mark K Transtrum
- Department of Physics and Astronomy, Brigham Young University, Provo, UT, United States of America
| | - Benjamin B Machta
- Department of Physics and Systems Biology Institute, Yale University, New Haven, CT, United States of America
| | - James P Sethna
- Department of Physics, Cornell University, Ithaca, NY, United States of America
| |
Collapse
|
10
|
Amigó JM, Tempesta P. Information Geometry, Complexity Measures and Data Analysis. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1797. [PMID: 36554202 PMCID: PMC9777868 DOI: 10.3390/e24121797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Accepted: 12/07/2022] [Indexed: 06/17/2023]
Abstract
In the last several years, a new approach to information theory, called information geometry, has emerged [...].
Collapse
Affiliation(s)
- José M. Amigó
- Centro de Investigación Operativa, Universidad Miguel Hernández, 03202 Elche, Spain
| | - Piergiulio Tempesta
- Departamento de Física Teórica, Facultad de Ciencias Físicas, Universidad Complutense, 28040 Madrid, Spain
- Instituto de Ciencias Matemáticas, C/ Nicolás Cabrera, No 13–15, 28049 Madrid, Spain
| |
Collapse
|
11
|
Li W, Rubio F. On a prior based on the Wasserstein information matrix. Stat Probab Lett 2022. [DOI: 10.1016/j.spl.2022.109645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/31/2022]
|
12
|
Information Geometry in Roegenian Economics. ENTROPY 2022; 24:e24070932. [PMID: 35885155 PMCID: PMC9318952 DOI: 10.3390/e24070932] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 07/03/2022] [Accepted: 07/03/2022] [Indexed: 12/04/2022]
Abstract
We characterise the geometry of the statistical Roegenian manifold that arises from the equilibrium distribution of an income of noninteracting identical economic actors. The main results for ideal income are included in three subsections: partition function in distribution, scalar curvature, and geodesics. Although this system displays no phase transition, its analysis provides an enlightening contrast with the results of Van der Waals Income in Roegenian Economics, where we shall examine the geometry of the economic Van der Waals income, which does exhibit a “monetary policy as liquidity—income” transition. Here we focus on three subsections: canonical partition function, economic limit, and information geometry of the economic Van der Waals manifold.
Collapse
|
13
|
A Fisher Information-Based Incompatibility Criterion for Quantum Channels. ENTROPY 2022; 24:e24060805. [PMID: 35741526 PMCID: PMC9222584 DOI: 10.3390/e24060805] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 06/06/2022] [Accepted: 06/07/2022] [Indexed: 02/05/2023]
Abstract
We introduce a new incompatibility criterion for quantum channels based on the notion of (quantum) Fisher information. Our construction is based on a similar criterion for quantum measurements put forward by H. Zhu. We then study the power of the incompatibility criterion in different scenarios. First, we prove the first analytical conditions for the incompatibility of two Schur channels. Then, we study the incompatibility structure of a tuple of depolarizing channels, comparing the newly introduced criterion with the known results from asymmetric quantum cloning.
Collapse
|
14
|
Kontio J, Soñora VR, Pesola V, Lamba R, Dittmann A, Navarro AD, Koivunen J, Pihlajaniemi T, Izzi V. Analysis of extracellular matrix network dynamics in cancer using the MatriNet database. Matrix Biol 2022; 110:141-150. [DOI: 10.1016/j.matbio.2022.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Revised: 04/23/2022] [Accepted: 05/10/2022] [Indexed: 10/18/2022]
|
15
|
Louwerse MD, Sivak DA. Information Thermodynamics of the Transition-Path Ensemble. PHYSICAL REVIEW LETTERS 2022; 128:170602. [PMID: 35570424 DOI: 10.1103/physrevlett.128.170602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Accepted: 03/10/2022] [Indexed: 06/15/2023]
Abstract
The reaction coordinate describing a transition between reactant and product is a fundamental concept in the theory of chemical reactions. Within transition-path theory, a quantitative definition of the reaction coordinate is found in the committor, which is the probability that a trajectory initiated from a given microstate first reaches the product before the reactant. Here we develop an information-theoretic origin for the committor and show how selecting transition paths from a long ergodic equilibrium trajectory induces entropy production which exactly equals the information that system dynamics provide about the reactivity of trajectories. This equality of entropy production and dynamical information generation also holds at the level of arbitrary individual coordinates, providing parallel measures of the coordinate's relevance to the reaction, each of which is maximized by the committor.
Collapse
Affiliation(s)
- Miranda D Louwerse
- Department of Chemistry, Simon Fraser University, Burnaby, British Columbia V5A1S6, Canada
| | - David A Sivak
- Department of Physics, Simon Fraser University, Burnaby, British Columbia V5A1S6, Canada
| |
Collapse
|
16
|
Sharp JA, Browning AP, Burrage K, Simpson MJ. Parameter estimation and uncertainty quantification using information geometry. J R Soc Interface 2022; 19:20210940. [PMID: 35472269 PMCID: PMC9042578 DOI: 10.1098/rsif.2021.0940] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
In this work, we: (i) review likelihood-based inference for parameter estimation and the construction of confidence regions; and (ii) explore the use of techniques from information geometry, including geodesic curves and Riemann scalar curvature, to supplement typical techniques for uncertainty quantification, such as Bayesian methods, profile likelihood, asymptotic analysis and bootstrapping. These techniques from information geometry provide data-independent insights into uncertainty and identifiability, and can be used to inform data collection decisions. All code used in this work to implement the inference and information geometry techniques is available on GitHub.
Collapse
Affiliation(s)
- Jesse A Sharp
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Alexander P Browning
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Kevin Burrage
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,ARC Centre of Excellence for Mathematical and Statistical Frontiers, Queensland University of Technology, Brisbane, Queensland, Australia.,Department of Computer Science, University of Oxford, Oxford, UK
| | - Matthew J Simpson
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Queensland, Australia.,Centre for Data Science, Queensland University of Technology, Brisbane, Queensland, Australia
| |
Collapse
|
17
|
Conformal Control Tools for Statistical Manifolds and for γ-Manifolds. MATHEMATICS 2022. [DOI: 10.3390/math10071061] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/10/2022]
Abstract
The theory of statistical manifolds w.r.t. a conformal structure is reviewed in a creative manner and developed. By analogy, the γ-manifolds are introduced. New conformal invariant tools are defined. A necessary condition for the f-conformal equivalence of γ-manifolds is found, extending that for the α-conformal equivalence for statistical manifolds. Certain examples of these new defined geometrical objects are given in the theory of Iinformation.
Collapse
|
18
|
Ishibashi H, Akaho S. Principal Component Analysis for Gaussian Process Posteriors. Neural Comput 2022; 34:1189-1219. [PMID: 35344991 DOI: 10.1162/neco_a_01489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Accepted: 12/20/2021] [Indexed: 11/04/2022]
Abstract
This letter proposes an extension of principal component analysis for gaussian process (GP) posteriors, denoted by GP-PCA. Since GP-PCA estimates a low-dimensional space of GP posteriors, it can be used for metalearning, a framework for improving the performance of target tasks by estimating a structure of a set of tasks. The issue is how to define a structure of a set of GPs with an infinite-dimensional parameter, such as coordinate system and a divergence. In this study, we reduce the infiniteness of GP to the finite-dimensional case under the information geometrical framework by considering a space of GP posteriors that have the same prior. In addition, we propose an approximation method of GP-PCA based on variational inference and demonstrate the effectiveness of GP-PCA as meta-learning through experiments.
Collapse
Affiliation(s)
| | - Shotaro Akaho
- National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8568, Japan.,RIKEN AIP, Tokyo 103-0027, Japan
| |
Collapse
|
19
|
Nielsen F. Statistical Divergences between Densities of Truncated Exponential Families with Nested Supports: Duo Bregman and Duo Jensen Divergences. ENTROPY 2022; 24:e24030421. [PMID: 35327931 PMCID: PMC8947456 DOI: 10.3390/e24030421] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 03/14/2022] [Accepted: 03/16/2022] [Indexed: 11/28/2022]
Abstract
By calculating the Kullback–Leibler divergence between two probability measures belonging to different exponential families dominated by the same measure, we obtain a formula that generalizes the ordinary Fenchel–Young divergence. Inspired by this formula, we define the duo Fenchel–Young divergence and report a majorization condition on its pair of strictly convex generators, which guarantees that this divergence is always non-negative. The duo Fenchel–Young divergence is also equivalent to a duo Bregman divergence. We show how to use these duo divergences by calculating the Kullback–Leibler divergence between densities of truncated exponential families with nested supports, and report a formula for the Kullback–Leibler divergence between truncated normal distributions. Finally, we prove that the skewed Bhattacharyya distances between truncated exponential families amount to equivalent skewed duo Jensen divergences.
Collapse
Affiliation(s)
- Frank Nielsen
- Sony Computer Science Laboratories, Tokyo 141-0022, Japan
| |
Collapse
|
20
|
|
21
|
Weighted Relative Group Entropies and Associated Fisher Metrics. ENTROPY 2022; 24:e24010120. [PMID: 35052146 PMCID: PMC8775049 DOI: 10.3390/e24010120] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Revised: 01/08/2022] [Accepted: 01/11/2022] [Indexed: 11/17/2022]
Abstract
A large family of new α-weighted group entropy functionals is defined and associated Fisher-like metrics are considered. All these notions are well-suited semi-Riemannian tools for the geometrization of entropy-related statistical models, where they may act as sensitive controlling invariants. The main result of the paper establishes a link between such a metric and a canonical one. A sufficient condition is found, in order that the two metrics be conformal (or homothetic). In particular, we recover a recent result, established for α=1 and for non-weighted relative group entropies. Our conformality condition is “universal”, in the sense that it does not depend on the group exponential.
Collapse
|
22
|
Zhen YZ, Egloff D, Modi K, Dahlsten O. Universal Bound on Energy Cost of Bit Reset in Finite Time. PHYSICAL REVIEW LETTERS 2021; 127:190602. [PMID: 34797137 DOI: 10.1103/physrevlett.127.190602] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Accepted: 09/28/2021] [Indexed: 06/13/2023]
Abstract
We consider how the energy cost of bit reset scales with the time duration of the protocol. Bit reset necessarily takes place in finite time, where there is an extra penalty on top of the quasistatic work cost derived by Landauer. This extra energy is dissipated as heat in the computer, inducing a fundamental limit on the speed of irreversible computers. We formulate a hardware-independent expression for this limit in the framework of stochastic processes. We derive a closed-form lower bound on the work penalty as a function of the time taken for the protocol and bit reset error. It holds for discrete as well as continuous systems, assuming only that the master equation respects detailed balance.
Collapse
Affiliation(s)
- Yi-Zheng Zhen
- Shenzhen Institute for Quantum Science and Engineering and Department of Physics, Southern University of Science and Technology, Shenzhen 518055, China
- Hefei National Laboratory for Physical Sciences at Microscale and Department of Modern Physics, University of Science and Technology of China, Hefei 230026, China
| | - Dario Egloff
- Institute of Theoretical Physics, Technische Universität Dresden, D-01062 Dresden, Germany
- Max Planck Institute for the Physics of Complex Systems, Nöthnitzer Strasse 38, 01187 Dresden, Germany
| | - Kavan Modi
- Shenzhen Institute for Quantum Science and Engineering and Department of Physics, Southern University of Science and Technology, Shenzhen 518055, China
- School of Physics and Astronomy, Monash University, Clayton, Victoria 3800, Australia
| | - Oscar Dahlsten
- Shenzhen Institute for Quantum Science and Engineering and Department of Physics, Southern University of Science and Technology, Shenzhen 518055, China
| |
Collapse
|
23
|
Affiliation(s)
- Nandini Kannan
- Indo‐U.S. Science and Technology Forum New Delhi 110001 India
| | - Debasis Kundu
- Department of Mathematics and Statistics Indian Institute of Technology Kanpur Pin 208016 India
| |
Collapse
|
24
|
Alquier P. Approximate Bayesian Inference. ENTROPY 2020; 22:e22111272. [PMID: 33287041 PMCID: PMC7711853 DOI: 10.3390/e22111272] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Accepted: 11/06/2020] [Indexed: 11/16/2022]
Abstract
This is the Editorial article summarizing the scope of the Special Issue: Approximate Bayesian Inference.
Collapse
Affiliation(s)
- Pierre Alquier
- Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo 103-0027, Japan
| |
Collapse
|