1
|
Razo-Mejia M, Mani M, Petrov D. Bayesian inference of relative fitness on high-throughput pooled competition assays. PLoS Comput Biol 2024; 20:e1011937. [PMID: 38489348 PMCID: PMC10971673 DOI: 10.1371/journal.pcbi.1011937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 03/27/2024] [Accepted: 02/21/2024] [Indexed: 03/17/2024] Open
Abstract
The tracking of lineage frequencies via DNA barcode sequencing enables the quantification of microbial fitness. However, experimental noise coming from biotic and abiotic sources complicates the computation of a reliable inference. We present a Bayesian pipeline to infer relative microbial fitness from high-throughput lineage tracking assays. Our model accounts for multiple sources of noise and propagates uncertainties throughout all parameters in a systematic way. Furthermore, using modern variational inference methods based on automatic differentiation, we are able to scale the inference to a large number of unique barcodes. We extend this core model to analyze multi-environment assays, replicate experiments, and barcodes linked to genotypes. On simulations, our method recovers known parameters within posterior credible intervals. This work provides a generalizable Bayesian framework to analyze lineage tracking experiments. The accompanying open-source software library enables the adoption of principled statistical methods in experimental evolution.
Collapse
Affiliation(s)
- Manuel Razo-Mejia
- Department of Biology, Stanford University, Stanford, California, United States of America
| | - Madhav Mani
- NSF-Simons Center for Quantitative Biology, Northwestern University, Chicago, Illinois, United States of America
- Department of Engineering Sciences and Applied Mathematics, Northwestern University, Chicago, Illinois, United States of America
| | - Dmitri Petrov
- Department of Biology, Stanford University, Stanford, California, United States of America
- Stanford Cancer Institute, Stanford University School of Medicine, Stanford, California, United States of America
- Chan Zuckerberg Biohub, San Francisco, California, United States of America
| |
Collapse
|
2
|
Razo-Mejia M, Mani M, Petrov D. Bayesian inference of relative fitness on high-throughput pooled competition assays. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.14.562365. [PMID: 37904971 PMCID: PMC10614806 DOI: 10.1101/2023.10.14.562365] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/01/2023]
Abstract
The tracking of lineage frequencies via DNA barcode sequencing enables the quantification of microbial fitness. However, experimental noise coming from biotic and abiotic sources complicates the computation of a reliable inference. We present a Bayesian pipeline to infer relative microbial fitness from high-throughput lineage tracking assays. Our model accounts for multiple sources of noise and propagates uncertainties throughout all parameters in a systematic way. Furthermore, using modern variational inference methods based on automatic differentiation, we are able to scale the inference to a large number of unique barcodes. We extend this core model to analyze multi-environment assays, replicate experiments, and barcodes linked to genotypes. On simulations, our method recovers known parameters within posterior credible intervals. This work provides a generalizable Bayesian framework to analyze lineage tracking experiments. The accompanying open-source software library enables the adoption of principled statistical methods in experimental evolution.
Collapse
Affiliation(s)
| | - Madhav Mani
- NSF-Simons Center for Quantitative Biology, Northwestern University
- Department of Molecular Biosciences, Northwestern University
| | - Dmitri Petrov
- Department of Biology, Stanford University
- Stanford Cancer Institute, Stanford University School of Medicine
- Chan Zuckerberg Biohub
| |
Collapse
|
3
|
He M, Das P, Hotan G, Purdon PL. Switching state-space modeling of neural signal dynamics. PLoS Comput Biol 2023; 19:e1011395. [PMID: 37639391 PMCID: PMC10491408 DOI: 10.1371/journal.pcbi.1011395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 09/08/2023] [Accepted: 07/28/2023] [Indexed: 08/31/2023] Open
Abstract
Linear parametric state-space models are a ubiquitous tool for analyzing neural time series data, providing a way to characterize the underlying brain dynamics with much greater statistical efficiency than non-parametric data analysis approaches. However, neural time series data are frequently time-varying, exhibiting rapid changes in dynamics, with transient activity that is often the key feature of interest in the data. Stationary methods can be adapted to time-varying scenarios by employing fixed-duration windows under an assumption of quasi-stationarity. But time-varying dynamics can be explicitly modeled by switching state-space models, i.e., by using a pool of state-space models with different dynamics selected by a probabilistic switching process. Unfortunately, exact solutions for state inference and parameter learning with switching state-space models are intractable. Here we revisit a switching state-space model inference approach first proposed by Ghahramani and Hinton. We provide explicit derivations for solving the inference problem iteratively after applying a variational approximation on the joint posterior of the hidden states and the switching process. We introduce a novel initialization procedure using an efficient leave-one-out strategy to compare among candidate models, which significantly improves performance compared to the existing method that relies on deterministic annealing. We then utilize this state inference solution within a generalized expectation-maximization algorithm to estimate model parameters of the switching process and the linear state-space models with dynamics potentially shared among candidate models. We perform extensive simulations under different settings to benchmark performance against existing switching inference methods and further validate the robustness of our switching inference solution outside the generative switching model class. Finally, we demonstrate the utility of our method for sleep spindle detection in real recordings, showing how switching state-space models can be used to detect and extract transient spindles from human sleep electroencephalograms in an unsupervised manner.
Collapse
Affiliation(s)
- Mingjian He
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard-MIT Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Proloy Das
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
- Department of Anesthesia, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California, United States of America
| | - Gladia Hotan
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore
| | - Patrick L. Purdon
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, United States of America
- Harvard Medical School, Boston, Massachusetts, United States of America
- Department of Anesthesia, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California, United States of America
| |
Collapse
|
4
|
Jeon I, Kim T. Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network. Front Comput Neurosci 2023; 17:1092185. [PMID: 37449083 PMCID: PMC10336230 DOI: 10.3389/fncom.2023.1092185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 06/12/2023] [Indexed: 07/18/2023] Open
Abstract
Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.
Collapse
Affiliation(s)
| | - Taegon Kim
- Brain Science Institute, Korea Institute of Science and Technology, Seoul, Republic of Korea
| |
Collapse
|
5
|
Konaka Y, Naoki H. Decoding reward-curiosity conflict in decision-making from irrational behaviors. NATURE COMPUTATIONAL SCIENCE 2023; 3:418-432. [PMID: 38177842 PMCID: PMC10768639 DOI: 10.1038/s43588-023-00439-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 03/29/2023] [Indexed: 01/06/2024]
Abstract
Humans and animals are not always rational. They not only rationally exploit rewards but also explore an environment owing to their curiosity. However, the mechanism of such curiosity-driven irrational behavior is largely unknown. Here, we developed a decision-making model for a two-choice task based on the free energy principle, which is a theory integrating recognition and action selection. The model describes irrational behaviors depending on the curiosity level. We also proposed a machine learning method to decode temporal curiosity from behavioral data. By applying it to rat behavioral data, we found that the rat had negative curiosity, reflecting conservative selection sticking to more certain options and that the level of curiosity was upregulated by the expected future information obtained from an uncertain environment. Our decoding approach can be a fundamental tool for identifying the neural basis for reward-curiosity conflicts. Furthermore, it could be effective in diagnosing mental disorders.
Collapse
Affiliation(s)
- Yuki Konaka
- Laboratory of Data-Driven Biology, Graduate School of Integrated Sciences for Life, Hiroshima University, Hiroshima, Japan
| | - Honda Naoki
- Laboratory of Data-Driven Biology, Graduate School of Integrated Sciences for Life, Hiroshima University, Hiroshima, Japan.
- Kansei-Brain Informatics Group, Center for Brain, Mind and Kansei Sciences Research, Hiroshima University, Hiroshima, Japan.
- Theoretical Biology Research Group, Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences, Okazaki, Japan.
- Laboratory of Theoretical Biology, Graduate School of Biostudies, Kyoto University, Kyoto, Japan.
| |
Collapse
|
6
|
Prabhushankar M, AlRegib G. Stochastic surprisal: An inferential measurement of free energy in neural networks. Front Neurosci 2023; 17:926418. [PMID: 36998731 PMCID: PMC10043257 DOI: 10.3389/fnins.2023.926418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 02/09/2023] [Indexed: 03/16/2023] Open
Abstract
This paper conjectures and validates a framework that allows for action during inference in supervised neural networks. Supervised neural networks are constructed with the objective to maximize their performance metric in any given task. This is done by reducing free energy and its associated surprisal during training. However, the bottom-up inference nature of supervised networks is a passive process that renders them fallible to noise. In this paper, we provide a thorough background of supervised neural networks, both generative and discriminative, and discuss their functionality from the perspective of free energy principle. We then provide a framework for introducing action during inference. We introduce a new measurement called stochastic surprisal that is a function of the network, the input, and any possible action. This action can be any one of the outputs that the neural network has learnt, thereby lending stochasticity to the measurement. Stochastic surprisal is validated on two applications: Image Quality Assessment and Recognition under noisy conditions. We show that, while noise characteristics are ignored to make robust recognition, they are analyzed to estimate image quality scores. We apply stochastic surprisal on two applications, three datasets, and as a plug-in on 12 networks. In all, it provides a statistically significant increase among all measures. We conclude by discussing the implications of the proposed stochastic surprisal in other areas of cognitive psychology including expectancy-mismatch and abductive reasoning.
Collapse
Affiliation(s)
- Mohit Prabhushankar
- Omni Lab for Intelligent Visual Engineering and Science (OLIVES), Georgia Institute of Technology, Electrical and Computer Engineering, Atlanta, GA, United States
| | | |
Collapse
|
7
|
Clark KB. Neural Field Continuum Limits and the Structure–Function Partitioning of Cognitive–Emotional Brain Networks. BIOLOGY 2023; 12:biology12030352. [PMID: 36979044 PMCID: PMC10045557 DOI: 10.3390/biology12030352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/07/2023] [Accepted: 02/13/2023] [Indexed: 02/25/2023]
Abstract
In The cognitive-emotional brain, Pessoa overlooks continuum effects on nonlinear brain network connectivity by eschewing neural field theories and physiologically derived constructs representative of neuronal plasticity. The absence of this content, which is so very important for understanding the dynamic structure-function embedding and partitioning of brains, diminishes the rich competitive and cooperative nature of neural networks and trivializes Pessoa’s arguments, and similar arguments by other authors, on the phylogenetic and operational significance of an optimally integrated brain filled with variable-strength neural connections. Riemannian neuromanifolds, containing limit-imposing metaplastic Hebbian- and antiHebbian-type control variables, simulate scalable network behavior that is difficult to capture from the simpler graph-theoretic analysis preferred by Pessoa and other neuroscientists. Field theories suggest the partitioning and performance benefits of embedded cognitive-emotional networks that optimally evolve between exotic classical and quantum computational phases, where matrix singularities and condensations produce degenerate structure-function homogeneities unrealistic of healthy brains. Some network partitioning, as opposed to unconstrained embeddedness, is thus required for effective execution of cognitive-emotional network functions and, in our new era of neuroscience, should be considered a critical aspect of proper brain organization and operation.
Collapse
Affiliation(s)
- Kevin B. Clark
- Cures Within Reach, Chicago, IL 60602, USA;
- Felidae Conservation Fund, Mill Valley, CA 94941, USA
- Campus and Domain Champions Program, Multi-Tier Assistance, Training, and Computational Help (MATCH) Track, National Science Foundation’s Advanced Cyberinfrastructure Coordination Ecosystem: Services and Support (ACCESS), https://access-ci.org/
- Expert Network, Penn Center for Innovation, University of Pennsylvania, Philadelphia, PA 19104, USA
- Network for Life Detection (NfoLD), NASA Astrobiology Program, NASA Ames Research Center, Mountain View, CA 94035, USA
- Multi-Omics and Systems Biology & Artificial Intelligence and Machine Learning Analysis Working Groups, NASA GeneLab, NASA Ames Research Center, Mountain View, CA 94035, USA
- Frontier Development Lab, NASA Ames Research Center, Mountain View, CA 94035, USA & SETI Institute, Mountain View, CA 94043, USA
- Peace Innovation Institute, The Hague 2511, Netherlands & Stanford University, Palo Alto, CA 94305, USA
- Shared Interest Group for Natural and Artificial Intelligence (sigNAI), Max Planck Alumni Association, 14057 Berlin, Germany
- Biometrics and Nanotechnology Councils, Institute for Electrical and Electronics Engineers (IEEE), New York, NY 10016, USA
| |
Collapse
|
8
|
Thermodynamic fluctuation theorems govern human sensorimotor learning. Sci Rep 2023; 13:869. [PMID: 36650215 PMCID: PMC9845310 DOI: 10.1038/s41598-023-27736-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Accepted: 01/06/2023] [Indexed: 01/18/2023] Open
Abstract
The application of thermodynamic reasoning in the study of learning systems has a long tradition. Recently, new tools relating perfect thermodynamic adaptation to the adaptation process have been developed. These results, known as fluctuation theorems, have been tested experimentally in several physical scenarios and, moreover, they have been shown to be valid under broad mathematical conditions. Hence, although not experimentally challenged yet, they are presumed to apply to learning systems as well. Here we address this challenge by testing the applicability of fluctuation theorems in learning systems, more specifically, in human sensorimotor learning. In particular, we relate adaptive movement trajectories in a changing visuomotor rotation task to fully adapted steady-state behavior of individual participants. We find that human adaptive behavior in our task is generally consistent with fluctuation theorem predictions and discuss the merits and limitations of the approach.
Collapse
|
9
|
Hack P, Gottwald S, Braun DA. Jarzyski's Equality and Crooks' Fluctuation Theorem for General Markov Chains with Application to Decision-Making Systems. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1731. [PMID: 36554136 PMCID: PMC9777588 DOI: 10.3390/e24121731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Revised: 11/19/2022] [Accepted: 11/24/2022] [Indexed: 06/17/2023]
Abstract
We define common thermodynamic concepts purely within the framework of general Markov chains and derive Jarzynski's equality and Crooks' fluctuation theorem in this setup. In particular, we regard the discrete-time case, which leads to an asymmetry in the definition of work that appears in the usual formulation of Crooks' fluctuation theorem. We show how this asymmetry can be avoided with an additional condition regarding the energy protocol. The general formulation in terms of Markov chains allows transferring the results to other application areas outside of physics. Here, we discuss how this framework can be applied in the context of decision-making. This involves the definition of the relevant quantities, the assumptions that need to be made for the different fluctuation theorems to hold, as well as the consideration of discrete trajectories instead of the continuous trajectories, which are relevant in physics.
Collapse
|
10
|
Ilan Y. The constrained disorder principle defines living organisms and provides a method for correcting disturbed biological systems. Comput Struct Biotechnol J 2022; 20:6087-6096. [DOI: 10.1016/j.csbj.2022.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 10/26/2022] [Accepted: 11/06/2022] [Indexed: 11/11/2022] Open
|
11
|
Schach S, Lindner A, Braun DA. Bounded rational decision-making models suggest capacity-limited concurrent motor planning in human posterior parietal and frontal cortex. PLoS Comput Biol 2022; 18:e1010585. [PMID: 36227842 PMCID: PMC9560147 DOI: 10.1371/journal.pcbi.1010585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 09/18/2022] [Indexed: 11/05/2022] Open
Abstract
While traditional theories of sensorimotor processing have often assumed a serial decision-making pipeline, more recent approaches have suggested that multiple actions may be planned concurrently and vie for execution. Evidence for the latter almost exclusively stems from electrophysiological studies in posterior parietal and premotor cortex of monkeys. Here we study concurrent prospective motor planning in humans by recording functional magnetic resonance imaging (fMRI) during a delayed response task engaging movement sequences towards multiple potential targets. We find that also in human posterior parietal and premotor cortex delay activity modulates both with sequence complexity and the number of potential targets. We tested the hypothesis that this modulation is best explained by concurrent prospective planning as opposed to the mere maintenance of potential targets in memory. We devise a bounded rationality model with information constraints that optimally assigns information resources for planning and memory for this task and determine predicted information profiles according to the two hypotheses. When regressing delay activity on these model predictions, we find that the concurrent prospective planning strategy provides a significantly better explanation of the fMRI-signal modulations. Moreover, we find that concurrent prospective planning is more costly and thus limited for most subjects, as expressed by the best fitting information capacities. We conclude that bounded rational decision-making models allow relating both behavior and neural representations to utilitarian task descriptions based on bounded optimal information-processing assumptions. When the future is uncertain, it can be beneficial to concurrently plan several action possibilities in advance. Electrophysiological research found evidence in monkeys that brain regions in posterior parietal and promotor cortex are indeed capable of planning several actions in parallel. We now used fMRI to study brain activity in these brain regions in humans. For our analyses we applied bounded rationality models that optimally assign information resources to fMRI activity in a complex motor planning task. We find that theoretical information costs of concurrent prospective planning explained fMRI activity profiles significantly better than assuming alternative memory-based strategies. Moreover, exploiting the model allowed us to quantify the individual capacity limit for concurrent planning and to relate these individual limits to both subjects’ behavior and to their neural representations of planning.
Collapse
Affiliation(s)
- Sonja Schach
- Institute of Neural Information Processing, University of Ulm, Ulm, Germany
- * E-mail:
| | - Axel Lindner
- Tübingen Center for Mental Health, Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
- Centre of Neurology, Division of Neuropsychology, Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | | |
Collapse
|
12
|
Hack P, Braun DA, Gottwald S. Representing preorders with injective monotones. THEORY AND DECISION 2022; 93:663-690. [PMID: 36245967 PMCID: PMC9550776 DOI: 10.1007/s11238-021-09861-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 12/13/2021] [Indexed: 06/16/2023]
Abstract
We introduce a new class of real-valued monotones in preordered spaces, injective monotones. We show that the class of preorders for which they exist lies in between the class of preorders with strict monotones and preorders with countable multi-utilities, improving upon the known classification of preordered spaces through real-valued monotones. We extend several well-known results for strict monotones (Richter-Peleg functions) to injective monotones, we provide a construction of injective monotones from countable multi-utilities, and relate injective monotones to classic results concerning Debreu denseness and order separability. Along the way, we connect our results to Shannon entropy and the uncertainty preorder, obtaining new insights into how they are related. In particular, we show how injective monotones can be used to generalize some appealing properties of Jaynes' maximum entropy principle, which is considered a basis for statistical inference and serves as a justification for many regularization techniques that appear throughout machine learning and decision theory.
Collapse
Affiliation(s)
- Pedro Hack
- Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany
| | - Daniel A. Braun
- Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany
| | - Sebastian Gottwald
- Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany
| |
Collapse
|
13
|
Yufik Y, Malhotra R. Situational Understanding in the Human and the Machine. Front Syst Neurosci 2021; 15:786252. [PMID: 35002643 PMCID: PMC8733725 DOI: 10.3389/fnsys.2021.786252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 11/24/2021] [Indexed: 11/13/2022] Open
Abstract
The Air Force research programs envision developing AI technologies that will ensure battlespace dominance, by radical increases in the speed of battlespace understanding and decision-making. In the last half century, advances in AI have been concentrated in the area of machine learning. Recent experimental findings and insights in systems neuroscience, the biophysics of cognition, and other disciplines provide converging results that set the stage for technologies of machine understanding and machine-augmented Situational Understanding. This paper will review some of the key ideas and results in the literature, and outline new suggestions. We define situational understanding and the distinctions between understanding and awareness, consider examples of how understanding-or lack of it-manifest in performance, and review hypotheses concerning the underlying neuronal mechanisms. Suggestions for further R&D are motivated by these hypotheses and are centered on the notions of Active Inference and Virtual Associative Networks.
Collapse
Affiliation(s)
- Yan Yufik
- Virtual Structures Research, Inc., Potomac, MD, United States
| | - Raj Malhotra
- United States Air Force Sensor Directorate, Dayton, OH, United States
| |
Collapse
|
14
|
Lindig-León C, Schmid G, Braun DA. Bounded rational response equilibria in human sensorimotor interactions. Proc Biol Sci 2021; 288:20212094. [PMID: 34727714 PMCID: PMC8564607 DOI: 10.1098/rspb.2021.2094] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
The Nash equilibrium is one of the most central solution concepts to study strategic interactions between multiple players and has recently also been shown to capture sensorimotor interactions between players that are haptically coupled. While previous studies in behavioural economics have shown that systematic deviations from Nash equilibria in economic decision-making can be explained by the more general quantal response equilibria, such deviations have not been reported for the sensorimotor domain. Here we investigate haptically coupled dyads across three different sensorimotor games corresponding to the classic symmetric and asymmetric Prisoner's Dilemma, where the quantal response equilibrium predicts characteristic shifts across the three games, although the Nash equilibrium stays the same. We find that subjects exhibit the predicted deviations from the Nash solution. Furthermore, we show that taking into account subjects' priors for the games, we arrive at a more accurate description of bounded rational response equilibria that can be regarded as a quantal response equilibrium with non-uniform prior. Our results suggest that bounded rational response equilibria provide a general tool to explain sensorimotor interactions that include the Nash equilibrium as a special case in the absence of information processing limitations.
Collapse
Affiliation(s)
- Cecilia Lindig-León
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Germany
| | - Gerrit Schmid
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Germany
| | - Daniel A Braun
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Germany
| |
Collapse
|
15
|
Lindig-León C, Schmid G, Braun DA. Nash equilibria in human sensorimotor interactions explained by Q-learning with intrinsic costs. Sci Rep 2021; 11:20779. [PMID: 34675336 PMCID: PMC8531365 DOI: 10.1038/s41598-021-99428-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Accepted: 09/01/2021] [Indexed: 11/09/2022] Open
Abstract
The Nash equilibrium concept has previously been shown to be an important tool to understand human sensorimotor interactions, where different actors vie for minimizing their respective effort while engaging in a multi-agent motor task. However, it is not clear how such equilibria are reached. Here, we compare different reinforcement learning models to human behavior engaged in sensorimotor interactions with haptic feedback based on three classic games, including the prisoner's dilemma, and the symmetric and asymmetric matching pennies games. We find that a discrete analysis that reduces the continuous sensorimotor interaction to binary choices as in classical matrix games does not allow to distinguish between the different learning algorithms, but that a more detailed continuous analysis with continuous formulations of the learning algorithms and the game-theoretic solutions affords different predictions. In particular, we find that Q-learning with intrinsic costs that disfavor deviations from average behavior explains the observed data best, even though all learning algorithms equally converge to admissible Nash equilibrium solutions. We therefore conclude that it is important to study different learning algorithms for understanding sensorimotor interactions, as such behavior cannot be inferred from a game-theoretic analysis alone, that simply focuses on the Nash equilibrium concept, as different learning algorithms impose preferences on the set of possible equilibrium solutions due to the inherent learning dynamics.
Collapse
Affiliation(s)
- Cecilia Lindig-León
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Ulm, Germany.
| | - Gerrit Schmid
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Ulm, Germany
| | - Daniel A Braun
- Institute of Neural Information Processing, Faculty of Engineering, Computer Science and Psychology, Ulm University, Ulm, Germany
| |
Collapse
|
16
|
Haken H, Portugali J. Information and Self-Organization II: Steady State and Phase Transition. ENTROPY 2021; 23:e23060707. [PMID: 34199648 PMCID: PMC8229616 DOI: 10.3390/e23060707] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Revised: 05/29/2021] [Accepted: 05/29/2021] [Indexed: 01/22/2023]
Abstract
This paper starts from Schrödinger’s famous question “what is life” and elucidates answers that invoke, in particular, Friston’s free energy principle and its relation to the method of Bayesian inference and to Synergetics 2nd foundation that utilizes Jaynes’ maximum entropy principle. Our presentation reflects the shift from the emphasis on physical principles to principles of information theory and Synergetics. In view of the expected general audience of this issue, we have chosen a somewhat tutorial style that does not require special knowledge on physics but familiarizes the reader with concepts rooted in information theory and Synergetics.
Collapse
Affiliation(s)
- Hermann Haken
- Center of Synergetics, Institute for Theoretical Physics, Stuttgart University, 70550 Stuttgart, Germany;
| | - Juval Portugali
- Department of Geography and the Human Environment, The Raymond and Beverly Sackler Faculty of Exact Sciences, School of Geosciences, Tel Aviv University, Tel Aviv 69978, Israel
- Correspondence:
| |
Collapse
|
17
|
A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback. ENTROPY 2021; 23:e23060669. [PMID: 34073330 PMCID: PMC8227139 DOI: 10.3390/e23060669] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Revised: 05/20/2021] [Accepted: 05/21/2021] [Indexed: 11/24/2022]
Abstract
Bounded rationality is an important consideration stemming from the fact that agents often have limits on their processing abilities, making the assumption of perfect rationality inapplicable to many real tasks. We propose an information-theoretic approach to the inference of agent decisions under Smithian competition. The model explicitly captures the boundedness of agents (limited in their information-processing capacity) as the cost of information acquisition for expanding their prior beliefs. The expansion is measured as the Kullblack–Leibler divergence between posterior decisions and prior beliefs. When information acquisition is free, the homo economicus agent is recovered, while in cases when information acquisition becomes costly, agents instead revert to their prior beliefs. The maximum entropy principle is used to infer least biased decisions based upon the notion of Smithian competition formalised within the Quantal Response Statistical Equilibrium framework. The incorporation of prior beliefs into such a framework allowed us to systematically explore the effects of prior beliefs on decision-making in the presence of market feedback, as well as importantly adding a temporal interpretation to the framework. We verified the proposed model using Australian housing market data, showing how the incorporation of prior knowledge alters the resulting agent decisions. Specifically, it allowed for the separation of past beliefs and utility maximisation behaviour of the agent as well as the analysis into the evolution of agent beliefs.
Collapse
|