1
|
Charlton JA, Młynarski WF, Bai YH, Hermundstad AM, Goris RLT. Environmental dynamics shape perceptual decision bias. PLoS Comput Biol 2023; 19:e1011104. [PMID: 37289753 DOI: 10.1371/journal.pcbi.1011104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 04/13/2023] [Indexed: 06/10/2023] Open
Abstract
To interpret the sensory environment, the brain combines ambiguous sensory measurements with knowledge that reflects context-specific prior experience. But environmental contexts can change abruptly and unpredictably, resulting in uncertainty about the current context. Here we address two questions: how should context-specific prior knowledge optimally guide the interpretation of sensory stimuli in changing environments, and do human decision-making strategies resemble this optimum? We probe these questions with a task in which subjects report the orientation of ambiguous visual stimuli that were drawn from three dynamically switching distributions, representing different environmental contexts. We derive predictions for an ideal Bayesian observer that leverages knowledge about the statistical structure of the task to maximize decision accuracy, including knowledge about the dynamics of the environment. We show that its decisions are biased by the dynamically changing task context. The magnitude of this decision bias depends on the observer's continually evolving belief about the current context. The model therefore not only predicts that decision bias will grow as the context is indicated more reliably, but also as the stability of the environment increases, and as the number of trials since the last context switch grows. Analysis of human choice data validates all three predictions, suggesting that the brain leverages knowledge of the statistical structure of environmental change when interpreting ambiguous sensory signals.
Collapse
Affiliation(s)
- Julie A Charlton
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas, United States of America
| | | | - Yoon H Bai
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas, United States of America
| | - Ann M Hermundstad
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, United States of America
| | - Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, Texas, United States of America
| |
Collapse
|
2
|
Abstract
Inferring hidden structure from noisy observations is a problem addressed by Bayesian statistical learning, which aims to identify optimal models of the process that generated the observations given assumptions that constrain the space of potential solutions. Animals and machines face similar "model-selection" problems to infer latent properties and predict future states of the world. Here we review recent attempts to explain how intelligent agents address these challenges and how their solutions relate to Bayesian principles. We focus on how constraints on available information and resources affect inference and propose a general framework that uses benefit(accuracy) and accuracy(cost) curves to assess optimality under these constraints.
Collapse
Affiliation(s)
- Gaia Tavoni
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104.,Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA 19104
| | - Vijay Balasubramanian
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104.,Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA 19104
| | - Joshua I Gold
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA 19104
| |
Collapse
|
3
|
Harun R, Jun E, Park HH, Ganupuru P, Goldring AB, Hanks TD. Timescales of Evidence Evaluation for Decision Making and Associated Confidence Judgments Are Adapted to Task Demands. Front Neurosci 2020; 14:826. [PMID: 32903672 PMCID: PMC7438826 DOI: 10.3389/fnins.2020.00826] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2020] [Accepted: 07/15/2020] [Indexed: 01/29/2023] Open
Abstract
Decision making often involves choosing actions based on relevant evidence. This can benefit from focussing evidence evaluation on the timescale of greatest relevance based on the situation. Here, we use an auditory change detection task to determine how people adjust their timescale of evidence evaluation depending on task demands for detecting changes in their environment and assessing their internal confidence in those decisions. We confirm previous results that people adopt shorter timescales of evidence evaluation for detecting changes in contexts with shorter signal durations, while bolstering those results with model-free analyses not previously used and extending the results to the auditory domain. We also extend these results to show that in contexts with shorter signal durations, people also adopt correspondingly shorter timescales of evidence evaluation for assessing confidence in their decision about detecting a change. These results provide important insights into adaptability and flexible control of evidence evaluation for decision making.
Collapse
Affiliation(s)
- Rashed Harun
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| | - Elizabeth Jun
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| | - Heui Hye Park
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| | - Preetham Ganupuru
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| | - Adam B Goldring
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| | - Timothy D Hanks
- Department of Neurology and Center for Neuroscience, University of California, Davis, Davis, CA, United States
| |
Collapse
|
4
|
Pasturel C, Montagnini A, Perrinet LU. Humans adapt their anticipatory eye movements to the volatility of visual motion properties. PLoS Comput Biol 2020; 16:e1007438. [PMID: 32282790 PMCID: PMC7179935 DOI: 10.1371/journal.pcbi.1007438] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2019] [Revised: 04/23/2020] [Accepted: 02/27/2020] [Indexed: 12/20/2022] Open
Abstract
Animal behavior constantly adapts to changes, for example when the statistical properties of the environment change unexpectedly. For an agent that interacts with this volatile setting, it is important to react accurately and as quickly as possible. It has already been shown that when a random sequence of motion ramps of a visual target is biased to one direction (e.g. right or left), human observers adapt their eye movements to accurately anticipate the target's expected direction. Here, we prove that this ability extends to a volatile environment where the probability bias could change at random switching times. In addition, we also recorded the explicit prediction of the next outcome as reported by observers using a rating scale. Both results were compared to the estimates of a probabilistic agent that is optimal in relation to the assumed generative model. Compared to the classical leaky integrator model, we found a better match between our probabilistic agent and the behavioral responses, both for the anticipatory eye movements and the explicit task. Furthermore, by controlling the level of preference between exploitation and exploration in the model, we were able to fit for each individual's experimental dataset the most likely level of volatility and analyze inter-individual variability across participants. These results prove that in such an unstable environment, human observers can still represent an internal belief about the environmental contingencies, and use this representation both for sensory-motor control and for explicit judgments. This work offers an innovative approach to more generically test the diversity of human cognitive abilities in uncertain and dynamic environments.
Collapse
Affiliation(s)
- Chloé Pasturel
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille Univ, CNRS, Marseille, France
| | - Anna Montagnini
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille Univ, CNRS, Marseille, France
| | - Laurent Udo Perrinet
- Institut de Neurosciences de la Timone (UMR 7289), Aix Marseille Univ, CNRS, Marseille, France
- * E-mail:
| |
Collapse
|
5
|
FitzGerald THB, Penny WD, Bonnici HM, Adams RA. Retrospective Inference as a Form of Bounded Rationality, and Its Beneficial Influence on Learning. Front Artif Intell 2020; 3:2. [PMID: 33733122 PMCID: PMC7861256 DOI: 10.3389/frai.2020.00002] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2019] [Accepted: 01/14/2020] [Indexed: 12/22/2022] Open
Abstract
Probabilistic models of cognition typically assume that agents make inferences about current states by combining new sensory information with fixed beliefs about the past, an approach known as Bayesian filtering. This is computationally parsimonious, but, in general, leads to suboptimal beliefs about past states, since it ignores the fact that new observations typically contain information about the past as well as the present. This is disadvantageous both because knowledge of past states may be intrinsically valuable, and because it impairs learning about fixed or slowly changing parameters of the environment. For these reasons, in offline data analysis it is usual to infer on every set of states using the entire time series of observations, an approach known as (fixed-interval) Bayesian smoothing. Unfortunately, however, this is impractical for real agents, since it requires the maintenance and updating of beliefs about an ever-growing set of states. We propose an intermediate approach, finite retrospective inference (FRI), in which agents perform update beliefs about a limited number of past states (Formally, this represents online fixed-lag smoothing with a sliding window). This can be seen as a form of bounded rationality in which agents seek to optimize the accuracy of their beliefs subject to computational and other resource costs. We show through simulation that this approach has the capacity to significantly increase the accuracy of both inference and learning, using a simple variational scheme applied to both randomly generated Hidden Markov models (HMMs), and a specific application of the HMM, in the form of the widely used probabilistic reversal task. Our proposal thus constitutes a theoretical contribution to normative accounts of bounded rationality, which makes testable empirical predictions that can be explored in future work.
Collapse
Affiliation(s)
- Thomas H B FitzGerald
- School of Psychology, University of East Anglia, Norwich, United Kingdom.,The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom.,Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London, United Kingdom
| | - Will D Penny
- School of Psychology, University of East Anglia, Norwich, United Kingdom.,The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| | - Heidi M Bonnici
- School of Psychology, University of East Anglia, Norwich, United Kingdom
| | - Rick A Adams
- The Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom.,Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London, United Kingdom.,Department of Computer Science, University College London, London, United Kingdom
| |
Collapse
|
6
|
Karamched B, Stolarczyk S, Kilpatrick ZP, Josić K. Bayesian Evidence Accumulation on Social Networks. SIAM JOURNAL ON APPLIED DYNAMICAL SYSTEMS 2020; 19:1884-1919. [PMID: 36051948 PMCID: PMC9432802 DOI: 10.1137/19m1283793] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
To make decisions we are guided by the evidence we collect and the opinions of friends and neighbors. How do we combine our private beliefs with information we obtain from our social network? To understand the strategies humans use to do so, it is useful to compare them to observers that optimally integrate all evidence. Here we derive network models of rational (Bayes optimal) agents who accumulate private measurements and observe the decisions of their neighbors to make an irreversible choice between two options. The resulting information exchange dynamics has interesting properties: When decision thresholds are asymmetric, the absence of a decision can be increasingly informative over time. In a recurrent network of two agents, the absence of a decision can lead to a sequence of belief updates akin to those in the literature on common knowledge. On the other hand, in larger networks a single decision can trigger a cascade of agreements and disagreements that depend on the private information agents have gathered. Our approach provides a bridge between social decision making models in the economics literature, which largely ignore the temporal dynamics of decisions, and the single-observer evidence accumulator models used widely in neuroscience and psychology.
Collapse
Affiliation(s)
| | - Simon Stolarczyk
- Department of Mathematics, University of Houston, Houston, TX 77204
| | | | - Krešimir Josić
- Department of Mathematics, Department of Biology and Biochemistry, University of Houston, Houston, TX 77204, and Department of BioSciences, Rice University, Houston, TX 77005
| |
Collapse
|
7
|
Barendregt NW, Josić K, Kilpatrick ZP. Analyzing dynamic decision-making models using Chapman-Kolmogorov equations. J Comput Neurosci 2019; 47:205-222. [PMID: 31734803 PMCID: PMC7137388 DOI: 10.1007/s10827-019-00733-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2019] [Revised: 09/25/2019] [Accepted: 10/01/2019] [Indexed: 11/28/2022]
Abstract
Decision-making in dynamic environments typically requires adaptive evidence accumulation that weights new evidence more heavily than old observations. Recent experimental studies of dynamic decision tasks require subjects to make decisions for which the correct choice switches stochastically throughout a single trial. In such cases, an ideal observer's belief is described by an evolution equation that is doubly stochastic, reflecting stochasticity in the both observations and environmental changes. In these contexts, we show that the probability density of the belief can be represented using differential Chapman-Kolmogorov equations, allowing efficient computation of ensemble statistics. This allows us to reliably compare normative models to near-normative approximations using, as model performance metrics, decision response accuracy and Kullback-Leibler divergence of the belief distributions. Such belief distributions could be obtained empirically from subjects by asking them to report their decision confidence. We also study how response accuracy is affected by additional internal noise, showing optimality requires longer integration timescales as more noise is added. Lastly, we demonstrate that our method can be applied to tasks in which evidence arrives in a discrete, pulsatile fashion, rather than continuously.
Collapse
Affiliation(s)
- Nicholas W Barendregt
- Department of Applied Mathematics, University of Colorado Boulder, Boulder, CO, 80309, USA
| | - Krešimir Josić
- Department of Mathematics, University of Houston, Houston, TX, 77204, USA
| | - Zachary P Kilpatrick
- Department of Applied Mathematics, University of Colorado Boulder, Boulder, CO, 80309, USA.
| |
Collapse
|
8
|
Kilpatrick ZP, Holmes WR, Eissa TL, Josić K. Optimal models of decision-making in dynamic environments. Curr Opin Neurobiol 2019; 58:54-60. [PMID: 31326724 PMCID: PMC6859206 DOI: 10.1016/j.conb.2019.06.006] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Accepted: 06/22/2019] [Indexed: 11/16/2022]
Abstract
Nature is in constant flux, so animals must account for changes in their environment when making decisions. How animals learn the timescale of such changes and adapt their decision strategies accordingly is not well understood. Recent psychophysical experiments have shown humans and other animals can achieve near-optimal performance at two alternative forced choice (2AFC) tasks in dynamically changing environments. Characterization of performance requires the derivation and analysis of computational models of optimal decision-making policies on such tasks. We review recent theoretical work in this area, and discuss how models compare with subjects' behavior in tasks where the correct choice or evidence quality changes in dynamic, but predictable, ways.
Collapse
Affiliation(s)
| | - William R Holmes
- Department of Physics and Astronomy, Vanderbilt University, Nashville, TN, USA; Department of Mathematics, Vanderbilt University, Nashville, TN, USA; Quantitative Systems Biology Center, Vanderbilt University, Nashville, TN, USA
| | - Tahra L Eissa
- Department of Applied Mathematics, University of Colorado, Boulder, CO, USA
| | - Krešimir Josić
- Department of Mathematics, University of Houston, Houston, TX, USA; Department of Biology and Biochemistry, University of Houston, Houston, TX, USA; Department of BioSciences, Rice University, Houston, TX, USA.
| |
Collapse
|
9
|
Roxin A. Drift-diffusion models for multiple-alternative forced-choice decision making. JOURNAL OF MATHEMATICAL NEUROSCIENCE 2019; 9:5. [PMID: 31270706 PMCID: PMC6609930 DOI: 10.1186/s13408-019-0073-4] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/09/2019] [Accepted: 06/10/2019] [Indexed: 06/01/2023]
Abstract
The canonical computational model for the cognitive process underlying two-alternative forced-choice decision making is the so-called drift-diffusion model (DDM). In this model, a decision variable keeps track of the integrated difference in sensory evidence for two competing alternatives. Here I extend the notion of a drift-diffusion process to multiple alternatives. The competition between n alternatives takes place in a linear subspace of [Formula: see text] dimensions; that is, there are [Formula: see text] decision variables, which are coupled through correlated noise sources. I derive the multiple-alternative DDM starting from a system of coupled, linear firing rate equations. I also show that a Bayesian sequential probability ratio test for multiple alternatives is, in fact, equivalent to these same linear DDMs, but with time-varying thresholds. If the original neuronal system is nonlinear, one can once again derive a model describing a lower-dimensional diffusion process. The dynamics of the nonlinear DDM can be recast as the motion of a particle on a potential, the general form of which is given analytically for an arbitrary number of alternatives.
Collapse
Affiliation(s)
- Alex Roxin
- Centre de Recerca Matemàtica, Bellaterra, Spain.
- Barcelona Graduate School of Mathematics, Barcelona, Spain.
| |
Collapse
|
10
|
Ganupuru P, Goldring AB, Harun R, Hanks TD. Flexibility of Timescales of Evidence Evaluation for Decision Making. Curr Biol 2019; 29:2091-2097.e4. [PMID: 31178325 DOI: 10.1016/j.cub.2019.05.037] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2018] [Revised: 04/05/2019] [Accepted: 05/15/2019] [Indexed: 12/13/2022]
Abstract
To understand the neural mechanisms that support decision making, it is critical to characterize the timescale of evidence evaluation. Recent work has shown that subjects can adaptively adjust the timescale of evidence evaluation across blocks of trials depending on context [1]. However, it's currently unknown if adjustments to evidence evaluation occur online during deliberations based on a single stream of evidence. To examine this question, we employed a change-detection task in which subjects report their level of confidence in judging whether there has been a change in a stochastic auditory stimulus. Using a combination of psychophysical reverse correlation analyses and single-trial behavioral modeling, we compared the time period over which sensory information has leverage on detection report choices versus confidence. We demonstrate that the length of this period differs on separate sets of trials based on what's being reported. Surprisingly, confidence judgments on trials with no detection report are influenced by evidence occurring earlier than the time period of influence for detection reports. Our findings call into question models of decision formation involving static parameters that yield a singular timescale of evidence evaluation and instead suggest that the brain represents and utilizes multiple timescales of evidence evaluation during deliberation.
Collapse
Affiliation(s)
- Preetham Ganupuru
- Department of Neurology and Center for Neuroscience, University of California Davis, 1544 Newton Ct., Davis, CA 95618, USA
| | - Adam B Goldring
- Department of Neurology and Center for Neuroscience, University of California Davis, 1544 Newton Ct., Davis, CA 95618, USA
| | - Rashed Harun
- Department of Neurology and Center for Neuroscience, University of California Davis, 1544 Newton Ct., Davis, CA 95618, USA
| | - Timothy D Hanks
- Department of Neurology and Center for Neuroscience, University of California Davis, 1544 Newton Ct., Davis, CA 95618, USA.
| |
Collapse
|
11
|
Nguyen KP, Josić K, Kilpatrick ZP. Optimizing sequential decisions in the drift-diffusion model. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2019; 88:32-47. [PMID: 31564753 PMCID: PMC6764782 DOI: 10.1016/j.jmp.2018.11.001] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
To make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, thus providing a mathematically principled foundation for evidence-accumulation models of sequential decisions.
Collapse
Affiliation(s)
- Khanh P Nguyen
- Department of Mathematics, University of Houston, Houston TX 77204 (, )
| | - Krešimir Josić
- Department of Mathematics, University of Houston, Houston TX 77204 (, )
- Department of Biology and Biochemistry, University of Houston, Houston TX 77204
- Department of BioSciences, Rice University, Houston TX 77005
- equal authorship
| | - Zachary P Kilpatrick
- Department of Applied Mathematics, University of Colorado, Boulder, Colorado 80309, USA
- Department of Physiology and Biophysics, University of Colorado School of Medicine, Aurora, CO 80045
- equal authorship
| |
Collapse
|
12
|
Młynarski WF, Hermundstad AM. Adaptive coding for dynamic sensory inference. eLife 2018; 7:32055. [PMID: 29988020 PMCID: PMC6039184 DOI: 10.7554/elife.32055] [Citation(s) in RCA: 45] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2017] [Accepted: 04/11/2018] [Indexed: 12/30/2022] Open
Abstract
Behavior relies on the ability of sensory systems to infer properties of the environment from incoming stimuli. The accuracy of inference depends on the fidelity with which behaviorally relevant properties of stimuli are encoded in neural responses. High-fidelity encodings can be metabolically costly, but low-fidelity encodings can cause errors in inference. Here, we discuss general principles that underlie the tradeoff between encoding cost and inference error. We then derive adaptive encoding schemes that dynamically navigate this tradeoff. These optimal encodings tend to increase the fidelity of the neural representation following a change in the stimulus distribution, and reduce fidelity for stimuli that originate from a known distribution. We predict dynamical signatures of such encoding schemes and demonstrate how known phenomena, such as burst coding and firing rate adaptation, can be understood as hallmarks of optimal coding for accurate inference.
Collapse
Affiliation(s)
- Wiktor F Młynarski
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, United States
| | - Ann M Hermundstad
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States
| |
Collapse
|
13
|
Glaze CM, Filipowicz ALS, Kable JW, Balasubramanian V, Gold JI. A bias–variance trade-off governs individual differences in on-line learning in an unpredictable environment. Nat Hum Behav 2018. [DOI: 10.1038/s41562-018-0297-4] [Citation(s) in RCA: 45] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
|