1
|
Xie T, Adamek M, Cho H, Adamo MA, Ritaccio AL, Willie JT, Brunner P, Kubanek J. Graded decisions in the human brain. Nat Commun 2024; 15:4308. [PMID: 38773117 PMCID: PMC11109249 DOI: 10.1038/s41467-024-48342-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Accepted: 04/26/2024] [Indexed: 05/23/2024] Open
Abstract
Decision-makers objectively commit to a definitive choice, yet at the subjective level, human decisions appear to be associated with a degree of uncertainty. Whether decisions are definitive (i.e., concluding in all-or-none choices), or whether the underlying representations are graded, remains unclear. To answer this question, we recorded intracranial neural signals directly from the brain while human subjects made perceptual decisions. The recordings revealed that broadband gamma activity reflecting each individual's decision-making process, ramped up gradually while being graded by the accumulated decision evidence. Crucially, this grading effect persisted throughout the decision process without ever reaching a definite bound at the time of choice. This effect was most prominent in the parietal cortex, a brain region traditionally implicated in decision-making. These results provide neural evidence for a graded decision process in humans and an analog framework for flexible choice behavior.
Collapse
Affiliation(s)
- Tao Xie
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO, 63110, USA
| | - Markus Adamek
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO, 63110, USA
| | - Hohyun Cho
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO, 63110, USA
| | - Matthew A Adamo
- Department of Neurosurgery, Albany Medical College, Albany, NY, 12208, USA
| | - Anthony L Ritaccio
- Department of Neurology, Albany Medical College, Albany, NY, 12208, USA
- Department of Neurology, Mayo Clinic, Jacksonville, FL, 32224, USA
| | - Jon T Willie
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO, 63110, USA
| | - Peter Brunner
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, MO, 63110, USA.
- National Center for Adaptive Neurotechnologies, St. Louis, MO, 63110, USA.
- Department of Neurology, Albany Medical College, Albany, NY, 12208, USA.
| | - Jan Kubanek
- Department of Biomedical Engineering, University of Utah, Salt Lake City, UT, 84112, USA.
| |
Collapse
|
2
|
Calder-Travis J, Bogacz R, Yeung N. Expressions for Bayesian confidence of drift diffusion observers in fluctuating stimuli tasks. JOURNAL OF MATHEMATICAL PSYCHOLOGY 2023; 117:102815. [PMID: 38188903 PMCID: PMC7615478 DOI: 10.1016/j.jmp.2023.102815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/09/2024]
Abstract
We introduce a new approach to modelling decision confidence, with the aim of enabling computationally cheap predictions while taking into account, and thereby exploiting, trial-by-trial variability in stochastically fluctuating stimuli. Using the framework of the drift diffusion model of decision making, along with time-dependent thresholds and the idea of a Bayesian confidence readout, we derive expressions for the probability distribution over confidence reports. In line with current models of confidence, the derivations allow for the accumulation of "pipeline" evidence that has been received but not processed by the time of response, the effect of drift rate variability, and metacognitive noise. The expressions are valid for stimuli that change over the course of a trial with normally-distributed fluctuations in the evidence they provide. A number of approximations are made to arrive at the final expressions, and we test all approximations via simulation. The derived expressions contain only a small number of standard functions, and require evaluating only once per trial, making trial-by-trial modelling of confidence data in stochastically fluctuating stimuli tasks more feasible. We conclude by using the expressions to gain insight into the confidence of optimal observers, and empirically observed patterns.
Collapse
Affiliation(s)
| | - Rafal Bogacz
- MRC Brain Network Dynamics Unit, Nuffield Department of Clinical Neuroscience, University of Oxford, UK
| | - Nick Yeung
- Department of Experimental Psychology, University of Oxford, UK
| |
Collapse
|
3
|
Tsetsos K. Unlocking a new dimension in the speed-accuracy trade-off. Trends Cogn Sci 2023; 27:510-511. [PMID: 36959078 DOI: 10.1016/j.tics.2023.03.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Accepted: 03/14/2023] [Indexed: 03/25/2023]
Abstract
Why do we sometimes spend too much time on seemingly impossible-to-solve tasks instead of just moving on? Masís et al. provide a new perspective on the speed-accuracy trade-off (SAT), showing that, although prolonging deliberation looks suboptimal in the short run, it is a long-term investment that helps organisms reach proficient performance more rapidly.
Collapse
Affiliation(s)
- Konstantinos Tsetsos
- School of Psychological Science, University of Bristol, Bristol, UK; Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
| |
Collapse
|
4
|
Li X, Su R, Chen Y, Yang T. Optimal policy for uncertainty estimation concurrent with decision making. Cell Rep 2023; 42:112232. [PMID: 36924497 DOI: 10.1016/j.celrep.2023.112232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 01/30/2023] [Accepted: 02/23/2023] [Indexed: 03/17/2023] Open
Abstract
Decision making often depends on vague information that leads to uncertainty, which is a quantity contingent not on choice but on probability distributions of sensory evidence and other cognitive variables. Uncertainty may be computed in parallel and interact with decision making. Here, we adapt the classic random-dot motion direction discrimination task to allow subjects to indicate their uncertainty without having to form a decision first. The subjects' choices and reaction times for perceptual decisions and uncertainty responses are measured, respectively. We then build a value-based model in which decisions are based on optimizing value computed from a drift-diffusion process. The model accounts for key features of subjects' behavior and the variation across the individuals. It explains how the addition of the uncertainty option affects perceptual decision making. Our work establishes a value-based theoretical framework for studying uncertainty and perceptual decisions that can be readily applied in future investigations of the underlying neural mechanism.
Collapse
Affiliation(s)
- Xiaodong Li
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Ruixin Su
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China; University of Chinese Academy of Sciences, Beijing 100049, China
| | - Yilin Chen
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China
| | - Tianming Yang
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China.
| |
Collapse
|
5
|
Masís J, Chapman T, Rhee JY, Cox DD, Saxe AM. Strategically managing learning during perceptual decision making. eLife 2023; 12:64978. [PMID: 36786427 PMCID: PMC9928425 DOI: 10.7554/elife.64978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Accepted: 01/15/2023] [Indexed: 02/15/2023] Open
Abstract
Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats' strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process.
Collapse
Affiliation(s)
- Javier Masís
- Department of Molecular and Cellular Biology, Harvard UniversityCambridgeUnited States,Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Travis Chapman
- Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Juliana Y Rhee
- Department of Molecular and Cellular Biology, Harvard UniversityCambridgeUnited States,Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - David D Cox
- Department of Molecular and Cellular Biology, Harvard UniversityCambridgeUnited States,Center for Brain Science, Harvard UniversityCambridgeUnited States
| | - Andrew M Saxe
- Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
| |
Collapse
|
6
|
Yi L, Sekuler R. Audiovisual interaction with rate-varying signals. Iperception 2022; 13:20416695221116653. [PMID: 36467124 PMCID: PMC9716610 DOI: 10.1177/20416695221116653] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Accepted: 07/06/2022] [Indexed: 08/18/2023] Open
Abstract
A task-irrelevant, amplitude-modulating sound influences perception of a size-modulating visual stimulus. To probe the limits of this audiovisual interaction we vary the second temporal derivative of object size and of sound amplitude. In the study's first phase subjects see a visual stimulus size-modulating with f ″ ( x ) > 0, 0, or <0, and judge each one's rate as increasing, constant, or decreasing. Visual stimuli are accompanied by a steady, non-modulated auditory stimulus. The novel combination of multiple stimuli and multi-alternative responses allows subjects' similarity space to be estimated from the stimulus-response confusion matrix. In the study's second phase, rate-varying visual stimuli are presented in concert with auditory stimuli whose second derivative also varied. Subjects identified each visual stimuli as one of the three types, while trying to ignore the accompanying sound. Unlike some previous results with f ″ ( x ) fixed at 0, performance benefits relatively little when visual and auditory stimuli share the same directional change in modulation. However, performance does drop when visual and auditory stimului differ in their directions of rate change. Our task's computational demands may make it particularly vulnerable to the effects of a dynamic task-irrelevant stimulus.
Collapse
Affiliation(s)
- Long Yi
- Volen Center for Complex Systems, Brandeis University,
Waltham, MA, USA
| | - Robert Sekuler
- Volen Center for Complex Systems, Brandeis University,
Waltham, MA, USA
| |
Collapse
|
7
|
Barendregt NW, Gold JI, Josić K, Kilpatrick ZP. Normative decision rules in changing environments. eLife 2022; 11:e79824. [PMID: 36282065 PMCID: PMC9754630 DOI: 10.7554/elife.79824] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 10/20/2022] [Indexed: 11/13/2022] Open
Abstract
Models based on normative principles have played a major role in our understanding of how the brain forms decisions. However, these models have typically been derived for simple, stable conditions, and their relevance to decisions formed under more naturalistic, dynamic conditions is unclear. We previously derived a normative decision model in which evidence accumulation is adapted to fluctuations in the evidence-generating process that occur during a single decision (Glaze et al., 2015), but the evolution of commitment rules (e.g. thresholds on the accumulated evidence) under dynamic conditions is not fully understood. Here, we derive a normative model for decisions based on changing contexts, which we define as changes in evidence quality or reward, over the course of a single decision. In these cases, performance (reward rate) is maximized using decision thresholds that respond to and even anticipate these changes, in contrast to the static thresholds used in many decision models. We show that these adaptive thresholds exhibit several distinct temporal motifs that depend on the specific predicted and experienced context changes and that adaptive models perform robustly even when implemented imperfectly (noisily). We further show that decision models with adaptive thresholds outperform those with constant or urgency-gated thresholds in accounting for human response times on a task with time-varying evidence quality and average reward. These results further link normative and neural decision-making while expanding our view of both as dynamic, adaptive processes that update and use expectations to govern both deliberation and commitment.
Collapse
Affiliation(s)
- Nicholas W Barendregt
- Department of Applied Mathematics, University of Colorado BoulderBoulderUnited States
| | - Joshua I Gold
- Department of Neuroscience, University of PennsylvaniaPhiladelphiaUnited States
| | - Krešimir Josić
- Department of Mathematics, University of HoustonHoustonUnited States
| | - Zachary P Kilpatrick
- Department of Applied Mathematics, University of Colorado BoulderBoulderUnited States
| |
Collapse
|
8
|
Marshall JAR, Reina A, Hay C, Dussutour A, Pirrone A. Magnitude-sensitive reaction times reveal non-linear time costs in multi-alternative decision-making. PLoS Comput Biol 2022; 18:e1010523. [PMID: 36191032 PMCID: PMC9560628 DOI: 10.1371/journal.pcbi.1010523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 10/13/2022] [Accepted: 08/28/2022] [Indexed: 11/07/2022] Open
Abstract
Optimality analysis of value-based decisions in binary and multi-alternative choice settings predicts that reaction times should be sensitive only to differences in stimulus magnitudes, but not to overall absolute stimulus magnitude. Yet experimental work in the binary case has shown magnitude sensitive reaction times, and theory shows that this can be explained by switching from linear to multiplicative time costs, but also by nonlinear subjective utility. Thus disentangling explanations for observed magnitude sensitive reaction times is difficult. Here for the first time we extend the theoretical analysis of geometric time-discounting to ternary choices, and present novel experimental evidence for magnitude-sensitivity in such decisions, in both humans and slime moulds. We consider the optimal policies for all possible combinations of linear and geometric time costs, and linear and nonlinear utility; interestingly, geometric discounting emerges as the predominant explanation for magnitude sensitivity. Analysis of decisions based on option value (e.g. which pile of coins would you like?) suggests that the optimal rules correspond to simple mechanisms also known to be optimal for perceptual decisions (e.g. which light is brighter?) But, crucially, these analyses assume that the cost of time is linear—when the more usual assumption is made that time discounts multiplicatively (e.g. ‘a bird in the hand is worth two in the bush (and so two in the hand are worth four in the bush)’) then optimal decision-making looks quite different—in particular, the theory predicts that decision-making should be sensitive to the absolute magnitude of the opportunities, such as coin pile sizes, under consideration, in a way that the optimal perceptual mechanisms are not. As well as the theory, we present novel experimental evidence from human decision-making experiments, and foraging slime mould, of precisely such magnitude-sensitivity. This is a rare example of theory in behaviour making a falsifiable prediction that is confirmed in two, highly divergent, species, one with a brain and one without.
Collapse
Affiliation(s)
- James A. R. Marshall
- Department of Computer Science, University of Sheffield, Sheffield, United Kingdom
- Opteran Technologies, Sheffield, United Kingdom
- * E-mail:
| | - Andreagiovanni Reina
- Department of Computer Science, University of Sheffield, Sheffield, United Kingdom
- IRIDIA, Université Libre de Bruxelles, Brussels, Belgium
| | - Célia Hay
- Research Centre for Animal Cognition (CRCA), Centre for Integrative Biology (CBI), Toulouse University, Toulouse, France
| | - Audrey Dussutour
- Research Centre for Animal Cognition (CRCA), Centre for Integrative Biology (CBI), Toulouse University, Toulouse, France
| | - Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, United Kingdom
| |
Collapse
|
9
|
Trueblood JS. Theories of Context Effects in Multialternative, Multiattribute Choice. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 2022. [DOI: 10.1177/09637214221109587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Over the past several decades, researchers in psychology, neuroscience, marketing, and economics have been keen to understand context effects in multialternative, multiattribute decision making. These effects occur when choices among existing alternatives are altered by the addition of a new alternative to the choice set. The effects violate classic decision theories and have led to the development of computational and mathematical models that explain how underlying cognitive and neural mechanisms give rise to the effects. This article reviews dynamic models of these effects, comparing mechanisms across models. Most models of context effects incorporate an attention mechanism, which suggests that attention plays an important role in multialternative, multiattribute decision making. I conclude by discussing recent empirical studies of attention and context effects and hypothesize that changes in attention could be responsible for recently observed reversals in context effects.
Collapse
|
10
|
Degenerate boundaries for multiple-alternative decisions. Nat Commun 2022; 13:5066. [PMID: 36038538 PMCID: PMC9424291 DOI: 10.1038/s41467-022-32741-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 08/15/2022] [Indexed: 11/08/2022] Open
Abstract
Integration-to-threshold models of two-choice perceptual decision making have guided our understanding of human and animal behavior and neural processing. Although such models seem to extend naturally to multiple-choice decision making, consensus on a normative framework has yet to emerge, and hence the implications of threshold characteristics for multiple choices have only been partially explored. Here we consider sequential Bayesian inference and a conceptualisation of decision making as a particle diffusing in n-dimensions. We show by simulation that, within a parameterised subset of time-independent boundaries, the optimal decision boundaries comprise a degenerate family of nonlinear structures that jointly depend on the state of multiple accumulators and speed-accuracy trade-offs. This degeneracy is contrary to current 2-choice results where there is a single optimal threshold. Such boundaries support both stationary and collapsing thresholds as optimal strategies for decision-making, both of which result from stationary representations of nonlinear boundaries. Our findings point towards a normative theory of multiple-choice decision making, provide a characterisation of optimal decision thresholds under this framework, and inform the debate between stationary and dynamic decision boundaries for optimal decision making.
Collapse
|
11
|
Ramírez-Ruiz J, Moreno-Bote R. Optimal Allocation of Finite Sampling Capacity in Accumulator Models of Multialternative Decision Making. Cogn Sci 2022; 46:e13143. [PMID: 35523123 PMCID: PMC9285422 DOI: 10.1111/cogs.13143] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Revised: 02/07/2022] [Accepted: 04/16/2022] [Indexed: 11/28/2022]
Abstract
When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here, we study the problem of how to optimally allocate limited sampling time to multiple options, modeled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sublinearly, closely following a power law as a function of capacity for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multialternative decisions are endowed with breadth–depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.
Collapse
Affiliation(s)
- Jorge Ramírez-Ruiz
- Center for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra
| | - Rubén Moreno-Bote
- Center for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra.,Serra Húnter Fellow Programme, Universitat Pompeu Fabra
| |
Collapse
|
12
|
Dennison JB, Sazhin D, Smith DV. Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2022; 13:e1589. [PMID: 35137549 PMCID: PMC9124684 DOI: 10.1002/wcs.1589] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Revised: 11/28/2021] [Accepted: 12/21/2021] [Indexed: 01/10/2023]
Abstract
In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.
Collapse
Affiliation(s)
- Jeffrey B Dennison
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - Daniel Sazhin
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - David V Smith
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
13
|
Efficiently irrational: deciphering the riddle of human choice. Trends Cogn Sci 2022; 26:669-687. [PMID: 35643845 PMCID: PMC9283329 DOI: 10.1016/j.tics.2022.04.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Revised: 04/18/2022] [Accepted: 04/20/2022] [Indexed: 11/18/2022]
Abstract
For the past half-century, cognitive and social scientists have struggled with the irrationalities of human choice behavior; people consistently make choices that are logically inconsistent. Is human choice behavior evolutionarily adaptive or is it an inefficient patchwork of competing mechanisms? In this review, I present an interdisciplinary synthesis arguing for a novel interpretation: choice is efficiently irrational. Connecting findings across disciplines suggests that observed choice behavior reflects a precise optimization of the trade-off between the costs of increasing the precision of the choice mechanism and the declining benefits that come as precision increases. Under these constraints, a rationally imprecise strategy emerges that works toward optimal efficiency rather than toward optimal rationality. This approach rationalizes many of the puzzling inconsistencies of human choice behavior, explaining why these inconsistencies arise as an optimizing solution in biological choosers.
Collapse
|
14
|
Callaway F, van Opheusden B, Gul S, Das P, Krueger PM, Lieder F, Griffiths TL. Rational use of cognitive resources in human planning. Nat Hum Behav 2022; 6:1112-1125. [PMID: 35484209 DOI: 10.1038/s41562-022-01332-8] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 03/03/2022] [Indexed: 12/19/2022]
Abstract
Making good decisions requires thinking ahead, but the huge number of actions and outcomes one could consider makes exhaustive planning infeasible for computationally constrained agents, such as humans. How people are nevertheless able to solve novel problems when their actions have long-reaching consequences is thus a long-standing question in cognitive science. To address this question, we propose a model of resource-constrained planning that allows us to derive optimal planning strategies. We find that previously proposed heuristics such as best-first search are near optimal under some circumstances but not others. In a mouse-tracking paradigm, we show that people adapt their planning strategies accordingly, planning in a manner that is broadly consistent with the optimal model but not with any single heuristic model. We also find systematic deviations from the optimal model that might result from additional cognitive constraints that are yet to be uncovered.
Collapse
Affiliation(s)
| | | | - Sayan Gul
- Department of Psychology, University of California, Berkeley, CA, USA
| | - Priyam Das
- Department of Cognitive Sciences, University of California, Irvine, CA, USA
| | - Paul M Krueger
- Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Falk Lieder
- Max Planck Institute for Intelligent Systems, Tübingen, Germany
| | | |
Collapse
|
15
|
Frömer R, Shenhav A. Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making. Neurosci Biobehav Rev 2022; 134:104483. [PMID: 34902441 PMCID: PMC8844247 DOI: 10.1016/j.neubiorev.2021.12.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 12/01/2021] [Accepted: 12/04/2021] [Indexed: 12/26/2022]
Abstract
While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.
Collapse
Affiliation(s)
- R Frömer
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| | - A Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| |
Collapse
|
16
|
Alternative female and male developmental trajectories in the dynamic balance of human visual perception. Sci Rep 2022; 12:1674. [PMID: 35102227 PMCID: PMC8803928 DOI: 10.1038/s41598-022-05620-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2021] [Accepted: 12/17/2021] [Indexed: 12/21/2022] Open
Abstract
The numerous multistable phenomena in vision, hearing and touch attest that the inner workings of perception are prone to instability. We investigated a visual example-binocular rivalry-with an accurate no-report paradigm, and uncovered developmental and maturational lifespan trajectories that were specific for age and sex. To interpret these trajectories, we hypothesized that conflicting objectives of visual perception-such as stability of appearance, sensitivity to visual detail, and exploration of fundamental alternatives-change in relative importance over the lifespan. Computational modelling of our empirical results allowed us to estimate this putative development of stability, sensitivity, and exploration over the lifespan. Our results confirmed prior findings of developmental psychology and appear to quantify important aspects of neurocognitive phenotype. Additionally, we report atypical function of binocular rivalry in autism spectrum disorder and borderline personality disorder. Our computational approach offers new ways of quantifying neurocognitive phenotypes both in development and in dysfunction.
Collapse
|
17
|
Bond K, Dunovan K, Porter A, Rubin JE, Verstynen T. Dynamic decision policy reconfiguration under outcome uncertainty. eLife 2021; 10:e65540. [PMID: 34951589 PMCID: PMC8806193 DOI: 10.7554/elife.65540] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 12/23/2021] [Indexed: 11/18/2022] Open
Abstract
In uncertain or unstable environments, sometimes the best decision is to change your mind. To shed light on this flexibility, we evaluated how the underlying decision policy adapts when the most rewarding action changes. Human participants performed a dynamic two-armed bandit task that manipulated the certainty in relative reward (conflict) and the reliability of action-outcomes (volatility). Continuous estimates of conflict and volatility contributed to shifts in exploratory states by changing both the rate of evidence accumulation (drift rate) and the amount of evidence needed to make a decision (boundary height), respectively. At the trialwise level, following a switch in the optimal choice, the drift rate plummets and the boundary height weakly spikes, leading to a slow exploratory state. We find that the drift rate drives most of this response, with an unreliable contribution of boundary height across experiments. Surprisingly, we find no evidence that pupillary responses associated with decision policy changes. We conclude that humans show a stereotypical shift in their decision policies in response to environmental changes.
Collapse
Affiliation(s)
- Krista Bond
- Department of Psychology, Carnegie Mellon UniversityPittsburghUnited States
- Center for the Neural Basis of CognitionPittsburghUnited States
- Carnegie Mellon Neuroscience InstitutePittsburghUnited States
| | - Kyle Dunovan
- Department of Psychology, Carnegie Mellon UniversityPittsburghUnited States
| | - Alexis Porter
- Department of Psychology, Northwestern UniversityEvanstonUnited States
| | - Jonathan E Rubin
- Center for the Neural Basis of CognitionPittsburghUnited States
- Department of Mathematics, University of PittsburghPittsburghUnited States
| | - Timothy Verstynen
- Department of Psychology, Carnegie Mellon UniversityPittsburghUnited States
- Center for the Neural Basis of CognitionPittsburghUnited States
- Carnegie Mellon Neuroscience InstitutePittsburghUnited States
- Department of Biomedical Engineering, Carnegie Mellon UniversityPittsburghUnited States
| |
Collapse
|
18
|
Fiore VG, Gu X. Similar network compositions, but distinct neural dynamics underlying belief updating in environments with and without explicit outcomes. Neuroimage 2021; 247:118821. [PMID: 34920087 PMCID: PMC8823284 DOI: 10.1016/j.neuroimage.2021.118821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Revised: 12/10/2021] [Accepted: 12/13/2021] [Indexed: 11/19/2022] Open
Abstract
Classic decision theories typically assume the presence of explicit value-based outcomes after action selections to update beliefs about action-outcome contingencies. However, ecological environments are often opaque, and it remains unclear whether the neural dynamics underlying belief updating vary under conditions characterized by the presence or absence of such explicit value-based information, after each choice selection. We investigated this question in healthy humans (n = 28) using Bayesian inference and two multi-option fMRI tasks: a multi-armed bandit task, and a probabilistic perceptual task, respectively with and without explicit value-based feedback after choice selections. Model-based fMRI analysis revealed a network encoding belief updating which did not change depending on the task. More precisely, we found a confidence-building network that included anterior hippocampus, amygdala, and medial prefrontal cortex (mPFC), which became more active as beliefs about action-outcome probabilities were confirmed by newly acquired information. Despite these consistent responses across tasks, dynamic causal modeling estimated that the network dynamics changed depending on the presence or absence of trial-by-trial value-based outcomes. In the task deprived of immediate feedback, the hippocampus increased its influence towards both amygdala and mPFC, in association with increased strength in the confidence signal. However, the opposite causal relations were found (i.e., from both mPFC and amygdala towards the hippocampus), in presence of immediate outcomes. This finding revealed an asymmetric relationship between decision confidence computations, which were based on similar computational models across tasks, and neural implementation, which varied depending on the availability of outcomes after choice selections.
Collapse
Affiliation(s)
- Vincenzo G Fiore
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States; Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10027, United States.
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States; Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10027, United States; Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States.
| |
Collapse
|
19
|
Pirrone A, Reina A, Stafford T, Marshall JAR, Gobet F. Magnitude-sensitivity: rethinking decision-making. Trends Cogn Sci 2021; 26:66-80. [PMID: 34750080 DOI: 10.1016/j.tics.2021.10.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 10/05/2021] [Accepted: 10/05/2021] [Indexed: 11/25/2022]
Abstract
Magnitude-sensitivity refers to the result that performance in decision-making, across domains and organisms, is affected by the total value of the possible alternatives. This simple result offers a window into fundamental issues in decision-making and has led to a reconsideration of ecological decision-making, prominent computational models of decision-making, and optimal decision-making. Moreover, magnitude-sensitivity has inspired the design of new robotic systems that exploit natural solutions and apply optimal decision-making policies. In this article, we review the key theoretical and empirical results about magnitude-sensitivity and highlight the importance that this phenomenon has for the understanding of decision-making. Furthermore, we discuss open questions and ideas for future research.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK.
| | - Andreagiovanni Reina
- Institute for Interdisciplinary Studies on Artificial Intelligence (IRIDIA), Université Libre de Bruxelles, Brussels, Belgium
| | - Tom Stafford
- Department of Psychology, University of Sheffield, Sheffield, UK
| | | | - Fernand Gobet
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK
| |
Collapse
|
20
|
Wang S, Feng SF, Bornstein AM. Mixing memory and desire: How memory reactivation supports deliberative decision-making. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2021; 13:e1581. [PMID: 34665529 DOI: 10.1002/wcs.1581] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 08/24/2021] [Accepted: 09/16/2021] [Indexed: 11/09/2022]
Abstract
Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.
Collapse
Affiliation(s)
- Shaoming Wang
- Department of Psychology, New York University, New York, New York, USA
| | - Samuel F Feng
- Department of Mathematics, Khalifa University of Science and Technology, Abu Dhabi, UAE.,Khalifa University Centre for Biotechnology, Khalifa University of Science and Technology, Abu Dhabi, UAE
| | - Aaron M Bornstein
- Department of Cognitive Sciences, University of California-Irvine, Irvine, California, USA.,Center for the Neurobiology of Learning & Memory, University of California-Irvine, Irvine, California, USA.,Institute for Mathematical Behavioral Sciences, University of California-Irvine, Irvine, California, USA
| |
Collapse
|
21
|
Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning. Psychon Bull Rev 2021; 28:20-39. [PMID: 32710256 DOI: 10.3758/s13423-020-01774-z] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
What determines the speed of our decisions? Various models of decision-making have focused on perceptual evidence, past experience, and task complexity as important factors determining the degree of deliberation needed for a decision. Here, we build on a sequential sampling decision-making framework to develop a new model that captures a range of reaction time (RT) effects by accounting for both working memory and instrumental learning processes. The model captures choices and RTs at various stages of learning, and in learning environments with varying complexity. Moreover, the model generalizes from tasks with deterministic reward contingencies to probabilistic ones. The model succeeds in part by incorporating prior uncertainty over actions when modeling RT. This straightforward process model provides a parsimonious account of decision dynamics during instrumental learning and makes unique predictions about internal representations of action values.
Collapse
|
22
|
The cherry effect or the issue behind well-being. Cogn Process 2021; 22:711-713. [PMID: 34047894 DOI: 10.1007/s10339-021-01032-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Accepted: 04/26/2021] [Indexed: 10/21/2022]
Abstract
Humans define well-being on predefined assumptions, based on inner and outer criteria as references. As illustrated, these criteria are subject to constant change, even in a situation when one is acting freely and is in control of all possible external influences. Even in scenarios that seemingly allow autonomy with one variable to analyse, underlying "irrationality" affects our ability to define and operationalize any desirable trait or state, such as well-being, euthymia or health. Before eating a bowl full of cherries, one creates an idea of how much cherries he/she will eat. However, as one starts eating, perception and following assumptions change. As cherries labeled as most desirable disappear, other cherries start to appear more alluring. The cherry effect could be of relevance in defining the terms such as well-being, euthymia and basically any other term encompassing a complex category of the human condition dependent on our perceived reality.
Collapse
|
23
|
Lee DG, Daunizeau J. Trading mental effort for confidence in the metacognitive control of value-based decision-making. eLife 2021; 10:e63282. [PMID: 33900198 PMCID: PMC8128438 DOI: 10.7554/elife.63282] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2020] [Accepted: 04/23/2021] [Indexed: 01/08/2023] Open
Abstract
Why do we sometimes opt for actions or items that we do not value the most? Under current neurocomputational theories, such preference reversals are typically interpreted in terms of errors that arise from the unreliable signaling of value to brain decision systems. But, an alternative explanation is that people may change their mind because they are reassessing the value of alternative options while pondering the decision. So, why do we carefully ponder some decisions, but not others? In this work, we derive a computational model of the metacognitive control of decisions or MCD. In brief, we assume that fast and automatic processes first provide initial (and largely uncertain) representations of options' values, yielding prior estimates of decision difficulty. These uncertain value representations are then refined by deploying cognitive (e.g., attentional, mnesic) resources, the allocation of which is controlled by an effort-confidence tradeoff. Importantly, the anticipated benefit of allocating resources varies in a decision-by-decision manner according to the prior estimate of decision difficulty. The ensuing MCD model predicts response time, subjective feeling of effort, choice confidence, changes of mind, as well as choice-induced preference change and certainty gain. We test these predictions in a systematic manner, using a dedicated behavioral paradigm. Our results provide a quantitative link between mental effort, choice confidence, and preference reversals, which could inform interpretations of related neuroimaging findings.
Collapse
Affiliation(s)
- Douglas G Lee
- Sorbonne UniversityParisFrance
- Paris Brain Institute (ICM)ParisFrance
- Institute of Cognitive Sciences and Technologies, National Research Council of ItalyRomeItaly
| | - Jean Daunizeau
- Paris Brain Institute (ICM)ParisFrance
- Translational Neuromodeling Unit (TNU), ETHZurichSwitzerland
| |
Collapse
|
24
|
Jang AI, Sharma R, Drugowitsch J. Optimal policy for attention-modulated decisions explains human fixation behavior. eLife 2021; 10:e63436. [PMID: 33769284 PMCID: PMC8064754 DOI: 10.7554/elife.63436] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2020] [Accepted: 03/17/2021] [Indexed: 01/23/2023] Open
Abstract
Traditional accumulation-to-bound decision-making models assume that all choice options are processed with equal attention. In real life decisions, however, humans alternate their visual fixation between individual items to efficiently gather relevant information (Yang et al., 2016). These fixations also causally affect one's choices, biasing them toward the longer-fixated item (Krajbich et al., 2010). We derive a normative decision-making model in which attention enhances the reliability of information, consistent with neurophysiological findings (Cohen and Maunsell, 2009). Furthermore, our model actively controls fixation changes to optimize information gathering. We show that the optimal model reproduces fixation-related choice biases seen in humans and provides a Bayesian computational rationale for this phenomenon. This insight led to additional predictions that we could confirm in human data. Finally, by varying the relative cognitive advantage conferred by attention, we show that decision performance is benefited by a balanced spread of resources between the attended and unattended items.
Collapse
Affiliation(s)
- Anthony I Jang
- Department of Neurobiology, Harvard Medical SchoolBostonUnited States
| | - Ravi Sharma
- Division of Biostatistics and Bioinformatics, Department of Family Medicine and Public Health, UC San Diego School of MedicineLa JollaUnited States
| | - Jan Drugowitsch
- Department of Neurobiology, Harvard Medical SchoolBostonUnited States
| |
Collapse
|
25
|
Callaway F, Rangel A, Griffiths TL. Fixation patterns in simple choice reflect optimal information sampling. PLoS Comput Biol 2021; 17:e1008863. [PMID: 33770069 PMCID: PMC8026028 DOI: 10.1371/journal.pcbi.1008863] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2020] [Revised: 04/07/2021] [Accepted: 03/10/2021] [Indexed: 11/24/2022] Open
Abstract
Simple choices (e.g., eating an apple vs. an orange) are made by integrating noisy evidence that is sampled over time and influenced by visual attention; as a result, fluctuations in visual attention can affect choices. But what determines what is fixated and when? To address this question, we model the decision process for simple choice as an information sampling problem, and approximate the optimal sampling policy. We find that it is optimal to sample from options whose value estimates are both high and uncertain. Furthermore, the optimal policy provides a reasonable account of fixations and choices in binary and trinary simple choice, as well as the differences between the two cases. Overall, the results show that the fixation process during simple choice is influenced dynamically by the value estimates computed during the decision process, in a manner consistent with optimal information sampling.
Collapse
Affiliation(s)
- Frederick Callaway
- Department of Psychology, Princeton University, Princeton, New Jersey, United States of America
| | - Antonio Rangel
- Departments of Humanities and Social Sciences and Computation and Neural Systems, California Institute of Technology, Pasadena, California, United States of America
| | - Thomas L. Griffiths
- Department of Psychology, Princeton University, Princeton, New Jersey, United States of America
- Department of Computer Science, Princeton University, Princeton, New Jersey, United States of America
| |
Collapse
|
26
|
Roy NA, Bak JH, Akrami A, Brody CD, Pillow JW. Extracting the dynamics of behavior in sensory decision-making experiments. Neuron 2021; 109:597-610.e6. [PMID: 33412101 PMCID: PMC7897255 DOI: 10.1016/j.neuron.2020.12.004] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Revised: 10/23/2020] [Accepted: 12/03/2020] [Indexed: 11/21/2022]
Abstract
Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.
Collapse
Affiliation(s)
- Nicholas A Roy
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| | - Ji Hyun Bak
- Korea Institute for Advanced Study, Seoul 02455, South Korea; Redwood Center for Theoretical Neuroscience, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Athena Akrami
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Sainsbury Wellcome Centre, University College London, London W1T 4JG, UK
| | - Carlos D Brody
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Howard Hughes Medical Institute, Princeton University, Princeton, NJ 08544, USA
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Psychology, Princeton University, Princeton, NJ 08544, USA.
| |
Collapse
|
27
|
Fiore VG, Guertler ACV, Yu JC, Tatineni CC, Gu X. A change of mind: Globus pallidus activity and effective connectivity during changes in choice selections. Eur J Neurosci 2021; 53:2774-2787. [PMID: 33556221 DOI: 10.1111/ejn.15142] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Revised: 01/14/2021] [Accepted: 02/02/2021] [Indexed: 11/29/2022]
Abstract
The basal ganglia are a group of interconnected subcortical nuclei that plays a key role in multiple motor and cognitive processes, in a close interplay with several cortical regions. Two conflicting theories postulate that the basal ganglia pathways can either foster or suppress the cortico-striatal output or, alternatively, they can stabilize or destabilize the cortico-striatal circuit dynamics. These different approaches significantly impact the understanding of observable behaviours and cognitive processes in healthy, as well as clinical populations. We investigated the predictions of these models in healthy participants (N = 28), using dynamic causal modeling of fMRI BOLD activity to estimate time- and context-dependent changes in the indirect pathway effective connectivity, in association with repetitions or changes of choice selections. We used two multi-option tasks that required the participants to adapt to uncontrollable environmental changes, by performing sequential choice selections, with and without value-based feedbacks. We found that, irrespective of the task, the trials that were characterized by changes in choice selections (switch trials) were associated with a neural response that mostly overlapped with a network commonly described for the encoding of uncertainty. More interestingly, dynamic causal modeling and family-wise model comparison identified with high likelihood a directed causal relation from the external to the internal part of the globus pallidus (i.e., the short indirect pathway in the basal ganglia), in association with the switch trials. This finding supports the hypothesis that the short indirect pathway in the basal ganglia drives instability in the network dynamics, resulting in changes in choice selection.
Collapse
Affiliation(s)
- Vincenzo G Fiore
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Ann-Cathrin V Guertler
- School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, Texas, USA
| | - Ju-Chi Yu
- School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, Texas, USA
| | - Chandana C Tatineni
- The Texas College of Osteopathic Medicine at University of North Texas Health Science Center, Fort Worth, Texas, USA
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, New York, USA.,Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
28
|
Abstract
The drift-diffusion model (DDM) is a model of sequential sampling with diffusion signals, where the decision maker accumulates evidence until the process hits either an upper or lower stopping boundary and then stops and chooses the alternative that corresponds to that boundary. In perceptual tasks, the drift of the process is related to which choice is objectively correct, whereas in consumption tasks, the drift is related to the relative appeal of the alternatives. The simplest version of the DDM assumes that the stopping boundaries are constant over time. More recently, a number of papers have used nonconstant boundaries to better fit the data. This paper provides a statistical test for DDMs with general, nonconstant boundaries. As a by-product, we show that the drift and the boundary are uniquely identified. We use our condition to nonparametrically estimate the drift and the boundary and construct a test statistic based on finite samples.
Collapse
|
29
|
Pirrone A, Gobet F. Modeling Value-Based Decision-Making Policies Using Genetic Programming. SWISS JOURNAL OF PSYCHOLOGY 2020. [DOI: 10.1024/1421-0185/a000241] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
Abstract. An important way to develop models in psychology and cognitive science is to express them as computer programs. However, computational modeling is not an easy task. To address this issue, some have proposed using artificial-intelligence (AI) techniques, such as genetic programming (GP) to semiautomatically generate models. In this paper, we establish whether models used to generate data can be recovered when GP evolves models accounting for such data. As an example, we use an experiment from decision-making which addresses a central question in decision-making research, namely, to understand what strategy, or “policy,” agents adopt in order to make a choice. In decision-making, this often means understanding the policy that best explains the distribution of choices and/or reaction times of two-alternative forced-choice decisions. We generated data from three models using different psychologically plausible policies and then evaluated the ability and extent of GP to correctly identify the true generating model among the class of virtually infinite candidate models. Our results show that, regardless of the complexity of the policy, GP can correctly identify the true generating process. Given these results, we discuss implications for cognitive science research and computational scientific discovery as well as possible future applications.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, United Kingdom
| | - Fernand Gobet
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, United Kingdom
| |
Collapse
|
30
|
Abstract
Animals frequently need to choose the best alternative from a set of possibilities, whether it is which direction to swim in or which food source to favor. How long should a network of neurons take to choose the best of N options? Theoretical results suggest that the optimal time grows as log(N), if the values of each option are imperfectly perceived. However, standard self-terminating neural network models of decision-making cannot achieve this optimal behavior. We show how using certain additional nonlinear response properties in neurons, which are ignored in standard models, results in a decision-making architecture that both achieves the optimal scaling of decision time and accounts for multiple experimentally observed features of neural decision-making. An elemental computation in the brain is to identify the best in a set of options and report its value. It is required for inference, decision-making, optimization, action selection, consensus, and foraging. Neural computing is considered powerful because of its parallelism; however, it is unclear whether neurons can perform this max-finding operation in a way that improves upon the prohibitively slow optimal serial max-finding computation (which takes ∼Nlog(N) time for N noisy candidate options) by a factor of N, the benchmark for parallel computation. Biologically plausible architectures for this task are winner-take-all (WTA) networks, where individual neurons inhibit each other so only those with the largest input remain active. We show that conventional WTA networks fail the parallelism benchmark and, worse, in the presence of noise, altogether fail to produce a winner when N is large. We introduce the nWTA network, in which neurons are equipped with a second nonlinearity that prevents weakly active neurons from contributing inhibition. Without parameter fine-tuning or rescaling as N varies, the nWTA network achieves the parallelism benchmark. The network reproduces experimentally observed phenomena like Hick’s law without needing an additional readout stage or adaptive N-dependent thresholds. Our work bridges scales by linking cellular nonlinearities to circuit-level decision-making, establishes that distributed computation saturating the parallelism benchmark is possible in networks of noisy, finite-memory neurons, and shows that Hick’s law may be a symptom of near-optimal parallel decision-making with noisy input.
Collapse
|
31
|
Lee D, Coricelli G. An Empirical Test of the Role of Value Certainty in Decision Making. Front Psychol 2020; 11:574473. [PMID: 33192874 PMCID: PMC7605174 DOI: 10.3389/fpsyg.2020.574473] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2020] [Accepted: 08/31/2020] [Indexed: 11/13/2022] Open
Abstract
Most contemporary models of value-based decisions are built on value estimates that are typically self-reported by the decision maker. Such models have been successful in accounting for choice accuracy and response time, and more recently choice confidence. The fundamental driver of such models is choice difficulty, which is almost always defined as the absolute value difference between the subjective value ratings of the options in a choice set. Yet a decision maker is not necessarily able to provide a value estimate with the same degree of certainty for each option that he encounters. We propose that choice difficulty is determined not only by absolute value distance of choice options, but also by their value certainty. In this study, we first demonstrate the reliability of the concept of an option-specific value certainty using three different experimental measures. We then demonstrate the influence that value certainty has on choice, including accuracy (consistency), choice confidence, response time, and choice-induced preference change (i.e., the degree to which value estimates change from pre- to post-choice evaluation). We conclude with a suggestion of how popular contemporary models of choice (e.g., race model, drift-diffusion model) could be improved by including option-specific value certainty as one of their inputs.
Collapse
Affiliation(s)
- Douglas Lee
- Department of Economics, University of Southern California, Los Angeles, CA, United States
| | | |
Collapse
|
32
|
Gluth S, Kern N, Kortmann M, Vitali CL. Value-based attention but not divisive normalization influences decisions with multiple alternatives. Nat Hum Behav 2020; 4:634-645. [PMID: 32015490 PMCID: PMC7306407 DOI: 10.1038/s41562-020-0822-0] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2019] [Accepted: 01/07/2020] [Indexed: 01/13/2023]
Abstract
Violations of economic rationality principles in choices between three or more options are critical for understanding the neural and cognitive mechanisms of decision-making. A recent study reported that the relative choice accuracy between two options decreases as the value of a third (distractor) option increases and attributed this effect to divisive normalization of neural value representations. In two preregistered experiments, a direct replication and an eye-tracking experiment, we assessed the replicability of this effect and tested an alternative account that assumes value-based attention to mediate the distractor effect. Surprisingly, we could not replicate the distractor effect in our experiments. However, we found a dynamic influence of distractor value on fixations to distractors as predicted by the value-based attention theory. Computationally, we show that extending an established sequential sampling decision-making model by a value-based attention mechanism offers a comprehensive account of the interplay between value, attention, response times and decisions.
Collapse
Affiliation(s)
- Sebastian Gluth
- Department of Psychology, University of Basel, Basel, Switzerland.
| | - Nadja Kern
- Department of Psychology, University of Basel, Basel, Switzerland
| | - Maria Kortmann
- Department of Psychology, University of Basel, Basel, Switzerland
| | - Cécile L Vitali
- Department of Psychology, University of Basel, Basel, Switzerland
| |
Collapse
|
33
|
Mysore SP, Kothari NB. Mechanisms of competitive selection: A canonical neural circuit framework. eLife 2020; 9:e51473. [PMID: 32431293 PMCID: PMC7239658 DOI: 10.7554/elife.51473] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Accepted: 04/02/2020] [Indexed: 01/25/2023] Open
Abstract
Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.
Collapse
Affiliation(s)
- Shreesh P Mysore
- Department of Psychological and Brain Sciences, Johns Hopkins UniversityBaltimoreUnited States
- The Solomon H. Snyder Department of Neuroscience, Johns Hopkins UniversityBaltimoreUnited States
| | - Ninad B Kothari
- Department of Psychological and Brain Sciences, Johns Hopkins UniversityBaltimoreUnited States
| |
Collapse
|
34
|
Keung W, Hagen TA, Wilson RC. A divisive model of evidence accumulation explains uneven weighting of evidence over time. Nat Commun 2020; 11:2160. [PMID: 32358501 PMCID: PMC7195479 DOI: 10.1038/s41467-020-15630-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2019] [Accepted: 03/12/2020] [Indexed: 12/21/2022] Open
Abstract
Divisive normalization has long been used to account for computations in various neural processes and behaviours. The model proposes that inputs into a neural system are divisively normalized by the system’s total activity. More recently, dynamical versions of divisive normalization have been shown to account for how neural activity evolves over time in value-based decision making. Despite its ubiquity, divisive normalization has not been studied in decisions that require evidence to be integrated over time. Such decisions are important when the information is not all available at once. A key feature of such decisions is how evidence is weighted over time, known as the integration kernel. Here, we provide a formal expression for the integration kernel in divisive normalization, and show that divisive normalization quantitatively accounts for 133 human participants’ perceptual decision making behaviour, performing as well as the state-of-the-art Drift Diffusion Model, the predominant model for perceptual evidence accumulation. Divisive normalization is thought to be a ubiquitous computation in the brain, but has not been studied in decisions that require integrating evidence over time. Here, the authors show in humans that dynamic divisive normalization accounts for the uneven weighting of perceptual evidence over time.
Collapse
Affiliation(s)
- Waitsang Keung
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA.
| | - Todd A Hagen
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA
| | - Robert C Wilson
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA.,Cognitive Science Program, University of Arizona, Tucson, AZ, 85719, USA
| |
Collapse
|