1
|
Hall AF, Browning M, Huys QJM. The computational structure of consummatory anhedonia. Trends Cogn Sci 2024; 28:541-553. [PMID: 38423829 DOI: 10.1016/j.tics.2024.01.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 01/18/2024] [Accepted: 01/18/2024] [Indexed: 03/02/2024]
Abstract
Anhedonia is a reduction in enjoyment, motivation, or interest. It is common across mental health disorders and a harbinger of poor treatment outcomes. The enjoyment aspect, termed 'consummatory anhedonia', in particular poses fundamental questions about how the brain constructs rewards: what processes determine how intensely a reward is experienced? Here, we outline limitations of existing computational conceptualisations of consummatory anhedonia. We then suggest a richer reinforcement learning (RL) account of consummatory anhedonia with a reconceptualisation of subjective hedonic experience in terms of goal progress. This accounts qualitatively for the impact of stress, dysfunctional cognitions, and maladaptive beliefs on hedonic experience. The model also offers new views on the treatments for anhedonia.
Collapse
Affiliation(s)
- Anna F Hall
- Applied Computational Psychiatry Lab, Mental Health Neuroscience Department, Division of Psychiatry and Max Planck Centre for Computational Psychiatry and Ageing Research, Queen Square Institute of Neurology, University College London, London, UK
| | - Michael Browning
- Department of Psychiatry, University of Oxford, Oxford, UK; Oxford Health NHS Trust, Oxford, UK
| | - Quentin J M Huys
- Applied Computational Psychiatry Lab, Mental Health Neuroscience Department, Division of Psychiatry and Max Planck Centre for Computational Psychiatry and Ageing Research, Queen Square Institute of Neurology, University College London, London, UK.
| |
Collapse
|
2
|
Koslov SR, Kable JW, Foster BL. Dissociable Contributions of the Medial Parietal Cortex to Recognition Memory. J Neurosci 2024; 44:e2220232024. [PMID: 38527809 PMCID: PMC11063824 DOI: 10.1523/jneurosci.2220-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 03/04/2024] [Accepted: 03/15/2024] [Indexed: 03/27/2024] Open
Abstract
Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its distinct functional characteristics during different retrieval tasks. Specifically, while recognition and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct spatial patterns of response within MPC. However, other studies have emphasized alternate MPC functional dissociations in terms of brain network connectivity profiles or stimulus category selectivity. As the unique contributions of MPC to episodic memory remain unclear, adjudicating between these different accounts can provide better consensus regarding MPC function. Therefore, we used a precision-neuroimaging dataset (7T functional magnetic resonance imaging) to examine how MPC regions are differentially engaged during recognition memory and how these task-related dissociations may also reflect distinct connectivity and stimulus category functional profiles. We observed interleaved, though spatially distinct, subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli. In addition, this dissociation was further accentuated by functional subregions displaying distinct profiles of connectivity with the hippocampus during task and rest. Finally, we show that recent observations of dissociable person and place selectivity within the MPC reflect category-specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, by examining precision functional mapping within individuals, these data suggest that previously distinct observations of functional dissociation within MPC conform to a common principle of organization throughout hippocampal-neocortical memory systems.
Collapse
Affiliation(s)
- Seth R Koslov
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | - Joseph W Kable
- Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| | - Brett L Foster
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania 19104
| |
Collapse
|
3
|
Holton E, Grohn J, Ward H, Manohar SG, O'Reilly JX, Kolling N. Goal commitment is supported by vmPFC through selective attention. Nat Hum Behav 2024:10.1038/s41562-024-01844-5. [PMID: 38632389 DOI: 10.1038/s41562-024-01844-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 02/01/2024] [Indexed: 04/19/2024]
Abstract
When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.
Collapse
Affiliation(s)
- Eleanor Holton
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Jan Grohn
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
| | - Harry Ward
- Centre for Experimental Medicine and Rheumatology, Queen Mary University London (QMUL), London, UK
| | - Sanjay G Manohar
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Jill X O'Reilly
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
- Stem Cell and Brain Research Institute U1208, Inserm, Université Claude Bernard Lyon 1, Bron, France
| |
Collapse
|
4
|
Leng X, Frömer R, Summe T, Shenhav A. Mutual inclusivity improves decision-making by smoothing out choice's competitive edge. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.05.12.540529. [PMID: 37425763 PMCID: PMC10327072 DOI: 10.1101/2023.05.12.540529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Decisions form a central bottleneck to most tasks, one that people often experience as costly. Past work proposes mitigating those costs by lowering one's threshold for deciding. Here, we test an alternative solution, one that targets the basis for most choice costs: that choosing one option sacrifices others (mutual exclusivity). Across 5 studies (N = 462), we test whether this tension can be relieved by framing choices as inclusive (allowing selection of more than one option, as in buffets). We find that inclusivity makes choices more efficient, by selectively reducing competition between potential responses as participants accumulate information for each of their options. Inclusivity also made participants feel less conflicted, especially when they couldn't decide which good option to keep or which bad option to get rid of. These inclusivity benefits were also distinguishable from the effects of manipulating decision threshold (increased urgency), which improved choices but not experiences thereof.
Collapse
Affiliation(s)
- Xiamin Leng
- Department of Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Sciences Brown University, Providence, RI, USA
| | - Romy Frömer
- Department of Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Sciences Brown University, Providence, RI, USA
- School of Psychology, Centre for Human Brain Health, University of Birmingham, Birmingham, UK
| | - Thomas Summe
- Department of Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Sciences Brown University, Providence, RI, USA
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Sciences Brown University, Providence, RI, USA
| |
Collapse
|
5
|
Sakamoto Y, Miyoshi K. A confidence framing effect: Flexible use of evidence in metacognitive monitoring. Conscious Cogn 2024; 118:103636. [PMID: 38244396 DOI: 10.1016/j.concog.2024.103636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 12/20/2023] [Accepted: 01/05/2024] [Indexed: 01/22/2024]
Abstract
Human behavior is flexibly regulated by specific goals of cognitive tasks. One notable example is goal-directed modulation of metacognitive behavior, where logically equivalent decision-making problems can yield different patterns of introspective confidence depending on the frame in which they are presented. While this observation highlights the important heuristic nature of metacognitive monitoring, computational mechanisms underlying this phenomenon remain elusive. We confirmed the confidence framing effect in two-alternative dot-number discrimination and in previously published preference-choice data, demonstrating distinctive confidence patterns between "choose more" or "choose less" frames. Formal model comparisons revealed a simple confidence heuristic behind this phenomenon, which assigns greater weight to chosen than unchosen stimulus evidence. This computation appears to be based on internal evidence constituted under specific task demands rather than physical stimulus intensity itself, a view justified in terms of ecological rationality. These results shed light on the adaptive nature of human decision-making and metacognitive monitoring.
Collapse
Affiliation(s)
- Yosuke Sakamoto
- Graduate School of Frontier Biosciences, Osaka University, 1-3 Yamadaoka, Suita, Osaka 565-0871, Japan
| | - Kiyofumi Miyoshi
- Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo, Kyoto 606-8501, Japan.
| |
Collapse
|
6
|
Hoy CW, Quiroga-Martinez DR, Sandoval E, King-Stephens D, Laxer KD, Weber P, Lin JJ, Knight RT. Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex. Nat Commun 2023; 14:8520. [PMID: 38129440 PMCID: PMC10739882 DOI: 10.1038/s41467-023-44248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 12/05/2023] [Indexed: 12/23/2023] Open
Abstract
The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.
Collapse
Affiliation(s)
- Colin W Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - David R Quiroga-Martinez
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Aarhus, Denmark
| | - Eduardo Sandoval
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
7
|
Fahey MP, Yee DM, Leng X, Tarlow M, Shenhav A. Motivational context determines the impact of aversive outcomes on mental effort allocation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.27.564461. [PMID: 37961466 PMCID: PMC10634922 DOI: 10.1101/2023.10.27.564461] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
It is well known that people will exert effort on a task if sufficiently motivated, but how they distribute these efforts across different strategies (e.g., efficiency vs. caution) remains uncertain. Past work has shown that people invest effort differently for potential positive outcomes (rewards) versus potential negative outcomes (penalties). However, this research failed to account for differences in the context in which negative outcomes motivate someone - either as punishment or reinforcement. It is therefore unclear whether effort profiles differ as a function of outcome valence, motivational context, or both. Using computational modeling and our novel Multi-Incentive Control Task, we show that the influence of aversive outcomes on one's effort profile is entirely determined by their motivational context. Participants (N:91) favored increased caution in response to larger penalties for incorrect responses, and favored increased efficiency in response to larger reinforcement for correct responses, whether positively or negatively incentivized.
Collapse
Affiliation(s)
- Mahalia Prater Fahey
- Cognitive, Linguistic, and Psychological Sciences, Brown University Carney Institute for Brain Science, Brown University
| | - Debbie M Yee
- Cognitive, Linguistic, and Psychological Sciences, Brown University Carney Institute for Brain Science, Brown University
| | - Xiamin Leng
- Cognitive, Linguistic, and Psychological Sciences, Brown University Carney Institute for Brain Science, Brown University
| | - Maisy Tarlow
- Cognitive, Linguistic, and Psychological Sciences, Brown University Carney Institute for Brain Science, Brown University
| | - Amitai Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Brown University Carney Institute for Brain Science, Brown University
| |
Collapse
|
8
|
Molinaro G, Collins AGE. A goal-centric outlook on learning. Trends Cogn Sci 2023; 27:1150-1164. [PMID: 37696690 DOI: 10.1016/j.tics.2023.08.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 08/11/2023] [Accepted: 08/14/2023] [Indexed: 09/13/2023]
Abstract
Goals play a central role in human cognition. However, computational theories of learning and decision-making often take goals as given. Here, we review key empirical findings showing that goals shape the representations of inputs, responses, and outcomes, such that setting a goal crucially influences the central aspects of any learning process: states, actions, and rewards. We thus argue that studying goal selection is essential to advance our understanding of learning. By following existing literature in framing goal selection within a hierarchy of decision-making problems, we synthesize important findings on the principles underlying goal value attribution and exploration strategies. Ultimately, we propose that a goal-centric perspective will help develop more complete accounts of learning in both biological and artificial agents.
Collapse
Affiliation(s)
- Gaia Molinaro
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA.
| | - Anne G E Collins
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA; Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
9
|
Koslov SR, Kable JW, Foster BL. Dissociable contributions of the medial parietal cortex to recognition memory. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.09.12.557048. [PMID: 37745317 PMCID: PMC10515876 DOI: 10.1101/2023.09.12.557048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/26/2023]
Abstract
Human neuroimaging studies of episodic memory retrieval routinely observe the engagement of specific cortical regions beyond the medial temporal lobe. Of these, medial parietal cortex (MPC) is of particular interest given its ubiquitous, and yet distinct, functional characteristics during different types of retrieval tasks. Specifically, while recognition memory and autobiographical recall tasks are both used to probe episodic retrieval, these paradigms consistently drive distinct patterns of response within MPC. This dissociation adds to growing evidence suggesting a common principle of functional organization across memory related brain structures, specifically regarding the control or content demands of memory-based decisions. To carefully examine this putative organization, we used a high-resolution fMRI dataset collected at ultra-high field (7T) while subjects performed thousands of recognition-memory trials to identify MPC regions responsive to recognition-decisions or semantic content of stimuli within and across individuals. We observed interleaving, though distinct, functional subregions of MPC where responses were sensitive to either recognition decisions or the semantic representation of stimuli, but rarely both. In addition, this functional dissociation within MPC was further accentuated by distinct profiles of connectivity bias with the hippocampus during task and rest. Finally, we show that recent observations of person and place selectivity within MPC reflect category specific responses from within identified semantic regions that are sensitive to mnemonic demands. Together, these data better account for how distinct patterns of MPC responses can occur as a result of task demands during episodic retrieval and may reflect a common principle of organization throughout hippocampal-neocortical memory systems.
Collapse
Affiliation(s)
- Seth R. Koslov
- Department of Neurosurgery, Perelman School of Medicine; University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | - Joseph W. Kable
- Department of Psychology; University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | - Brett L. Foster
- Department of Neurosurgery, Perelman School of Medicine; University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| |
Collapse
|
10
|
Molinaro G, Collins AGE. Intrinsic rewards explain context-sensitive valuation in reinforcement learning. PLoS Biol 2023; 21:e3002201. [PMID: 37459394 PMCID: PMC10374061 DOI: 10.1371/journal.pbio.3002201] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2022] [Revised: 07/27/2023] [Accepted: 06/15/2023] [Indexed: 07/28/2023] Open
Abstract
When observing the outcome of a choice, people are sensitive to the choice's context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms-reflecting a different theoretical viewpoint-may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new "intrinsically enhanced" RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.
Collapse
Affiliation(s)
- Gaia Molinaro
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
| | - Anne G E Collins
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| |
Collapse
|
11
|
Moneta N, Garvert MM, Heekeren HR, Schuck NW. Task state representations in vmPFC mediate relevant and irrelevant value signals and their behavioral influence. Nat Commun 2023; 14:3156. [PMID: 37258534 PMCID: PMC10232498 DOI: 10.1038/s41467-023-38709-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2021] [Accepted: 05/09/2023] [Indexed: 06/02/2023] Open
Abstract
The ventromedial prefrontal-cortex (vmPFC) is known to contain expected value signals that inform our choices. But expected values even for the same stimulus can differ by task. In this study, we asked how the brain flexibly switches between such value representations in a task-dependent manner. Thirty-five participants alternated between tasks in which either stimulus color or motion predicted rewards. We show that multivariate vmPFC signals contain a rich representation that includes the current task state or context (motion/color), the associated expected value, and crucially, the irrelevant value of the alternative context. We also find that irrelevant value representations in vmPFC compete with relevant value signals, interact with task-state representations and relate to behavioral signs of value competition. Our results shed light on vmPFC's role in decision making, bridging between its role in mapping observations onto the task states of a mental map, and computing expected values for multiple states.
Collapse
Affiliation(s)
- Nir Moneta
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195, Berlin, Germany.
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, 14195, Berlin, Germany.
- Einstein Center for Neurosciences Berlin, Charité Universitätsmedizin Berlin, 10117, Berlin, Germany.
| | - Mona M Garvert
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195, Berlin, Germany
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, 14195, Berlin, Germany
- Department of Psychology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103, Leipzig, Germany
| | - Hauke R Heekeren
- Einstein Center for Neurosciences Berlin, Charité Universitätsmedizin Berlin, 10117, Berlin, Germany
- Department of Education and Psychology, Freie Universität Berlin, 14195, Berlin, Germany
- Institute of Psychology, Universität Hamburg, 20146, Hamburg, Germany
| | - Nicolas W Schuck
- Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, 14195, Berlin, Germany.
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, 14195, Berlin, Germany.
- Institute of Psychology, Universität Hamburg, 20146, Hamburg, Germany.
| |
Collapse
|
12
|
Kraemer PM, Gluth S. Episodic Memory Retrieval Affects the Onset and Dynamics of Evidence Accumulation during Value-based Decisions. J Cogn Neurosci 2023; 35:692-714. [PMID: 36724395 DOI: 10.1162/jocn_a_01968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
In neuroeconomics, there is much interest in understanding simple value-based choices where agents choose between visually presented goods, comparable to a shopping scenario in a supermarket. However, many everyday decisions are made in the physical absence of the considered goods, requiring agents to recall information about the goods from memory. Here, we asked whether and how this reliance on an internal memory representation affects the temporal dynamics of decision making on a cognitive and neural level. Participants performed a remember-and-decide task in which they made simple purchasing decisions between money offers and snack items while undergoing EEG. Snack identity was presented either visually (value trials) or had to be recalled from memory (memory trials). Behavioral data indicated comparable choice consistency across both trial types, but considerably longer RTs in memory trials. Drift-diffusion modeling suggested that this RT difference was because of longer nondecision time of decision processes as well as altered evidence accumulation dynamics (lower accumulation rate and higher decision threshold). The nondecision time effect was supported by a delayed onset of the lateralized readiness potential. These results show that both decision and nondecision processes are prolonged when participants need to resort to internal memory representations during value-based decisions.
Collapse
|
13
|
Foster BL, Koslov SR, Aponik-Gremillion L, Monko ME, Hayden BY, Heilbronner SR. A tripartite view of the posterior cingulate cortex. Nat Rev Neurosci 2023; 24:173-189. [PMID: 36456807 PMCID: PMC10041987 DOI: 10.1038/s41583-022-00661-x] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/04/2022] [Indexed: 12/03/2022]
Abstract
The posterior cingulate cortex (PCC) is one of the least understood regions of the cerebral cortex. By contrast, the anterior cingulate cortex has been the subject of intensive investigation in humans and model animal systems, leading to detailed behavioural and computational theoretical accounts of its function. The time is right for similar progress to be made in the PCC given its unique anatomical and physiological properties and demonstrably important contributions to higher cognitive functions and brain diseases. Here, we describe recent progress in understanding the PCC, with a focus on convergent findings across species and techniques that lay a foundation for establishing a formal theoretical account of its functions. Based on this converging evidence, we propose that the broader PCC region contains three major subregions - the dorsal PCC, ventral PCC and retrosplenial cortex - that respectively support the integration of executive, mnemonic and spatial processing systems. This tripartite subregional view reconciles inconsistencies in prior unitary theories of PCC function and offers promising new avenues for progress.
Collapse
Affiliation(s)
- Brett L Foster
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| | - Seth R Koslov
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Lyndsey Aponik-Gremillion
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, USA.,Department of Health Sciences, Dumke College for Health Professionals, Weber State University, Ogden, UT, USA
| | - Megan E Monko
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.,Center for Magnetic Resonance Research and Center for Neural Engineering, University of Minnesota, Minneapolis, MN, USA
| | | |
Collapse
|
14
|
Self-judgment dissected: A computational modeling analysis of self-referential processing and its relationship to trait mindfulness facets and depression symptoms. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:171-189. [PMID: 36168080 PMCID: PMC9931629 DOI: 10.3758/s13415-022-01033-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/29/2022] [Indexed: 11/08/2022]
Abstract
Cognitive theories of depression, and mindfulness theories of well-being, converge on the notion that self-judgment plays a critical role in mental health. However, these theories have rarely been tested via tasks and computational modeling analyses that can disentangle the information processes operative in self-judgments. We applied a drift-diffusion computational model to the self-referential encoding task (SRET) collected before and after an 8-week mindfulness intervention (n = 96). A drift-rate regression parameter representing positive-relative to negative-self-referential judgment strength positively related to mindful awareness and inversely related to depression, both at baseline and over time; however, this parameter did not significantly relate to the interaction between mindful awareness and nonjudgmentalness. At the level of individual depression symptoms, at baseline, a spectrum of symptoms (inversely) correlated with the drift-rate regression parameter, suggesting that many distinct depression symptoms relate to valenced self-judgment between subjects. By contrast, over the intervention, changes in only a smaller subset of anhedonia-related depression symptoms showed substantial relationships with this parameter. Both behavioral and model-derived measures showed modest split-half and test-retest correlations. Results support cognitive theories that implicate self-judgment in depression and mindfulness theories, which imply that mindful awareness should lead to more positive self-views.
Collapse
|
15
|
Ritz H, Frömer R, Shenhav A. Phantom controllers: Misspecified models create the false appearance of adaptive control during value-based choice. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.18.524640. [PMID: 36711762 PMCID: PMC9882254 DOI: 10.1101/2023.01.18.524640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Abstract
Decision scientists have grown increasingly interested in how people adaptively control their decision making. Researchers have demonstrated that parameters governing the accumulation of evidence towards a choice, such as the decision threshold, are shaped by information available prior to or in parallel with one's evaluation of an option set (e.g., recent outcomes or choice conflict). A recent account has taken a bold leap forward in this approach, suggesting that adjustments in decision parameters can be motivated by the value of the options under consideration. This motivated control account predicts that when faced with difficult choices (similarly valued options) under time pressure, people will adaptively lower their decision threshold to ensure that they make a choice in time. This account was supported by drift diffusion modeling of a deadlined choice task, demonstrating that decision thresholds decrease for difficult relative to easy choices. Here, we reanalyze the data from this experiment, and show that evidence for this novel account does not hold up to further scrutiny. Using a more systematic and comprehensive modeling approach, we show that this previously observed threshold adjustment disappears (or even reverses) under a more complete model of the data. Importantly, we further show how this and other apparent evidence for motivated control arises as an artifact of model (mis)specification, where one model's putatively controlled decision process (e.g., value-driven threshold adjustments) can mimic another model's stimulus-driven decision processes (e.g., accumulator competition or collapsing bounds). Collectively, this work reveals crucial insights and constraints in the pursuit of understanding how control guides decision-making, and when it doesn't.
Collapse
Affiliation(s)
- H Ritz
- Cognitive, Linguistic, and Psychological Sciences, Brown University
- Carney Institute for Brain Sciences, Brown University
- Princeton Neuroscience Institute, Princeton University
| | - R Frömer
- Cognitive, Linguistic, and Psychological Sciences, Brown University
- Carney Institute for Brain Sciences, Brown University
- School of Psychology, University of Birmingham
- Centre for Human Brain Health, University of Birmingham
| | - A Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Brown University
- Carney Institute for Brain Sciences, Brown University
| |
Collapse
|
16
|
De Martino B, Cortese A. Goals, usefulness and abstraction in value-based choice. Trends Cogn Sci 2023; 27:65-80. [PMID: 36446707 DOI: 10.1016/j.tics.2022.11.001] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 10/26/2022] [Accepted: 11/01/2022] [Indexed: 11/27/2022]
Abstract
Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.
Collapse
Affiliation(s)
- Benedetto De Martino
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK; Computational Neuroscience Laboratories, ATR Institute International, 619-0288 Kyoto, Japan.
| | - Aurelio Cortese
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK; Computational Neuroscience Laboratories, ATR Institute International, 619-0288 Kyoto, Japan.
| |
Collapse
|
17
|
Social, affective, and non-motoric bodily cues to the Sense of Agency: A systematic review of the experience of control. Neurosci Biobehav Rev 2022; 142:104900. [DOI: 10.1016/j.neubiorev.2022.104900] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Revised: 09/15/2022] [Accepted: 09/29/2022] [Indexed: 10/31/2022]
|
18
|
Bublatzky F, Schellhaas S, Paret C. Aversive anticipations modulate electrocortical correlates of decision-making and reward reversal learning, but not behavioral performance. Front Behav Neurosci 2022; 16:908454. [PMID: 35990730 PMCID: PMC9389167 DOI: 10.3389/fnbeh.2022.908454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Predicting the consequences of one’s own decisions is crucial for organizing future behavior. However, when reward contingencies vary frequently, flexible adaptation of decisions is likely to depend on the situation. We examined the effects of an instructed threat context on choice behavior (i.e., reversal learning) and its electrocortical correlates. In a probabilistic decision-making task, 30 participants had to choose between two options that were either contingent on monetary gains or losses. Reward contingencies were reversed after reaching a probabilistic threshold. Decision-making and reversal learning were examined with two contextual background colors, which were instructed as signals for threat-of-shock or safety. Self-report data confirmed the threat context as more unpleasant, arousing, and threatening relative to safety condition. However, against our expectations, behavioral performance was comparable during the threat and safety conditions (i.e., errors-to-criterion, number of reversal, error rates, and choice times). Regarding electrocortical activity, feedback processing changed throughout the visual processing stream. The feedback-related negativity (FRN) reflected expectancy-driven processing (unexpected vs. congruent losses and gains), and the threat-selective P3 component revealed non-specific discrimination of gains vs. losses. Finally, the late positive potentials (LPP) showed strongly valence-specific processing (unexpected and congruent losses vs. gains). Thus, regardless of contextual threat, early and late cortical activity reflects an attentional shift from expectation- to outcome-based feedback processing. Findings are discussed in terms of reward, threat, and reversal-learning mechanisms with implications for emotion regulation and anxiety disorders.
Collapse
Affiliation(s)
- Florian Bublatzky
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- *Correspondence: Florian Bublatzky,
| | - Sabine Schellhaas
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Christian Paret
- Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health Mannheim, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
- Tel-Aviv Sourasky Medical Center, School of Psychological Sciences, Sagol Brain Institute, Wohl Institute for Advanced Imaging, Tel-Aviv University, Tel Aviv, Israel
- Christian Paret,
| |
Collapse
|
19
|
Kane GA, James MH, Shenhav A, Daw ND, Cohen JD, Aston-Jones G. Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task. J Neurosci 2022; 42:5730-5744. [PMID: 35688627 PMCID: PMC9302469 DOI: 10.1523/jneurosci.1940-21.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Revised: 04/24/2022] [Accepted: 04/28/2022] [Indexed: 01/22/2023] Open
Abstract
In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.
Collapse
Affiliation(s)
- Gary A Kane
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts 02155
| | - Morgan H James
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers University, Piscataway, New Jersey 08854
- Brain Health Institute, Rutgers University, Pisccataway, New Jersey 08854
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, Rhode Island 02912
| | - Nathaniel D Daw
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
| | - Jonathan D Cohen
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
| | - Gary Aston-Jones
- Brain Health Institute, Rutgers University, Pisccataway, New Jersey 08854
| |
Collapse
|
20
|
Lin E, Schmid PC. Does power increase attention to rewards? Examining the brain and behavior. JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY 2022. [DOI: 10.1016/j.jesp.2022.104332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
21
|
Frömer R, Shenhav A. Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making. Neurosci Biobehav Rev 2022; 134:104483. [PMID: 34902441 PMCID: PMC8844247 DOI: 10.1016/j.neubiorev.2021.12.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 12/01/2021] [Accepted: 12/04/2021] [Indexed: 12/26/2022]
Abstract
While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.
Collapse
Affiliation(s)
- R Frömer
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| | - A Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| |
Collapse
|
22
|
High-value decisions are fast and accurate, inconsistent with diminishing value sensitivity. Proc Natl Acad Sci U S A 2022; 119:2101508119. [PMID: 35105801 PMCID: PMC8832986 DOI: 10.1073/pnas.2101508119] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/20/2021] [Indexed: 11/22/2022] Open
Abstract
What information about economic value is incorporated into decision-makers’ choices? Across the decision sciences, several prominent models ignore average value, assuming that only value differences are incorporated into the decision-making process, while others assume diminishing sensitivity to value, suggesting that it should be more difficult to choose between high-value options. Other models suggest that high-value decisions should, if anything, be treated as less important (holding value difference constant). Across three experiments with very different types of choices (food, art, and learned stimuli), we find violations of these predictions. Contrary to expectations, the presence of high-value options makes decisions easier while also inducing more effort to get them right. It is a widely held belief that people’s choices are less sensitive to changes in value as value increases. For example, the subjective difference between $11 and $12 is believed to be smaller than between $1 and $2. This idea is consistent with applications of the Weber-Fechner Law and divisive normalization to value-based choice and with psychological interpretations of diminishing marginal utility. According to random utility theory in economics, smaller subjective differences predict less accurate choices. Meanwhile, in the context of sequential sampling models in psychology, smaller subjective differences also predict longer response times. Based on these models, we would predict decisions between high-value options to be slower and less accurate. In contrast, some have argued on normative grounds that choices between high-value options should be made with less caution, leading to faster and less accurate choices. Here, we model the dynamics of the choice process across three different choice domains, accounting for both discriminability and response caution. Contrary to predictions, we mostly observe faster and more accurate decisions (i.e., higher drift rates) between high-value options. We also observe that when participants are alerted about incoming high-value decisions, they exert more caution and not less. We rule out several explanations for these results, using tasks with both subjective and objective values. These results cast doubt on the notion that increasing value reduces discriminability.
Collapse
|
23
|
Collins AGE, Shenhav A. Advances in modeling learning and decision-making in neuroscience. Neuropsychopharmacology 2022; 47:104-118. [PMID: 34453117 PMCID: PMC8617262 DOI: 10.1038/s41386-021-01126-y] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2021] [Revised: 07/14/2021] [Accepted: 07/22/2021] [Indexed: 02/07/2023]
Abstract
An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.
Collapse
Affiliation(s)
- Anne G E Collins
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA.
| |
Collapse
|
24
|
McDougle SD, Ballard IC, Baribault B, Bishop SJ, Collins AGE. Executive Function Assigns Value to Novel Goal-Congruent Outcomes. Cereb Cortex 2021; 32:231-247. [PMID: 34231854 PMCID: PMC8634563 DOI: 10.1093/cercor/bhab205] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Revised: 05/10/2021] [Accepted: 06/04/2021] [Indexed: 11/14/2022] Open
Abstract
People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from abstract novel outcomes harnesses the same circuitry that supports learning from familiar secondary reinforcers. Behavior and neuroimaging revealed that novel images can act as a substitute for rewards during instrumental learning, producing reliable reward-like signals in dopaminergic circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. These results suggest that learning from novel outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow for human reinforcement learning over arbitrarily abstract reward functions.
Collapse
Affiliation(s)
| | - Ian C Ballard
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
| | - Beth Baribault
- Department of Psychology, University of California, Berkeley, CA 94704, USA
| | - Sonia J Bishop
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
- Department of Psychology, University of California, Berkeley, CA 94704, USA
| | - Anne G E Collins
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA
- Department of Psychology, University of California, Berkeley, CA 94704, USA
| |
Collapse
|
25
|
Pirrone A, Reina A, Stafford T, Marshall JAR, Gobet F. Magnitude-sensitivity: rethinking decision-making. Trends Cogn Sci 2021; 26:66-80. [PMID: 34750080 DOI: 10.1016/j.tics.2021.10.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 10/05/2021] [Accepted: 10/05/2021] [Indexed: 11/25/2022]
Abstract
Magnitude-sensitivity refers to the result that performance in decision-making, across domains and organisms, is affected by the total value of the possible alternatives. This simple result offers a window into fundamental issues in decision-making and has led to a reconsideration of ecological decision-making, prominent computational models of decision-making, and optimal decision-making. Moreover, magnitude-sensitivity has inspired the design of new robotic systems that exploit natural solutions and apply optimal decision-making policies. In this article, we review the key theoretical and empirical results about magnitude-sensitivity and highlight the importance that this phenomenon has for the understanding of decision-making. Furthermore, we discuss open questions and ideas for future research.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK.
| | - Andreagiovanni Reina
- Institute for Interdisciplinary Studies on Artificial Intelligence (IRIDIA), Université Libre de Bruxelles, Brussels, Belgium
| | - Tom Stafford
- Department of Psychology, University of Sheffield, Sheffield, UK
| | | | - Fernand Gobet
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK
| |
Collapse
|
26
|
Bejjani C, Egner T. Evaluating the learning of stimulus-control associations through incidental memory of reinforcement events. J Exp Psychol Learn Mem Cogn 2021; 47:1599-1621. [PMID: 34498904 PMCID: PMC8758512 DOI: 10.1037/xlm0001058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Cognitive control describes the ability to use internal goals to strategically guide how we process and respond to our environment. Changes in the environment lead to adaptation in control strategies. This type of control learning can be observed in performance adjustments in response to varying proportions of easy to hard trials over blocks of trials on classic control tasks. Known as the list-wide proportion congruent (LWPC) effect, increased difficulty is met with enhanced attentional control. Recent research has shown that motivational manipulations may enhance the LWPC effect, but the underlying mechanisms are not yet understood. We manipulated Stroop proportion congruency over blocks of trials and, after each trial, provided participants with either performance-contingent feedback ("correct/incorrect") or noncontingent feedback ("response logged") above trial-unique, task-irrelevant images (reinforcement events). The LWPC task was followed by a surprise recognition memory task, which allowed us to test whether attention to feedback (incidental memory for the images) varies as a function of proportion congruency, time, performance contingency, and individual differences. We replicated a robust LWPC effect in a large sample (N = 402) but observed no differences in behavior between feedback groups. Importantly, the memory data revealed better encoding of feedback images from context-defining trials (e.g., congruent trials in a mostly congruent block), especially early in a new context and in congruent conditions. Individual differences in reward and punishment sensitivity were not strongly associated with control-learning effects. These results suggest that statistical learning of contextual demand may have a larger impact on control learning than individual differences in motivation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Collapse
Affiliation(s)
- Christina Bejjani
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708
- Center for Cognitive Neuroscience, Duke University, Durham, NC 27708
| | - Tobias Egner
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708
- Center for Cognitive Neuroscience, Duke University, Durham, NC 27708
| |
Collapse
|
27
|
Eckstein MK, Wilbrecht L, Collins AGE. What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience. Curr Opin Behav Sci 2021; 41:128-137. [PMID: 34984213 PMCID: PMC8722372 DOI: 10.1016/j.cobeha.2021.06.004] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Reinforcement learning (RL) is a concept that has been invaluable to fields including machine learning, neuroscience, and cognitive science. However, what RL entails differs between fields, leading to difficulties when interpreting and translating findings. After laying out these differences, this paper focuses on cognitive (neuro)science to discuss how we as a field might over-interpret RL modeling results. We too often assume-implicitly-that modeling results generalize between tasks, models, and participant populations, despite negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique (neuro)cognitive processes, a concept we call interpretability, when evidence suggests that they capture different functions across studies and tasks. We conclude that future computational research needs to pay increased attention to implicit assumptions when using RL models, and suggest that a more systematic understanding of contextual factors will help address issues and improve the ability of RL to explain brain and behavior.
Collapse
Affiliation(s)
- Maria K Eckstein
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
| | - Linda Wilbrecht
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| | - Anne G E Collins
- Department of Psychology, UC Berkeley, 2121 Berkeley Way West, Berkeley, 94720, CA, USA
- Helen Wills Neuroscience Institute, UC Berkeley, 175 Li Ka Shing Center, Berkeley, 94720, CA, USA
| |
Collapse
|
28
|
Silston B, Wise T, Qi S, Sui X, Dayan P, Mobbs D. Neural encoding of perceived patch value during competitive and hazardous virtual foraging. Nat Commun 2021; 12:5478. [PMID: 34531399 PMCID: PMC8446065 DOI: 10.1038/s41467-021-25816-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Accepted: 08/26/2021] [Indexed: 11/16/2022] Open
Abstract
Natural observations suggest that in safe environments, organisms avoid competition to maximize gain, while in hazardous environments the most effective survival strategy is to congregate with competition to reduce the likelihood of predatory attack. We probed the extent to which survival decisions in humans follow these patterns, and examined the factors that determined individual-level decision-making. In a virtual foraging task containing changing levels of competition in safe and hazardous patches with virtual predators, we demonstrate that human participants inversely select competition avoidant and risk diluting strategies depending on perceived patch value (PPV), a computation dependent on reward, threat, and competition. We formulate a mathematically grounded quantification of PPV in social foraging environments and show using multivariate fMRI analyses that PPV is encoded by mid-cingulate cortex (MCC) and ventromedial prefrontal cortices (vMPFC), regions that integrate action and value signals. Together, these results suggest humans utilize and integrate multidimensional information to adaptively select patches highest in PPV, and that MCC and vMPFC play a role in adapting to both competitive and predatory threats in a virtual foraging setting. Humans adapt decision strategies in response to environmental demands. Here the authors show that decisions in a virtual foraging task can be modelled based on perceived patch value, which includes reward, competition and threat, and is associated with activity in ventromedial prefrontal and medial cingulate cortices.
Collapse
Affiliation(s)
- Brian Silston
- Columbia University, Department of Psychology, 406 Schermerhorn Hall 1190 Amsterdam Ave., New York, NY, 10027, USA
| | - Toby Wise
- Department of Humanities and Social Sciences and California Institute of Technology, 1200 E California Blvd, HSS 228-77, Pasadena, CA, 91125, USA.,Wellcome Centre for Human Neuroimaging, University College London, London, UK.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Song Qi
- Department of Humanities and Social Sciences and California Institute of Technology, 1200 E California Blvd, HSS 228-77, Pasadena, CA, 91125, USA
| | - Xin Sui
- Department of Humanities and Social Sciences and California Institute of Technology, 1200 E California Blvd, HSS 228-77, Pasadena, CA, 91125, USA
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany.,University of Tübingen, Tübingen, Germany
| | - Dean Mobbs
- Department of Humanities and Social Sciences and California Institute of Technology, 1200 E California Blvd, HSS 228-77, Pasadena, CA, 91125, USA. .,Computation and Neural Systems Program at the California Institute of Technology, 1200 E California Blvd, HSS, 228-77, Pasadena, CA, USA.
| |
Collapse
|
29
|
Comparison of the determinants for positive and negative affect proposed by appraisal theories, goal-directed theories, and predictive processing theories. Curr Opin Behav Sci 2021. [DOI: 10.1016/j.cobeha.2021.03.015] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
30
|
Mental representations distinguish value-based decisions from perceptual decisions. Psychon Bull Rev 2021; 28:1413-1422. [PMID: 33821461 DOI: 10.3758/s13423-021-01911-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/05/2021] [Indexed: 11/08/2022]
Abstract
In our daily lives, we make a wide variety of decisions. One major distinction that has been made is between perceptual decisions and value-based (economic) decisions. We argue that this distinction is ill-defined, because these decisions vary on multiple dimensions. We present an alternative way to categorize decisions, based on two dimensions: subjective versus objective criteria, and evaluation of a stimulus versus a representation. We experimentally study the decision-making process (with eye-tracking) in each of the four resulting categories, using the same stimulus set of food images. Using a combination of individual-level and group-level modeling, we find surprisingly consistent patterns of behavior across the categories. However, we find stronger similarities between the subjective and objective categories, and stronger differences between the stimulus and representation categories.
Collapse
|
31
|
Callaway F, Rangel A, Griffiths TL. Fixation patterns in simple choice reflect optimal information sampling. PLoS Comput Biol 2021; 17:e1008863. [PMID: 33770069 PMCID: PMC8026028 DOI: 10.1371/journal.pcbi.1008863] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2020] [Revised: 04/07/2021] [Accepted: 03/10/2021] [Indexed: 11/24/2022] Open
Abstract
Simple choices (e.g., eating an apple vs. an orange) are made by integrating noisy evidence that is sampled over time and influenced by visual attention; as a result, fluctuations in visual attention can affect choices. But what determines what is fixated and when? To address this question, we model the decision process for simple choice as an information sampling problem, and approximate the optimal sampling policy. We find that it is optimal to sample from options whose value estimates are both high and uncertain. Furthermore, the optimal policy provides a reasonable account of fixations and choices in binary and trinary simple choice, as well as the differences between the two cases. Overall, the results show that the fixation process during simple choice is influenced dynamically by the value estimates computed during the decision process, in a manner consistent with optimal information sampling.
Collapse
Affiliation(s)
- Frederick Callaway
- Department of Psychology, Princeton University, Princeton, New Jersey, United States of America
| | - Antonio Rangel
- Departments of Humanities and Social Sciences and Computation and Neural Systems, California Institute of Technology, Pasadena, California, United States of America
| | - Thomas L. Griffiths
- Department of Psychology, Princeton University, Princeton, New Jersey, United States of America
- Department of Computer Science, Princeton University, Princeton, New Jersey, United States of America
| |
Collapse
|
32
|
Mormann M, Russo JE. Does Attention Increase the Value of Choice Alternatives? Trends Cogn Sci 2021; 25:305-315. [PMID: 33549495 DOI: 10.1016/j.tics.2021.01.004] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Revised: 01/10/2021] [Accepted: 01/11/2021] [Indexed: 12/17/2022]
Abstract
A growing recognition of the role of attention in decision-making has been driven by both the technology of eye tracking and the development of models that explicitly incorporate attention. One result of this convergence is the arresting claim that attention, by itself, can increase the perceived value of a decision alternative. In this review, we cover the origins of that claim, its empirical foundation, and the reasoning that supports it. The conclusion is that, to date, there is not sufficient evidence to support the claim. Alternative explanations for the extant evidentiary base are discussed, as is the balance between the bottom-up influence of empirical evidence and the top-down commitment to a conceptual framework.
Collapse
Affiliation(s)
- Milica Mormann
- Cox School of Business, Southern Methodist University, Dallas, TX 75205, USA.
| | - J Edward Russo
- S.C. Johnson College of Business, Cornell University, Ithaca, NY 14853, USA
| |
Collapse
|
33
|
Sepulveda P, Usher M, Davies N, Benson AA, Ortoleva P, De Martino B. Visual attention modulates the integration of goal-relevant evidence and not value. eLife 2020; 9:e60705. [PMID: 33200982 PMCID: PMC7723413 DOI: 10.7554/elife.60705] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 11/16/2020] [Indexed: 11/30/2022] Open
Abstract
When choosing between options, such as food items presented in plain view, people tend to choose the option they spend longer looking at. The prevailing interpretation is that visual attention increases value. However, in previous studies, 'value' was coupled to a behavioural goal, since subjects had to choose the item they preferred. This makes it impossible to discern if visual attention has an effect on value, or, instead, if attention modulates the information most relevant for the goal of the decision-maker. Here, we present the results of two independent studies-a perceptual and a value-based task-that allow us to decouple value from goal-relevant information using specific task-framing. Combining psychophysics with computational modelling, we show that, contrary to the current interpretation, attention does not boost value, but instead it modulates goal-relevant information. This work provides a novel and more general mechanism by which attention interacts with choice.
Collapse
Affiliation(s)
- Pradyumna Sepulveda
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| | - Marius Usher
- School of Psychological Sciences and Sagol School of Neuroscience, Tel Aviv UniversityTel AvivIsrael
| | - Ned Davies
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| | - Amy A Benson
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
| | - Pietro Ortoleva
- Department of Economics and Woodrow Wilson School, Princeton UniversityPrincetonUnited States
| | - Benedetto De Martino
- Institute of Cognitive Neuroscience, University College LondonLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| |
Collapse
|
34
|
Abstract
Momentary feelings of confidence accompany many of our actions and decisions. In addition to such “local” feelings of confidence, we also construct “global” confidence estimates about our skills and abilities (global self-performance estimates or SPEs). Distorted SPEs may have a pervasive impact on motivation and self-evaluation, for instance affecting estimates of our competitiveness at work or in a sports team. Here, we found that components of a brain network previously implicated in the tracking of local confidence was additionally modulated by SPE level, whereas ventral striatum tracked SPEs irrespective of confidence. Our findings of a neurocognitive basis for global SPEs lay the groundwork for understanding how distorted SPEs arise in educational and clinical settings. Humans create metacognitive beliefs about their performance across many levels of abstraction—from local confidence in individual decisions to global estimates of our skills and abilities. Despite a rich literature on the neural basis of local confidence judgements, how global self-performance estimates (SPEs) are constructed remains unknown. Using functional magnetic resonance imaging, we scanned human subjects while they performed several short blocks of tasks and reported on which task they think they performed best, providing a behavioral proxy for global SPEs. In a frontoparietal network sensitive to fluctuations in local confidence, we found that activity within ventromedial prefrontal cortex and precuneus was additionally modulated by global SPEs. In contrast, activity in ventral striatum was associated with subjects’ global SPEs irrespective of fluctuations in local confidence, and predicted the extent to which global SPEs tracked objective task difficulty across individuals. Our findings reveal neural representations of global SPEs that go beyond the tracking of local confidence, and lay the groundwork for understanding how a formation of global self-beliefs may go awry in conditions characterized by distorted self-evaluation.
Collapse
|
35
|
Mow JL, Gandhi A, Fulford D. Imaging the "social brain" in schizophrenia: A systematic review of neuroimaging studies of social reward and punishment. Neurosci Biobehav Rev 2020; 118:704-722. [PMID: 32841653 DOI: 10.1016/j.neubiorev.2020.08.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2020] [Revised: 06/20/2020] [Accepted: 08/07/2020] [Indexed: 10/23/2022]
Abstract
Decreased social functioning and high levels of loneliness and social isolation are common in schizophrenia spectrum disorders (SSD), contributing to reduced quality of life. One key contributor to social impairment is low social motivation, which may stem from aberrant neural processing of socially rewarding or punishing stimuli. To summarize research on the neurobiology of social motivation in SSD, we performed a systematic literature review of neuroimaging studies involving the presentation of social stimuli intended to elicit feelings of reward and/or punishment. Across 11 studies meeting criteria, people with SSD demonstrated weaker modulation of brain activity in regions within a proposed social interaction network, including prefrontal, cingulate, and striatal regions, as well as the amygdala and insula. Firm conclusions regarding neural differences in SSD in these regions, as well as connections within networks, are limited due to conceptual and methodological inconsistencies across the available studies. We conclude by making recommendations for the study of social reward and punishment processing in SSD in future research.
Collapse
Affiliation(s)
- Jessica L Mow
- Department of Psychological & Brain Sciences, Boston University, 64 Cummington Mall, Boston, MA, 02215, United States.
| | - Arti Gandhi
- Sargent College of Health and Rehabilitation Sciences, Boston University, 635 Commonwealth Avenue, Boston, MA, 02215, United States
| | - Daniel Fulford
- Department of Psychological & Brain Sciences, Boston University, 64 Cummington Mall, Boston, MA, 02215, United States; Sargent College of Health and Rehabilitation Sciences, Boston University, 635 Commonwealth Avenue, Boston, MA, 02215, United States
| |
Collapse
|
36
|
Vaidya AR, Fellows LK. Under construction: ventral and lateral frontal lobe contributions to value-based decision-making and learning. F1000Res 2020; 9:F1000 Faculty Rev-158. [PMID: 32161644 PMCID: PMC7050269 DOI: 10.12688/f1000research.21946.1] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/19/2020] [Indexed: 11/20/2022] Open
Abstract
Even apparently simple choices, like selecting a dessert in a pastry shop, involve options characterized by multiple motivationally relevant attributes. Neuroeconomic research suggests that the human brain may track the subjective value of such options, allowing disparate reward-predictive information to be compared in a common currency. However, the brain mechanisms involved in identifying value-predictive features and combining these to assess the value of each decision option remain unclear. Here, we review recent evidence from studies of multi-attribute decision-making in people with focal frontal lobe damage and in healthy people undergoing functional magnetic resonance imaging. This work suggests that ventromedial and lateral prefrontal cortex and orbitofrontal cortex are important for forming value judgments under conditions of complexity. We discuss studies supporting the involvement of these regions in selecting among and evaluating option attributes during value judgment and decision-making and when learning from reward feedback. These findings are consistent with roles for these regions in guiding value construction. They argue for a more nuanced understanding of how ventral and lateral prefrontal cortex contribute to discovering and recognizing value, processes that are required under the complex conditions typical of many everyday decisions.
Collapse
Affiliation(s)
- Avinash R Vaidya
- Department of Cognitive, Linguistic and Psychological Studies, Brown University, Providence, RI, USA
| | - Lesley K Fellows
- Montreal Neurological Institute, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada
| |
Collapse
|