1
|
Zylberberg A, Bakkour A, Shohamy D, Shadlen MN. Value construction through sequential sampling explains serial dependencies in decision making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.13.575363. [PMID: 39416151 PMCID: PMC11482742 DOI: 10.1101/2024.01.13.575363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
Many decisions are expressed as a preference for one item over another. When these items are familiar, it is often assumed that the decision maker assigns a value to each of the items and chooses the item with the highest value. These values may be imperfectly recalled, but are assumed to be stable over the course of an interview or psychological experiment. Choices that are inconsistent with a stated valuation are thought to occur because of unspecified noise that corrupts the neural representation of value. Assuming that the noise is uncorrelated over time, the pattern of choices and response times in value-based decisions are modeled within the framework of Bounded Evidence Accumulation (BEA), similar to that used in perceptual decision-making. In BEA, noisy evidence samples accumulate over time until the accumulated evidence for one of the options reaches a threshold. Here, we argue that the assumption of temporally uncorrelated noise, while reasonable for perceptual decisions, is not reasonable for value-based decisions. Subjective values depend on the internal state of the decision maker, including their desires, needs, priorities, attentional state, and goals. These internal states may change over time, or undergo revaluation, as will the subjective values. We reasoned that these hypothetical value changes should be detectable in the pattern of choices made over a sequence of decisions. We reanalyzed data from a well-studied task in which participants were presented with pairs of snacks and asked to choose the one they preferred. Using a novel algorithm (Reval), we show that the subjective value of the items changes significantly during a short experimental session (about 1 hour). Values derived with Reval explain choice and response time better than explicitly stated values. They also better explain the BOLD signal in the ventromedial prefrontal cortex, known to represent the value of decision alternatives. Revaluation is also observed in a BEA model in which successive evidence samples are not assumed to be independent. We argue that revaluation is a consequence of the process by which values are constructed during deliberation to resolve preference choices.
Collapse
Affiliation(s)
- Ariel Zylberberg
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States
| | - Akram Bakkour
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States
- Department of Psychology, University of Chicago, Illinois, United States
- Neuroscience Institute, University of Chicago, Illinois, United States
| | - Daphna Shohamy
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States
- Department of Neuroscience, Columbia University, New York, United States
- The Kavli Institute for Brain Science, Columbia University, New York, United States
| | - Michael N Shadlen
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia University, New York, United States
- Department of Neuroscience, Columbia University, New York, United States
- The Kavli Institute for Brain Science, Columbia University, New York, United States
- Howard Hughes Medical Institute, Chevy Chase, United States
| |
Collapse
|
2
|
Cecchini G, DePass M, Baspinar E, Andujar M, Ramawat S, Pani P, Ferraina S, Destexhe A, Moreno-Bote R, Cos I. Cognitive mechanisms of learning in sequential decision-making under uncertainty: an experimental and theoretical approach. Front Behav Neurosci 2024; 18:1399394. [PMID: 39188591 PMCID: PMC11346247 DOI: 10.3389/fnbeh.2024.1399394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Accepted: 07/19/2024] [Indexed: 08/28/2024] Open
Abstract
Learning to make adaptive decisions involves making choices, assessing their consequence, and leveraging this assessment to attain higher rewarding states. Despite vast literature on value-based decision-making, relatively little is known about the cognitive processes underlying decisions in highly uncertain contexts. Real world decisions are rarely accompanied by immediate feedback, explicit rewards, or complete knowledge of the environment. Being able to make informed decisions in such contexts requires significant knowledge about the environment, which can only be gained via exploration. Here we aim at understanding and formalizing the brain mechanisms underlying these processes. To this end, we first designed and performed an experimental task. Human participants had to learn to maximize reward while making sequences of decisions with only basic knowledge of the environment, and in the absence of explicit performance cues. Participants had to rely on their own internal assessment of performance to reveal a covert relationship between their choices and their subsequent consequences to find a strategy leading to the highest cumulative reward. Our results show that the participants' reaction times were longer whenever the decision involved a future consequence, suggesting greater introspection whenever a delayed value had to be considered. The learning time varied significantly across participants. Second, we formalized the neurocognitive processes underlying decision-making within this task, combining mean-field representations of competing neural populations with a reinforcement learning mechanism. This model provided a plausible characterization of the brain dynamics underlying these processes, and reproduced each aspect of the participants' behavior, from their reaction times and choices to their learning rates. In summary, both the experimental results and the model provide a principled explanation to how delayed value may be computed and incorporated into the neural dynamics of decision-making, and to how learning occurs in these uncertain scenarios.
Collapse
Affiliation(s)
- Gloria Cecchini
- Facultat de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Spain
- Center for Brain and Cognition, DTIC, Universitat Pompeu Fabra, Barcelona, Spain
| | - Michael DePass
- Center for Brain and Cognition, DTIC, Universitat Pompeu Fabra, Barcelona, Spain
| | - Emre Baspinar
- CNRS, Institute of Neuroscience (NeuroPSI), Paris-Saclay University, Saclay, France
| | - Marta Andujar
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Surabhi Ramawat
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Pierpaolo Pani
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Stefano Ferraina
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Alain Destexhe
- CNRS, Institute of Neuroscience (NeuroPSI), Paris-Saclay University, Saclay, France
| | - Rubén Moreno-Bote
- Center for Brain and Cognition, DTIC, Universitat Pompeu Fabra, Barcelona, Spain
- Serra-Hunter Fellow Programme, Barcelona, Spain
| | - Ignasi Cos
- Facultat de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Spain
- Serra-Hunter Fellow Programme, Barcelona, Spain
| |
Collapse
|
3
|
Feng YY, Bromberg-Martin ES, Monosov IE. Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making. Cell Rep 2024; 43:114341. [PMID: 38878290 DOI: 10.1016/j.celrep.2024.114341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 03/27/2024] [Accepted: 05/23/2024] [Indexed: 06/25/2024] Open
Abstract
The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.
Collapse
Affiliation(s)
- Yang-Yang Feng
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA; Department of Biomedical Engineering, Washington University, St. Louis, MO, USA
| | | | - Ilya E Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA; Department of Biomedical Engineering, Washington University, St. Louis, MO, USA; Washington University Pain Center, Washington University, St. Louis, MO, USA; Department of Neurosurgery, Washington University, St. Louis, MO, USA; Department of Electrical Engineering, Washington University, St. Louis, MO, USA.
| |
Collapse
|
4
|
Martinez CA, Pantazopoulos H, Gisabella B, Stephens ET, Garteiser J, Del Arco A. Choice impulsivity after repeated social stress is associated with increased perineuronal nets in the medial prefrontal cortex. Sci Rep 2024; 14:7093. [PMID: 38528075 DOI: 10.1038/s41598-024-57599-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2024] [Accepted: 03/20/2024] [Indexed: 03/27/2024] Open
Abstract
Repeated stress can predispose to substance abuse. However, behavioral and neurobiological adaptations that link stress to substance abuse remain unclear. This study investigates whether intermittent social defeat (ISD), a stress protocol that promotes drug-seeking behavior, alters intertemporal decision-making and cortical inhibitory function in the medial prefrontal cortex (mPFC). Male long evans rats were trained in a delay discounting task (DDT) where rats make a choice between a fast (1 s) small reward (1 sugar pellet) and a large reward (3 sugar pellets) that comes with a time delay (10 s or 20 s). A decreased preference for delayed rewards was used as an index of choice impulsivity. Rats were exposed to ISD and tested in the DDT 24 h after each stress episode, and one- and two-weeks after the last stress episode. Immunohistochemistry was performed in rat's brains to evaluate perineuronal nets (PNNs) and parvalbumin GABA interneurons (PV) labeling as markers of inhibitory function in mPFC. ISD significantly decreased the preference for delayed large rewards in low impulsive, but not high impulsive, animals. ISD also increased the density of PNNs in the mPFC. These results suggest that increased choice impulsivity and cortical inhibition predispose animals to seek out rewards after stress.
Collapse
Affiliation(s)
| | - Harry Pantazopoulos
- Department of Psychiatry and Human Behavior, Medical School, University of Mississippi Medical Center, Jackson, MS, USA
| | - Barbara Gisabella
- Department of Psychiatry and Human Behavior, Medical School, University of Mississippi Medical Center, Jackson, MS, USA
| | - Emily T Stephens
- Department of Psychiatry and Human Behavior, Medical School, University of Mississippi Medical Center, Jackson, MS, USA
| | - Jacob Garteiser
- Department of Psychiatry and Human Behavior, Medical School, University of Mississippi Medical Center, Jackson, MS, USA
| | - Alberto Del Arco
- HESRM, School of Applied Sciences, University of Mississippi, Oxford, MS, USA.
- Department of Psychiatry and Human Behavior, Medical School, University of Mississippi Medical Center, Jackson, MS, USA.
| |
Collapse
|
5
|
Wenzel JM, Zlebnik NE, Patton MH, Smethells JR, Ayvazian VM, Dantrassy HM, Zhang LY, Mathur BN, Cheer JF. Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity. Neuropsychopharmacology 2023; 48:1821-1831. [PMID: 37208501 PMCID: PMC10579332 DOI: 10.1038/s41386-023-01604-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 05/02/2023] [Accepted: 05/05/2023] [Indexed: 05/21/2023]
Abstract
Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.
Collapse
Affiliation(s)
- Jennifer M Wenzel
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA.
- Department of Psychological Sciences, University of San Diego, San Diego, CA, 92110, USA.
| | - Natalie E Zlebnik
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA.
- Division of Biomedical Sciences, University of California, Riverside School of Medicine, Riverside, CA, 92521, USA.
| | - Mary H Patton
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
- St. Jude Children's Research Hospital, Memphis, TN, 38105, USA
| | - John R Smethells
- Hennepin Healthcare Research Institute, Minneapolis, MN, 55404, USA
| | - Victoria M Ayvazian
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Hannah M Dantrassy
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Lan-Yuan Zhang
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Brian N Mathur
- Department of Psychiatry, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
- Department of Pharmacology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| | - Joseph F Cheer
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
- Department of Psychiatry, University of Maryland School of Medicine, Baltimore, MD, 21201, USA
| |
Collapse
|
6
|
Feng YY, Bromberg-Martin ES, Monosov IE. Dorsal raphe neurons signal integrated value during multi-attribute decision-making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.17.553745. [PMID: 37662243 PMCID: PMC10473596 DOI: 10.1101/2023.08.17.553745] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.
Collapse
Affiliation(s)
- Yang-Yang Feng
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
| | | | - Ilya E. Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
- Washington University Pain Center, Washington University, St. Louis, Missouri, USA
- Department of Neurosurgery, Washington University, St. Louis, Missouri, USA
- Department of Electrical Engineering, Washington University, St. Louis, Missouri, USA
| |
Collapse
|
7
|
Newton-Fenner A, Hewitt D, Henderson J, Roberts H, Mari T, Gu Y, Gorelkina O, Giesbrecht T, Fallon N, Roberts C, Stancak A. Economic value in the Brain: A meta-analysis of willingness-to-pay using the Becker-DeGroot-Marschak auction. PLoS One 2023; 18:e0286969. [PMID: 37428744 DOI: 10.1371/journal.pone.0286969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Accepted: 05/29/2023] [Indexed: 07/12/2023] Open
Abstract
Forming and comparing subjective values (SVs) of choice options is a critical stage of decision-making. Previous studies have highlighted a complex network of brain regions involved in this process by utilising a diverse range of tasks and stimuli, varying in economic, hedonic and sensory qualities. However, the heterogeneity of tasks and sensory modalities may systematically confound the set of regions mediating the SVs of goods. To identify and delineate the core brain valuation system involved in processing SV, we utilised the Becker-DeGroot-Marschak (BDM) auction, an incentivised demand-revealing mechanism which quantifies SV through the economic metric of willingness-to-pay (WTP). A coordinate-based activation likelihood estimation meta-analysis analysed twenty-four fMRI studies employing a BDM task (731 participants; 190 foci). Using an additional contrast analysis, we also investigated whether this encoding of SV would be invariant to the concurrency of auction task and fMRI recordings. A fail-safe number analysis was conducted to explore potential publication bias. WTP positively correlated with fMRI-BOLD activations in the left ventromedial prefrontal cortex with a sub-cluster extending into anterior cingulate cortex, bilateral ventral striatum, right dorsolateral prefrontal cortex, right inferior frontal gyrus, and right anterior insula. Contrast analysis identified preferential engagement of the mentalizing-related structures in response to concurrent scanning. Together, our findings offer succinct empirical support for the core structures participating in the formation of SV, separate from the hedonic aspects of reward and evaluated in terms of WTP using BDM, and show the selective involvement of inhibition-related brain structures during active valuation.
Collapse
Affiliation(s)
- Alice Newton-Fenner
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
- Institute of Risk and Uncertainty, University of Liverpool, Liverpool, United Kingdom
| | - Danielle Hewitt
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom
| | - Jessica Henderson
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
| | - Hannah Roberts
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
| | - Tyler Mari
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
| | - Yiquan Gu
- Henley Business School, University of Reading, Reading, United Kingdom
| | - Olga Gorelkina
- Management School, University of Liverpool, Liverpool, United Kingdom
| | - Timo Giesbrecht
- Unilever, Research and Development, Port Sunlight, United Kingdom
| | - Nicolas Fallon
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
| | - Carl Roberts
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
| | - Andrej Stancak
- Department of Psychology, University of Liverpool, Liverpool, United Kingdom
- Institute of Risk and Uncertainty, University of Liverpool, Liverpool, United Kingdom
| |
Collapse
|
8
|
Pasquereau B, Turner RS. Neural dynamics underlying self-control in the primate subthalamic nucleus. eLife 2023; 12:e83971. [PMID: 37204300 PMCID: PMC10259453 DOI: 10.7554/elife.83971] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 05/18/2023] [Indexed: 05/20/2023] Open
Abstract
The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost-benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.
Collapse
Affiliation(s)
- Benjamin Pasquereau
- Institut des Sciences Cognitives Marc Jeannerod, UMR 5229, Centre National de la Recherche Scientifique, 69675 Bron CedexBronFrance
- Université Claude Bernard Lyon 1, 69100 VilleurbanneVilleurbanneFrance
| | - Robert S Turner
- Department of Neurobiology, Center for Neuroscience and The Center for the Neural Basis of Cognition, University of PittsburghPittsburghUnited States
| |
Collapse
|
9
|
Yu L, Gao J, Kong Y, Huang L. Impact of perceived scarcity on delay of gratification: meditation effects of self-efficacy and self-control. CURRENT PSYCHOLOGY 2023:1-9. [PMID: 37359664 PMCID: PMC10031180 DOI: 10.1007/s12144-023-04455-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/22/2023] [Indexed: 03/24/2023]
Abstract
Most people worldwide suffer from resource scarcity in their lives. Perceived scarcity plays an important role in cognitive abilities and decision-making. This study employed the perceived scarcity, self-control, self-efficacy, and delayed gratification scales to explore the relationship among perceived scarcity, self-efficacy, self-control, and delayed gratification as well as the mediating role of self-efficacy and self-control between perceived scarcity and delayed gratification. A cross-sectional survey was conducted via an online survey platform with 1,109 Chinese college students. The results showed that perceived scarcity was negatively correlated with individual self-efficacy, self-control, and delayed gratification, while self-efficacy and self-control played a partial parallel-mediated role between perceived scarcity and delayed gratification. The mediation model accounted for 28% of the variance in delayed gratification. Moreover, the results indicated that perceived scarcity can reduce the delay in gratification through its negative impact on individual self-efficacy and self-control. To some extent, this result explains how perceived scarcity delays gratification from the perspective of motivation and cognition and provides support for further research on the intervention of perceived scarcity's psychological and behavioral consequences.
Collapse
Affiliation(s)
- Liang Yu
- School of Humanities and Management, Wannan Medical College, Wuhu, China
| | - Jie Gao
- School of Public Fundamentals, Wannan Medical College, Wuhu, China
| | - Yan Kong
- Department of Psychology, University of Science and Technology of China, Hefei, China
| | - Long Huang
- School of Humanities and Management, Wannan Medical College, Wuhu, China
| |
Collapse
|
10
|
Suhaimi A, Lim AWH, Chia XW, Li C, Makino H. Representation learning in the artificial and biological neural networks underlying sensorimotor integration. SCIENCE ADVANCES 2022; 8:eabn0984. [PMID: 35658033 PMCID: PMC9166289 DOI: 10.1126/sciadv.abn0984] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 04/18/2022] [Indexed: 06/15/2023]
Abstract
The integration of deep learning and theories of reinforcement learning (RL) is a promising avenue to explore novel hypotheses on reward-based learning and decision-making in humans and other animals. Here, we trained deep RL agents and mice in the same sensorimotor task with high-dimensional state and action space and studied representation learning in their respective neural networks. Evaluation of thousands of neural network models with extensive hyperparameter search revealed that learning-dependent enrichment of state-value and policy representations of the task-performance-optimized deep RL agent closely resembled neural activity of the posterior parietal cortex (PPC). These representations were critical for the task performance in both systems. PPC neurons also exhibited representations of the internally defined subgoal, a feature of deep RL algorithms postulated to improve sample efficiency. Such striking resemblance between the artificial and biological networks and their functional convergence in sensorimotor integration offers new opportunities to better understand respective intelligent systems.
Collapse
|
11
|
Padoa-Schioppa C. Logistic analysis of choice data: A primer. Neuron 2022; 110:1615-1630. [PMID: 35334232 PMCID: PMC9119943 DOI: 10.1016/j.neuron.2022.03.002] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2022] [Revised: 02/25/2022] [Accepted: 02/28/2022] [Indexed: 11/21/2022]
Abstract
Logistic regressions were developed in economics to model individual choice behavior. In recent years, they have become an important tool in decision neuroscience. Here, I describe and discuss different logistic models, emphasizing the underlying assumptions and possible interpretations. Logistic models may be used to quantify a variety of behavioral traits, including the relative subjective value of different goods, the choice accuracy, risk attitudes, and choice biases. More complex logistic models can be used for choices between good bundles, in cases of nonlinear value functions, and for choices between multiple options. Finally, logistic models can quantify the explanatory power of neuronal activity on choices, thus providing a valid alternative to receiver operating characteristic (ROC) analyses.
Collapse
Affiliation(s)
- Camillo Padoa-Schioppa
- Department of Neuroscience, Department of Economics, and Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO 63110, USA.
| |
Collapse
|
12
|
Shi W, Ballesta S, Padoa-Schioppa C. Neuronal origins of reduced accuracy and biases in economic choices under sequential offers. eLife 2022; 11:e75910. [PMID: 35416775 PMCID: PMC9045815 DOI: 10.7554/elife.75910] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Accepted: 04/08/2022] [Indexed: 02/03/2023] Open
Abstract
Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here, we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in the orbitofrontal cortex revealed that these phenomena emerged at different computational stages. Lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey.
Collapse
Affiliation(s)
- Weikang Shi
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Sebastien Ballesta
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
| | - Camillo Padoa-Schioppa
- Department of Neuroscience, Washington University in St. LouisSt. LouisUnited States
- Department of Economics, Washington University in St. LouisSt. LouisUnited States
- Department of Biomedical Engineering, Washington University in St. LouisSt. LouisUnited States
| |
Collapse
|
13
|
Subramanian A, Chitlangia S, Baths V. Reinforcement learning and its connections with neuroscience and psychology. Neural Netw 2021; 145:271-287. [PMID: 34781215 DOI: 10.1016/j.neunet.2021.10.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 09/26/2021] [Accepted: 10/01/2021] [Indexed: 11/19/2022]
Abstract
Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.
Collapse
Affiliation(s)
- Ajay Subramanian
- Department of Psychology, New York University, New York, New York, 10003, USA; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Sharad Chitlangia
- Amazon; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Veeky Baths
- Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India; Department of Biological Sciences, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| |
Collapse
|
14
|
Meirhaeghe N, Sohn H, Jazayeri M. A precise and adaptive neural mechanism for predictive temporal processing in the frontal cortex. Neuron 2021; 109:2995-3011.e5. [PMID: 34534456 PMCID: PMC9737059 DOI: 10.1016/j.neuron.2021.08.025] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Revised: 07/02/2021] [Accepted: 08/18/2021] [Indexed: 12/14/2022]
Abstract
The theory of predictive processing posits that the brain computes expectations to process information predictively. Empirical evidence in support of this theory, however, is scarce and largely limited to sensory areas. Here, we report a precise and adaptive mechanism in the frontal cortex of non-human primates consistent with predictive processing of temporal events. We found that the speed of neural dynamics is precisely adjusted according to the average time of an expected stimulus. This speed adjustment, in turn, enables neurons to encode stimuli in terms of deviations from expectation. This lawful relationship was evident across multiple experiments and held true during learning: when temporal statistics underwent covert changes, neural responses underwent predictable changes that reflected the new mean. Together, these results highlight a precise mathematical relationship between temporal statistics in the environment and neural activity in the frontal cortex that may serve as a mechanism for predictive temporal processing.
Collapse
Affiliation(s)
- Nicolas Meirhaeghe
- Harvard-MIT Division of Health Sciences & Technology, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| | - Hansem Sohn
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA,Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
| |
Collapse
|
15
|
Bennett MS. Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans. Front Neuroanat 2021; 15:693346. [PMID: 34489649 PMCID: PMC8418099 DOI: 10.3389/fnana.2021.693346] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2021] [Accepted: 07/13/2021] [Indexed: 11/13/2022] Open
Abstract
Retracing the evolutionary steps by which human brains evolved can offer insights into the underlying mechanisms of human brain function as well as the phylogenetic origin of various features of human behavior. To this end, this article presents a model for interpreting the physical and behavioral modifications throughout major milestones in human brain evolution. This model introduces the concept of a "breakthrough" as a useful tool for interpreting suites of brain modifications and the various adaptive behaviors these modifications enabled. This offers a unique view into the ordered steps by which human brains evolved and suggests several unique hypotheses on the mechanisms of human brain function.
Collapse
|
16
|
Hori Y, Mimura K, Nagai Y, Fujimoto A, Oyama K, Kikuchi E, Inoue KI, Takada M, Suhara T, Richmond BJ, Minamimoto T. Single caudate neurons encode temporally discounted value for formulating motivation for action. eLife 2021; 10:e61248. [PMID: 34328413 PMCID: PMC8352586 DOI: 10.7554/elife.61248] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2020] [Accepted: 07/29/2021] [Indexed: 11/27/2022] Open
Abstract
The term 'temporal discounting' describes both choice preferences and motivation for delayed rewards. Here we show that neuronal activity in the dorsal part of the primate caudate head (dCDh) signals the temporally discounted value needed to compute the motivation for delayed rewards. Macaque monkeys performed an instrumental task, in which visual cues indicated the forthcoming size and delay duration before reward. Single dCDh neurons represented the temporally discounted value without reflecting changes in the animal's physiological state. Bilateral pharmacological or chemogenetic inactivation of dCDh markedly distorted the normal task performance based on the integration of reward size and delay, but did not affect the task performance for different reward sizes without delay. These results suggest that dCDh is involved in encoding the integrated multi-dimensional information critical for motivation.
Collapse
Affiliation(s)
- Yukiko Hori
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Koki Mimura
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Yuji Nagai
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Atsushi Fujimoto
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Kei Oyama
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Erika Kikuchi
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Ken-ichi Inoue
- Systems Neuroscience Section, Primate Research Institute, Kyoto UniversityInuyamaJapan
| | - Masahiko Takada
- Systems Neuroscience Section, Primate Research Institute, Kyoto UniversityInuyamaJapan
| | - Tetsuya Suhara
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| | - Barry J Richmond
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Department of Health and Human ServicesBethesdaUnited States
| | - Takafumi Minamimoto
- Department of Functional Brain Imaging, National Institutes for Quantum and Radiological Science and TechnologyChibaJapan
| |
Collapse
|
17
|
Bennett MS. What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence. Front Psychol 2021; 12:685853. [PMID: 34393912 PMCID: PMC8358274 DOI: 10.3389/fpsyg.2021.685853] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 06/16/2021] [Indexed: 01/24/2023] Open
Abstract
This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.
Collapse
|
18
|
Zhang B, Kan JYY, Yang M, Wang X, Tu J, Dorris MC. Transforming absolute value to categorical choice in primate superior colliculus during value-based decision making. Nat Commun 2021; 12:3410. [PMID: 34099726 PMCID: PMC8184840 DOI: 10.1038/s41467-021-23747-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 04/23/2021] [Indexed: 11/09/2022] Open
Abstract
Value-based decision making involves choosing from multiple options with different values. Despite extensive studies on value representation in various brain regions, the neural mechanism for how multiple value options are converted to motor actions remains unclear. To study this, we developed a multi-value foraging task with varying menu of items in non-human primates using eye movements that dissociates value and choice, and conducted electrophysiological recording in the midbrain superior colliculus (SC). SC neurons encoded "absolute" value, independent of available options, during late fixation. In addition, SC neurons also represent value threshold, modulated by available options, different from conventional motor threshold. Electrical stimulation of SC neurons biased choices in a manner predicted by the difference between the value representation and the value threshold. These results reveal a neural mechanism directly transforming absolute values to categorical choices within SC, supporting highly efficient value-based decision making critical for real-world economic behaviors.
Collapse
Affiliation(s)
- Beizhen Zhang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China.
- University of Chinese Academy of Sciences, Beijing, China.
| | - Janis Ying Ying Kan
- Department of Biomedical and Molecular Sciences, Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada
| | - Mingpo Yang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Xiaochun Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Jiahao Tu
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Michael Christopher Dorris
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
19
|
Hunt LT. Frontal circuit specialisations for decision making. Eur J Neurosci 2021; 53:3654-3671. [PMID: 33864305 DOI: 10.1111/ejn.15236] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2021] [Revised: 03/15/2021] [Accepted: 04/04/2021] [Indexed: 11/29/2022]
Abstract
There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.
Collapse
Affiliation(s)
- Laurence T Hunt
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
20
|
Shin EJ, Jang Y, Kim S, Kim H, Cai X, Lee H, Sul JH, Lee SH, Chung Y, Lee D, Jung MW. Robust and distributed neural representation of action values. eLife 2021; 10:53045. [PMID: 33876728 PMCID: PMC8104958 DOI: 10.7554/elife.53045] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Accepted: 04/19/2021] [Indexed: 11/13/2022] Open
Abstract
Studies in rats, monkeys, and humans have found action-value signals in multiple regions of the brain. These findings suggest that action-value signals encoded in these brain structures bias choices toward higher expected rewards. However, previous estimates of action-value signals might have been inflated by serial correlations in neural activity and also by activity related to other decision variables. Here, we applied several statistical tests based on permutation and surrogate data to analyze neural activity recorded from the striatum, frontal cortex, and hippocampus. The results show that previously identified action-value signals in these brain areas cannot be entirely accounted for by concurrent serial correlations in neural activity and action value. We also found that neural activity related to action value is intermixed with signals related to other decision variables. Our findings provide strong evidence for broadly distributed neural signals related to action value throughout the brain.
Collapse
Affiliation(s)
- Eun Ju Shin
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Yunsil Jang
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Soyoun Kim
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea
| | - Hoseok Kim
- Department of Neuroscience, Biomedicum, Karolinska Institutet, Stockholm, Sweden
| | - Xinying Cai
- New York University Shanghai, NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, and Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
| | - Hyunjung Lee
- Department of Anatomy, Kyungpook National University School of Medicine, Daegu, Republic of Korea
| | - Jung Hoon Sul
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea
| | - Sung-Hyun Lee
- Neuroscience Graduate Program, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Yeonseung Chung
- Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Daeyeol Lee
- The Zanvyl Krieger Mind/Brain Institute, Kavli Neuroscience Discovery Institute, Department of Neuroscience, and Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, United States
| | - Min Whan Jung
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| |
Collapse
|
21
|
Azab H, Hayden BY. Partial integration of the components of value in anterior cingulate cortex. Behav Neurosci 2021; 134:296-308. [PMID: 32658523 DOI: 10.1037/bne0000382] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Evaluation often involves integrating multiple determinants of value, such as the different possible outcomes in risky choice. A brain region can be placed either before or after a presumed evaluation stage by measuring how responses of its neurons depend on multiple determinants of value. A brain region could also, in principle, show partial integration, which would indicate that it occupies a middle position between (preevaluative) nonintegration and (postevaluative) full integration. Existing mathematical techniques cannot distinguish full from partial integration and therefore risk misidentifying regional function. Here we use a new Bayesian regression-based approach to analyze responses of neurons in dorsal anterior cingulate cortex (dACC) to risky offers. We find that dACC neurons only partially integrate across outcome dimensions, indicating that dACC cannot be assigned to either a pre- or postevaluative position. Neurons in dACC also show putative signatures of value comparison, thereby demonstrating that comparison does not require complete evaluation before proceeding. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
- Habiba Azab
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| |
Collapse
|
22
|
Coley AA, Padilla-Coreano N, Patel R, Tye KM. Valence processing in the PFC: Reconciling circuit-level and systems-level views. INTERNATIONAL REVIEW OF NEUROBIOLOGY 2021; 158:171-212. [PMID: 33785145 DOI: 10.1016/bs.irn.2020.12.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
An essential component in animal behavior is the ability to process emotion and dissociate among positive and negative valence in response to a rewarding or aversive stimulus. The medial prefrontal cortex (mPFC)-responsible for higher order executive functions that include cognition, learning, and working memory; and is also involved in sociability-plays a major role in emotional processing and control. Although the amygdala is widely regarded as the "emotional hub," the mPFC encodes for context-specific salience and elicits top-down control over limbic circuitry. The mPFC can then conduct behavioral responses, via cortico-striatal and cortico-brainstem pathways, that correspond to emotional stimuli. Evidence shows that abnormalities within the mPFC lead to sociability deficits, working memory impairments, and drug-seeking behavior that include addiction and compulsive disorders; as well as conditions such as anhedonia. Recent studies investigate the effects of aberrant salience processing on cortical circuitry and neuronal populations associated with these behaviors. In this chapter, we discuss mPFC valence processing, neuroanatomical connections, and physiological substrates involved in mPFC-associated behavior. We review neurocomputational and theoretical models such as "mixed selectivity," that describe cognitive control, attentiveness, and motivational drives. Using this knowledge, we describe the effects of valence imbalances and its influence on mPFC neural pathways that contribute to deficits in social cognition, while understanding the effects in addiction/compulsive behaviors and anhedonia.
Collapse
Affiliation(s)
- Austin A Coley
- Salk Institute for Biological Studies, La Jolla, CA, United States
| | | | - Reesha Patel
- Salk Institute for Biological Studies, La Jolla, CA, United States
| | - Kay M Tye
- Salk Institute for Biological Studies, La Jolla, CA, United States.
| |
Collapse
|
23
|
Li Y, Liu C, Ji M, You X. Shape of progress bar effect on subjective evaluation, duration perception and physiological reaction. INTERNATIONAL JOURNAL OF INDUSTRIAL ERGONOMICS 2021; 81:103031. [DOI: 10.1016/j.ergon.2020.103031] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/13/2023]
|
24
|
Abeles D, Amit R, Tal-Perry N, Carrasco M, Yuval-Greenberg S. Oculomotor inhibition precedes temporally expected auditory targets. Nat Commun 2020; 11:3524. [PMID: 32665559 PMCID: PMC7360783 DOI: 10.1038/s41467-020-17158-9] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2019] [Accepted: 05/27/2020] [Indexed: 11/09/2022] Open
Abstract
Eye movements are inhibited prior to the onset of temporally-predictable visual targets. This oculomotor inhibition effect could be considered a marker for the formation of temporal expectations and the allocation of temporal attention in the visual domain. Here we show that eye movements are also inhibited before predictable auditory targets. In two experiments, we manipulate the period between a cue and an auditory target to be either predictable or unpredictable. The findings show that although there is no perceptual gain from avoiding gaze-shifts in this procedure, saccades and blinks are inhibited prior to predictable relative to unpredictable auditory targets. These findings show that oculomotor inhibition occurs prior to auditory targets. This link between auditory expectation and oculomotor behavior reveals a multimodal perception action coupling, which has a central role in temporal expectations.
Collapse
Affiliation(s)
- Dekel Abeles
- School of Psychological Sciences, Tel-Aviv University, Ramat Aviv, 6997801, Tel Aviv-Yafo, Israel
| | - Roy Amit
- Sagol School of Neuroscience, Tel-Aviv University, Ramat Aviv, 6997801, Tel Aviv-Yafo, Israel
| | - Noam Tal-Perry
- School of Psychological Sciences, Tel-Aviv University, Ramat Aviv, 6997801, Tel Aviv-Yafo, Israel
| | - Marisa Carrasco
- Department of Psychology and Center for Neural Science, New York University, 6 Washington Place, New York, NY, 10003, USA
| | - Shlomit Yuval-Greenberg
- School of Psychological Sciences, Tel-Aviv University, Ramat Aviv, 6997801, Tel Aviv-Yafo, Israel. .,Sagol School of Neuroscience, Tel-Aviv University, Ramat Aviv, 6997801, Tel Aviv-Yafo, Israel.
| |
Collapse
|
25
|
Kimmel DL, Elsayed GF, Cunningham JP, Newsome WT. Value and choice as separable and stable representations in orbitofrontal cortex. Nat Commun 2020; 11:3466. [PMID: 32651373 PMCID: PMC7351792 DOI: 10.1038/s41467-020-17058-y] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2020] [Accepted: 06/10/2020] [Indexed: 12/13/2022] Open
Abstract
Value-based decision-making requires different variables-including offer value, choice, expected outcome, and recent history-at different times in the decision process. Orbitofrontal cortex (OFC) is implicated in value-based decision-making, but it is unclear how downstream circuits read out complex OFC responses into separate representations of the relevant variables to support distinct functions at specific times. We recorded from single OFC neurons while macaque monkeys made cost-benefit decisions. Using a novel analysis, we find separable neural dimensions that selectively represent the value, choice, and expected reward of the present and previous offers. The representations are generally stable during periods of behavioral relevance, then transition abruptly at key task events and between trials. Applying new statistical methods, we show that the sensitivity, specificity and stability of the representations are greater than expected from the population's low-level features-dimensionality and temporal smoothness-alone. The separability and stability suggest a mechanism-linear summation over static synaptic weights-by which downstream circuits can select for specific variables at specific times.
Collapse
Affiliation(s)
- Daniel L Kimmel
- Mortimer Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, 10027, USA.
- Department of Psychiatry, Columbia University, New York, NY, 10032, USA.
| | | | - John P Cunningham
- Mortimer Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, 10027, USA
- Department of Statistics, Columbia University, New York, NY, 10027, USA
| | - William T Newsome
- Department of Neurobiology and Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, 94305, USA
| |
Collapse
|
26
|
Bouchacourt F, Palminteri S, Koechlin E, Ostojic S. Temporal chunking as a mechanism for unsupervised learning of task-sets. eLife 2020; 9:50469. [PMID: 32149602 PMCID: PMC7108869 DOI: 10.7554/elife.50469] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Accepted: 02/24/2020] [Indexed: 12/26/2022] Open
Abstract
Depending on environmental demands, humans can learn and exploit multiple concurrent sets of stimulus-response associations. Mechanisms underlying the learning of such task-sets remain unknown. Here we investigate the hypothesis that task-set learning relies on unsupervised chunking of stimulus-response associations that occur in temporal proximity. We examine behavioral and neural data from a task-set learning experiment using a network model. We first show that task-set learning can be achieved provided the timescale of chunking is slower than the timescale of stimulus-response learning. Fitting the model to behavioral data on a subject-by-subject basis confirmed this expectation and led to specific predictions linking chunking and task-set retrieval that were borne out by behavioral performance and reaction times. Comparing the model activity with BOLD signal allowed us to identify neural correlates of task-set retrieval in a functional network involving ventral and dorsal prefrontal cortex, with the dorsal system preferentially engaged when retrievals are used to improve performance.
Collapse
Affiliation(s)
- Flora Bouchacourt
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sante et de la Recherche Medicale, Paris, France.,Departement d'Etudes Cognitives, Ecole Normale Superieure, Paris, France
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sante et de la Recherche Medicale, Paris, France.,Departement d'Etudes Cognitives, Ecole Normale Superieure, Paris, France.,Institut d'Etudes de la Cognition, Universite de Recherche Paris Sciences et Lettres, Paris, France
| | - Etienne Koechlin
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sante et de la Recherche Medicale, Paris, France.,Departement d'Etudes Cognitives, Ecole Normale Superieure, Paris, France
| | - Srdjan Ostojic
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sante et de la Recherche Medicale, Paris, France.,Departement d'Etudes Cognitives, Ecole Normale Superieure, Paris, France.,Institut d'Etudes de la Cognition, Universite de Recherche Paris Sciences et Lettres, Paris, France
| |
Collapse
|
27
|
Weidacker K, Johnston SJ, Mullins PG, Boy F, Dymond S. Impulsive decision-making and gambling severity: The influence of γ-amino-butyric acid (GABA) and glutamate-glutamine (Glx). Eur Neuropsychopharmacol 2020; 32:36-46. [PMID: 31901336 DOI: 10.1016/j.euroneuro.2019.12.110] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/15/2019] [Revised: 11/15/2019] [Accepted: 12/13/2019] [Indexed: 11/28/2022]
Abstract
Discounting larger, delayed rewards for smaller, immediate rewards is a stable psychological trait known to be impaired in gambling disorder (GD). Neuroimaging with non-GD populations indicates involvement of anterior cingulate (ACC) and dorsolateral prefrontal cortex (dlPFC) in delay discounting. However, little is known about the role of intrinsic properties of brain functioning, such as neurotransmitter action, in impaired discounting in GD. Here, we used magnetic resonance spectroscopy to assess glutamate-glutamine (Glx) and γ-amino-butyric acid (GABA+) concentrations in the dorsal ACC (dACC), dlPFC and occipital cortex of human males with and without GD. Gambling symptom severity correlated negatively with Glx levels in the dACC and occipital voxels. Discounting of small and medium delayed rewards was negatively associated with GABA+ in the dACC, while the discounting of large delayed rewards was negatively associated with GABA+/Glx ratios in the dlPFC. Additionally, in GD, discounting of large delayed rewards was negatively correlated with occipital GABA+ levels. Overall, these findings show that high gambling symptom severity is associated with low levels of Glx and that dACC (GABA+), right dlPFC (GABA+/Glx), and occipital areas (GABA+) track the magnitude of delayed rewards during discounting.
Collapse
Affiliation(s)
- Kathrin Weidacker
- Department of Psychology, Swansea University, Singleton Campus, Swansea SA2 8PP, United Kingdom; Department of Psychiatry, University of Cambridge, Box 189, Level E4, Addenbrooke's Hospital, Cambridge CB2 0QQ, United Kingdom
| | - Stephen J Johnston
- Department of Psychology, Swansea University, Singleton Campus, Swansea SA2 8PP, United Kingdom
| | - Paul G Mullins
- School of Psychology, Bangor University, Bangor, Gwynedd LL57 2AS, United Kingdom
| | - Fred Boy
- Department of Psychology, Swansea University, Singleton Campus, Swansea SA2 8PP, United Kingdom; School of Management, Swansea University, Bay Campus, Swansea SA1 8EN, United Kingdom
| | - Simon Dymond
- Department of Psychology, Swansea University, Singleton Campus, Swansea SA2 8PP, United Kingdom; Department of Psychology, Reykjavík University, Menntavegur 1, Nauthólsvík, 101 Reykjavík, Iceland.
| |
Collapse
|
28
|
Ventral striatum supports Methylphenidate therapeutic effects on impulsive choices expressed in temporal discounting task. Sci Rep 2020; 10:716. [PMID: 31959838 PMCID: PMC6971276 DOI: 10.1038/s41598-020-57595-6] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2019] [Accepted: 01/03/2020] [Indexed: 12/04/2022] Open
Abstract
Methylphenidate (MPH) is a dopamine transporter (DAT) inhibitor used to treat attention-deficit/hyperactivity-disorder (ADHD). ADHD patients make impulsive choices in delay discounting tasks (DDT) and MPH reduces such impulsivity, but its therapeutic site of action remains unknown. Based on the high density of DAT in the striatum, we hypothesized that the striatum, especially the ventral striatum (VS) and caudate nucleus which both encode temporal discounting, can be preferential MPH action sites. To determine whether one of these striatal territories is predominantly involved in the effect of MPH, we trained monkeys to make choices during DDT. First, consistent with clinical observations, we found an overall reduction of impulsive choices with a low dose of MPH administered via intramuscular injections, whereas we reported sedative-like effects with a higher dose. Then, using PET-imaging, we found that the therapeutic reduction of impulsive choices was associated with selective DAT occupancy of MPH in the VS. Finally, we confirmed the selective involvement of the VS in the effect of MPH by testing the animals’ impulsivity with microinjections of the drug in distinct striatal territories. Together, these results show that the therapeutic effect of MPH on impulsive decisions is mainly restricted to its action in the VS.
Collapse
|
29
|
Martinez E, Pasquereau B, Saga Y, Météreau É, Tremblay L. The anterior caudate nucleus supports impulsive choices triggered by pramipexole. Mov Disord 2019; 35:296-305. [PMID: 31737954 DOI: 10.1002/mds.27898] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2019] [Revised: 09/30/2019] [Accepted: 10/06/2019] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND Pramipexole is a dopamine agonist used as a treatment in PD and restless legs syndrome to reduce motor symptoms, but it often induces impulse control disorders. In particular, patients with impulse control disorders tend to make more impulsive choices in the delay discounting task, that is, they choose small immediate rewards over larger delayed ones more often than patients without impulse control disorders and healthy subjects do. Yet the site of action of pramipexole that produces these impulsive choices remains unknown. Based on the heterogeneity of corticostriatal projections and the massive dopamine innervation of the striatum, we hypothesized that impulsive choices triggered by dopamine treatments may be supported by a specific striatal territory. OBJECTIVES This study aims to determine by which anteriorstriatum territory the Pramipexole trigger impulsive choices; the caudate nucleus, the ventral striatum or the putamen. METHODS We compared pramipexole intramuscular injections to intracerebral microinjections within the three striatal territories in healthy monkeys trained to execute the delay discounting task, a behavioral paradigm typically used to evaluate impulsive choices. RESULTS We found that pramipexole intramuscular injections induced impulsive choices in all monkeys. Local microinjections were performed inside the anterior caudate nucleus, ventral striatum, and anterior putamen and reproduced those impulsive choices when pramipexole was directly injected into the caudate nucleus, whereas injections into the ventral striatum or putamen had no effect on monkeys' choices. CONCLUSIONS These results, consistent with clinical studies, suggest that impulsive choices triggered by pramipexole are supported by the caudate nucleus, allowing us to emphasize the importance of dopamine modulation inside this striatal territory in decision processes underlying impulsive behaviors. © 2019 International Parkinson and Movement Disorder Society.
Collapse
Affiliation(s)
- Eva Martinez
- Institut des Sciences Cognitives Marc Jeannerod, UMR-5229, Bron, France.,Université Claude Bernard Lyon 1, Villeurbanne, France
| | | | - Yosuke Saga
- Institut des Sciences Cognitives Marc Jeannerod, UMR-5229, Bron, France
| | - Élise Météreau
- Institut des Sciences Cognitives Marc Jeannerod, UMR-5229, Bron, France
| | - Léon Tremblay
- Institut des Sciences Cognitives Marc Jeannerod, UMR-5229, Bron, France.,Université Claude Bernard Lyon 1, Villeurbanne, France
| |
Collapse
|
30
|
Pastor-Bernier A, Stasiak A, Schultz W. Orbitofrontal signals for two-component choice options comply with indifference curves of Revealed Preference Theory. Nat Commun 2019; 10:4885. [PMID: 31653852 PMCID: PMC6814743 DOI: 10.1038/s41467-019-12792-4] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2019] [Accepted: 10/01/2019] [Indexed: 11/08/2022] Open
Abstract
Economic choice options contain multiple components and constitute vectorial bundles. The question arises how they are represented by single-dimensional, scalar neuronal signals that are suitable for economic decision-making. Revealed Preference Theory provides formalisms for establishing preference relations between such bundles, including convenient graphic indifference curves. During stochastic choice between bundles with the same two juice components, we identified neuronal signals for vectorial, multi-component bundles in the orbitofrontal cortex of monkeys. A scalar signal integrated the values from all bundle components in the structured manner of the Theory; it followed the behavioral indifference curves within their confidence limits, was indistinguishable between differently composed but equally revealed preferred bundles, predicted bundle choice and complied with an optimality axiom. Further, distinct signals in other neurons coded the option components separately but followed indifference curves as a population. These data demonstrate how scalar signals represent vectorial, multi-component choice options.
Collapse
Affiliation(s)
- Alexandre Pastor-Bernier
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, CB2 3DY, UK
| | - Arkadiusz Stasiak
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, CB2 3DY, UK
| | - Wolfram Schultz
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, CB2 3DY, UK.
| |
Collapse
|
31
|
Wang Y, Qin Y, Li H, Yao D, Sun B, Li Z, Li X, Dai Y, Wen C, Zhang L, Zhang C, Zhu T, Luo C. Abnormal Functional Connectivity in Cognitive Control Network, Default Mode Network, and Visual Attention Network in Internet Addiction: A Resting-State fMRI Study. Front Neurol 2019; 10:1006. [PMID: 31620077 PMCID: PMC6759465 DOI: 10.3389/fneur.2019.01006] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2019] [Accepted: 09/03/2019] [Indexed: 01/01/2023] Open
Abstract
Internet addiction (IA) has become a global mental and social problem, which may lead to a series of psychiatric symptoms including uncontrolled use of internet, and lack of concentration. However, the exact pathophysiology of IA remains unclear. Most of functional connectivity studies were based on pre-selected regions of interest (ROI), which could not provide a comprehensive picture of the communication abnormalities in IA, and might lead to limited or bias observations. Using local functional connectivity density (lFCD), this study aimed to explore the whole-brain abnormalities of functional connectivity in IA. We evaluated the whole-brain lFCD resulting from resting-state fMRI data in 28 IA individuals and 30 demographically matched healthy control subjects (HCs). The correlations between clinical characteristics and aberrant lFCD were also assessed. Compared with HCs, subjects with IA exhibited heightened lFCD values in the right dorsolateral prefrontal cortex (DLPFC), left parahippocampal gyrus (PHG), and cerebellum, and the bilateral middle cingulate cortex (MCC) and superior temporal pole (STP), as well as decreased lFCD values in the right inferior parietal lobe (IPL), and bilateral calcarine and lingual gyrus. Voxel-based correlation analysis revealed the significant correlations between the Young's Internet Addiction Test (IAT) score and altered lFCD values in the left PHG and bilateral STP. These findings revealed the hyper-connectivity in cognitive control network and default mode network as well as the hypo-connectivity in visual attention network, verifying the common mechanism in IA and substance addiction, and the underlying association between IA, and attention deficit/hyperactivity disorder in terms of neurobiology.
Collapse
Affiliation(s)
- Yang Wang
- School of Acupuncture and Tuina, Chengdu University of Traditional Chinese Medicine, Chengdu, China.,Department of Rehabilitation, Shuangliu Maternal and Child Health Care Hospital, Chengdu, China
| | - Yun Qin
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Hui Li
- School of Medicine, Chengdu University, Chengdu, China
| | - Dezhong Yao
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Bo Sun
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Zhiliang Li
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Xin Li
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Yu Dai
- School of Rehabilitation and Health Preservation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Chao Wen
- Department of Rehabilitation, Zigong Fifth People's Hospital, Zigong, China
| | - Lingrui Zhang
- School of Rehabilitation and Health Preservation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Chenchen Zhang
- School of Rehabilitation and Health Preservation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Tianmin Zhu
- School of Rehabilitation and Health Preservation, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Cheng Luo
- Key Laboratory for NeuroInformation of Ministry of Education, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
32
|
Datta D, Arnsten AFT. Loss of Prefrontal Cortical Higher Cognition with Uncontrollable Stress: Molecular Mechanisms, Changes with Age, and Relevance to Treatment. Brain Sci 2019; 9:brainsci9050113. [PMID: 31108855 PMCID: PMC6562841 DOI: 10.3390/brainsci9050113] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2019] [Revised: 05/13/2019] [Accepted: 05/13/2019] [Indexed: 01/11/2023] Open
Abstract
The newly evolved prefrontal cortex (PFC) generates goals for "top-down" control of behavior, thought, and emotion. However, these circuits are especially vulnerable to uncontrollable stress, with powerful, intracellular mechanisms that rapidly take the PFC "off-line." High levels of norepinephrine and dopamine released during stress engage α1-AR and D1R, which activate feedforward calcium-cAMP signaling pathways that open nearby potassium channels to weaken connectivity and reduce PFC cell firing. Sustained weakening with chronic stress leads to atrophy of dendrites and spines. Understanding these signaling events helps to explain the increased susceptibility of the PFC to stress pathology during adolescence, when dopamine expression is increased in the PFC, and with advanced age, when the molecular "brakes" on stress signaling are diminished by loss of phosphodiesterases. These mechanisms have also led to pharmacological treatments for stress-related disorders, including guanfacine treatment of childhood trauma, and prazosin treatment of veterans and civilians with post-traumatic stress disorder.
Collapse
Affiliation(s)
- Dibyadeep Datta
- Department Neuroscience, Yale Medical School, New Haven, CT 06510, USA.
| | - Amy F T Arnsten
- Department Neuroscience, Yale Medical School, New Haven, CT 06510, USA.
| |
Collapse
|
33
|
Setogawa T, Mizuhiki T, Matsumoto N, Akizawa F, Kuboki R, Richmond BJ, Shidara M. Neurons in the monkey orbitofrontal cortex mediate reward value computation and decision-making. Commun Biol 2019; 2:126. [PMID: 30963114 PMCID: PMC6451015 DOI: 10.1038/s42003-019-0363-0] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2018] [Accepted: 03/01/2019] [Indexed: 11/28/2022] Open
Abstract
Choice reflects the values of available alternatives; more valuable options are chosen more often than less valuable ones. Here we studied whether neuronal responses in orbitofrontal cortex (OFC) reflect the value difference between options, and whether there is a causal link between OFC neuronal activity and choice. Using a decision-making task where two visual stimuli were presented sequentially, each signifying a value, we showed that when the second stimulus appears many neurons encode the value difference between alternatives. Later when the choice occurs, that difference signal disappears and a signal indicating the chosen value emerges. Pharmacological inactivation of OFC neurons coding for choice-related values increases the monkey's latency to make a choice and the likelihood that it will choose the less valuable alternative, when the value difference is small. Thus, OFC neurons code for value information that could be used to directly influence choice.
Collapse
Affiliation(s)
- Tsuyoshi Setogawa
- Faculty of Medicine, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
- Department of Health and Human Services, Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415 USA
| | - Takashi Mizuhiki
- Faculty of Medicine, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
- Doctoral Program in Kansei, Behavioral and Brain Science, Graduate School of Comprehensive Human Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
| | - Narihisa Matsumoto
- Human Informatics Research Institute, AIST, 1-1-1 Umezono, Tsukuba, Ibaraki 305-8568 Japan
| | - Fumika Akizawa
- Doctoral Program in Kansei, Behavioral and Brain Science, Graduate School of Comprehensive Human Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
| | - Ryosuke Kuboki
- Doctoral Program in Kansei, Behavioral and Brain Science, Graduate School of Comprehensive Human Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
| | - Barry J. Richmond
- Department of Health and Human Services, Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415 USA
| | - Munetaka Shidara
- Faculty of Medicine, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
- Doctoral Program in Kansei, Behavioral and Brain Science, Graduate School of Comprehensive Human Sciences, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8577 Japan
| |
Collapse
|
34
|
Jung MW, Lee H, Jeong Y, Lee JW, Lee I. Remembering rewarding futures: A simulation-selection model of the hippocampus. Hippocampus 2018; 28:913-930. [PMID: 30155938 PMCID: PMC6587829 DOI: 10.1002/hipo.23023] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2018] [Revised: 07/06/2018] [Accepted: 08/23/2018] [Indexed: 02/06/2023]
Abstract
Despite tremendous progress, the neural circuit dynamics underlying hippocampal mnemonic processing remain poorly understood. We propose a new model for hippocampal function-the simulation-selection model-based on recent experimental findings and neuroecological considerations. Under this model, the mammalian hippocampus evolved to simulate and evaluate arbitrary navigation sequences. Specifically, we suggest that CA3 simulates unexperienced navigation sequences in addition to remembering experienced ones, and CA1 selects from among these CA3-generated sequences, reinforcing those that are likely to maximize reward during offline idling states. High-value sequences reinforced in CA1 may allow flexible navigation toward a potential rewarding location during subsequent navigation. We argue that the simulation-selection functions of the hippocampus have evolved in mammals mostly because of the unique navigational needs of land mammals. Our model may account for why the mammalian hippocampus has evolved not only to remember, but also to imagine episodes, and how this might be implemented in its neural circuits.
Collapse
Affiliation(s)
- Min Whan Jung
- Center for Synaptic Brain Dysfunctions, Institute for Basic ScienceDaejeonSouth Korea
- Department of Biological SciencesKorea Advanced Institute of Science and TechnologyDaejeonSouth Korea
| | - Hyunjung Lee
- Department of AnatomyKyungpook National University School of MedicineDaeguSouth Korea
| | - Yeongseok Jeong
- Center for Synaptic Brain Dysfunctions, Institute for Basic ScienceDaejeonSouth Korea
- Department of Biological SciencesKorea Advanced Institute of Science and TechnologyDaejeonSouth Korea
| | - Jong Won Lee
- Center for Synaptic Brain Dysfunctions, Institute for Basic ScienceDaejeonSouth Korea
| | - Inah Lee
- Department of Brain and Cognitive SciencesSeoul National UniversitySeoulSouth Korea
| |
Collapse
|
35
|
Brockett AT, Pribut HJ, Vázquez D, Roesch MR. The impact of drugs of abuse on executive function: characterizing long-term changes in neural correlates following chronic drug exposure and withdrawal in rats. Learn Mem 2018; 25:461-473. [PMID: 30115768 PMCID: PMC6097763 DOI: 10.1101/lm.047001.117] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Accepted: 06/25/2018] [Indexed: 12/21/2022]
Abstract
Addiction has long been characterized by diminished executive function, control, and impulsivity management. In particular, these deficits often manifest themselves as impairments in reversal learning, delay discounting, and response inhibition. Understanding the neurobiological substrates of these behavioral deficits is of paramount importance to our understanding of addiction. Within the cycle of addiction, periods during and after withdrawal represent a particularly difficult point of intervention in that the negative physical symptoms associated with drug removal and drug craving increase the likelihood that the patient will relapse and return to drug use in order to abate these symptoms. Moreover, it is often during this time that drug induced deficits in executive function hinder the ability of the patient to refrain from drug use. Thus, it is necessary to understand the physiological and behavioral changes associated with withdrawal and drug craving-largely manifesting as deficits in executive control-to develop more effective treatment strategies. In this review, we address the long-term impact that drugs of abuse have on the behavioral and neural correlates that give rise to executive control as measured by reversal learning, delay discounting, and stop-signal tasks, focusing particularly on our work using rats as a model system.
Collapse
Affiliation(s)
- Adam T Brockett
- Department of Psychology, University of Maryland, College Park, Maryland 20742, USA
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, USA
| | - Heather J Pribut
- Department of Psychology, University of Maryland, College Park, Maryland 20742, USA
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, USA
| | - Daniela Vázquez
- Department of Psychology, University of Maryland, College Park, Maryland 20742, USA
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, USA
| | - Matthew R Roesch
- Department of Psychology, University of Maryland, College Park, Maryland 20742, USA
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742, USA
| |
Collapse
|
36
|
Cavanagh SE, Towers JP, Wallis JD, Hunt LT, Kennerley SW. Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex. Nat Commun 2018; 9:3498. [PMID: 30158519 PMCID: PMC6115433 DOI: 10.1038/s41467-018-05873-3] [Citation(s) in RCA: 77] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2017] [Accepted: 07/31/2018] [Indexed: 01/17/2023] Open
Abstract
Competing accounts propose that working memory (WM) is subserved either by persistent activity in single neurons or by dynamic (time-varying) activity across a neural population. Here, we compare these hypotheses across four regions of prefrontal cortex (PFC) in an oculomotor-delayed-response task, where an intervening cue indicated the reward available for a correct saccade. WM representations were strongest in ventrolateral PFC neurons with higher intrinsic temporal stability (time-constant). At the population-level, although a stable mnemonic state was reached during the delay, this tuning geometry was reversed relative to cue-period selectivity, and was disrupted by the reward cue. Single-neuron analysis revealed many neurons switched to coding reward, rather than maintaining task-relevant spatial selectivity until saccade. These results imply WM is fulfilled by dynamic, population-level activity within high time-constant neurons. Rather than persistent activity supporting stable mnemonic representations that bridge subsequent salient stimuli, PFC neurons may stabilise a dynamic population-level process supporting WM.
Collapse
Affiliation(s)
- Sean E Cavanagh
- Sobell Department of Motor Neuroscience, University College London, London WC1N 3BG, UK.
| | - John P Towers
- Sobell Department of Motor Neuroscience, University College London, London WC1N 3BG, UK
| | - Joni D Wallis
- Department of Psychology, University of California at Berkeley, Berkeley, CA 94720, United States.,Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA 94720, United States
| | - Laurence T Hunt
- Sobell Department of Motor Neuroscience, University College London, London WC1N 3BG, UK.,Max Planck-UCL Centre for Computational Psychiatry and Aging, University College London, London WC1B 5EH, UK.,Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford OX37JX, UK
| | - Steven W Kennerley
- Sobell Department of Motor Neuroscience, University College London, London WC1N 3BG, UK. .,Department of Psychology, University of California at Berkeley, Berkeley, CA 94720, United States. .,Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA 94720, United States.
| |
Collapse
|
37
|
Parro C, Dixon ML, Christoff K. The neural basis of motivational influences on cognitive control. Hum Brain Mapp 2018; 39:5097-5111. [PMID: 30120846 DOI: 10.1002/hbm.24348] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2017] [Revised: 07/16/2018] [Accepted: 07/30/2018] [Indexed: 12/22/2022] Open
Abstract
Cognitive control mechanisms support the deliberate regulation of thought and behavior based on current goals. Recent work suggests that motivational incentives improve cognitive control and has begun to elucidate critical neural substrates. We conducted a quantitative meta-analysis of neuroimaging studies of motivated cognitive control using activation likelihood estimation (ALE) and Neurosynth to delineate the brain regions that are consistently activated across studies. The analysis included studies that investigated changes in brain activation during cognitive control tasks when reward incentives were present versus absent. The ALE analysis revealed consistent recruitment in regions associated with the frontoparietal control network including the inferior frontal sulcus and intraparietal sulcus, as well as regions associated with the salience network including the anterior insula and anterior mid-cingulate cortex. As a complementary analysis, we performed a large-scale exploratory meta-analysis using Neurosynth to identify regions that are recruited in studies using of the terms cognitive control and incentive. This analysis replicated the ALE results and also identified the rostrolateral prefrontal cortex, caudate nucleus, nucleus accumbens, medial thalamus, inferior frontal junction, premotor cortex, and hippocampus. Finally, we separately compared recruitment during cue and target periods, which tap into proactive engagement of rule-outcome associations, and the mobilization of appropriate viscero-motor states to execute a response, respectively. We found that largely distinct sets of brain regions are recruited during cue and target periods. Altogether, these findings suggest that flexible interactions between frontoparietal, salience, and dopaminergic midbrain-striatal networks may allow control demands to be precisely tailored based on expected value.
Collapse
Affiliation(s)
- Cameron Parro
- Department of Psychology, University of British Columbia, Vancouver, British Columbia, Canada
| | - Matthew L Dixon
- Department of Psychology, University of British Columbia, Vancouver, British Columbia, Canada
| | - Kalina Christoff
- Department of Psychology, University of British Columbia, Vancouver, British Columbia, Canada.,Centre for Brain Health, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
38
|
Separate Neural Networks for Gains and Losses in Intertemporal Choice. Neurosci Bull 2018; 34:725-735. [PMID: 30088149 PMCID: PMC6129240 DOI: 10.1007/s12264-018-0267-x] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2017] [Accepted: 06/10/2018] [Indexed: 12/31/2022] Open
Abstract
An important and unresolved question is how human brain regions process information and interact with each other in intertemporal choice related to gains and losses. Using psychophysiological interaction and dynamic causal modeling analyses, we investigated the functional interactions between regions involved in the decision-making process while participants performed temporal discounting tasks in both the gains and losses domains. We found two distinct intrinsic valuation systems underlying temporal discounting in the gains and losses domains: gains were specifically evaluated in the medial regions, including the medial prefrontal and orbitofrontal cortices, and losses were evaluated in the lateral dorsolateral prefrontal cortex. In addition, immediate reward or punishment was found to modulate the functional interactions between the dorsolateral prefrontal cortex and distinct regions in both the gains and losses domains: in the gains domain, the mesolimbic regions; in the losses domain, the medial prefrontal cortex, anterior cingulate cortex, and insula. These findings suggest that intertemporal choice of gains and losses might involve distinct valuation systems, and more importantly, separate neural interactions may implement the intertemporal choices of gains and losses. These findings may provide a new biological perspective for understanding the neural mechanisms underlying intertemporal choice of gains and losses.
Collapse
|
39
|
Massi B, Donahue CH, Lee D. Volatility Facilitates Value Updating in the Prefrontal Cortex. Neuron 2018; 99:598-608.e4. [PMID: 30033151 DOI: 10.1016/j.neuron.2018.06.033] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2018] [Revised: 05/13/2018] [Accepted: 06/21/2018] [Indexed: 01/24/2023]
Abstract
Adaptation of learning and decision-making might depend on the regulation of activity in the prefrontal cortex. Here we examined how volatility of reward probabilities influences learning and neural activity in the primate prefrontal cortex. We found that animals selected recently rewarded targets more often when reward probabilities of different options fluctuated across trials than when they were fixed. Additionally, neurons in the orbitofrontal cortex displayed more sustained activity related to the outcomes of their previous choices when reward probabilities changed over time. Such volatility also enhanced signals in the dorsolateral prefrontal cortex related to the current but not the previous location of the previously rewarded target. These results suggest that prefrontal activity related to choice and reward is dynamically regulated by the volatility of the environment and underscore the role of the prefrontal cortex in identifying aspects of the environment that are responsible for previous outcomes and should be learned.
Collapse
Affiliation(s)
- Bart Massi
- Interdeparmental Neuroscience Program, Yale School of Medicine, New Haven, CT 06510, USA
| | | | - Daeyeol Lee
- Interdeparmental Neuroscience Program, Yale School of Medicine, New Haven, CT 06510, USA; Department of Neuroscience, Yale School of Medicine, New Haven, CT 06510, USA; Department of Psychiatry, Yale School of Medicine, New Haven, CT 06510, USA; Kavli Institute for Neuroscience, Yale School of Medicine, New Haven, CT 06510, USA; Department of Psychology, Yale University, New Haven, CT 06520, USA.
| |
Collapse
|
40
|
Apps MAJ, Ramnani N. Contributions of the Medial Prefrontal Cortex to Social Influence in Economic Decision-Making. Cereb Cortex 2018; 27:4635-4648. [PMID: 28922858 DOI: 10.1093/cercor/bhx183] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2016] [Indexed: 01/10/2023] Open
Abstract
Economic decisions are guided by highly subjective reward valuations (SVs). Often these SVs are over-ridden when individuals conform to social norms. Yet, the neural mechanisms that underpin the distinct processing of such normative reward valuations (NVs) are poorly understood. The dorsomedial and ventromedial portions of the prefrontal cortex (dmPFC/vmPFC) are putatively key regions for processing social and economic information respectively. However, the contribution of these regions to economic decisions guided by social norms is unclear. Using functional magnetic resonance imaging and computational modeling we examine the neural mechanisms underlying the processing of SVs and NVs. Subjects (n = 15) indicated either their own economic preferences or made similar choices based on a social norm-learnt during a training session. We found that that the vmPFC and dmPFC make dissociable contributions to the processing of SV and NV. Regions of the dmPFC processed "only" the value of rewards when making normative choices. In contrast, we identify a novel mechanism in the vmPFC for the coding of value. This region signaled both subjective and normative valuations, but activity was scaled positively for SV and negatively for NV. These results highlight some of the key mechanisms that underpin conformity and social influence in economic decision-making.
Collapse
Affiliation(s)
- M A J Apps
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK.,Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford OX3 9DU, UK.,Department of Psychology, Royal Holloway, University of London TWO 0EX, UK
| | - N Ramnani
- Department of Psychology, Royal Holloway, University of London TWO 0EX, UK
| |
Collapse
|
41
|
Fluid network dynamics in the prefrontal cortex during multiple strategy switching. Nat Commun 2018; 9:309. [PMID: 29358717 PMCID: PMC5778086 DOI: 10.1038/s41467-017-02764-x] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2017] [Accepted: 12/26/2017] [Indexed: 01/08/2023] Open
Abstract
Coordinated shifts of neuronal activity in the prefrontal cortex are associated with strategy adaptations in behavioural tasks, when animals switch from following one rule to another. However, network dynamics related to multiple-rule changes are scarcely known. We show how firing rates of individual neurons in the prelimbic and cingulate cortex correlate with the performance of rats trained to change their navigation multiple times according to allocentric and egocentric strategies. The concerted population activity exhibits a stable firing during the performance of one rule but shifted to another neuronal firing state when a new rule is learnt. Interestingly, when the same rule is presented a second time within the same session, neuronal firing does not revert back to the original neuronal firing state, but a new activity-state is formed. Our data indicate that neuronal firing of prefrontal cortical neurons represents changes in strategy and task-performance rather than specific strategies or rules. Population activity of neurons in the prefrontal cortex has been shown to represent cognitive strategy and rules. Here the authors report that when the same rule is repeated on multiple occasions in the task, it is accompanied each time by a new prefrontal firing rate state.
Collapse
|
42
|
Padoa-Schioppa C, Conen KE. Orbitofrontal Cortex: A Neural Circuit for Economic Decisions. Neuron 2017; 96:736-754. [PMID: 29144973 PMCID: PMC5726577 DOI: 10.1016/j.neuron.2017.09.031] [Citation(s) in RCA: 146] [Impact Index Per Article: 20.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2017] [Revised: 09/14/2017] [Accepted: 09/20/2017] [Indexed: 11/24/2022]
Abstract
Economic choice behavior entails the computation and comparison of subjective values. A central contribution of neuroeconomics has been to show that subjective values are represented explicitly at the neuronal level. With this result at hand, the field has increasingly focused on the difficult question of where in the brain and how exactly subjective values are compared to make a decision. Here, we review a broad range of experimental and theoretical results suggesting that good-based decisions are generated in a neural circuit within the orbitofrontal cortex (OFC). The main lines of evidence supporting this proposal include the fact that goal-directed behavior is specifically disrupted by OFC lesions, the fact that different groups of neurons in this area encode the input and the output of the decision process, the fact that activity fluctuations in each of these cell groups correlate with choice variability, and the fact that these groups of neurons are computationally sufficient to generate decisions. Results from other brain regions are consistent with the idea that good-based decisions take place in OFC and indicate that value signals inform a variety of mental functions. We also contrast the present proposal with other leading models for the neural mechanisms of economic decisions. Finally, we indicate open questions and suggest possible directions for future research.
Collapse
Affiliation(s)
- Camillo Padoa-Schioppa
- Department of Neuroscience, Washington University in St. Louis, St. Louis, MO 63110, USA; Department of Economics, Washington University in St. Louis, St. Louis, MO 63110, USA; Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO 63110, USA.
| | - Katherine E Conen
- Department of Neuroscience, Washington University in St. Louis, St. Louis, MO 63110, USA
| |
Collapse
|
43
|
Xu S, Das G, Hueske E, Tonegawa S. Dorsal Raphe Serotonergic Neurons Control Intertemporal Choice under Trade-off. Curr Biol 2017; 27:3111-3119.e3. [PMID: 28988863 PMCID: PMC5691357 DOI: 10.1016/j.cub.2017.09.008] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Revised: 08/11/2017] [Accepted: 09/06/2017] [Indexed: 11/28/2022]
Abstract
Appropriate choice about delayed reward is fundamental to the survival of animals. Although animals tend to prefer immediate reward, delaying gratification is often advantageous. The dorsal raphe (DR) serotonergic neurons have long been implicated in the processing of delayed reward, but it has been unclear whether or when their activity causally directs choice. Here, we transiently augmented or reduced the activity of DR serotonergic neurons, while mice decided between differently delayed rewards as they performed a novel odor-guided intertemporal choice task. We found that these manipulations, precisely targeted at the decision point, were sufficient to bidirectionally influence impulsive choice. The manipulation specifically affected choices with more difficult trade-off. Similar effects were observed when we manipulated the serotonergic projections to the nucleus accumbens (NAc). We propose that DR serotonergic neurons preempt reward delays at the decision point and play a critical role in suppressing impulsive choice by regulating decision trade-off.
Collapse
Affiliation(s)
- Sangyu Xu
- RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Agency for Science, Technology and Research, Singapore 138632, Singapore.
| | - Gishnu Das
- RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Emily Hueske
- RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Susumu Tonegawa
- RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Biology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Howard Hughes Medical Institute, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Brain Science Institute, RIKEN, Wako, Saitama 351-0198, Japan.
| |
Collapse
|
44
|
Frost R, McNaughton N. The neural basis of delay discounting: A review and preliminary model. Neurosci Biobehav Rev 2017; 79:48-65. [DOI: 10.1016/j.neubiorev.2017.04.022] [Citation(s) in RCA: 68] [Impact Index Per Article: 9.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2017] [Revised: 04/20/2017] [Accepted: 04/21/2017] [Indexed: 12/30/2022]
|
45
|
McGraw JJ, Zona LC, Cromwell HC. The effects of ethanol on diverse components of choice in the rat: reward discrimination, preference and relative valuation. Eur J Neurosci 2017. [PMID: 28639261 DOI: 10.1111/ejn.13627] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Alcohol consumption impairs judgment and choice. How alcohol alters these crucial processes is primarily unknown. Choice can be fractionated into different components including reward discrimination, preference and relative valuation that can function together or in isolation depending upon diverse factors including choice context. We examined the diverse components and contextual effects by analyzing the effects of alcohol drinking on choice behavior in a task with a reduced level of temporal and spatial constraints. Rats were trained to drink 10% ethanol during 6 weeks of behavior testing using a combined sucrose-fade and two-bottle free-choice procedure. Two different sucrose pellet outcomes (e.g., constant vs. variable) were presented each week to examine the impact of voluntary drinking on reward-based decision-making. Behavioral contexts of single option, free choice and extinction were examined for each outcome set. Comparisons were made between alcohol and control groups and within the alcohol group over time to inspect choice profiles. Between-group results showed alcohol drinking animals expressed altered place preference and modified sucrose reward approach latencies. The within-group profile showed that alcohol drinking animals can express adequate reward discrimination, preference and incentive contrast during free choice. All of these components were significantly reduced during the context of extinction. Control animals were also impacted by extinction but not as severely. The findings point to a need for a greater focus on the context and the diverse components of choice when examining external and internal factors influencing decision-making during alcohol or other substance of abuse exposure.
Collapse
Affiliation(s)
- Justin J McGraw
- J.P. Scott Center for Neuroscience, Mind and Behavior and Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| | - Luke C Zona
- J.P. Scott Center for Neuroscience, Mind and Behavior and Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| | - Howard C Cromwell
- J.P. Scott Center for Neuroscience, Mind and Behavior and Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| |
Collapse
|
46
|
Hunt LT, Hayden BY. A distributed, hierarchical and recurrent framework for reward-based choice. Nat Rev Neurosci 2017; 18:172-182. [PMID: 28209978 PMCID: PMC5621622 DOI: 10.1038/nrn.2017.7] [Citation(s) in RCA: 137] [Impact Index Per Article: 19.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.
Collapse
Affiliation(s)
- Laurence T Hunt
- Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK
| | - Benjamin Y Hayden
- Department of Brain and Cognitive Sciences, University of Rochester, 309 Meliora Hall, Rochester, New York 14618, USA
| |
Collapse
|
47
|
Leathers ML, Olson CR. In monkeys making value-based decisions, amygdala neurons are sensitive to cue value as distinct from cue salience. J Neurophysiol 2017; 117:1499-1511. [PMID: 28077664 DOI: 10.1152/jn.00564.2016] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2016] [Revised: 01/05/2017] [Accepted: 01/05/2017] [Indexed: 11/22/2022] Open
Abstract
Neurons in the lateral intraparietal (LIP) area of macaque monkey parietal cortex respond to cues predicting rewards and penalties of variable size in a manner that depends on the motivational salience of the predicted outcome (strong for both large reward and large penalty) rather than on its value (positive for large reward and negative for large penalty). This finding suggests that LIP mediates the capture of attention by salient events and does not encode value in the service of value-based decision making. It leaves open the question whether neurons elsewhere in the brain encode value in the identical task. To resolve this issue, we recorded neuronal activity in the amygdala in the context of the task employed in the LIP study. We found that responses to reward-predicting cues were similar between areas, with the majority of reward-sensitive neurons responding more strongly to cues that predicted large reward than to those that predicted small reward. Responses to penalty-predicting cues were, however, markedly different. In the amygdala, unlike LIP, few neurons were sensitive to penalty size, few penalty-sensitive neurons favored large over small penalty, and the dependence of firing rate on penalty size was negatively correlated with its dependence on reward size. These results indicate that amygdala neurons encoded cue value under circumstances in which LIP neurons exhibited sensitivity to motivational salience. However, the representation of negative value, as reflected in sensitivity to penalty size, was weaker than the representation of positive value, as reflected in sensitivity to reward size.NEW & NOTEWORTHY This is the first study to characterize amygdala neuronal responses to cues predicting rewards and penalties of variable size in monkeys making value-based choices. Manipulating reward and penalty size allowed distinguishing activity dependent on motivational salience from activity dependent on value. This approach revealed in a previous study that neurons of the lateral intraparietal (LIP) area encode motivational salience. Here, it reveals that amygdala neurons encode value. The results establish a sharp functional distinction between the two areas.
Collapse
Affiliation(s)
- Marvin L Leathers
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania; and .,Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Carl R Olson
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania; and.,Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|
48
|
Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation. eNeuro 2016; 3:eN-NWR-0022-16. [PMID: 27822506 PMCID: PMC5089537 DOI: 10.1523/eneuro.0022-16.2016] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2016] [Revised: 10/06/2016] [Accepted: 10/07/2016] [Indexed: 11/21/2022] Open
Abstract
The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.
Collapse
|
49
|
Decision Making: A Role for the Amygdala in Translating Goals into Choices. Curr Biol 2016; 26:R1177-R1179. [DOI: 10.1016/j.cub.2016.09.050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
50
|
Cavanagh SE, Wallis JD, Kennerley SW, Hunt LT. Autocorrelation structure at rest predicts value correlates of single neurons during reward-guided choice. eLife 2016; 5. [PMID: 27705742 PMCID: PMC5052031 DOI: 10.7554/elife.18937] [Citation(s) in RCA: 64] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2016] [Accepted: 09/15/2016] [Indexed: 01/28/2023] Open
Abstract
Correlates of value are routinely observed in the prefrontal cortex (PFC) during reward-guided decision making. In previous work (Hunt et al., 2015), we argued that PFC correlates of chosen value are a consequence of varying rates of a dynamical evidence accumulation process. Yet within PFC, there is substantial variability in chosen value correlates across individual neurons. Here we show that this variability is explained by neurons having different temporal receptive fields of integration, indexed by examining neuronal spike rate autocorrelation structure whilst at rest. We find that neurons with protracted resting temporal receptive fields exhibit stronger chosen value correlates during choice. Within orbitofrontal cortex, these neurons also sustain coding of chosen value from choice through the delivery of reward, providing a potential neural mechanism for maintaining predictions and updating stored values during learning. These findings reveal that within PFC, variability in temporal specialisation across neurons predicts involvement in specific decision-making computations. DOI:http://dx.doi.org/10.7554/eLife.18937.001
Collapse
Affiliation(s)
- Sean E Cavanagh
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom
| | - Joni D Wallis
- Department of Psychology, University of California, Berkeley, Berkeley, United States.,Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
| | - Steven W Kennerley
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom.,Department of Psychology, University of California, Berkeley, Berkeley, United States.,Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
| | - Laurence T Hunt
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom.,Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| |
Collapse
|