1
|
Burk DC, Taswell C, Tang H, Averbeck BB. Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers. J Neurosci 2024; 44:e1873232024. [PMID: 38670805 PMCID: PMC11170943 DOI: 10.1523/jneurosci.1873-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 02/27/2024] [Accepted: 04/11/2024] [Indexed: 04/28/2024] Open
Abstract
Reinforcement learning is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g., money, points) that are later exchanged for primary reinforcers (e.g., food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g., the current number of accumulated tokens, choice options, task epoch, trials since the last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n = 5 monkeys, three males and two females). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in the state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.
Collapse
Affiliation(s)
- Diana C Burk
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892-4415
| | - Craig Taswell
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892-4415
| | - Hua Tang
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892-4415
| | - Bruno B Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland 20892-4415
| |
Collapse
|
2
|
Tang H, Bartolo-Orozco R, Averbeck BB. Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.03.587097. [PMID: 38617219 PMCID: PMC11014508 DOI: 10.1101/2024.04.03.587097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and the mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, OFC leads the process of symbolic reinforcement learning in the ventral frontostriatal circuitry.
Collapse
|
3
|
Sypré L, Sharma S, Mantini D, Nelissen K. Intrinsic functional clustering of the macaque insular cortex. Front Integr Neurosci 2024; 17:1272529. [PMID: 38250745 PMCID: PMC10797002 DOI: 10.3389/fnint.2023.1272529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Accepted: 12/18/2023] [Indexed: 01/23/2024] Open
Abstract
The functional organization of the primate insula has been studied using a variety of techniques focussing on regional differences in either architecture, connectivity, or function. These complementary methods offered insights into the complex organization of the insula and proposed distinct parcellation schemes at varying levels of detail and complexity. The advent of imaging techniques that allow non-invasive assessment of structural and functional connectivity, has popularized data-driven connectivity-based parcellation methods to investigate the organization of the human insula. Yet, it remains unclear if the subdivisions derived from these data-driven clustering methods reflect meaningful descriptions of the functional specialization of the insula. In this study, we employed hierarchical clustering to examine the cluster parcellations of the macaque insula. As our aim was exploratory, we examined parcellations consisting of two up to ten clusters. Three different cluster validation methods (fingerprinting, silhouette, elbow) converged on a four-cluster solution as the most optimal representation of our data. Examining functional response properties of these clusters, in addition to their brain-wide functional connectivity suggested a functional specialization related to processing gustatory, somato-motor, vestibular and social visual cues. However, a more detailed functional differentiation aligning with previous functional investigations of insula subfields became evident at higher cluster numbers beyond the proposed optimal four clusters. Overall, our findings demonstrate that resting-state-based hierarchical clustering can provide a meaningful description of the insula's functional organization at some level of detail. Nonetheless, cluster parcellations derived from this method are best combined with data obtained through other modalities, to provide a more comprehensive and detailed account of the insula's complex functional organization.
Collapse
Affiliation(s)
- Lotte Sypré
- Laboratory for Neuro- & Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
- Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | | | - Dante Mantini
- Leuven Brain Institute, KU Leuven, Leuven, Belgium
- Movement Control & Neuroplasticity Research Group, KU Leuven, Leuven, Belgium
| | - Koen Nelissen
- Laboratory for Neuro- & Psychophysiology, Department of Neurosciences, KU Leuven, Leuven, Belgium
- Leuven Brain Institute, KU Leuven, Leuven, Belgium
| |
Collapse
|
4
|
Gugushvili A, Jarosz E. A longitudinal study of perceived social position and health-related quality of life. Soc Sci Med 2024; 340:116446. [PMID: 38042026 DOI: 10.1016/j.socscimed.2023.116446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Revised: 11/12/2023] [Accepted: 11/18/2023] [Indexed: 12/04/2023]
Abstract
BACKGROUND A large body of evidence suggests that there is a social gradient in the association between perceived social position and various health outcomes. Yet only a fraction of this research uses longitudinal data, and these studies usually rely on two data points in time, consider a single health outcome measure, overlook non-linear effects of perceived social position, and come almost exclusively from the Western welfare democracies. METHODS Using data for 1921 individuals from three waves (2008, 2013, 2018) of the Polish Panel Survey (POLPAN), we fit between- and within-individuals hybrid-effects models with cluster-robust standard errors to investigate the association between one's perceived social position (self-placement on a socioeconomic hierarchy scale varying 1 to 10) and subsequent health-related quality of life (HRQoL) measured using the Nottingham Health Profile (NHP) and its six components. RESULTS We find that the association between perceived social position and health-related quality of life is larger when estimated between individuals than within individuals, yet in fixed- and hybrid-effects models perceived social position remains significantly and negatively linked with both the aggregated NHP measure as well as with its components such as emotional reaction, physical abilities, sleep, and social isolation. We also identify that starting to perceive oneself at the lower end of the social hierarchy is associated with a deteriorating health-related quality of life but a change at the top of the perceived social hierarchy is not linked with an improvement in NHP scores. CONCLUSIONS We provide new evidence on the significant and non-linear links between perceived social position and health-related quality of life and highlight possible pathways linking these two aspects of individuals' lives.
Collapse
Affiliation(s)
- Alexi Gugushvili
- Department of Sociology and Human Geography, University of Oslo, Norway.
| | - Ewa Jarosz
- Faculty of Economic Sciences, University of Warsaw, Poland
| |
Collapse
|
5
|
Hoy CW, Quiroga-Martinez DR, Sandoval E, King-Stephens D, Laxer KD, Weber P, Lin JJ, Knight RT. Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex. Nat Commun 2023; 14:8520. [PMID: 38129440 PMCID: PMC10739882 DOI: 10.1038/s41467-023-44248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 12/05/2023] [Indexed: 12/23/2023] Open
Abstract
The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.
Collapse
Affiliation(s)
- Colin W Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - David R Quiroga-Martinez
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Aarhus, Denmark
| | - Eduardo Sandoval
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
6
|
Grabenhorst F, Ponce-Alvarez A, Battaglia-Mayer A, Deco G, Schultz W. A view-based decision mechanism for rewards in the primate amygdala. Neuron 2023; 111:3871-3884.e14. [PMID: 37725980 PMCID: PMC10914681 DOI: 10.1016/j.neuron.2023.08.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 07/12/2023] [Accepted: 08/23/2023] [Indexed: 09/21/2023]
Abstract
Primates make decisions visually by shifting their view from one object to the next, comparing values between objects, and choosing the best reward, even before acting. Here, we show that when monkeys make value-guided choices, amygdala neurons encode their decisions in an abstract, purely internal representation defined by the monkey's current view but not by specific object or reward properties. Across amygdala subdivisions, recorded activity patterns evolved gradually from an object-specific value code to a transient, object-independent code in which currently viewed and last-viewed objects competed to reflect the emerging view-based choice. Using neural-network modeling, we identified a sequence of computations by which amygdala neurons implemented view-based decision making and eventually recovered the chosen object's identity when the monkeys acted on their choice. These findings reveal a neural mechanism in the amygdala that derives object choices from abstract, view-based computations, suggesting an efficient solution for decision problems with many objects.
Collapse
Affiliation(s)
- Fabian Grabenhorst
- Department of Experimental Psychology, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK; Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK.
| | - Adrián Ponce-Alvarez
- Center for Brain and Cognition, Department of Technology and Information, Universitat Pompeu Fabra, Carrer Ramón Trias Fargas, 25-27, 08005 Barcelona, Spain; Departament de Matemàtiques, EPSEB, Universitat Politècnica de Catalunya, Barcelona, 08028 Barcelona, Spain
| | | | - Gustavo Deco
- Center for Brain and Cognition, Department of Technology and Information, Universitat Pompeu Fabra, Carrer Ramón Trias Fargas, 25-27, 08005 Barcelona, Spain; Institució Catalana de la Recerca i Estudis Avançats, Universitat Barcelona, Passeig Lluís Companys 23, 08010 Barcelona, Spain
| | - Wolfram Schultz
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK
| |
Collapse
|
7
|
Shih WY, Yu HY, Lee CC, Chou CC, Chen C, Glimcher PW, Wu SW. Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex. Nat Commun 2023; 14:7821. [PMID: 38016973 PMCID: PMC10684521 DOI: 10.1038/s41467-023-42092-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 09/28/2023] [Indexed: 11/30/2023] Open
Abstract
Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.
Collapse
Affiliation(s)
- Wan-Yu Shih
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| | - Hsiang-Yu Yu
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Cheng-Chia Lee
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
| | - Chien-Chen Chou
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Chien Chen
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Paul W Glimcher
- Neuroscience Institute, NYU Grossman School of Medicine, New York, NY, USA.
- Department of Neuroscience and Physiology, NYU Grossman School of Medicine, New York, NY, USA.
| | - Shih-Wei Wu
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| |
Collapse
|
8
|
Burk DC, Taswell C, Tang H, Averbeck BB. Computational mechanisms underlying motivation to earn symbolic reinforcers. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.11.561900. [PMID: 37873311 PMCID: PMC10592730 DOI: 10.1101/2023.10.11.561900] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Reinforcement learning (RL) is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g. money, points) that can later be exchanged for primary reinforcers (e.g. food, drink). Although symbolic reinforcers are motivating, little is understood about the neural or computational mechanisms underlying the motivation to earn them. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g. current number of accumulated tokens, choice options, task epoch, trials since last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to capture the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n=5 monkeys). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.
Collapse
Affiliation(s)
- Diana C. Burk
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda MD, 20892-4415
| | - Craig Taswell
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda MD, 20892-4415
| | - Hua Tang
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda MD, 20892-4415
| | - Bruno B. Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda MD, 20892-4415
| |
Collapse
|
9
|
Sypré L, Durand JB, Nelissen K. Functional characterization of macaque insula using task-based and resting-state fMRI. Neuroimage 2023; 276:120217. [PMID: 37271304 DOI: 10.1016/j.neuroimage.2023.120217] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 05/13/2023] [Accepted: 06/01/2023] [Indexed: 06/06/2023] Open
Abstract
Neurophysiological investigations over the past decades have demonstrated the involvement of the primate insula in a wide array of sensory, cognitive, affective and regulatory functions, yet the complex functional organization of the insula remains unclear. Here we examined to what extent non-invasive task-based and resting-state fMRI provides support for functional specialization and integration of sensory and motor information in the macaque insula. Task-based fMRI experiments suggested a functional specialization related to processing of ingestive/taste/distaste information in anterior insula, grasping-related sensorimotor responses in middle insula and vestibular information in posterior insula. Visual stimuli depicting social information involving conspecific`s lip-smacking gestures yielded responses in middle and anterior portions of dorsal and ventral insula, overlapping partially with the sensorimotor and ingestive/taste/distaste fields. Functional specialization/integration of the insula was further corroborated by seed-based whole brain resting-state analyses, showing distinct functional connectivity gradients across the anterio-posterior extent of both dorsal and ventral insula. Posterior insula showed functional correlations in particular with vestibular/optic flow network regions, mid-dorsal insula with vestibular/optic flow as well as parieto-frontal regions of the sensorimotor grasping network, mid-ventral insula with social/affiliative network regions in temporal, cingulate and prefrontal cortices and anterior insula with taste and mouth motor networks including premotor and frontal opercular regions.
Collapse
Affiliation(s)
- Lotte Sypré
- Laboratory for Neuro- & Psychophysiology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | | | - Koen Nelissen
- Laboratory for Neuro- & Psychophysiology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium.
| |
Collapse
|
10
|
McNally GP, Jean-Richard-Dit-Bressel P, Millan EZ, Lawrence AJ. Pathways to the persistence of drug use despite its adverse consequences. Mol Psychiatry 2023; 28:2228-2237. [PMID: 36997610 PMCID: PMC10611585 DOI: 10.1038/s41380-023-02040-z] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Revised: 03/10/2023] [Accepted: 03/15/2023] [Indexed: 04/01/2023]
Abstract
The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.
Collapse
Affiliation(s)
- Gavan P McNally
- School of Psychology, UNSW Sydney, Sydney, NSW, 2052, Australia.
| | | | - E Zayra Millan
- School of Psychology, UNSW Sydney, Sydney, NSW, 2052, Australia
| | - Andrew J Lawrence
- Florey Institute of Neuroscience and Mental Health, Parkville, VIC, 3010, Australia
- Florey Department of Neuroscience and Mental Health, University of Melbourne, Melbourne, VIC, 3010, Australia
| |
Collapse
|
11
|
Seak LCU, Ferrari-Toniolo S, Jain R, Nielsen K, Schultz W. Systematic comparison of risky choices in humans and monkeys. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.07.527517. [PMID: 36798272 PMCID: PMC9934584 DOI: 10.1101/2023.02.07.527517] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
The past decades have seen tremendous progress in fundamental studies on economic choice in humans. However, elucidation of the underlying neuronal processes requires invasive neurophysiological studies that are met with difficulties in humans. Monkeys as evolutionary closest relatives offer a solution. The animals display sophisticated and well-controllable behavior that allows to implement key constructs of proven economic choice theories. However, the similarity of economic choice between the two species has never been systematically investigated. We investigated compliance with the independence axiom (IA) of expected utility theory as one of the most demanding choice tests and compared IA violations between humans and monkeys. Using generalized linear modeling and cumulative prospect theory (CPT), we found that humans and monkeys made comparable risky choices, although their subjective values (utilities) differed. These results suggest similar fundamental choice mechanism across these primate species and encourage to study their underlying neurophysiological mechanisms.
Collapse
Affiliation(s)
- Leo Chi U Seak
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| | - Simone Ferrari-Toniolo
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| | - Ritesh Jain
- Management School, University of Liverpool, Liverpool L697ZY, United Kingdom
| | - Kirby Nielsen
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena CA 91125, USA
| | - Wolfram Schultz
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| |
Collapse
|
12
|
A neuronal prospect theory model in the brain reward circuitry. Nat Commun 2022; 13:5855. [PMID: 36195765 PMCID: PMC9532451 DOI: 10.1038/s41467-022-33579-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Accepted: 09/22/2022] [Indexed: 11/23/2022] Open
Abstract
Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk. It is unclear how the activity of individual neurons conform to prospect theory. Here, the authors demonstrate that the activity of single neurons in various reward-related regions in the monkey brain can be described as encoding a multiplicative combination of utility and probability weighting, and that this subjective valuation process is achieved via a distributed coding scheme.
Collapse
|