1
|
Goudar V, Kim JW, Liu Y, Dede AJO, Jutras MJ, Skelin I, Ruvalcaba M, Chang W, Ram B, Fairhall AL, Lin JJ, Knight RT, Buffalo EA, Wang XJ. A Comparison of Rapid Rule-Learning Strategies in Humans and Monkeys. J Neurosci 2024; 44:e0231232024. [PMID: 38871463 PMCID: PMC11236592 DOI: 10.1523/jneurosci.0231-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 05/28/2024] [Accepted: 05/31/2024] [Indexed: 06/15/2024] Open
Abstract
Interspecies comparisons are key to deriving an understanding of the behavioral and neural correlates of human cognition from animal models. We perform a detailed comparison of the strategies of female macaque monkeys to male and female humans on a variant of the Wisconsin Card Sorting Test (WCST), a widely studied and applied task that provides a multiattribute measure of cognitive function and depends on the frontal lobe. WCST performance requires the inference of a rule change given ambiguous feedback. We found that well-trained monkeys infer new rules three times more slowly than minimally instructed humans. Input-dependent hidden Markov model-generalized linear models were fit to their choices, revealing hidden states akin to feature-based attention in both species. Decision processes resembled a win-stay, lose-shift strategy with interspecies similarities as well as key differences. Monkeys and humans both test multiple rule hypotheses over a series of rule-search trials and perform inference-like computations to exclude candidate choice options. We quantitatively show that perseveration, random exploration, and poor sensitivity to negative feedback account for the slower task-switching performance in monkeys.
Collapse
Affiliation(s)
- Vishwa Goudar
- Center for Neural Science, New York University, New York 10003
| | - Jeong-Woo Kim
- Center for Neural Science, New York University, New York 10003
| | - Yue Liu
- Center for Neural Science, New York University, New York 10003
| | - Adam J O Dede
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington 98195
| | - Michael J Jutras
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington 98195
| | - Ivan Skelin
- Department of Neurology, University of California, Davis, California 95616
- The Center for Mind and Brain, University of California, Davis, California 95616
| | - Michael Ruvalcaba
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
| | - William Chang
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
| | - Bhargavi Ram
- Department of Neurology, University of California, Davis, California 95616
- The Center for Mind and Brain, University of California, Davis, California 95616
| | - Adrienne L Fairhall
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington 98195
| | - Jack J Lin
- Department of Neurology, University of California, Davis, California 95616
- The Center for Mind and Brain, University of California, Davis, California 95616
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720
- Department of Psychology, University of California, Berkeley, California 94720
| | - Elizabeth A Buffalo
- Department of Physiology and Biophysics, University of Washington, Seattle, Washington 98195
- Washington Primate Research Center, University of Washington, Seattle, Washington 98195
| | - Xiao-Jing Wang
- Center for Neural Science, New York University, New York 10003
| |
Collapse
|
2
|
Feng YY, Bromberg-Martin ES, Monosov IE. Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making. Cell Rep 2024; 43:114341. [PMID: 38878290 DOI: 10.1016/j.celrep.2024.114341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 03/27/2024] [Accepted: 05/23/2024] [Indexed: 06/25/2024] Open
Abstract
The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.
Collapse
Affiliation(s)
- Yang-Yang Feng
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA; Department of Biomedical Engineering, Washington University, St. Louis, MO, USA
| | | | - Ilya E Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA; Department of Biomedical Engineering, Washington University, St. Louis, MO, USA; Washington University Pain Center, Washington University, St. Louis, MO, USA; Department of Neurosurgery, Washington University, St. Louis, MO, USA; Department of Electrical Engineering, Washington University, St. Louis, MO, USA.
| |
Collapse
|
3
|
Raizel MF, de Oliveira Sergio T, Starski PA, Hopf FW. Heart rate variability: A primer for alcohol researchers. Alcohol 2024:S0741-8329(24)00085-5. [PMID: 38906390 DOI: 10.1016/j.alcohol.2024.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Revised: 06/04/2024] [Accepted: 06/05/2024] [Indexed: 06/23/2024]
Abstract
Problem alcohol drinking remains a major cost and burden for society. Also, rates of problem drinking in women have dramatically increased in recent decades, and women are at risk for more alcohol problems and comorbidities. The purpose of this commentary is to discuss the potential utility of cardiac measures, including heart rate (HR) and HR variability (HRV), as markers of individual and sex differences in the drive to drink alcohol. We recently used cardiac telemetry in female and male adult rats to determine whether different cardiac markers, including HR and HRV, would differently predict alcohol and anxiety-like behavior across the sexes. Indeed, female behaviors related to HRV measures that indicate more parasympathetic (PNS) influence (the "rest and digest" system). In contrast, male behaviors are associated more with sympathetic (SNS) indicators (the activation system). Remarkably, similar sex differences in PNS versus SNS engagement under challenge are seen in human studies, suggesting strong cross-species convergence in differential autonomic regulation in females and males. Here, we describe the larger challenges that alcohol addiction presents, and how HRV measures may provide new biomarkers to help enhance development of more individualized and sex-specific treatments. We briefly explain the physiological systems underlying cardiac PNS and SNS states, and how specific HRV metrics are defined and validated, especially why particular HRV measures are considered to reflect more PNS versus SNS influence. Finally, we describe hormonal influences and sex differences in brain circuits related to cardiac autonomic regulation. Together, these findings show that HR and HRV have potential for uncovering key underlying mechanisms of sex and individual differences in autonomic drivers, which could guide more personalized treatment.
Collapse
Affiliation(s)
- M Frasier Raizel
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana, USA; Indiana University School of Medicine, Medical Scientist Training Program, Indianapolis, Indiana, USA
| | - Thatiane de Oliveira Sergio
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana, USA; Stark Neurosciences Research Institute, Indiana University, Indianapolis, Indiana, USA
| | - Phillip A Starski
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana, USA; Stark Neurosciences Research Institute, Indiana University, Indianapolis, Indiana, USA
| | - F Woodward Hopf
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, Indiana, USA; Stark Neurosciences Research Institute, Indiana University, Indianapolis, Indiana, USA.
| |
Collapse
|
4
|
Lockwood PL, Cutler J, Drew D, Abdurahman A, Jeyaretna DS, Apps MAJ, Husain M, Manohar SG. Human ventromedial prefrontal cortex is necessary for prosocial motivation. Nat Hum Behav 2024:10.1038/s41562-024-01899-4. [PMID: 38802539 DOI: 10.1038/s41562-024-01899-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 04/23/2024] [Indexed: 05/29/2024]
Abstract
Ventromedial prefrontal cortex (vmPFC) is vital for decision-making. Functional neuroimaging links vmPFC to processing rewards and effort, while parallel work suggests vmPFC involvement in prosocial behaviour. However, the necessity of vmPFC for these functions is unknown. Patients with rare focal vmPFC lesions (n = 25), patients with lesions elsewhere (n = 15) and healthy controls (n = 40) chose between rest and exerting effort to earn rewards for themselves or another person. vmPFC damage decreased prosociality across behavioural and computational measures. vmPFC patients earned less, discounted rewards by effort more, and exerted less force when another person benefited, compared to both control groups. Voxel-based lesion mapping revealed dissociations between vmPFC subregions. While medial damage led to antisocial behaviour, lateral damage increased prosocial behaviour relative to patients with damage elsewhere. vmPFC patients also showed reduced effort sensitivity overall, but reward sensitivity was limited to specific subregions. These results reveal multiple causal contributions of vmPFC to prosocial behaviour, effort and reward.
Collapse
Affiliation(s)
- Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - Jo Cutler
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - Daniel Drew
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Ayat Abdurahman
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Psychology, University of Cambridge, Cambridge, UK
| | - Deva Sanjeeva Jeyaretna
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Neurology, John Radcliffe Hospital, Oxford, UK
| | - Matthew A J Apps
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Masud Husain
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Neurology, John Radcliffe Hospital, Oxford, UK
| | - Sanjay G Manohar
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Neurology, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
5
|
Fouragnan EF, Hosking B, Cheung Y, Prakash B, Rushworth M, Sel A. Timing along the cardiac cycle modulates neural signals of reward-based learning. Nat Commun 2024; 15:2976. [PMID: 38582905 PMCID: PMC10998831 DOI: 10.1038/s41467-024-46921-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Accepted: 03/14/2024] [Indexed: 04/08/2024] Open
Abstract
Natural fluctuations in cardiac activity modulate brain activity associated with sensory stimuli, as well as perceptual decisions about low magnitude, near-threshold stimuli. However, little is known about the relationship between fluctuations in heart activity and other internal representations. Here we investigate whether the cardiac cycle relates to learning-related internal representations - absolute and signed prediction errors. We combined machine learning techniques with electroencephalography with both simple, direct indices of task performance and computational model-derived indices of learning. Our results demonstrate that just as people are more sensitive to low magnitude, near-threshold sensory stimuli in certain cardiac phases, so are they more sensitive to low magnitude absolute prediction errors in the same cycles. However, this occurs even when the low magnitude prediction errors are associated with clearly suprathreshold sensory events. In addition, participants exhibiting stronger differences in their prediction error representations between cardiac cycles exhibited higher learning rates and greater task accuracy.
Collapse
Affiliation(s)
- Elsa F Fouragnan
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK.
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK.
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK.
| | - Billy Hosking
- Brain Research Imaging Centre (BRIC), Faculty of Health, University of Plymouth, Plymouth, PL6 8BU, UK
- School of Psychology, Faculty of Health, University of Plymouth, Plymouth, PL4 8AA, UK
| | - Yin Cheung
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Brooke Prakash
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Matthew Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
| | - Alejandra Sel
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK
- Centre for Brain Science, Department of Psychology, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
- Essex ESNEFT Psychological Research Unit for Behaviour, Health and Wellbeing, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK
| |
Collapse
|
6
|
Starski P, Morningstar MD, Katner SN, Frasier RM, De Oliveira Sergio T, Wean S, Lapish CC, Hopf FW. Neural Activity in the Anterior Insula at Drinking Onset and Licking Relates to Compulsion-Like Alcohol Consumption. J Neurosci 2024; 44:e1490232023. [PMID: 38242696 PMCID: PMC10904088 DOI: 10.1523/jneurosci.1490-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 11/20/2023] [Accepted: 12/10/2023] [Indexed: 01/21/2024] Open
Abstract
Much remains unknown about the etiology of compulsion-like alcohol drinking, where consumption persists despite adverse consequences. The role of the anterior insula (AIC) in emotion, motivation, and interoception makes this brain region a likely candidate to drive challenge-resistant behavior, including compulsive drinking. Indeed, subcortical projections from the AIC promote compulsion-like intake in rats and are recruited in heavy-drinking humans during compulsion for alcohol, highlighting the importance of and need for more information about AIC activity patterns that support aversion-resistant responding. Single-unit activity was recorded in the AIC from 15 male rats during alcohol-only and compulsion-like consumption. We found three sustained firing phenotypes, sustained-increase, sustained-decrease, and drinking-onset cells, as well as several firing patterns synchronized with licking. While many AIC neurons had session-long activity changes, only neurons with firing increases at drinking onset had greater activity under compulsion-like conditions. Further, only cells with persistent firing increases maintained activity during pauses in licking, suggesting roles in maintaining drive for alcohol during breaks. AIC firing was not elevated during saccharin drinking, similar to lack of effect of AIC inhibition on sweet fluid intake in many studies. In addition, we observed subsecond changes in AIC neural activity tightly entrained to licking. One lick-synched firing pattern (determined for all licks in a session) predicted compulsion-like drinking, while a separate lick-associated pattern correlated with greater consumption across alcohol intake conditions. Collectively, these data provide a more integrated model for the role of AIC firing in compulsion-like drinking, with important relevance for how the AIC promotes sustained motivated responding more generally.
Collapse
Affiliation(s)
- Phillip Starski
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis 46202, Indiana
| | - Mitch D Morningstar
- Department of Psychology, IU-Purdue University Indianapolis, Indianapolis 46202, Indiana
| | - Simon N Katner
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis 46202, Indiana
| | - Raizel M Frasier
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis 46202, Indiana
| | | | - Sarah Wean
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis 46202, Indiana
| | - Christopher C Lapish
- Department of Anatomy, Cell Biology, and Physiology, IU School of Medicine, Indianapolis 46202, Indiana
- Stark Neurosciences Research Institute, Indianapolis 46202, Indiana
| | - F Woodward Hopf
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis 46202, Indiana
- Stark Neurosciences Research Institute, Indianapolis 46202, Indiana
| |
Collapse
|
7
|
Luo Q, Kanen JW, Bari A, Skandali N, Langley C, Knudsen GM, Alsiö J, Phillips BU, Sahakian BJ, Cardinal RN, Robbins TW. Comparable roles for serotonin in rats and humans for computations underlying flexible decision-making. Neuropsychopharmacology 2024; 49:600-608. [PMID: 37914893 PMCID: PMC10789782 DOI: 10.1038/s41386-023-01762-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 09/22/2023] [Accepted: 10/17/2023] [Indexed: 11/03/2023]
Abstract
Serotonin is critical for adapting behavior flexibly to meet changing environmental demands. Cognitive flexibility is important for successful attainment of goals, as well as for social interactions, and is frequently impaired in neuropsychiatric disorders, including obsessive-compulsive disorder. However, a unifying mechanistic framework accounting for the role of serotonin in behavioral flexibility has remained elusive. Here, we demonstrate common effects of manipulating serotonin function across two species (rats and humans) on latent processes supporting choice behavior during probabilistic reversal learning, using computational modelling. The findings support a role of serotonin in behavioral flexibility and plasticity, indicated, respectively, by increases or decreases in choice repetition ('stickiness') or reinforcement learning rates following manipulations intended to increase or decrease serotonin function. More specifically, the rate at which expected value increased following reward and decreased following punishment (reward and punishment 'learning rates') was greatest after sub-chronic administration of the selective serotonin reuptake inhibitor (SSRI) citalopram (5 mg/kg for 7 days followed by 10 mg/kg twice a day for 5 days) in rats. Conversely, humans given a single dose of an SSRI (20 mg escitalopram), which can decrease post-synaptic serotonin signalling, and rats that received the neurotoxin 5,7-dihydroxytryptamine (5,7-DHT), which destroys forebrain serotonergic neurons, exhibited decreased reward learning rates. A basic perseverative tendency ('stickiness'), or choice repetition irrespective of the outcome produced, was likewise increased in rats after the 12-day SSRI regimen and decreased after single dose SSRI in humans and 5,7-DHT in rats. These common effects of serotonergic manipulations on rats and humans-identified via computational modelling-suggest an evolutionarily conserved role for serotonin in plasticity and behavioral flexibility and have clinical relevance transdiagnostically for neuropsychiatric disorders.
Collapse
Affiliation(s)
- Qiang Luo
- National Clinical Research Center for Aging and Medicine at Huashan Hospital, State Key Laboratory of Medical Neurobiology and Ministry of Education Frontiers Center for Brain Science, Institutes of Brain Science and Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, P. R. China.
- Center for Computational Psychiatry, Ministry of Education Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Human Phenome Institute, Fudan University, Shanghai, 200433, China.
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK.
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK.
| | - Jonathan W Kanen
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK
| | | | - Nikolina Skandali
- Department of Psychiatry, University of Cambridge, Cambridge, CB2 0SZ, UK
- Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, CB21 5EF, UK
- NIHR Cambridge Biomedical Research Centre, University of Cambridge, Cambridge, CB2 0QQ, UK
| | - Christelle Langley
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK
- Department of Psychiatry, University of Cambridge, Cambridge, CB2 0SZ, UK
| | - Gitte Moos Knudsen
- Neurobiology Research Unit, the Neuroscience Centre, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Johan Alsiö
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK
| | - Benjamin U Phillips
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK
| | - Barbara J Sahakian
- National Clinical Research Center for Aging and Medicine at Huashan Hospital, State Key Laboratory of Medical Neurobiology and Ministry of Education Frontiers Center for Brain Science, Institutes of Brain Science and Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, P. R. China
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK
- Department of Psychiatry, University of Cambridge, Cambridge, CB2 0SZ, UK
| | - Rudolf N Cardinal
- Department of Psychiatry, University of Cambridge, Cambridge, CB2 0SZ, UK
- Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, CB21 5EF, UK
| | - Trevor W Robbins
- National Clinical Research Center for Aging and Medicine at Huashan Hospital, State Key Laboratory of Medical Neurobiology and Ministry of Education Frontiers Center for Brain Science, Institutes of Brain Science and Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, 200433, P. R. China.
- Department of Psychology, University of Cambridge, Cambridge, CB2 3EB, UK.
- Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, CB2 3EB, UK.
| |
Collapse
|
8
|
Hoy CW, Quiroga-Martinez DR, Sandoval E, King-Stephens D, Laxer KD, Weber P, Lin JJ, Knight RT. Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex. Nat Commun 2023; 14:8520. [PMID: 38129440 PMCID: PMC10739882 DOI: 10.1038/s41467-023-44248-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2022] [Accepted: 12/05/2023] [Indexed: 12/23/2023] Open
Abstract
The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.
Collapse
Affiliation(s)
- Colin W Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA.
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| | - David R Quiroga-Martinez
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Aarhus, Denmark
| | - Eduardo Sandoval
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA
| | - Kenneth D Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA
| | - Jack J Lin
- Department of Neurology, University of California, Davis, Davis, CA, USA
- Center for Mind and Brain, University of California, Davis, Davis, CA, USA
| | - Robert T Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA
- Department of Psychology, University of California, Berkeley, Berkeley, CA, USA
| |
Collapse
|
9
|
Hoy CW, de Hemptinne C, Wang SS, Harmer CJ, Apps MAJ, Husain M, Starr PA, Little S. Beta and theta oscillations track effort and previous reward in human basal ganglia and prefrontal cortex during decision making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.05.570285. [PMID: 38106063 PMCID: PMC10723308 DOI: 10.1101/2023.12.05.570285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we investigate this with chronic intracranial recordings from prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12-20 Hz) oscillations tracking subjective effort on a single trial basis and PFC theta (4-7 Hz) signaling previous trial reward. Stimulation of PFC increased overall acceptance of offers in addition to increasing the impact of reward on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, as well as supporting a causal role of PFC for such choices.
Collapse
Affiliation(s)
- Colin W. Hoy
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| | - Coralie de Hemptinne
- Norman Fixel Institute for Neurological Diseases, University of Florida, Gainesville, FL, USA
- Department of Neurology, University of Florida, Gainesville, FL, USA
| | - Sarah S. Wang
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| | | | - Mathew A. J. Apps
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Masud Husain
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Philip A. Starr
- Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA
| | - Simon Little
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| |
Collapse
|
10
|
Lloyd A, Viding E, McKay R, Furl N. Understanding patch foraging strategies across development. Trends Cogn Sci 2023; 27:1085-1098. [PMID: 37500422 DOI: 10.1016/j.tics.2023.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 07/29/2023]
Abstract
Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.
Collapse
Affiliation(s)
- Alex Lloyd
- Clinical, Educational, and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK.
| | - Essi Viding
- Clinical, Educational, and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK
| | - Ryan McKay
- Department of Psychology, Royal Holloway, University of London, Egham Hill, Egham, TW20 0EX, UK
| | - Nicholas Furl
- Department of Psychology, Royal Holloway, University of London, Egham Hill, Egham, TW20 0EX, UK
| |
Collapse
|
11
|
Pauli R, Brazil IA, Kohls G, Klein-Flügge MC, Rogers JC, Dikeos D, Dochnal R, Fairchild G, Fernández-Rivas A, Herpertz-Dahlmann B, Hervas A, Konrad K, Popma A, Stadler C, Freitag CM, De Brito SA, Lockwood PL. Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable. Nat Commun 2023; 14:5689. [PMID: 37709750 PMCID: PMC10502052 DOI: 10.1038/s41467-023-41124-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 08/24/2023] [Indexed: 09/16/2023] Open
Abstract
Theoretical and empirical accounts suggest that adolescence is associated with heightened reward learning and impulsivity. Experimental tasks and computational models that can dissociate reward learning from the tendency to initiate actions impulsively (action initiation bias) are thus critical to characterise the mechanisms that drive developmental differences. However, existing work has rarely quantified both learning ability and action initiation, or it has relied on small samples. Here, using computational modelling of a learning task collected from a large sample (N = 742, 9-18 years, 11 countries), we test differences in reward and punishment learning and action initiation from childhood to adolescence. Computational modelling reveals that whilst punishment learning rates increase with age, reward learning remains stable. In parallel, action initiation biases decrease with age. Results are similar when considering pubertal stage instead of chronological age. We conclude that heightened reward responsivity in adolescence can reflect differences in action initiation rather than enhanced reward learning.
Collapse
Affiliation(s)
- Ruth Pauli
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
| | - Inti A Brazil
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - Gregor Kohls
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- Department of Child and Adolescent Psychiatry, Faculty of Medicine, TU, Dresden, Germany
| | - Miriam C Klein-Flügge
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Jack C Rogers
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Dimitris Dikeos
- Department of Psychiatry, Medical School, National and Kapodistrian University of Athens, Athens, Greece
| | - Roberta Dochnal
- Faculty of Medicine, Child and Adolescent Psychiatry, Department of the Child Health Center, Szeged University, Szeged, Hungary
| | | | | | - Beate Herpertz-Dahlmann
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
| | - Amaia Hervas
- University Hospital Mutua Terrassa, Barcelona, Spain
| | - Kerstin Konrad
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, RWTH Aachen University, Aachen, Germany
- JARA-Brain Institute II, Molecular Neuroscience and Neuroimaging, RWTH Aachen and Research Centre Jülich, Jülich, Germany
| | - Arne Popma
- Department of Child and Adolescent Psychiatry, VU University Medical Center, Amsterdam, Netherlands
| | - Christina Stadler
- Department of Child and Adolescent Psychiatry, Psychiatric University Hospital, University of Basel, Basel, Switzerland
| | - Christine M Freitag
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital Frankfurt, Goethe University, Frankfurt am Main, Germany
| | - Stephane A De Brito
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
| |
Collapse
|
12
|
Starski PA, De Oliveira Sergio T, Hopf FW. Using lickometry to infer differential contributions of salience network regions during compulsion-like alcohol drinking. ADDICTION NEUROSCIENCE 2023; 7:100102. [PMID: 38736902 PMCID: PMC11086682 DOI: 10.1016/j.addicn.2023.100102] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 05/14/2024]
Abstract
Alcohol use disorder extracts substantial personal, social and clinical costs, and continued intake despite negative consequences (compulsion-like consumption) can contribute strongly. Here we discuss lickometry, a simple method where lick times are determined across a session, while analysis across many aspects of licking can offer important insights into underlying psychological and action strategies, including their brain mechanisms. We first describe studies implicating anterior insula (AIC) and dorsal medial prefrontal cortex (dMPF) in compulsion-like responding for alcohol, then review work suggesting that AIC/ventral frontal cortex versus dMPF regulate different aspects of behavior (oral control and overall response strategy, versus moment-to-moment action organization). We then detail our lickometer work comparing alcohol-only drinking (AOD) and compulsion-like drinking under moderate- or higher-challenge (ModChD or HiChD, using quinine-alcohol). Many studies have suggested utilization of one of two main strategies, with higher motivation indicated by more bouts, and greater palatability suggested by longer, faster bouts. Instead, ModChD shows decreased variability in many lick measures, which is unexpected but consistent with the suggested importance of automaticity for addiction. Also surprising is that HiChD retains several behavior changes seen with ModChD, reduced tongue variability and earlier bout start, even though intake is otherwise disrupted. Since AIC-related measures are retained under both moderate- and higher-challenge, we propose a novel hypothesis that AIC sustains overall commitment regardless of challenge level, while disordered licking during HiChD mirrors the effects of dMPF inhibition. Thus, while AIC provides overall drive despite challenge, the ability to act is ultimately determined within the dMPF.
Collapse
Affiliation(s)
- Phillip A. Starski
- Indiana University School of Medicine, Department of Psychiatry, Indianapolis IN, USA
| | | | - Frederic W. Hopf
- Indiana University School of Medicine, Department of Psychiatry, Indianapolis IN, USA
- Stark Neurosciences Research Institute, Indianapolis IN, USA
| |
Collapse
|
13
|
Feng YY, Bromberg-Martin ES, Monosov IE. Dorsal raphe neurons signal integrated value during multi-attribute decision-making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.17.553745. [PMID: 37662243 PMCID: PMC10473596 DOI: 10.1101/2023.08.17.553745] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when grappling with reward uncertainty. However, whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider all these attributes to make a choice, is unclear. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes. Remarkably, these neurons commonly integrated offer attributes in a manner that reflected monkeys' overall preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how DRN participates in integrated value computations, guiding theories of DRN in decision-making and psychiatric disease.
Collapse
Affiliation(s)
- Yang-Yang Feng
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
| | | | - Ilya E. Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Department of Biomedical Engineering, Washington University, St. Louis, Missouri, USA
- Washington University Pain Center, Washington University, St. Louis, Missouri, USA
- Department of Neurosurgery, Washington University, St. Louis, Missouri, USA
- Department of Electrical Engineering, Washington University, St. Louis, Missouri, USA
| |
Collapse
|
14
|
Woo JH, Aguirre CG, Bari BA, Tsutsui KI, Grabenhorst F, Cohen JY, Schultz W, Izquierdo A, Soltani A. Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:600-619. [PMID: 36823249 PMCID: PMC10444905 DOI: 10.3758/s13415-022-01059-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 12/22/2022] [Indexed: 02/25/2023]
Abstract
Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.
Collapse
Affiliation(s)
- Jae Hyung Woo
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Claudia G Aguirre
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Bilal A Bari
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - Ken-Ichiro Tsutsui
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
- Laboratory of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, Sendai, Japan
| | - Fabian Grabenhorst
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Allen Institute for Neural Dynamics, Seattle, WA, USA
| | - Wolfram Schultz
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
| | - Alicia Izquierdo
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
- The Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA
| | - Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|
15
|
Jansen M, Lockwood PL, Cutler J, de Bruijn ERA. l-DOPA and oxytocin influence the neurocomputational mechanisms of self-benefitting and prosocial reinforcement learning. Neuroimage 2023; 270:119983. [PMID: 36848972 DOI: 10.1016/j.neuroimage.2023.119983] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 02/03/2023] [Accepted: 02/23/2023] [Indexed: 02/27/2023] Open
Abstract
Humans learn through reinforcement, particularly when outcomes are unexpected. Recent research suggests similar mechanisms drive how we learn to benefit other people, that is, how we learn to be prosocial. Yet the neurochemical mechanisms underlying such prosocial computations remain poorly understood. Here, we investigated whether pharmacological manipulation of oxytocin and dopamine influence the neurocomputational mechanisms underlying self-benefitting and prosocial reinforcement learning. Using a double-blind placebo-controlled cross-over design, we administered intranasal oxytocin (24 IU), dopamine precursor l-DOPA (100 mg + 25 mg carbidopa), or placebo over three sessions. Participants performed a probabilistic reinforcement learning task with potential rewards for themselves, another participant, or no one, during functional magnetic resonance imaging. Computational models of reinforcement learning were used to calculate prediction errors (PEs) and learning rates. Participants behavior was best explained by a model with different learning rates for each recipient, but these were unaffected by either drug. On the neural level, however, both drugs blunted PE signaling in the ventral striatum and led to negative signaling of PEs in the anterior mid-cingulate cortex, dorsolateral prefrontal cortex, inferior parietal gyrus, and precentral gyrus, compared to placebo, and regardless of recipient. Oxytocin (versus placebo) administration was additionally associated with opposing tracking of self-benefitting versus prosocial PEs in dorsal anterior cingulate cortex, insula and superior temporal gyrus. These findings suggest that both l-DOPA and oxytocin induce a context-independent shift from positive towards negative tracking of PEs during learning. Moreover, oxytocin may have opposing effects on PE signaling when learning to benefit oneself versus another.
Collapse
Affiliation(s)
- Myrthe Jansen
- Department of Clinical Psychology, Leiden University, the Netherlands; Leiden Institute for Brain and Cognition (LIBC), Leiden, the Netherlands.
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK; Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK; Centre for Developmental Science, School of Psychology, University of Birmingham, UK
| | - Jo Cutler
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK; Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK; Centre for Developmental Science, School of Psychology, University of Birmingham, UK
| | - Ellen R A de Bruijn
- Department of Clinical Psychology, Leiden University, the Netherlands; Leiden Institute for Brain and Cognition (LIBC), Leiden, the Netherlands
| |
Collapse
|
16
|
Huang FY, Grabenhorst F. Nutrient-Sensitive Reinforcement Learning in Monkeys. J Neurosci 2023; 43:1714-1730. [PMID: 36669886 PMCID: PMC10010454 DOI: 10.1523/jneurosci.0752-22.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2022] [Revised: 11/27/2022] [Accepted: 12/19/2022] [Indexed: 01/21/2023] Open
Abstract
In reinforcement learning (RL), animals choose by assigning values to options and learn by updating these values from reward outcomes. This framework has been instrumental in identifying fundamental learning variables and their neuronal implementations. However, canonical RL models do not explain how reward values are constructed from biologically critical intrinsic reward components, such as nutrients. From an ecological perspective, animals should adapt their foraging choices in dynamic environments to acquire nutrients that are essential for survival. Here, to advance the biological and ecological validity of RL models, we investigated how (male) monkeys adapt their choices to obtain preferred nutrient rewards under varying reward probabilities. We found that the nutrient composition of rewards strongly influenced learning and choices. Preferences of the animals for specific nutrients (sugar, fat) affected how they adapted to changing reward probabilities; the history of recent rewards influenced choices of the monkeys more strongly if these rewards contained the their preferred nutrients (nutrient-specific reward history). The monkeys also chose preferred nutrients even when they were associated with lower reward probability. A nutrient-sensitive RL model captured these processes; it updated the values of individual sugar and fat components of expected rewards based on experience and integrated them into subjective values that explained the choices of the monkeys. Nutrient-specific reward prediction errors guided this value-updating process. Our results identify nutrients as important reward components that guide learning and choice by influencing the subjective value of choice options. Extending RL models with nutrient-value functions may enhance their biological validity and uncover nutrient-specific learning and decision variables.SIGNIFICANCE STATEMENT RL is an influential framework that formalizes how animals learn from experienced rewards. Although reward is a foundational concept in RL theory, canonical RL models cannot explain how learning depends on specific reward properties, such as nutrients. Intuitively, learning should be sensitive to the nutrient components of the reward to benefit health and survival. Here, we show that the nutrient (fat, sugar) composition of rewards affects how the monkeys choose and learn in an RL paradigm and that key learning variables including reward history and reward prediction error should be modified with nutrient-specific components to account for the choice behavior observed in the monkeys. By incorporating biologically critical nutrient rewards into the RL framework, our findings help advance the ecological validity of RL models.
Collapse
Affiliation(s)
- Fei-Yang Huang
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3TA, United Kingdom
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| | - Fabian Grabenhorst
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3TA, United Kingdom
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| |
Collapse
|
17
|
Wittmann MK, Scheuplein M, Gibbons SG, Noonan MP. Local and global reward learning in the lateral frontal cortex show differential development during human adolescence. PLoS Biol 2023; 21:e3002010. [PMID: 36862726 PMCID: PMC10013901 DOI: 10.1371/journal.pbio.3002010] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 03/14/2023] [Accepted: 01/20/2023] [Indexed: 03/03/2023] Open
Abstract
Reward-guided choice is fundamental for adaptive behaviour and depends on several component processes supported by prefrontal cortex. Here, across three studies, we show that two such component processes, linking reward to specific choices and estimating the global reward state, develop during human adolescence and are linked to the lateral portions of the prefrontal cortex. These processes reflect the assignment of rewards contingently to local choices, or noncontingently, to choices that make up the global reward history. Using matched experimental tasks and analysis platforms, we show the influence of both mechanisms increase during adolescence (study 1) and that lesions to lateral frontal cortex (that included and/or disconnected both orbitofrontal and insula cortex) in human adult patients (study 2) and macaque monkeys (study 3) impair both local and global reward learning. Developmental effects were distinguishable from the influence of a decision bias on choice behaviour, known to depend on medial prefrontal cortex. Differences in local and global assignments of reward to choices across adolescence, in the context of delayed grey matter maturation of the lateral orbitofrontal and anterior insula cortex, may underlie changes in adaptive behaviour.
Collapse
Affiliation(s)
- Marco K. Wittmann
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Headington, Oxford, United Kingdom
- Department of Experimental Psychology, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, United Kingdom
| | - Maximilian Scheuplein
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Institute of Education and Child Studies, Leiden University, Leiden, the Netherlands
| | - Sophie G. Gibbons
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
| | - MaryAnn P. Noonan
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Department of Psychology, University of York, York, United Kingdom
- * E-mail:
| |
Collapse
|
18
|
Trudel N, Lockwood PL, Rushworth MFS, Wittmann MK. Neural activity tracking identity and confidence in social information. eLife 2023; 12:71315. [PMID: 36763582 PMCID: PMC9917428 DOI: 10.7554/elife.71315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2021] [Accepted: 12/15/2022] [Indexed: 02/11/2023] Open
Abstract
Humans learn about the environment either directly by interacting with it or indirectly by seeking information about it from social sources such as conspecifics. The degree of confidence in the information obtained through either route should determine the impact that it has on adapting and changing behaviour. We examined whether and how behavioural and neural computations differ during non-social learning as opposed to learning from social sources. Trial-wise confidence judgements about non-social and social information sources offered a window into this learning process. Despite matching exactly the statistical features of social and non-social conditions, confidence judgements were more accurate and less changeable when they were made about social as opposed to non-social information sources. In addition to subjective reports of confidence, differences were also apparent in the Bayesian estimates of participants' subjective beliefs. Univariate activity in dorsomedial prefrontal cortex and posterior temporoparietal junction more closely tracked confidence about social as opposed to non-social information sources. In addition, the multivariate patterns of activity in the same areas encoded identities of social information sources compared to non-social information sources.
Collapse
Affiliation(s)
- Nadescha Trudel
- Wellcome Centre of Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College LondonLondonUnited Kingdom
| | - Patricia L Lockwood
- Centre for Human Brain Health, School of Psychology, University of BirminghamBirminghamUnited Kingdom
- Institute for Mental Health, School of Psychology, University of BirminghamBirminghamUnited Kingdom
- Centre for Developmental Science, School of Psychology, University of BirminghamBirminghamUnited Kingdom
| | - Matthew FS Rushworth
- Wellcome Centre of Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Wellcome Centre of Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of OxfordOxfordUnited Kingdom
| | - Marco K Wittmann
- Wellcome Centre of Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College LondonLondonUnited Kingdom
- Department of Experimental Psychology, University College LondonLondonUnited Kingdom
| |
Collapse
|
19
|
Klein-Flügge MC, Jensen DEA, Takagi Y, Priestley L, Verhagen L, Smith SM, Rushworth MFS. Relationship between nuclei-specific amygdala connectivity and mental health dimensions in humans. Nat Hum Behav 2022; 6:1705-1722. [PMID: 36138220 PMCID: PMC7613949 DOI: 10.1038/s41562-022-01434-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2020] [Accepted: 07/14/2022] [Indexed: 01/14/2023]
Abstract
There has been increasing interest in using neuroimaging measures to predict psychiatric disorders. However, predictions usually rely on large brain networks and large disorder heterogeneity. Thus, they lack both anatomical and behavioural specificity, preventing the advancement of targeted interventions. Here we address both challenges. First, using resting-state functional magnetic resonance imaging, we parcellated the amygdala, a region implicated in mood disorders, into seven nuclei. Next, a questionnaire factor analysis provided subclinical mental health dimensions frequently altered in anxious-depressive individuals, such as negative emotions and sleep problems. Finally, for each behavioural dimension, we identified the most predictive resting-state functional connectivity between individual amygdala nuclei and highly specific regions of interest, such as the dorsal raphe nucleus in the brainstem or medial frontal cortical regions. Connectivity in circumscribed amygdala networks predicted behaviours in an independent dataset. Our results reveal specific relations between mental health dimensions and connectivity in precise subcortical networks.
Collapse
Affiliation(s)
- Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK.
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK.
| | - Daria E A Jensen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK
| | - Yu Takagi
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
- Department of Psychiatry, Warneford Hospital, University of Oxford, Oxford, UK
- Graduate School of Frontier Biosciences, Osaka University, Suita, Japan
| | - Luke Priestley
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Stephen M Smith
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB) and Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| |
Collapse
|
20
|
Klein-Flügge MC, Bongioanni A, Rushworth MFS. Medial and orbital frontal cortex in decision-making and flexible behavior. Neuron 2022; 110:2743-2770. [PMID: 35705077 DOI: 10.1016/j.neuron.2022.05.022] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 05/17/2022] [Accepted: 05/19/2022] [Indexed: 11/15/2022]
Abstract
The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.
Collapse
Affiliation(s)
- Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK; Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, Level 6, West Wing, John Radcliffe Hospital, Oxford OX3 9DU, UK; Department of Psychiatry, University of Oxford, Warneford Lane, Headington, Oxford OX3 7JX, UK.
| | - Alessandro Bongioanni
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK; Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, Level 6, West Wing, John Radcliffe Hospital, Oxford OX3 9DU, UK
| |
Collapse
|
21
|
Calapai A, Cabrera-Moreno J, Moser T, Jeschke M. Flexible auditory training, psychophysics, and enrichment of common marmosets with an automated, touchscreen-based system. Nat Commun 2022; 13:1648. [PMID: 35347139 PMCID: PMC8960775 DOI: 10.1038/s41467-022-29185-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 02/28/2022] [Indexed: 11/09/2022] Open
Abstract
Devising new and more efficient protocols to analyze the phenotypes of non-human primates, as well as their complex nervous systems, is rapidly becoming of paramount importance. This is because with genome-editing techniques, recently adopted to non-human primates, new animal models for fundamental and translational research have been established. One aspect in particular, namely cognitive hearing, has been difficult to assess compared to visual cognition. To address this, we devised autonomous, standardized, and unsupervised training and testing of auditory capabilities of common marmosets with a cage-based standalone, wireless system. All marmosets tested voluntarily operated the device on a daily basis and went from naïve to experienced at their own pace and with ease. Through a series of experiments, here we show, that animals autonomously learn to associate sounds with images; to flexibly discriminate sounds, and to detect sounds of varying loudness. The developed platform and training principles combine in-cage training of common marmosets for cognitive and psychoacoustic assessment with an enriched environment that does not rely on dietary restriction or social separation, in compliance with the 3Rs principle. The authors present a cage-based stand-alone platform for autonomous, standardized, and unsupervised training and testing of visuo-auditory-cued behaviours of common marmosets. The experiments do not require dietary restriction or social separation.
Collapse
Affiliation(s)
- A Calapai
- Cognitive Neuroscience Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Cognitive Hearing in Primates (CHiP) Group, Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Leibniz ScienceCampus "Primate Cognition", Göttingen, Germany
| | - J Cabrera-Moreno
- Cognitive Hearing in Primates (CHiP) Group, Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Institute for Auditory Neuroscience and InnerEarLab, University Medical Center Göttingen, 37075, Göttingen, Germany.,Göttingen Graduate School for Neurosciences, Biophysics and Molecular Biosciences, University of Göttingen, 37075, Göttingen, Germany
| | - T Moser
- Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany.,Institute for Auditory Neuroscience and InnerEarLab, University Medical Center Göttingen, 37075, Göttingen, Germany.,Göttingen Graduate School for Neurosciences, Biophysics and Molecular Biosciences, University of Göttingen, 37075, Göttingen, Germany.,Auditory Neuroscience Group and Synaptic Nanophysiology Group, Max Planck Institute for Multidisciplinary Sciences, 37077, Göttingen, Germany.,Cluster of Excellence "Multiscale Bioimaging: from Molecular Machines to Networks of Excitable Cells" (MBExC), University of Göttingen, 37075, Göttingen, Germany
| | - M Jeschke
- Cognitive Hearing in Primates (CHiP) Group, Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany. .,Auditory Neuroscience and Optogenetics Laboratory, German Primate Center - Leibniz-Institute for Primate Research, Göttingen, Germany. .,Leibniz ScienceCampus "Primate Cognition", Göttingen, Germany. .,Institute for Auditory Neuroscience and InnerEarLab, University Medical Center Göttingen, 37075, Göttingen, Germany.
| |
Collapse
|
22
|
Martins D, Lockwood P, Cutler J, Moran R, Paloyelis Y. Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning. Prog Neurobiol 2022; 213:102253. [DOI: 10.1016/j.pneurobio.2022.102253] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 01/19/2022] [Accepted: 02/28/2022] [Indexed: 12/18/2022]
|
23
|
Khalighinejad N, Manohar S, Husain M, Rushworth MFS. Complementary roles of serotonergic and cholinergic systems in decisions about when to act. Curr Biol 2022; 32:1150-1162.e7. [PMID: 35150603 PMCID: PMC8926843 DOI: 10.1016/j.cub.2022.01.042] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2021] [Revised: 11/15/2021] [Accepted: 01/17/2022] [Indexed: 11/23/2022]
Abstract
Decision-making not only involves deciding about which action to choose but when and whether to initiate an action in the first place. Macaque monkeys tracked number of dots on a screen and could choose when to make a response. The longer the animals waited before responding, the more dots appeared on the screen and the higher the probability of reward. Monkeys waited longer before making a response when a trial’s value was less than the environment’s average value. Recordings of brain activity with fMRI revealed that activity in dorsal raphe nucleus (DRN)—a key source of serotonin (5-HT)—tracked average value of the environment. By contrast, activity in the basal forebrain (BF)—an important source of acetylcholine (ACh)—was related to decision time to act as a function of immediate and recent past context. Interactions between DRN and BF and the anterior cingulate cortex (ACC), another region with action initiation-related activity, occurred as a function of the decision time to act. Next, we performed two psychopharmacological studies. Manipulating systemic 5-HT by citalopram prolonged the time macaques waited to respond for a given opportunity. This effect was more evident during blocks with long inter-trial intervals (ITIs) where good opportunities were sparse. Manipulating systemic acetylcholine (ACh) by rivastigmine reduced the time macaques waited to respond given the immediate and recent past context, a pattern opposite to the effect observed with 5-HT. These findings suggest complementary roles for serotonin/DRN and acetylcholine/BF in decisions about when to initiate an action. Both immediate context and wider environment influence decisions about when to act DRN and 5-HT mediate the influence of wider environment BF and ACh mediate the influence of immediate context
Collapse
Affiliation(s)
- Nima Khalighinejad
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Sanjay Manohar
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Masud Husain
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
24
|
Grossman CD, Bari BA, Cohen JY. Serotonin neurons modulate learning rate through uncertainty. Curr Biol 2022; 32:586-599.e7. [PMID: 34936883 PMCID: PMC8825708 DOI: 10.1016/j.cub.2021.12.006] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Revised: 10/11/2021] [Accepted: 12/03/2021] [Indexed: 12/20/2022]
Abstract
Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.
Collapse
Affiliation(s)
- Cooper D Grossman
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| | - Bilal A Bari
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA.
| |
Collapse
|
25
|
Cools R, Arnsten AFT. Neuromodulation of prefrontal cortex cognitive function in primates: the powerful roles of monoamines and acetylcholine. Neuropsychopharmacology 2022; 47:309-328. [PMID: 34312496 PMCID: PMC8617291 DOI: 10.1038/s41386-021-01100-8] [Citation(s) in RCA: 62] [Impact Index Per Article: 31.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Revised: 07/06/2021] [Accepted: 07/06/2021] [Indexed: 02/07/2023]
Abstract
The primate prefrontal cortex (PFC) subserves our highest order cognitive operations, and yet is tremendously dependent on a precise neurochemical environment for proper functioning. Depletion of noradrenaline and dopamine, or of acetylcholine from the dorsolateral PFC (dlPFC), is as devastating as removing the cortex itself, and serotonergic influences are also critical to proper functioning of the orbital and medial PFC. Most neuromodulators have a narrow inverted U dose response, which coordinates arousal state with cognitive state, and contributes to cognitive deficits with fatigue or uncontrollable stress. Studies in monkeys have revealed the molecular signaling mechanisms that govern the generation and modulation of mental representations by the dlPFC, allowing dynamic regulation of network strength, a process that requires tight regulation to prevent toxic actions, e.g., as occurs with advanced age. Brain imaging studies in humans have observed drug and genotype influences on a range of cognitive tasks and on PFC circuit functional connectivity, e.g., showing that catecholamines stabilize representations in a baseline-dependent manner. Research in monkeys has already led to new treatments for cognitive disorders in humans, encouraging future research in this important field.
Collapse
Affiliation(s)
- Roshan Cools
- Department of Psychiatry, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Amy F T Arnsten
- Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
| |
Collapse
|
26
|
Monosov IE, Rushworth MFS. Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change. Neuropsychopharmacology 2022; 47:196-210. [PMID: 34234288 PMCID: PMC8617208 DOI: 10.1038/s41386-021-01079-2] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/07/2021] [Revised: 05/27/2021] [Accepted: 06/15/2021] [Indexed: 02/06/2023]
Abstract
Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.
Collapse
Affiliation(s)
- Ilya E Monosov
- Department of Neuroscience, Washington University School of Medicine, St. Louis, MO, USA.
- Department of Biomedical Engineering, Washington University, St. Louis, MO, USA.
- Department of Electrical Engineering, Washington University, St. Louis, MO, USA.
- Department of Neurosurgery, Washington University, St. Louis, MO, USA.
- Pain Center, Washington University, St. Louis, MO, USA.
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.
| |
Collapse
|
27
|
Munoz F, Meaney A, Gross A, Liu K, Pouliopoulos AN, Liu D, Konofagou EE, Ferrera VP. Long term study of motivational and cognitive effects of low-intensity focused ultrasound neuromodulation in the dorsal striatum of nonhuman primates. Brain Stimul 2022; 15:360-372. [PMID: 35092823 PMCID: PMC9419899 DOI: 10.1016/j.brs.2022.01.014] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 01/18/2022] [Accepted: 01/24/2022] [Indexed: 12/20/2022] Open
Abstract
Noninvasive brain stimulation using transcranial focused ultrasound (FUS) has many potential applications as a research and clinical tool, including incorporation into neural prosthetics for cognitive rehabilitation. To develop this technology, it is necessary to evaluate the safety and efficacy of FUS neuromodulation for specific brain targets and cognitive functions. It is also important to test whether repeated long-term application of FUS to deep brain targets improves or degrades behavioral and cognitive function. To this end, we investigated the effects of FUS in the dorsal striatum of nonhuman primates (NHP) performing a visual-motor decision-making task for small or large rewards. Over the course of 2 years, we performed 129 and 147 FUS applications, respectively, in two NHP. FUS (0.5 MHz @ 0.2-0.8 MPa) was applied to the putamen and caudate in both hemispheres to evaluate the effects on movement accuracy, motivation, decision accuracy, and response time. Sonicating the caudate or the putamen unilaterally resulted in modest but statistically significant improvements in motivation and decision accuracy, but at the cost of slower reaction times. The effects were dose (i.e., FUS pressure) and reward dependent. There was no effect on reaching accuracy, nor was there long-term behavioral impairment or neurological trauma evident on T1-weighted, T2-weighted, or susceptibility-weighted MRI scans. Sonication also resulted in significant changes in resting state functional connectivity between the caudate and multiple cortical regions. The results indicate that applying FUS to the dorsal striatum can positively impact the motivational and cognitive aspects of decision making. The capability of FUS to improve motivation and cognition in NHPs points to its therapeutic potential in treating a wide variety of human neural diseases, and warrants further development as a novel technique for non-invasive deep brain stimulation.
Collapse
Affiliation(s)
- F Munoz
- Dept. of Neuroscience, Columbia University, United States; Zuckerman Mind Brain Behavior Institute, Columbia University, United States.
| | - A Meaney
- Zuckerman Mind Brain Behavior Institute, Columbia University
| | | | - K Liu
- Dept. of Biomedical Engineering, Columbia University
| | | | - D Liu
- Dept. of Neuroscience, Columbia University,Zuckerman Mind Brain Behavior Institute, Columbia University
| | - EE Konofagou
- Dept. of Biomedical Engineering, Columbia University,Dept. of Radiology, Columbia University
| | - VP Ferrera
- Dept. of Neuroscience, Columbia University,Zuckerman Mind Brain Behavior Institute, Columbia University,Dept. of Psychiatry, Columbia University
| |
Collapse
|
28
|
Soltani A, Koechlin E. Computational models of adaptive behavior and prefrontal cortex. Neuropsychopharmacology 2022; 47:58-71. [PMID: 34389808 PMCID: PMC8617006 DOI: 10.1038/s41386-021-01123-1] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 07/19/2021] [Accepted: 07/20/2021] [Indexed: 02/07/2023]
Abstract
The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus-action associations through rewards; (2) predictive models learning stimulus- and/or action-outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.
Collapse
Affiliation(s)
- Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Etienne Koechlin
- Institut National de la Sante et de la Recherche Medicale, Universite Pierre et Marie Curie, Ecole Normale Superieure, Paris, France.
| |
Collapse
|
29
|
Folloni D, Fouragnan E, Wittmann MK, Roumazeilles L, Tankelevitch L, Verhagen L, Attali D, Aubry JF, Sallet J, Rushworth MFS. Ultrasound modulation of macaque prefrontal cortex selectively alters credit assignment-related activity and behavior. SCIENCE ADVANCES 2021; 7:eabg7700. [PMID: 34910510 PMCID: PMC8673758 DOI: 10.1126/sciadv.abg7700] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 10/28/2021] [Indexed: 05/30/2023]
Abstract
Credit assignment is the association of specific instances of reward to the specific events, such as a particular choice, that caused them. Without credit assignment, choice values reflect an approximate estimate of how good the environment was when the choice was made—the global reward state—rather than exactly which outcome the choice caused. Combined transcranial ultrasound stimulation (TUS) and functional magnetic resonance imaging in macaques demonstrate credit assignment–related activity in prefrontal area 47/12o, and when this signal was disrupted with TUS, choice value representations across the brain were impaired. As a consequence, behavior was no longer guided by choice value, and decision-making was poorer. By contrast, global reward state–related activity in the adjacent anterior insula remained intact and determined decision-making after prefrontal disruption.
Collapse
Affiliation(s)
- Davide Folloni
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
| | - Elsa Fouragnan
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
- School of Psychology, University of Plymouth, Plymouth, UK
| | - Marco K. Wittmann
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
| | - Lea Roumazeilles
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
| | - Lev Tankelevitch
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, 6525 HR, Netherlands
| | - David Attali
- Physics for Medicine Paris, ESPCI Paris, INSERM, CNRS, PSL Research University, Paris, France
- GHU PARIS Psychiatrie and Neurosciences, site Sainte-Anne, Service Hospitalo-Universitaire, Pôle Hospitalo-Universitaire, Paris 15, F-75014 Paris, France
- Université de Paris, F-75005 Paris, France
| | - Jean-François Aubry
- Physics for Medicine Paris, ESPCI Paris, INSERM, CNRS, PSL Research University, Paris, France
| | - Jerome Sallet
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
- Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, 18 Avenue Doyen Lepine, 69500 Bron, France
| | - Matthew F. S. Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, Mansfield Road, Oxford OX1 3TA, University of Oxford, Oxford, UK
| |
Collapse
|
30
|
Trepka E, Spitmaan M, Bari BA, Costa VD, Cohen JY, Soltani A. Entropy-based metrics for predicting choice behavior based on local response to reward. Nat Commun 2021; 12:6567. [PMID: 34772943 PMCID: PMC8590026 DOI: 10.1038/s41467-021-26784-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 10/18/2021] [Indexed: 11/16/2022] Open
Abstract
For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.
Collapse
Affiliation(s)
- Ethan Trepka
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Mehran Spitmaan
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Bilal A Bari
- The Solomon H. Snyder Department of Neuroscience, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Brain Science Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Vincent D Costa
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR, USA
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Brain Science Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|
31
|
Better living through understanding the insula: Why subregions can make all the difference. Neuropharmacology 2021; 198:108765. [PMID: 34461066 DOI: 10.1016/j.neuropharm.2021.108765] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 07/19/2021] [Accepted: 08/23/2021] [Indexed: 02/07/2023]
Abstract
Insula function is considered critical for many motivated behaviors, with proposed functions ranging from attention, behavioral control, emotional regulation, goal-directed and aversion-resistant responding. Further, the insula is implicated in many neuropsychiatric conditions including substance abuse. More recently, multiple insula subregions have been distinguished based on anatomy, connectivity, and functional contributions. Generally, posterior insula is thought to encode more somatosensory inputs, which integrate with limbic/emotional information in middle insula, that in turn integrate with cognitive processes in anterior insula. Together, these regions provide rapid interoceptive information about the current or predicted situation, facilitating autonomic recruitment and quick, flexible action. Here, we seek to create a robust foundation from which to understand potential subregion differences, and provide direction for future studies. We address subregion differences across humans and rodents, so that the latter's mechanistic interventions can best mesh with clinical relevance of human conditions. We first consider the insula's suggested roles in humans, then compare subregional studies, and finally describe rodent work. One primary goal is to encourage precision in describing insula subregions, since imprecision (e.g. including both posterior and anterior studies when describing insula work) does a disservice to a larger understanding of insula contributions. Additionally, we note that specific task details can greatly impact recruitment of various subregions, requiring care and nuance in design and interpretation of studies. Nonetheless, the central ethological importance of the insula makes continued research to uncover mechanistic, mood, and behavioral contributions of paramount importance and interest. This article is part of the special Issue on 'Neurocircuitry Modulating Drug and Alcohol Abuse'.
Collapse
|
32
|
Choice history effects in mice and humans improve reward harvesting efficiency. PLoS Comput Biol 2021; 17:e1009452. [PMID: 34606493 PMCID: PMC8516315 DOI: 10.1371/journal.pcbi.1009452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Revised: 10/14/2021] [Accepted: 09/15/2021] [Indexed: 12/04/2022] Open
Abstract
Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias. Animals foraging for food in natural habitats compete to obtain better quality food patches. To achieve this goal, animals can rely on memory and choose the same patches that have provided higher quality of food in the past. However, in natural habitats simply identifying better food patches may not be sufficient to successfully compete with their conspecifics, as food resources can grow over time. Therefore, it makes sense to visit from time to time those patches that were associated with lower food quality in the past. This demands optimal foraging animals to keep in memory not only which food patches provided the best food quality, but also which food patches they visited recently. To see if animals track their history of visits and use it to maximize the food harvesting efficiency, we subjected them to experimental conditions that mimicked natural foraging behavior. In our behavioral tasks, we replaced food foraging behavior with a two choice task that provided rewards to mice and humans. By developing a new computational model and subjecting animals to various behavioral manipulations, we demonstrate that keeping a memory of past visits helps the animals to optimize the efficiency with which they can harvest rewards.
Collapse
|
33
|
Báez-Mendoza R, Vázquez Y, Mastrobattista EP, Williams ZM. Neuronal Circuits for Social Decision-Making and Their Clinical Implications. Front Neurosci 2021; 15:720294. [PMID: 34658766 PMCID: PMC8517320 DOI: 10.3389/fnins.2021.720294] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2021] [Accepted: 09/09/2021] [Indexed: 11/13/2022] Open
Abstract
Social living facilitates individual access to rewards, cognitive resources, and objects that would not be otherwise accessible. There are, however, some drawbacks to social living, particularly when competing for scarce resources. Furthermore, variability in our ability to make social decisions can be associated with neuropsychiatric disorders. The neuronal mechanisms underlying social decision-making are beginning to be understood. The momentum to study this phenomenon has been partially carried over by the study of economic decision-making. Yet, because of the similarities between these different types of decision-making, it is unclear what is a social decision. Here, we propose a definition of social decision-making as choices taken in a context where one or more conspecifics are involved in the decision or the consequences of it. Social decisions can be conceptualized as complex economic decisions since they are based on the subjective preferences between different goods. During social decisions, individuals choose based on their internal value estimate of the different alternatives. These are complex decisions given that conspecifics beliefs or actions could modify the subject's internal valuations at every choice. Here, we first review recent developments in our collective understanding of the neuronal mechanisms and circuits of social decision-making in primates. We then review literature characterizing populations with neuropsychiatric disorders showing deficits in social decision-making and the underlying neuronal circuitries associated with these deficits.
Collapse
Affiliation(s)
- Raymundo Báez-Mendoza
- Department of Neurosurgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Yuriria Vázquez
- Laboratory of Neural Systems, The Rockefeller University, New York, NY, United States
| | - Emma P. Mastrobattista
- Department of Neurosurgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Ziv M. Williams
- Department of Neurosurgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| |
Collapse
|
34
|
Russ BE, Petkov CI, Kwok SC, Zhu Q, Belin P, Vanduffel W, Hamed SB. Common functional localizers to enhance NHP & cross-species neuroscience imaging research. Neuroimage 2021; 237:118203. [PMID: 34048898 PMCID: PMC8529529 DOI: 10.1016/j.neuroimage.2021.118203] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 05/15/2021] [Accepted: 05/24/2021] [Indexed: 11/25/2022] Open
Abstract
Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.
Collapse
Affiliation(s)
- Brian E Russ
- Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY, United States; Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York City, NY, United States; Department of Psychiatry, New York University at Langone, New York City, NY, United States.
| | - Christopher I Petkov
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne, United Kingdom
| | - Sze Chai Kwok
- Shanghai Key Laboratory of Brain Functional Genomics, Key Laboratory of Brain Functional Genomics Ministry of Education, Shanghai Key Laboratory of Magnetic Resonance, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China; Division of Natural and Applied Sciences, Duke Kunshan University, Kunshan, Jiangsu, China; NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China
| | - Qi Zhu
- Cognitive Neuroimaging Unit, INSERM, CEA, Université Paris-Saclay, NeuroSpin Center, 91191 Gif/Yvette, France; Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium
| | - Pascal Belin
- Institut de Neurosciences de La Timone, Aix-Marseille Université et CNRS, Marseille, 13005, France
| | - Wim Vanduffel
- Laboratory for Neuro-and Psychophysiology, Department of Neurosciences, KU Leuven Medical School, Leuven, 3000, Belgium; Leuven Brain Institute, KU Leuven, Leuven, 3000, Belgium; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, United States; Department of Radiology, Harvard Medical School, Boston, MA 02144, United States.
| | - Suliann Ben Hamed
- Institut des Sciences Cognitives Marc Jeannerod, UMR 5229, Université de Lyon - CNRS, France.
| |
Collapse
|
35
|
Cutler J, Wittmann MK, Abdurahman A, Hargitai LD, Drew D, Husain M, Lockwood PL. Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved. Nat Commun 2021; 12:4440. [PMID: 34290236 PMCID: PMC8295324 DOI: 10.1038/s41467-021-24576-w] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 06/25/2021] [Indexed: 12/23/2022] Open
Abstract
Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18-36) and older (age 60-80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.
Collapse
Affiliation(s)
- Jo Cutler
- Centre for Human Brain Health and Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Marco K Wittmann
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Ayat Abdurahman
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Department of Psychology, University of Cambridge, Cambridge, UK
| | - Luca D Hargitai
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Daniel Drew
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Masud Husain
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Patricia L Lockwood
- Centre for Human Brain Health and Institute for Mental Health, School of Psychology, University of Birmingham, Birmingham, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
- Christ Church, University of Oxford, Oxford, UK.
| |
Collapse
|
36
|
Wittmann MK, Trudel N, Trier HA, Klein-Flügge MC, Sel A, Verhagen L, Rushworth MFS. Causal manipulation of self-other mergence in the dorsomedial prefrontal cortex. Neuron 2021; 109:2353-2361.e11. [PMID: 34171289 PMCID: PMC8326319 DOI: 10.1016/j.neuron.2021.05.027] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2020] [Revised: 03/30/2021] [Accepted: 05/19/2021] [Indexed: 11/15/2022]
Abstract
To navigate social environments, people must simultaneously hold representations about their own and others’ abilities. During self-other mergence, people estimate others’ abilities not only on the basis of the others’ past performance, but the estimates are also influenced by their own performance. For example, if we perform well, we overestimate the abilities of those with whom we are co-operating and underestimate competitors. Self-other mergence is associated with specific activity patterns in the dorsomedial prefrontal cortex (dmPFC). Using a combination of non-invasive brain stimulation, functional magnetic resonance imaging, and computational modeling, we show that dmPFC neurostimulation silences these neural signatures of self-other mergence in relation to estimation of others’ abilities. In consequence, self-other mergence behavior increases, and our assessments of our own performance are projected increasingly onto other people. This suggests an inherent tendency to form interdependent social representations and a causal role of the dmPFC in separating self and other representations. During self-other mergence (SOM), people confuse one’s own with another’s performance Brain stimulation over dorsomedial prefrontal cortex (dmPFC) alters neural SOM Brain stimulation over dmPFC simultaneously alters behavioral SOM This suggests a causal role of dmPFC in separating self and other representations
Collapse
Affiliation(s)
- Marco K Wittmann
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK.
| | - Nadescha Trudel
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK
| | - Hailey A Trier
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK
| | - Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK
| | - Alejandra Sel
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK; Centre for Brain Science, Department of Psychology, University of Essex, Wivenhoe Park, Colchester CO4 3SQ, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK; Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, Tinsley Building, University of Oxford, Mansfield Road, Oxford OX1 3TA, UK
| |
Collapse
|
37
|
Pribut HJ, Vázquez D, Brockett AT, Wei AD, Tennyson SS, Roesch MR. Prior Cocaine Exposure Increases Firing to Immediate Reward While Attenuating Cue and Context Signals Related to Reward Value in the Insula. J Neurosci 2021; 41:4667-4677. [PMID: 33849944 PMCID: PMC8260251 DOI: 10.1523/jneurosci.3025-20.2021] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 03/29/2021] [Accepted: 04/02/2021] [Indexed: 01/20/2023] Open
Abstract
The insula contributes to behavioral control and is disrupted by substance abuse, yet we know little about the neural signals underlying these functions or how they are disrupted after chronic drug self-administration. Here, male and female rats self-administered either cocaine (experimental group) or sucrose (control) for 12 consecutive days. After a 1 month withdrawal period, we recorded from insula while rats performed a previously learned reward-guided decision-making task. Cocaine-exposed rats were more sensitive to value manipulations and were faster to respond. These behavioral changes were accompanied by elevated counts of neurons in the insula that increased firing to reward. These neurons also fired more strongly at the start of long-delay trials, when a more immediate reward would be expected, and fired less strongly in anticipation of the actual delivery of delayed rewards. Although reward-related firing to immediate reward was enhanced after cocaine self-administration, reward-predicting cue and context signals were attenuated. In addition to revealing novel firing patterns unique to insula, our data suggest changes in such neural activity likely contribute to impaired decision making observed after drug use.SIGNIFICANCE STATEMENT The insula plays a clear role in drug addiction and drug-induced impairments of decision making, yet there is little understanding of its underlying neural signals. We found that chronic cocaine self-administration reduces cue and context encoding in insula while enhancing signals related to immediate reward. These changes in neural activity likely contribute to impaired decision making and impulsivity observed after drug use.
Collapse
Affiliation(s)
- Heather J Pribut
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Daniela Vázquez
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Adam T Brockett
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| | - Alice D Wei
- Department of Psychology, University of Maryland, College Park, Maryland 20742
| | - Stephen S Tennyson
- Department of Psychology, University of Maryland, College Park, Maryland 20742
| | - Matthew R Roesch
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| |
Collapse
|
38
|
Schoenbaum G, Ishii H, Walton ME, Panayi MC. Defining an orbitofrontal compass: Functional and anatomical heterogeneity across anterior-posterior and medial-lateral axes. Behav Neurosci 2021; 135:165-173. [PMID: 34060873 PMCID: PMC7613671 DOI: 10.1037/bne0000442] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The orbitofrontal cortex (OFC) plays a critical role in the flexible control of behaviors and has been the focus of increasing research interest. However, there have been a number of controversies around the exact theoretical role of the OFC. One potential source of these issues is the comparison of evidence from different studies, particularly across species, which focus on different specific sub-regions within the OFC. Furthermore, there is emerging evidence that there may be functional diversity across the OFC which may account for these theoretical differences. Therefore, in this review we consider evidence supporting functional heterogeneity within the OFC and how it relates to underlying anatomical heterogeneity. We highlight the importance of anatomical and functional distinctions within the traditionally defined OFC subregions across the medial-lateral axis, which are often not differentiated for practical and historical reasons. We then consider emerging evidence of even finer-grained distinctions within these defined subregions along the anterior-posterior axis. These fine-grained anatomical considerations reveal a pattern of dissociable, but often complementary functions within the OFC. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
Affiliation(s)
| | - Hironori Ishii
- Department of Experimental Psychology, University of Oxford
| | - Mark E Walton
- Department of Experimental Psychology, University of Oxford
| | | |
Collapse
|