1
|
Drossel G, Heilbronner SR, Zimmermann J, Zilverstand A. Neuroimaging of the effects of drug exposure or self-administration in rodents: A systematic review. Neurosci Biobehav Rev 2024; 164:105823. [PMID: 39094280 DOI: 10.1016/j.neubiorev.2024.105823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2024] [Revised: 06/19/2024] [Accepted: 07/20/2024] [Indexed: 08/04/2024]
Abstract
A systematic review of functional neuroimaging studies on drug (self-) administration in rodents is lacking. Here, we summarized effects of acute or chronic drug administration of various classes of drugs on brain function and determined consistency with human literature. We performed a systematic literature search and identified 125 studies on in vivo rodent resting-state functional magnetic resonance imaging (n = 84) or positron emission tomography (n = 41) spanning depressants (n = 27), opioids (n = 23), stimulants (n = 72), and cannabis (n = 3). Results primarily showed alterations in the striatum, consistent with the human literature. The anterior cingulate cortex and (nonspecific) prefrontal cortex were also frequently implicated. Upregulation was most often found after shorter administration and downregulation after long chronic administration, particularly in the striatum. Importantly, results were consistent across study design, administration models, imaging method, and animal states. Results provide evidence of altered resting-state brain function in rodents upon drug administration, implicating the brain's reward network analogous to human studies. However, alterations were more dynamic than previously known, with dynamic adaptation depending on the length of drug administration.
Collapse
Affiliation(s)
- Gunner Drossel
- Graduate Program in Neuroscience, University of Minnesota, Minneapolis, MN, USA; Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA
| | | | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA; Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| | - Anna Zilverstand
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA; Medical Discovery Team on Addiction, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
2
|
Bosulu J, Pezzulo G, Hétu S. Needing: An Active Inference Process for Physiological Motivation. J Cogn Neurosci 2024; 36:2011-2028. [PMID: 38940737 DOI: 10.1162/jocn_a_02209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/29/2024]
Abstract
Need states are internal states that arise from deprivation of crucial biological stimuli. They direct motivation, independently of external learning. Despite their separate origin, they interact with reward processing systems that respond to external stimuli. This article aims to illuminate the functioning of the needing system through the lens of active inference, a framework for understanding brain and cognition. We propose that need states exert a pervasive influence on the organism, which in active inference terms translates to a "pervasive surprise"-a measure of the distance from the organism's preferred state. Crucially, we define needing as an active inference process that seeks to reduce this pervasive surprise. Through a series of simulations, we demonstrate that our proposal successfully captures key aspects of the phenomenology and neurobiology of needing. We show that as need states increase, the tendency to occupy preferred states strengthens, independently of external reward prediction. Furthermore, need states increase the precision of states (stimuli and actions) leading to preferred states, suggesting their ability to amplify the value of reward cues and rewards themselves. Collectively, our model and simulations provide valuable insights into the directional and underlying influence of need states, revealing how this influence amplifies the wanting or liking associated with relevant stimuli.
Collapse
Affiliation(s)
- Juvenal Bosulu
- Université de Montréal
- Centre interdisciplinaire de recherche sur le cerveau et l'apprentissage (CIRCA), Montréal, Québec, Canada
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies (ISTC-CNR), Rome, Italy
| | - Sébastien Hétu
- Université de Montréal
- Centre interdisciplinaire de recherche sur le cerveau et l'apprentissage (CIRCA), Montréal, Québec, Canada
| |
Collapse
|
3
|
França TFA. Exploring undiscovered public knowledge in neuroscience. Eur J Neurosci 2024; 60:4723-4737. [PMID: 38782707 DOI: 10.1111/ejn.16396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Revised: 04/21/2024] [Accepted: 05/03/2024] [Indexed: 05/25/2024]
Abstract
In this essay, I argue that the combination of research synthesis and philosophical methods can fill an important methodological gap in neuroscience. While experimental research and formal modelling have seen their methods progressively increase in rigour and sophistication over the years, the task of analysing and synthesizing the vast literature reporting new results and models has lagged behind. The problem is aggravated because neuroscience has grown and expanded into a vast mosaic of related but partially independent subfields, each with their own literatures. This fragmentation not only makes it difficult to see the full picture emerging from neuroscience research but also limits progress in individual subfields. The current neuroscience literature has the perfect conditions to create what the information scientist Don Swanson called "undiscovered public knowledge"-knowledge that exists in the mutual implications of different published pieces of information but that is nonetheless undiscovered because those pieces have not been explicitly connected. Current methods for rigorous research synthesis, such as systematic reviews and meta-analyses, mostly focus on combining similar studies and are not suited for exploring undiscovered public knowledge. To that aim, they need to be adapted and supplemented. I argue that successful exploration of the hidden implications in the neuroscience literature will require the combination of these adapted research synthesis methods with philosophical methods for rigorous (and creative) analysis and synthesis.
Collapse
Affiliation(s)
- Thiago F A França
- Departamento de Psicobiologia, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| |
Collapse
|
4
|
Slifstein M, Qu W, Gil R, Weinstein JJ, Perlman G, Jaworski-Calara T, Meng J, Hu B, Moeller SJ, Horga G, Abi-Dargham A. Kappa opioid receptor availability predicts severity of anhedonia in schizophrenia. Neuropsychopharmacology 2024:10.1038/s41386-024-01975-3. [PMID: 39217267 DOI: 10.1038/s41386-024-01975-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2024] [Revised: 07/31/2024] [Accepted: 08/20/2024] [Indexed: 09/04/2024]
Abstract
The kappa opioid receptor (KOR) and its endogenous agonist dynorphin have been implicated in multiple psychiatric conditions including psychotic disorders. We tested the hypotheses that kappa expression is elevated and associated with psychotic symptoms in schizophrenia. We measured kappa expression in unmedicated patients with schizophrenia (7 female, 6 male) and matched controls (7 female, 6 male) with positron emission tomography (PET). We also acquired a measurement of cumulative dopamine activity over the life span in the same subjects using neuromelanin sensitive MRI. We hypothesized that neuromelanin accumulation would be higher in patients than controls and that in patients there would be a positive association between KOR availability and neuromelanin accumulation. Fourteen patients and thirteen controls were enrolled. Whole brain dynamic PET imaging data using the KOR selective tracer [18F]LY245998 were acquired. Distribution volume (VT) was measured with region of interest analysis in 14 brain regions. Neuromelanin accumulation in midbrain dopaminergic nuclei was assessed in the same subjects. Positive and negative symptoms were measured by a clinical psychologist. We did not observe group level differences in KOR expression, neuromelanin accumulation or relationships of these to positive symptoms. Unexpectedly, we did observe strong positive associations between KOR expression and symptoms of anhedonia in the patients (Pearson r > 0.7, uncorrected p < 0.01 in 8 cortical brain regions). We also observed moderate associations between KOR expression and neuromelanin levels in patients. In conclusion, we did not observe a relationship between kappa and symptoms of psychosis but the observed relationship to the negative symptom of anhedonia is in line with recent work testing kappa antagonism as a therapy for anhedonia in depression.
Collapse
Affiliation(s)
- Mark Slifstein
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA.
| | - Wenchao Qu
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Roberto Gil
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Jodi J Weinstein
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Greg Perlman
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | | | - Jiayan Meng
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Bao Hu
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Scott J Moeller
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| | - Guillermo Horga
- Vagelos College of Physicians and Surgeons, Columbia University, Stony Brook, New York, USA
- New York State Psychiatric Institute, Stony Brook, New York, USA
| | - Anissa Abi-Dargham
- Renaissance School of Medicine, Stony Brook University, Stony Brook, New York, USA
| |
Collapse
|
5
|
Taira M, Millard SJ, Verghese A, DiFazio LE, Hoang IB, Jia R, Sias A, Wikenheiser A, Sharpe MJ. Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning. J Neurosci 2024; 44:e0120242024. [PMID: 38969504 PMCID: PMC11358529 DOI: 10.1523/jneurosci.0120-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 06/18/2024] [Accepted: 06/20/2024] [Indexed: 07/07/2024] Open
Abstract
Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTADA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits. To understand whether the NAcC might regulate a unique component of learning, we recorded dopamine release in NAcC while male rats performed a backward conditioning task where a reward is followed by a neutral cue. We used this task because we can delineate different components of learning, which include sensory-specific inhibitory and general excitatory components. Furthermore, we have shown that VTADA neurons are necessary for both the specific and general components of backward associations. Here, we found that dopamine release in NAcC increased to the reward across learning while reducing to the cue that followed as it became more expected. This mirrors the dopamine prediction error signal seen during forward conditioning and cannot be accounted for temporal-difference reinforcement learning. Subsequent tests allowed us to dissociate these learning components and revealed that dopamine release in NAcC reflects the general excitatory component of backward associations, but not their sensory-specific component. These results emphasize the importance of examining distinct functions of different dopamine projections in reinforcement learning.
Collapse
Affiliation(s)
- Masakazu Taira
- Department of Psychology, University of Sydney, Camperdown, New South Wales 2006, Australia
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Samuel J Millard
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Anna Verghese
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Lauren E DiFazio
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Ivy B Hoang
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Ruiting Jia
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Ana Sias
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Andrew Wikenheiser
- Department of Psychology, University of California, Los Angeles 90095, California
| | - Melissa J Sharpe
- Department of Psychology, University of Sydney, Camperdown, New South Wales 2006, Australia
- Department of Psychology, University of California, Los Angeles 90095, California
| |
Collapse
|
6
|
Colom M, Kraev I, Stramek AK, Loza IB, Rostron CL, Heath CJ, Dommett EJ, Singer BF. Conditioning- and reward-related dendritic and presynaptic plasticity of nucleus accumbens neurons in male and female sign-tracker rats. Eur J Neurosci 2024. [PMID: 39193632 DOI: 10.1111/ejn.16513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 07/22/2024] [Accepted: 08/12/2024] [Indexed: 08/29/2024]
Abstract
For a subset of individuals known as sign-trackers, discrete Pavlovian cues associated with rewarding stimuli can acquire incentive properties and exert control over behaviour. Because responsiveness to cues is a feature of various neuropsychiatric conditions, rodent models of sign-tracking may prove useful for exploring the neurobiology of individual variation in psychiatric vulnerabilities. Converging evidence points towards the involvement of dopaminergic neurotransmission in the nucleus accumbens core (NAc) in the development of sign-tracking, yet whether this phenotype is associated with specific accumbal postsynaptic properties is unknown. Here, we examined dendritic spine structural organisation, as well as presynaptic and postsynaptic markers of activity, in the NAc core of male and female rats following a Pavlovian-conditioned approach procedure. In contrast to our prediction that cue re-exposure would increase spine density, experiencing the discrete lever-cue without reward delivery resulted in lower spine density than control rats for which the lever was unpaired with reward during training; this effect was tempered in the most robust sign-trackers. Interestingly, this same behavioural test (lever presentation without reward) resulted in increased levels of a marker of presynaptic activity (synaptophysin), and this effect was greatest in female rats. Whilst some behavioural differences were observed in females during initial Pavlovian training, final conditioning scores did not differ from males and were unaffected by the oestrous cycle. This work provides novel insights into how conditioning impacts the neuronal plasticity of the NAc core, whilst highlighting the importance of studying the behaviour and neurobiology of both male and female rats.
Collapse
Affiliation(s)
- Morgane Colom
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
- King's College, Institute of Psychiatry, Psychology and Neuroscience, London, UK
- Department of Psychology, University of Cambridge, Cambridge, UK
| | - Igor Kraev
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
| | - Agata K Stramek
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
| | - Iwona B Loza
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
| | - Claire L Rostron
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
| | - Christopher J Heath
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
| | - Eleanor J Dommett
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
- King's College, Institute of Psychiatry, Psychology and Neuroscience, London, UK
| | - Bryan F Singer
- School of Life, Health and Chemical Sciences, The Open University, Milton Keynes, UK
- School of Psychology, Sussex Neuroscience, Sussex Addiction Research and Intervention Centre, University of Sussex, Brighton, UK
| |
Collapse
|
7
|
Elum JE, Szelenyi ER, Juarez B, Murry AD, Loginov G, Zamorano CA, Gao P, Wu G, Ng-Evans S, Yee JX, Xu X, Golden SA, Zweifel LS. Distinct dynamics and intrinsic properties in ventral tegmental area populations mediate reward association and motivation. Cell Rep 2024; 43:114668. [PMID: 39207900 DOI: 10.1016/j.celrep.2024.114668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 07/04/2024] [Accepted: 08/06/2024] [Indexed: 09/04/2024] Open
Abstract
Ventral tegmental area (VTA) dopamine neurons regulate reward-related associative learning and reward-driven motivated behaviors, but how these processes are coordinated by distinct VTA neuronal subpopulations remains unresolved. Here, we compare the contribution of two primarily dopaminergic and largely non-overlapping VTA subpopulations, all VTA dopamine neurons and VTA GABAergic neurons of the mouse midbrain, to these processes. We find that the dopamine subpopulation that projects to the nucleus accumbens (NAc) core preferentially encodes reward-predictive cues and prediction errors. In contrast, the subpopulation that projects to the NAc shell preferentially encodes goal-directed actions and relative reward anticipation. VTA GABA neuron activity strongly contrasts VTA dopamine population activity and preferentially encodes reward outcome and retrieval. Electrophysiology, targeted optogenetics, and whole-brain input mapping reveal multiple convergent sources that contribute to the heterogeneity among VTA dopamine subpopulations that likely underlies their distinct encoding of reward-related associations and motivation that defines their functions in these contexts.
Collapse
Affiliation(s)
- Jordan E Elum
- Graduate Program in Neuroscience, University of Washington, Seattle, WA, USA
| | - Eric R Szelenyi
- Department of Biological Structure, University of Washington, Seattle, WA, USA; University of Washington Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), Seattle, WA, USA
| | - Barbara Juarez
- Department of Neurobiology, University of Maryland, Baltimore, MD, USA
| | - Alexandria D Murry
- Department of Biological Structure, University of Washington, Seattle, WA, USA
| | - Grigory Loginov
- Center for Integrative Brain Research, Seattle Children's Research Institute, Seattle, WA, USA
| | - Catalina A Zamorano
- Department of Pharmacology, University of Washington, Seattle, WA, USA; University of Washington Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), Seattle, WA, USA
| | - Pan Gao
- Department of Anatomy and Neurobiology, School of Medicine, University of California, Irvine, Irvine, CA, USA
| | - Ginny Wu
- Department of Anatomy and Neurobiology, School of Medicine, University of California, Irvine, Irvine, CA, USA
| | - Scott Ng-Evans
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Joshua X Yee
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Center, Seattle, WA, USA
| | - Xiangmin Xu
- Department of Anatomy and Neurobiology, School of Medicine, University of California, Irvine, Irvine, CA, USA
| | - Sam A Golden
- Graduate Program in Neuroscience, University of Washington, Seattle, WA, USA; Department of Biological Structure, University of Washington, Seattle, WA, USA; University of Washington Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), Seattle, WA, USA
| | - Larry S Zweifel
- Graduate Program in Neuroscience, University of Washington, Seattle, WA, USA; Department of Pharmacology, University of Washington, Seattle, WA, USA; Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA; University of Washington Center of Excellence in Neurobiology of Addiction, Pain, and Emotion (NAPE), Seattle, WA, USA.
| |
Collapse
|
8
|
Treutlein J, Löhlein S, Einenkel KE, Picotin R, Diekhof EK, Gruber O. Association of Unc-51-like Kinase 4 ( ULK4) with the reactivity of the extended reward system in response to conditioned stimuli. World J Biol Psychiatry 2024:1-8. [PMID: 39185807 DOI: 10.1080/15622975.2024.2393381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/21/2024] [Revised: 08/05/2024] [Accepted: 08/13/2024] [Indexed: 08/27/2024]
Abstract
OBJECTIVES ULK4 is an established candidate gene for mental disorders and antipsychotic treatment response. We investigated the association of functional genetic variation at the ULK4 locus with the human extended dopaminergic reward system using fMRI during the performance of a well-established reward paradigm. METHODS Two hundred and thirty-four patients were included in this study. Association of genetic variation in the ULK4 gene with reward system functioning were determined using the Desire-Reason-Dilemma (DRD) paradigm which allows to assess brain activation in response to conditioned reward stimuli. RESULTS Variant prioritisation revealed the strongest functional signatures for the ULK4 variant rs17215589, coding for amino acid exchange Ala715Thr. For rs17215589 minor allele carriers, we detected increased activation responses to conditioned reward stimuli in the ventral tegmental area, nucleus accumbens and several cortical brain regions of the extended reward system. CONCLUSIONS Our findings provide further evidence in humans that genetic variation in ULK4 may increase the vulnerability to mental disorders, by modulating the extended reward system function. Future studies are needed to confirm the modulation of the extended reward system by ULK4 and to specify the role of this mechanism in the pathogenesis of psychiatric disorders.
Collapse
Affiliation(s)
- Jens Treutlein
- Section for Experimental Psychopathology and Neuroimaging, Department of General Psychiatry, Heidelberg University, Heidelberg, Germany
| | - Simone Löhlein
- Section for Experimental Psychopathology and Neuroimaging, Department of General Psychiatry, Heidelberg University, Heidelberg, Germany
- Institute of Flight Systems, University of the Bundeswehr Munich, Munich, Germany
| | - Karolin E Einenkel
- Section for Experimental Psychopathology and Neuroimaging, Department of General Psychiatry, Heidelberg University, Heidelberg, Germany
| | - Rosanne Picotin
- Section for Experimental Psychopathology and Neuroimaging, Department of General Psychiatry, Heidelberg University, Heidelberg, Germany
| | - Esther K Diekhof
- Institute for Cell- and Systemsbiology of Animals, Department of Biology, Neuroendocrinology Unit, Universität Hamburg, Hamburg, Germany
- Department of Psychiatry and Psychotherapy, Center for Translational Research in Systems Neuroscience and Clinical Psychiatry, Georg August University Göttingen, Göttingen, Germany
| | - Oliver Gruber
- Section for Experimental Psychopathology and Neuroimaging, Department of General Psychiatry, Heidelberg University, Heidelberg, Germany
| |
Collapse
|
9
|
Simon RC, Loveless MC, Yee JX, Goh B, Cho SG, Nasir Z, Hashikawa K, Stuber GD, Zweifel LS, Soden ME. Opto-seq reveals input-specific immediate-early gene induction in ventral tegmental area cell types. Neuron 2024; 112:2721-2731.e5. [PMID: 38901431 PMCID: PMC11343674 DOI: 10.1016/j.neuron.2024.05.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 03/18/2024] [Accepted: 05/27/2024] [Indexed: 06/22/2024]
Abstract
The ventral tegmental area (VTA) is a critical node in circuits governing motivated behavior and is home to diverse populations of neurons that release dopamine, gamma-aminobutyric acid (GABA), glutamate, or combinations of these neurotransmitters. The VTA receives inputs from many brain regions, but a comprehensive understanding of input-specific activation of VTA neuronal subpopulations is lacking. To address this, we combined optogenetic stimulation of select VTA inputs with single-nucleus RNA sequencing (snRNA-seq) and highly multiplexed in situ hybridization to identify distinct neuronal clusters and characterize their spatial distribution and activation patterns. Quantification of immediate-early gene (IEG) expression revealed that different inputs activated select VTA subpopulations, which demonstrated cell-type-specific transcriptional programs. Within dopaminergic subpopulations, IEG induction levels correlated with differential expression of ion channel genes. This new transcriptomics-guided circuit analysis reveals the diversity of VTA activation driven by distinct inputs and provides a resource for future analysis of VTA cell types.
Collapse
Affiliation(s)
- Rhiana C Simon
- Graduate Program in Neuroscience, University of Washington, Seattle, WA 98195, USA; Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA
| | - Mary C Loveless
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Joshua X Yee
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Brandon Goh
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Su G Cho
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Zainab Nasir
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Koichi Hashikawa
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA; Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA 98195, USA
| | - Garret D Stuber
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA; Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA 98195, USA
| | - Larry S Zweifel
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA
| | - Marta E Soden
- Center for the Neurobiology of Addiction, Pain, and Emotion, University of Washington, Seattle, WA 98195, USA; Department of Pharmacology, University of Washington, Seattle, WA 98195, USA.
| |
Collapse
|
10
|
Howard JD, Edmonds D, Schoenbaum G, Kahnt T. Distributed midbrain responses signal the content of positive identity prediction errors. Curr Biol 2024:S0960-9822(24)01076-5. [PMID: 39197457 DOI: 10.1016/j.cub.2024.07.105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Revised: 06/12/2024] [Accepted: 07/31/2024] [Indexed: 09/01/2024]
Abstract
Recent work across species has shown that midbrain dopamine neurons signal not only errors in the prediction of reward value but also in the prediction of value-neutral sensory features. To support learning of associative structures in downstream areas, identity prediction errors (iPEs) should signal specific information about the mis-predicted outcome. Here, we used pattern-based analysis of functional magnetic resonance imaging (fMRI) data acquired during reversal learning to characterize the information content of iPE responses in the human midbrain. We find that fMRI responses to value-neutral identity errors contain information about the identity of the unexpectedly received reward (positive iPE+) but not about the identity of the omitted reward (negative iPE-). Exploratory analyses revealed representations of iPE- in the dorsomedial prefrontal cortex. These results demonstrate that ensemble midbrain responses to value-neutral identity errors convey information about the identity of unexpectedly received outcomes, which could shape the formation of novel stimulus-outcome associations that constitute cognitive maps.
Collapse
Affiliation(s)
- James D Howard
- Department of Psychology, Brandeis University, Waltham, MA 02453, USA.
| | - Donnisa Edmonds
- Department of Neurology, Northwestern University, Chicago, IL 60611, USA
| | - Geoffrey Schoenbaum
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD 21224, USA.
| |
Collapse
|
11
|
Maroto-Gómez M, Burguete-Alventosa J, Álvarez-Arias S, Malfaz M, Salichs MÁ. A Bio-Inspired Dopamine Model for Robots with Autonomous Decision-Making. Biomimetics (Basel) 2024; 9:504. [PMID: 39194483 DOI: 10.3390/biomimetics9080504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2024] [Revised: 08/18/2024] [Accepted: 08/19/2024] [Indexed: 08/29/2024] Open
Abstract
Decision-making systems allow artificial agents to adapt their behaviours, depending on the information they perceive from the environment and internal processes. Human beings possess unique decision-making capabilities, adapting to current situations and anticipating future challenges. Autonomous robots with adaptive and anticipatory decision-making emulating humans can bring robots with skills that users can understand more easily. Human decisions highly depend on dopamine, a brain substance that regulates motivation and reward, acknowledging positive and negative situations. Considering recent neuroscience studies about the dopamine role in the human brain and its influence on decision-making and motivated behaviour, this paper proposes a model based on how dopamine drives human motivation and decision-making. The model allows robots to behave autonomously in dynamic environments, learning the best action selection strategy and anticipating future rewards. The results show the model's performance in five scenarios, emphasising how dopamine levels vary depending on the robot's situation and stimuli perception. Moreover, we show the model's integration into the Mini social robot to provide insights into how dopamine levels drive motivated autonomous behaviour regulating biologically inspired internal processes emulated in the robot.
Collapse
Affiliation(s)
- Marcos Maroto-Gómez
- Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
| | - Javier Burguete-Alventosa
- Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
| | - Sofía Álvarez-Arias
- Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
| | - María Malfaz
- Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
| | - Miguel Ángel Salichs
- Department of Systems Engineering and Automation, University Carlos III of Madrid, Av. de la Universidad, 30, 28911 Leganes, Madrid, Spain
| |
Collapse
|
12
|
Tomberg C, Petagna M, de Selliers de Moranville LA. Spontaneous eye blinks in horses (Equus caballus) are modulated by attention. Sci Rep 2024; 14:19336. [PMID: 39164361 PMCID: PMC11336180 DOI: 10.1038/s41598-024-70141-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 08/13/2024] [Indexed: 08/22/2024] Open
Abstract
Spontaneous eye blinks are brief closures of both eyelids. The spontaneous eye blink rate (SEBR) exceeds physiological corneal needs and is modulated by emotions and cognitive states, including vigilance and attention, in humans. In several animal species, the SEBR is modulated by stress and antipredator vigilance, which may limit the loss of visual information due to spontaneous eye closing. Here, we investigated whether the SEBR is modulated by attention in the domestic horse (Equus caballus). Our data supported previous studies indicating a tonic SEBR specific to each individual. We also found that, superimposed on a tonic SEBR, phasic changes were induced by cognitive processing. Attention downmodulated the SEBR, with the magnitude of blink inhibition proportional to the degree of attentional selectivity. On the other hand, reward anticipation upregulated the SEBR. Our data also suggested that horses possess the cognitive property of object permanence: they understand that an object that is no longer in their visual field has not ceased to exist. In conclusion, our results suggested that spontaneous eye blinks in horses are modulated by attentional cognitive processing.
Collapse
Affiliation(s)
- Claude Tomberg
- Faculty of Medicine, Université libre de Bruxelles, 808, route de Lennik, CP 630, 1070, Brussels, Belgium.
| | - Maxime Petagna
- Faculty of Medicine, Université libre de Bruxelles, 808, route de Lennik, CP 630, 1070, Brussels, Belgium
| | | |
Collapse
|
13
|
Fritsche M, Majumdar A, Strickland L, Liebana Garcia S, Bogacz R, Lak A. Temporal regularities shape perceptual decisions and striatal dopamine signals. Nat Commun 2024; 15:7093. [PMID: 39154025 PMCID: PMC11330509 DOI: 10.1038/s41467-024-51393-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Accepted: 08/05/2024] [Indexed: 08/19/2024] Open
Abstract
Perceptual decisions should depend on sensory evidence. However, such decisions are also influenced by past choices and outcomes. These choice history biases may reflect advantageous strategies to exploit temporal regularities of natural environments. However, it is unclear whether and how observers can adapt their choice history biases to different temporal regularities, to exploit the multitude of temporal correlations that exist in nature. Here, we show that male mice adapt their perceptual choice history biases to different temporal regularities of visual stimuli. This adaptation was slow, evolving over hundreds of trials across several days. It occurred alongside a fast non-adaptive choice history bias, limited to a few trials. Both fast and slow trial history effects are well captured by a normative reinforcement learning algorithm with multi-trial belief states, comprising both current trial sensory and previous trial memory states. We demonstrate that dorsal striatal dopamine tracks predictions of the model and behavior, suggesting that striatal dopamine reports reward predictions associated with adaptive choice history biases. Our results reveal the adaptive nature of perceptual choice history biases and shed light on their underlying computational principles and neural correlates.
Collapse
Affiliation(s)
- Matthias Fritsche
- Department of Physiology, Anatomy & Genetics, University of Oxford, Oxford, UK.
| | - Antara Majumdar
- Department of Physiology, Anatomy & Genetics, University of Oxford, Oxford, UK
| | - Lauren Strickland
- Department of Physiology, Anatomy & Genetics, University of Oxford, Oxford, UK
- Institute of Behavioral Neuroscience, University College London, London, UK
| | | | - Rafal Bogacz
- MRC Brain Network Dynamics Unit, University of Oxford, Oxford, UK
| | - Armin Lak
- Department of Physiology, Anatomy & Genetics, University of Oxford, Oxford, UK.
| |
Collapse
|
14
|
Ferry RA, Shah VV, Jin J, Jarcho JM, Hajcak G, Nelson BD. Neural response to monetary and social rewards in adolescent girls and their parents. Neuroimage 2024; 297:120705. [PMID: 38914211 DOI: 10.1016/j.neuroimage.2024.120705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 06/14/2024] [Accepted: 06/21/2024] [Indexed: 06/26/2024] Open
Abstract
Functional magnetic resonance imaging (fMRI) studies have indicated that the mesocorticolimbic dopamine system is heavily involved in all stages of reward processing. However, the majority of research has been conducted using monetary rewards and it is unclear to what extent other types of rewards, such as social rewards, evoke similar or different neural activation. There have also been few investigations into potential differences or similarities between reward processing in parents and offspring. The present study examined fMRI neural activation in response to monetary and social reward in a sample of 14-22-year-old adolescent girls (N = 145) and a biological parent (N = 124) and compared activation across adolescent-parent dyads (N = 82). Across all participants, both monetary and social reward elicited bilateral striatal activation, which did not differ between reward types or between adolescents and their parents. Neural activation in response to the different reward types were positively correlated in the striatum among adolescents and in the mPFC and OFC among parents. Overall, the present study suggests that both monetary and social reward elicit striatal activation regardless of age and provides evidence that neural mechanisms underlying reward processing may converge differentially among youth and adults.
Collapse
Affiliation(s)
- Rachel A Ferry
- Department of Psychology, Stony Brook University, 100 Nicolls Rd, Stony Brook, NY 11794-2500, USA.
| | - Virja V Shah
- Department of Psychology, Stony Brook University, 100 Nicolls Rd, Stony Brook, NY 11794-2500, USA
| | - Jingwen Jin
- Department of Psychology, University of Hong Kong, The Jockey Club Tower, Centennial Campus, Pokfulam Road, Hong Kong
| | - Johanna M Jarcho
- Department of Psychology and Neuroscience, Temple University, 1701N 13th St, Philadelphia, PA 19122, USA
| | - Greg Hajcak
- School of Education and Counseling Psychology, Santa Clara University, 455 El Camino Real, Santa Clara, CA 95053, USA
| | - Brady D Nelson
- Department of Psychology, Stony Brook University, 100 Nicolls Rd, Stony Brook, NY 11794-2500, USA
| |
Collapse
|
15
|
Basu A, Yang JH, Yu A, Glaeser-Khan S, Rondeau JA, Feng J, Krystal JH, Li Y, Kaye AP. Frontal Norepinephrine Represents a Threat Prediction Error Under Uncertainty. Biol Psychiatry 2024; 96:256-267. [PMID: 38316333 PMCID: PMC11269024 DOI: 10.1016/j.biopsych.2024.01.025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 01/19/2024] [Accepted: 01/29/2024] [Indexed: 02/07/2024]
Abstract
BACKGROUND To adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified. METHODS We measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor. RESULTS We found that norepinephrine response to conditioned stimuli reflects aversive memory strength. When delays between auditory stimuli and footshock are introduced, norepinephrine acts as a prediction error signal. However, temporal difference prediction errors do not fully explain norepinephrine dynamics. To explain noradrenergic signaling, we used an updated reinforcement learning model with uncertainty about time and found that it explained norepinephrine dynamics across learning and variations in temporal and auditory task structure. CONCLUSIONS Norepinephrine thus combines cognitive and affective information into a predictive signal and links time with the anticipation of danger.
Collapse
Affiliation(s)
- Aakash Basu
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut; Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, Connecticut
| | - Jen-Hau Yang
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut
| | - Abigail Yu
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut
| | | | - Jocelyne A Rondeau
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut
| | - Jiesi Feng
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
| | - John H Krystal
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut; Clinical Neuroscience Division, Veterans Administration National Center for PTSD, West Haven, Connecticut
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China; Peking University-IDG/McGovern Institute for Brain Research, Beijing, China; Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Alfred P Kaye
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut; Clinical Neuroscience Division, Veterans Administration National Center for PTSD, West Haven, Connecticut; Wu Tsai Institute, Yale University, New Haven, Connecticut.
| |
Collapse
|
16
|
Roth RH, Ding JB. Cortico-basal ganglia plasticity in motor learning. Neuron 2024; 112:2486-2502. [PMID: 39002543 PMCID: PMC11309896 DOI: 10.1016/j.neuron.2024.06.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 05/29/2024] [Accepted: 06/17/2024] [Indexed: 07/15/2024]
Abstract
One key function of the brain is to control our body's movements, allowing us to interact with the world around us. Yet, many motor behaviors are not innate but require learning through repeated practice. Among the brain's motor regions, the cortico-basal ganglia circuit is particularly crucial for acquiring and executing motor skills, and neuronal activity in these regions is directly linked to movement parameters. Cell-type-specific adaptations of activity patterns and synaptic connectivity support the learning of new motor skills. Functionally, neuronal activity sequences become structured and associated with learned movements. On the synaptic level, specific connections become potentiated during learning through mechanisms such as long-term synaptic plasticity and dendritic spine dynamics, which are thought to mediate functional circuit plasticity. These synaptic and circuit adaptations within the cortico-basal ganglia circuitry are thus critical for motor skill acquisition, and disruptions in this plasticity can contribute to movement disorders.
Collapse
Affiliation(s)
- Richard H Roth
- Department of Neurosurgery, Stanford University, Stanford, CA 94305, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD 20815, USA.
| | - Jun B Ding
- Department of Neurosurgery, Stanford University, Stanford, CA 94305, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD 20815, USA; Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA 94305, USA; The Phil & Penny Knight Initiative for Brain Resilience at the Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
17
|
Schaaf JV, Weidinger L, Molleman L, van den Bos W. Test-retest reliability of reinforcement learning parameters. Behav Res Methods 2024; 56:4582-4599. [PMID: 37684495 PMCID: PMC11289054 DOI: 10.3758/s13428-023-02203-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/18/2023] [Indexed: 09/10/2023]
Abstract
It has recently been suggested that parameter estimates of computational models can be used to understand individual differences at the process level. One area of research in which this approach, called computational phenotyping, has taken hold is computational psychiatry. One requirement for successful computational phenotyping is that behavior and parameters are stable over time. Surprisingly, the test-retest reliability of behavior and model parameters remains unknown for most experimental tasks and models. The present study seeks to close this gap by investigating the test-retest reliability of canonical reinforcement learning models in the context of two often-used learning paradigms: a two-armed bandit and a reversal learning task. We tested independent cohorts for the two tasks (N = 69 and N = 47) via an online testing platform with a between-test interval of five weeks. Whereas reliability was high for personality and cognitive measures (with ICCs ranging from .67 to .93), it was generally poor for the parameter estimates of the reinforcement learning models (with ICCs ranging from .02 to .52 for the bandit task and from .01 to .71 for the reversal learning task). Given that simulations indicated that our procedures could detect high test-retest reliability, this suggests that a significant proportion of the variability must be ascribed to the participants themselves. In support of that hypothesis, we show that mood (stress and happiness) can partly explain within-participant variability. Taken together, these results are critical for current practices in computational phenotyping and suggest that individual variability should be taken into account in the future development of the field.
Collapse
Affiliation(s)
- Jessica V Schaaf
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands.
- Cognitive Neuroscience Department, Radboud University Medical Centre, Nijmegen, the Netherlands.
- Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands.
| | - Laura Weidinger
- DeepMind, London, United Kingdom
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| | - Lucas Molleman
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| | - Wouter van den Bos
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
| |
Collapse
|
18
|
Nicholas J, Amlang C, Lin CYR, Montaser-Kouhsari L, Desai N, Pan MK, Kuo SH, Shohamy D. The Role of the Cerebellum in Learning to Predict Reward: Evidence from Cerebellar Ataxia. CEREBELLUM (LONDON, ENGLAND) 2024; 23:1355-1368. [PMID: 38066397 PMCID: PMC11161554 DOI: 10.1007/s12311-023-01633-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/02/2023] [Indexed: 01/25/2024]
Abstract
Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia University, New York, NY, USA
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA
| | - Christian Amlang
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Chi-Ying R Lin
- Department of Neurology, Baylor College of Medicine, Houston, TX, USA
| | | | - Natasha Desai
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA
| | - Ming-Kai Pan
- Department of Medical Research, National Taiwan University Hospital, 100, Taipei, Taiwan
- Department and Graduate Institute of Pharmacology, National Taiwan University College of Medicine, 100, Taipei, Taiwan
- Cerebellar Research Center, National Taiwan University Hospital, Yun-Lin Branch, Yun-Lin, Taiwan
| | - Sheng-Han Kuo
- Department of Neurology, Columbia University Medical Center, 650 W. 168th St, Rm 305, New York, NY, 10032, USA.
- Initiative for Columbia Ataxia and Tremor, Columbia University Medical Center, New York, NY, USA.
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, NY, USA.
- Zuckerman Mind Brain Behavior Institute, Columbia University, Quad 3D, 3227 Broadway, New York, NY, 10027, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
19
|
Mograbi DC, Rodrigues R, Bienemann B, Huntley J. Brain Networks, Neurotransmitters and Psychedelics: Towards a Neurochemistry of Self-Awareness. Curr Neurol Neurosci Rep 2024; 24:323-340. [PMID: 38980658 PMCID: PMC11258181 DOI: 10.1007/s11910-024-01353-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/17/2024] [Indexed: 07/10/2024]
Abstract
PURPOSE OF REVIEW Self-awareness can be defined as the capacity of becoming the object of one's own awareness and, increasingly, it has been the target of scientific inquiry. Self-awareness has important clinical implications, and a better understanding of the neurochemical basis of self-awareness may help clarifying causes and developing interventions for different psychopathological conditions. The current article explores the relationship between neurochemistry and self-awareness, with special attention to the effects of psychedelics. RECENT FINDINGS The functioning of self-related networks, such as the default-mode network and the salience network, and how these are influenced by different neurotransmitters is discussed. The impact of psychedelics on self-awareness is reviewed in relation to specific processes, such as interoception, body ownership, agency, metacognition, emotional regulation and autobiographical memory, within a framework based on predictive coding. Improved outcomes in emotional regulation and autobiographical memory have been observed in association with the use of psychedelics, suggesting higher-order self-awareness changes, which can be modulated by relaxation of priors and improved coping mechanisms linked to cognitive flexibility. Alterations in bodily self-awareness are less consistent, being potentially impacted by doses employed, differences in acute/long-term effects and the presence of clinical conditions. Future studies investigating the effects of different molecules in rebalancing connectivity between resting-state networks may lead to novel therapeutic approaches and the refinement of existing treatments.
Collapse
Affiliation(s)
- Daniel C Mograbi
- Department of Psychology, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil.
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
| | - Rafael Rodrigues
- Department of Psychology, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil
| | - Bheatrix Bienemann
- Department of Psychology, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil
| | - Jonathan Huntley
- Division of Psychiatry, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
20
|
Keller GB, Sterzer P. Predictive Processing: A Circuit Approach to Psychosis. Annu Rev Neurosci 2024; 47:85-101. [PMID: 38424472 DOI: 10.1146/annurev-neuro-100223-121214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2024]
Abstract
Predictive processing is a computational framework that aims to explain how the brain processes sensory information by making predictions about the environment and minimizing prediction errors. It can also be used to explain some of the key symptoms of psychotic disorders such as schizophrenia. In recent years, substantial advances have been made in our understanding of the neuronal circuitry that underlies predictive processing in cortex. In this review, we summarize these findings and how they might relate to psychosis and to observed cell type-specific effects of antipsychotic drugs. We argue that quantifying the effects of antipsychotic drugs on specific neuronal circuit elements is a promising approach to understanding not only the mechanism of action of antipsychotic drugs but also psychosis. Finally, we outline some of the key experiments that should be done. The aims of this review are to provide an overview of the current circuit-based approaches to psychosis and to encourage further research in this direction.
Collapse
Affiliation(s)
- Georg B Keller
- Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland;
- Faculty of Natural Science, University of Basel, Basel, Switzerland
| | - Philipp Sterzer
- Department of Psychiatry, University of Basel, Basel, Switzerland
| |
Collapse
|
21
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
22
|
Tsay JS, Kim HE, McDougle SD, Taylor JA, Haith A, Avraham G, Krakauer JW, Collins AGE, Ivry RB. Fundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval. eLife 2024; 13:e91839. [PMID: 39087986 PMCID: PMC11293869 DOI: 10.7554/elife.91839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 07/22/2024] [Indexed: 08/02/2024] Open
Abstract
Motor learning is often viewed as a unitary process that operates outside of conscious awareness. This perspective has led to the development of sophisticated models designed to elucidate the mechanisms of implicit sensorimotor learning. In this review, we argue for a broader perspective, emphasizing the contribution of explicit strategies to sensorimotor learning tasks. Furthermore, we propose a theoretical framework for motor learning that consists of three fundamental processes: reasoning, the process of understanding action-outcome relationships; refinement, the process of optimizing sensorimotor and cognitive parameters to achieve motor goals; and retrieval, the process of inferring the context and recalling a control policy. We anticipate that this '3R' framework for understanding how complex movements are learned will open exciting avenues for future research at the intersection between cognition and action.
Collapse
Affiliation(s)
- Jonathan S Tsay
- Department of Psychology, Carnegie Mellon UniversityPittsburghUnited States
- Neuroscience Institute, Carnegie Mellon UniversityPittsburgUnited States
| | - Hyosub E Kim
- School of Kinesiology, University of British ColumbiaVancouverCanada
| | | | - Jordan A Taylor
- Department of Psychology, Princeton UniversityPrincetonUnited States
| | - Adrian Haith
- Department of Neurology, Johns Hopkins UniversityBaltimoreUnited States
| | - Guy Avraham
- Department of Psychology, University of California BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California BerkeleyBerkeleyUnited States
| | - John W Krakauer
- Department of Neurology, Johns Hopkins UniversityBaltimoreUnited States
- Department of Neuroscience, Johns Hopkins UniversityBaltimoreUnited States
- Santa Fe InstituteSanta FeUnited States
| | - Anne GE Collins
- Department of Psychology, University of California BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California BerkeleyBerkeleyUnited States
| | - Richard B Ivry
- Department of Psychology, University of California BerkeleyBerkeleyUnited States
- Helen Wills Neuroscience Institute, University of California BerkeleyBerkeleyUnited States
| |
Collapse
|
23
|
Peart DR, Nolan CJ, Stone AP, Williams MA, Karlovcec JM, Murray JE. Disruption of positive- and negative-feature morphine interoceptive occasion setters by dopamine receptor agonism and antagonism in male and female rats. Psychopharmacology (Berl) 2024; 241:1597-1615. [PMID: 38580732 DOI: 10.1007/s00213-024-06584-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 03/29/2024] [Indexed: 04/07/2024]
Abstract
RATIONALE Internally perceived stimuli evoked by morphine administration can form Pavlovian associations such that they can function as occasion setters (OSs) for externally perceived reward cues in rats, coming to modulate reward-seeking behaviour. Though much research has investigated mechanisms underlying opioid-related reinforcement and analgesia, neurotransmitter systems involved in the functioning of opioids as Pavlovian interoceptive discriminative stimuli remain to be disentangled despite documented differences in the development of tolerance to analgesic versus discriminative stimulus effects. OBJECTIVES Dopamine has been implicated in many opioid-related behaviours, so we aimed to investigate the role of this neurotransmitter in expression of morphine occasion setting. METHODS Male and female rats were assigned to positive- (FP) or negative-feature (FN) groups and received an injection of morphine or saline before each training session. A 15-s white noise conditioned stimulus (CS) was presented 8 times during every training session; offset of this stimulus was followed by 4-s access to liquid sucrose on morphine, but not saline, sessions for FP rats. FN rats learned the reverse contingency. Following stable discrimination, rats began generalization testing for expression of morphine-guided sucrose seeking after systemic pretreatment with different doses of the non-selective dopamine receptor antagonist, flupenthixol, and the non-selective dopamine receptor agonist, apomorphine, combined with training doses of morphine or saline in a Latin-square design. RESULTS The morphine discrimination was acquired under both FP and FN contingencies by males and females. Neither flupenthixol nor apomorphine at any dose substituted for morphine, but both apomorphine and flupenthixol disrupted expression of the morphine OS. This inhibition was specific to sucrose seeking during CS presentations rather than during the period before CS onset and, in the case of apomorphine more so than flupenthixol, to trials on which access to sucrose was anticipated. CONCLUSIONS Our findings lend support to a mechanism of occasion setting involving gating of CS-induced dopamine release rather than by direct dopaminergic modulation by the morphine stimulus.
Collapse
Affiliation(s)
- Davin R Peart
- Department of Psychology, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
- Collaborative Neurosciences Graduate Program, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
| | - Caitlin J Nolan
- Collaborative Neurosciences Graduate Program, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
- Department of Biomedical Sciences, University of Guelph, Guelph, ON, Canada
| | - Adiia P Stone
- Department of Psychology, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
- Collaborative Neurosciences Graduate Program, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
| | - Mckenna A Williams
- Department of Molecular and Cellular Biology, University of Guelph, Guelph, ON, Canada
| | - Jessica M Karlovcec
- Department of Psychology, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
- Collaborative Neurosciences Graduate Program, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada
| | - Jennifer E Murray
- Department of Psychology, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada.
- Collaborative Neurosciences Graduate Program, University of Guelph, 50 Stone Road East, Guelph, ON, N1G 2W1, Canada.
| |
Collapse
|
24
|
Ngetich R, Villalba-García C, Soborun Y, Vékony T, Czakó A, Demetrovics Z, Németh D. Learning and memory processes in behavioural addiction: A systematic review. Neurosci Biobehav Rev 2024; 163:105747. [PMID: 38870547 DOI: 10.1016/j.neubiorev.2024.105747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 05/28/2024] [Accepted: 06/01/2024] [Indexed: 06/15/2024]
Abstract
Similar to addictive substances, addictive behaviours such as gambling and gaming are associated with maladaptive modulation of key brain areas and functional networks implicated in learning and memory. Therefore, this review sought to understand how different learning and memory processes relate to behavioural addictions and to unravel their underlying neural mechanisms. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we systematically searched four databases - PsycINFO, PubMed, Scopus, and Web of Science using the agreed-upon search string. Findings suggest altered executive function-dependent learning processes and enhanced habit learning in behavioural addiction. Whereas the relationship between working memory and behavioural addiction is influenced by addiction type, working memory aspect, and task nature. Additionally, long-term memory is incoherent in individuals with addictive behaviours. Consistently, neurophysiological evidence indicates alterations in brain areas and networks implicated in learning and memory processes in behavioural addictions. Overall, the present review argues that, like substance use disorders, alteration in learning and memory processes may underlie the development and maintenance of behavioural addictions.
Collapse
Affiliation(s)
- Ronald Ngetich
- Centre of Excellence in Responsible Gaming, University of Gibraltar, Gibraltar, Gibraltar
| | | | - Yanisha Soborun
- Centre of Excellence in Responsible Gaming, University of Gibraltar, Gibraltar, Gibraltar
| | - Teodóra Vékony
- Centre de Recherche en Neurosciences de Lyon CRNL U1028 UMR5292, INSERM, CNRS, Université Claude Bernard Lyon 1, Bron, France; Department of Education and Psychology, Faculty of Social Sciences, University of Atlántico Medio, Las Palmas de Gran Canaria, Spain
| | - Andrea Czakó
- Centre of Excellence in Responsible Gaming, University of Gibraltar, Gibraltar, Gibraltar; Institute of Psychology, ELTE Eötvös Loránd University, Budapest, Hungary
| | - Zsolt Demetrovics
- Centre of Excellence in Responsible Gaming, University of Gibraltar, Gibraltar, Gibraltar; Institute of Psychology, ELTE Eötvös Loránd University, Budapest, Hungary; College of Education, Psychology and Social Work, Flinders University, Adelaide, Australia.
| | - Dezső Németh
- Centre de Recherche en Neurosciences de Lyon CRNL U1028 UMR5292, INSERM, CNRS, Université Claude Bernard Lyon 1, Bron, France; Department of Education and Psychology, Faculty of Social Sciences, University of Atlántico Medio, Las Palmas de Gran Canaria, Spain; BML-NAP Research Group, Institute of Psychology, Eötvös Loránd University & Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
| |
Collapse
|
25
|
Parvizi-Wayne D, Severs L. When the interoceptive and conceptual clash: The case of oppositional phenomenal self-modelling in Tourette syndrome. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:660-680. [PMID: 38777988 PMCID: PMC11233343 DOI: 10.3758/s13415-024-01189-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/15/2024] [Indexed: 05/25/2024]
Abstract
Tourette syndrome (TS) has been associated with a rich set of symptoms that are said to be uncomfortable, unwilled, and effortful to manage. Furthermore, tics, the canonical characteristic of TS, are multifaceted, and their onset and maintenance is complex. A formal account that integrates these features of TS symptomatology within a plausible theoretical framework is currently absent from the field. In this paper, we assess the explanatory power of hierarchical generative modelling in accounting for TS symptomatology from the perspective of active inference. We propose a fourfold analysis of sensory, motor, and cognitive phenomena associated with TS. In Section 1, we characterise tics as a form of action aimed at sensory attenuation. In Section 2, we introduce the notion of epistemic ticcing and describe such behaviour as the search for evidence that there is an agent (i.e., self) at the heart of the generative hierarchy. In Section 3, we characterise both epistemic (sensation-free) and nonepistemic (sensational) tics as habitual behaviour. Finally, in Section 4, we propose that ticcing behaviour involves an inevitable conflict between distinguishable aspects of selfhood; namely, between the minimal phenomenal sense of self-which is putatively underwritten by interoceptive inference-and the explicit preferences that constitute the individual's conceptual sense of self. In sum, we aim to provide an empirically informed analysis of TS symptomatology under active inference, revealing a continuity between covert and overt features of the condition.
Collapse
Affiliation(s)
- D Parvizi-Wayne
- Department of Psychology, Royal Holloway University of London, London, UK.
| | - L Severs
- Centre for the Philosophy of Science, Faculty of Sciences, University of Lisbon, Lisbon, Portugal
- Ruhr-Universität Bochum, Institute of Philosophy II, Bochum, Germany
| |
Collapse
|
26
|
Lee RS, Sagiv Y, Engelhard B, Witten IB, Daw ND. A feature-specific prediction error model explains dopaminergic heterogeneity. Nat Neurosci 2024; 27:1574-1586. [PMID: 38961229 DOI: 10.1038/s41593-024-01689-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 05/22/2024] [Indexed: 07/05/2024]
Abstract
The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.
Collapse
Affiliation(s)
- Rachel S Lee
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Yotam Sagiv
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | | | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
27
|
Cinotti F, Coutureau E, Khamassi M, Marchand AR, Girard B. Regulation of reinforcement learning parameters captures long-term changes in rat behaviour. Eur J Neurosci 2024; 60:4469-4490. [PMID: 38923238 DOI: 10.1111/ejn.16449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Revised: 05/14/2024] [Accepted: 06/05/2024] [Indexed: 06/28/2024]
Abstract
In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to stabilise learning and exploit what they currently believe to be their best option, or instead explore potential alternatives and learn fast from new observations. While such a trade-off has been extensively studied in pretrained animals facing non-stationary decision-making tasks, it is yet unknown how they progressively tune it while learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long-term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three-armed bandit task across 24 days of pretraining. We found that the day-by-day evolution of rat performance and win-shift tendency revealed a progressive stabilisation of the way they regulated reinforcement learning parameters. We successfully captured these behavioural adaptations using a meta-learning model in which either the learning rate or the inverse temperature was controlled by the average reward rate.
Collapse
Affiliation(s)
- François Cinotti
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, CNRS, Paris, France
- University of Reading, School of Psychology and Clinical Language Sciences, Whiteknights, Reading, UK
| | | | - Mehdi Khamassi
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, CNRS, Paris, France
| | | | - Benoît Girard
- Institut des Systèmes Intelligents et de Robotique, Sorbonne Université, CNRS, Paris, France
| |
Collapse
|
28
|
Kondo HM, Oba T, Ezaki T, Kochiyama T, Shimada Y, Ohira H. Striatal GABA levels correlate with risk sensitivity in monetary loss. Front Neurosci 2024; 18:1439656. [PMID: 39145302 PMCID: PMC11321969 DOI: 10.3389/fnins.2024.1439656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2024] [Accepted: 07/17/2024] [Indexed: 08/16/2024] Open
Abstract
Background Decision-making under risk is a common challenge. It is known that risk-taking behavior varies between contexts of reward and punishment, yet the mechanisms underlying this asymmetry in risk sensitivity remain unclear. Methods This study used a monetary task to investigate neurochemical mechanisms and brain dynamics underpinning risk sensitivity. Twenty-eight participants engaged in a task requiring selection of visual stimuli to maximize monetary gains and minimize monetary losses. We modeled participant trial-and-error processes using reinforcement learning. Results Participants with higher subjective utility parameters showed risk preference in the gain domain (r = -0.59) and risk avoidance in the loss domain (r = -0.77). Magnetic resonance spectroscopy (MRS) revealed that risk avoidance in the loss domain was associated with γ-aminobutyric acid (GABA) levels in the ventral striatum (r = -0.42), but not in the insula (r = -0.15). Using functional magnetic resonance imaging (fMRI), we tested whether risk-sensitive brain dynamics contribute to participant risky choices. Energy landscape analyses demonstrated that higher switching rates between brain states, including the striatum and insula, were correlated with risk avoidance in the loss domain (r = -0.59), a relationship not observed in the gain domain (r = -0.02). Conclusions These findings from MRS and fMRI suggest that distinct mechanisms are involved in gain/loss decision making, mediated by subcortical neurometabolite levels and brain dynamic transitions.
Collapse
Affiliation(s)
| | - Takeyuki Oba
- Graduate School of Informatics, Nagoya University, Nagoya, Aichi, Japan
| | - Takahiro Ezaki
- Precursory Research for Embryonic Science and Technology, Japan Science and Technology Agency, Kawaguchi, Saitama, Japan
- Research Center for Advanced Science and Technology, University of Tokyo, Tokyo, Japan
| | | | - Yasuhiro Shimada
- Advanced ICT Research Institute, National Institute of Information and Communications Technology, Osaka, Japan
| | - Hideki Ohira
- Graduate School of Informatics, Nagoya University, Nagoya, Aichi, Japan
| |
Collapse
|
29
|
Bin Ibrahim MZ, Wang Z, Sajikumar S. Synapses tagged, memories kept: synaptic tagging and capture hypothesis in brain health and disease. Philos Trans R Soc Lond B Biol Sci 2024; 379:20230237. [PMID: 38853570 PMCID: PMC11343274 DOI: 10.1098/rstb.2023.0237] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 01/29/2024] [Accepted: 02/13/2024] [Indexed: 06/11/2024] Open
Abstract
The synaptic tagging and capture (STC) hypothesis lays the framework on the synapse-specific mechanism of protein synthesis-dependent long-term plasticity upon synaptic induction. Activated synapses will display a transient tag that will capture plasticity-related products (PRPs). These two events, tag setting and PRP synthesis, can be teased apart and have been studied extensively-from their electrophysiological and pharmacological properties to the molecular events involved. Consequently, the hypothesis also permits interactions of synaptic populations that encode different memories within the same neuronal population-hence, it gives rise to the associativity of plasticity. In this review, the recent advances and progress since the experimental debut of the STC hypothesis will be shared. This includes the role of neuromodulation in PRP synthesis and tag integrity, behavioural correlates of the hypothesis and modelling in silico. STC, as a more sensitive assay for synaptic health, can also assess neuronal aberrations. We will also expound how synaptic plasticity and associativity are altered in ageing-related decline and pathological conditions such as juvenile stress, cancer, sleep deprivation and Alzheimer's disease. This article is part of a discussion meeting issue 'Long-term potentiation: 50 years on'.
Collapse
Affiliation(s)
- Mohammad Zaki Bin Ibrahim
- Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore117597, Singapore
- Neurobiology Programme, Life Sciences Institute, National University of Singapore, Singapore119077, Singapore
| | - Zijun Wang
- Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore117597, Singapore
- Neurobiology Programme, Life Sciences Institute, National University of Singapore, Singapore119077, Singapore
| | - Sreedharan Sajikumar
- Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore117597, Singapore
- Neurobiology Programme, Life Sciences Institute, National University of Singapore, Singapore119077, Singapore
- Healthy Longevity Translational Research Programme, Yong Loo Lin School of Medicine, National University of Singapore, Singapore117597, Singapore
| |
Collapse
|
30
|
Banuelos C, Creswell K, Walsh C, Manuck SB, Gianaros PJ, Verstynen T. D2 dopamine receptor expression, reactivity to rewards, and reinforcement learning in a complex value-based decision-making task. Soc Cogn Affect Neurosci 2024; 19:nsae050. [PMID: 38988197 PMCID: PMC11281849 DOI: 10.1093/scan/nsae050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 04/24/2024] [Accepted: 07/10/2024] [Indexed: 07/12/2024] Open
Abstract
Different dopamine (DA) subtypes have opposing dynamics at postsynaptic receptors, with the ratio of D1 to D2 receptors determining the relative sensitivity to gains and losses, respectively, during value-based learning. This effective sensitivity to different reward feedback interacts with phasic DA levels to determine the effectiveness of learning, particularly in dynamic feedback situations where the frequency and magnitude of rewards need to be integrated over time to make optimal decisions. We modeled this effect in simulations of the underlying basal ganglia pathways and then tested the predictions in individuals with a variant of the human dopamine receptor D2 (DRD2; -141C Ins/Del and Del/Del) gene that associates with lower levels of D2 receptor expression (N = 119) and compared their performance in the Iowa Gambling Task to noncarrier controls (N = 319). Ventral striatal (VS) reactivity to rewards was measured in the Cards task with fMRI. DRD2 variant carriers made less effective decisions than noncarriers, but this effect was not moderated by VS reward reactivity as is hypothesized by our model. These results suggest that the interaction between DA receptor subtypes and reactivity to rewards during learning may be more complex than originally thought.
Collapse
Affiliation(s)
- Cristina Banuelos
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, United States
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, United States
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, United States
| | - Kasey Creswell
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, United States
| | - Catherine Walsh
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA 15260, United States
| | - Stephen B Manuck
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA 15260, United States
| | - Peter J Gianaros
- Department of Psychology, University of Pittsburgh, Pittsburgh, PA 15260, United States
- Center for the Neural Basis of Cognition, University of Pittsburgh, Pittsburgh, PA 15260, United States
| | - Timothy Verstynen
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, United States
- Carnegie Mellon Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, United States
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA 15213, United States
- Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, United States
| |
Collapse
|
31
|
Kocum CG, Cam Y, Shay DA, Schweizer TA, Konrad ER, Houska TK, Sardina CA, Schachtman TR, Vieira-Potter VJ, Will MJ. Voluntary wheel running access produces opposite effects in male and female rats on both palatable diet consumption and associated ventral striatal opioid- and dopamine-related gene expression. Front Integr Neurosci 2024; 18:1426219. [PMID: 39131599 PMCID: PMC11310025 DOI: 10.3389/fnint.2024.1426219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Accepted: 07/03/2024] [Indexed: 08/13/2024] Open
Abstract
The relationship between physical activity levels and feeding behaviors has been a focus of preclinical research for decades, yet this interaction has only recently been explored for potential sex differences. The aim of the present study was to isolate sex-dependent effects of voluntary wheel running (RUN) vs. sedentary locked wheel (SED) home cage conditions on palatability-driven feeding behavior using a 2-diet choice task between standard chow and a high-fat diet. The sex-dependent effects of physical activity on feeding behavior were examined following a within-subject novel reversal design of physical activity conditions (i.e., RUN > SED > RUN), to assess temporal sensitivity of the interaction. Following the final 2 weeks of reestablished and sustained RUN vs. SED conditions in separate groups of both males and females, reward-related opioid and dopamine gene expression within the nucleus accumbens (Acb) brain region were analyzed. Results demonstrated that the initial RUN > SED transition led to sex-dependent effects of SED condition, as males increased, and females decreased their high fat consumption, compared to their respective high fat consumption during previous RUN condition phase. Following reintroduction to the RUN condition, males decreased, and females increased their high fat consumption, compared to their separate SED control group. Last, sex-dependent shifts in ventral striatal opioid- and dopamine-related gene expression were observed to parallel the behavioral effects. The major findings of the study reveal that SED and RUN home cage conditions shift palatability-driven feeding in the opposite direction for males and females, these effects are sensitive to reversal, and these sex-dependent feeding behaviors track sex-dependent changes to critical reward-related gene expression patterns in the Acb. Considering the present high rates of sedentary behavior and obesity, furthering our understanding of the interaction between physical activity (or lack thereof) and feeding behavior should be a priority, especially in the context of these divergent sex-dependent outcomes.
Collapse
Affiliation(s)
- Courtney G. Kocum
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
| | - Yonca Cam
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
| | - Dusti A. Shay
- Department of Nutrition and Exercise Physiology, University of Missouri, Columbia, MO, United States
| | - Tim A. Schweizer
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
- Department of Biological Sciences, University of Missouri, Columbia, MO, United States
| | - Ella R. Konrad
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
| | - Tabitha K. Houska
- Department of Biological Sciences, University of Missouri, Columbia, MO, United States
| | - Carlos A. Sardina
- Department of Philosophy, University of Missouri, Columbia, MO, United States
| | - Todd R. Schachtman
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
| | - Victoria J. Vieira-Potter
- Department of Nutrition and Exercise Physiology, University of Missouri, Columbia, MO, United States
| | - Matthew J. Will
- Department of Psychological Sciences, University of Missouri, Columbia, MO, United States
| |
Collapse
|
32
|
Gershman SJ, Assad JA, Datta SR, Linderman SW, Sabatini BL, Uchida N, Wilbrecht L. Explaining dopamine through prediction errors and beyond. Nat Neurosci 2024:10.1038/s41593-024-01705-4. [PMID: 39054370 DOI: 10.1038/s41593-024-01705-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 06/13/2024] [Indexed: 07/27/2024]
Abstract
The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic dopamine observed in behaving animals. This Perspective helps to resolve some of the conflicting interpretations of dopamine that currently exist in the literature. We focus on the following three empirical challenges to the RPE theory of dopamine: why does dopamine (1) ramp up as animals approach rewards, (2) respond to sensory and motor features and (3) influence action selection? We argue that the prediction error concept, once it has been suitably modified and generalized based on an analysis of each computational problem, answers each challenge. Nonetheless, there are a number of additional empirical findings that appear to demand fundamentally different theoretical explanations beyond encoding RPE. Therefore, looking forward, we discuss the prospects for a unifying theory that respects the diversity of dopamine signaling and function as well as the complex circuitry that both underlies and responds to dopaminergic transmission.
Collapse
Affiliation(s)
- Samuel J Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, USA.
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA.
| | - John A Assad
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Scott W Linderman
- Department of Statistics and Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Bernardo L Sabatini
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Linda Wilbrecht
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| |
Collapse
|
33
|
Dere E. Insights into conscious cognitive information processing. Front Behav Neurosci 2024; 18:1443161. [PMID: 39135748 PMCID: PMC11318070 DOI: 10.3389/fnbeh.2024.1443161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2024] [Accepted: 07/15/2024] [Indexed: 08/15/2024] Open
Abstract
For over a century, the neuro- and pathophysiological, behavioral, and cognitive correlates of consciousness have been an active field of theoretical considerations and empirical research in a wide range of modern disciplines. Conscious cognitive processing of information cannot be observed directly, but might be inferred from step-like discontinuities in learning performance or sudden insight-based improvements in problem solving behavior. It is assumed that a sudden step of knowledge associated with insight requires a creative reorganization of mental representations of task- or problem-relevant information and the restructuration of the task, respectively problem to overcome an cognitive dead-end or impasse. Discontinuities in learning performance or problem solving after an insight event can be used as time-tags to capture the time window in which conscious cognitive information processing must have taken place. According to the platform theory of conscious cognitive information processing, the reorganization and restructuration processes, require the maintenance of task- or problem-relevant information in working memory for the operation of executive functions on these mental representations. Electrophysiological evidence suggests that the reorganization and restructuration processes in working memory, that precede insight-based problem solutions are accompanied by an increase in the power of gamma oscillations in cortical areas including the prefrontal cortex. Empirical evidence and theoretical assumptions argue for an involvement of gap junction channels and connexin hemichannels in cortical gamma-oscillations and working memory processes. Discontinuities in learning or problem solving performance might be used as time-tags to investigate the implication of gap junction channels and hemichannels in conscious cognitive processing.
Collapse
Affiliation(s)
- Ekrem Dere
- Department of Behavioral and Clinical Neuroscience, Ruhr-University Bochum, Bochum, Germany
- Unité de Formation et de Recherche des Sciences de la Vie (UFR 927), Institut de Biologie Paris-Seine, Sorbonne Université, Paris, France
| |
Collapse
|
34
|
Engel L, Wolff AR, Blake M, Collins VL, Sinha S, Saunders BT. Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning. Curr Biol 2024; 34:3086-3101.e4. [PMID: 38925117 PMCID: PMC11279555 DOI: 10.1016/j.cub.2024.05.069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Revised: 04/25/2024] [Accepted: 05/29/2024] [Indexed: 06/28/2024]
Abstract
Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.
Collapse
Affiliation(s)
- Liv Engel
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Amy R Wolff
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Madelyn Blake
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Val L Collins
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Sonal Sinha
- Krieger School of Arts & Sciences, Johns Hopkins University, 3400 N. Charles St, Baltimore, MD 21218, USA
| | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA.
| |
Collapse
|
35
|
Lee D, Lau N, Liu L, Root CM. Transformation of valence signaling in a striatopallidal circuit. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.01.551547. [PMID: 37577586 PMCID: PMC10418236 DOI: 10.1101/2023.08.01.551547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/15/2023]
Abstract
The ways in which sensory stimuli acquire motivational valence through association with other stimuli is one of the simplest forms of learning. Though we have identified many brain nuclei that play various roles in reward processing, a significant gap remains in understanding how valence encoding transforms through the layers of sensory processing. To address this gap, we carried out a comparative investigation of the anteromedial olfactory tubercle (OT), and the ventral pallidum (VP) - 2 connected nuclei of the basal ganglia which have both been implicated in reward processing. First, using anterograde and retrograde tracing, we show that both D1 and D2 neurons of the anteromedial OT project primarily to the VP and minimally elsewhere. Using 2-photon calcium imaging, we then investigated how the identity of the odor and reward contingency of the odor are differently encoded by neurons in either structure during a classical conditioning paradigm. We find that VP neurons robustly encode reward contingency, but not identity, in low-dimensional space. In contrast, the OT neurons primarily encode odor identity in high-dimensional space. Although D1 OT neurons showed larger responses to rewarded odors than other odors, consistent with prior findings, we interpret this as identity encoding with enhanced contrast. Finally, using a novel conditioning paradigm that decouples reward contingency and licking vigor, we show that both features are encoded by non-overlapping VP neurons. These results provide a novel framework for the striatopallidal circuit in which a high-dimensional encoding of stimulus identity is collapsed onto a low-dimensional encoding of motivational valence.
Collapse
|
36
|
Bouabid S, Zhang L, Vu MAT, Tang K, Graham BM, Noggle CA, Howe MW. Spatially organized striatum-wide acetylcholine dynamics for the learning and extinction of Pavlovian cues and actions. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.10.602947. [PMID: 39071401 PMCID: PMC11275942 DOI: 10.1101/2024.07.10.602947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
Striatal acetylcholine (ACh) has been linked to behavioral flexibility. A key component of flexibility is down-regulating responding as valued cues and actions become decoupled from positive outcomes. We used array fiber photometry in mice to investigate how ACh release across the striatum evolves during learning and extinction of Pavlovian associations. Changes in multi-phasic release to cues and consummatory actions were bi-directional and region-specific. Following extinction, increases in cue-evoked ACh release emerged in the anterior dorsal striatum (aDS) which preceded a down-regulation of anticipatory behavior. Silencing ACh release from cholinergic interneurons in the aDS blocked behavioral extinction. Dopamine release dipped below baseline for down-shifted cues, but glutamate input onto cholinergic interneurons did not change, suggesting an intrastriatal mechanism for the emergence of ACh increases. Our large-scale mapping of striatal ACh dynamics during learning pinpoints region-specific elevations in ACh release positioned to down-regulate behavior during extinction, a central feature of flexible behavior.
Collapse
Affiliation(s)
- Safa Bouabid
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Liangzhu Zhang
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Mai-Anh T. Vu
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Kylie Tang
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Benjamin M. Graham
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Christian A. Noggle
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Mark W. Howe
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| |
Collapse
|
37
|
Burwell SC, Yan H, Lim SS, Shields BC, Tadross MR. Natural phasic inhibition of dopamine neurons signals cognitive rigidity. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.09.593320. [PMID: 38766037 PMCID: PMC11100816 DOI: 10.1101/2024.05.09.593320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
When animals unexpectedly fail, their dopamine neurons undergo phasic inhibition that canonically drives extinction learning-a cognitive-flexibility mechanism for discarding outdated strategies. However, the existing evidence equates natural and artificial phasic inhibition, despite their spatiotemporal differences. Addressing this gap, we targeted a GABAA-receptor antagonist precisely to dopamine neurons, yielding three unexpected findings. First, this intervention blocked natural phasic inhibition selectively, leaving tonic activity unaffected. Second, blocking natural phasic inhibition accelerated extinction learning-opposite to canonical mechanisms. Third, our approach selectively benefitted perseverative mice, restoring rapid extinction without affecting new reward learning. Our findings reveal that extinction learning is rapid by default and slowed by natural phasic inhibition-challenging foundational learning theories, while delineating a synaptic mechanism and therapeutic target for cognitive rigidity.
Collapse
Affiliation(s)
- Sasha C.V. Burwell
- Department of Neurobiology, Duke University, Durham, NC
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD
| | - Haidun Yan
- Department of Biomedical Engineering, Duke University, NC
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD
| | - Shaun S.X. Lim
- Department of Biomedical Engineering, Duke University, NC
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD
| | - Brenda C. Shields
- Department of Biomedical Engineering, Duke University, NC
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD
| | - Michael R. Tadross
- Department of Neurobiology, Duke University, Durham, NC
- Department of Biomedical Engineering, Duke University, NC
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD
| |
Collapse
|
38
|
Cone I, Clopath C, Shouval HZ. Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time. Nat Commun 2024; 15:5856. [PMID: 38997276 PMCID: PMC11245539 DOI: 10.1038/s41467-024-50205-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 07/02/2024] [Indexed: 07/14/2024] Open
Abstract
The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.
Collapse
Affiliation(s)
- Ian Cone
- Department of Bioengineering, Imperial College London, London, UK
- Department of Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, USA
- Applied Physics Program, Rice University, Houston, TX, USA
| | - Claudia Clopath
- Department of Bioengineering, Imperial College London, London, UK
| | - Harel Z Shouval
- Department of Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, USA.
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA.
| |
Collapse
|
39
|
Zid M, Laurie VJ, Levine-Champagne A, Shourkeshti A, Harrell D, Herman AB, Ebitz RB. Humans forage for reward in reinforcement learning tasks. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.08.602539. [PMID: 39026817 PMCID: PMC11257465 DOI: 10.1101/2024.07.08.602539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Abstract
How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic answer is that we calculate the value of each option and then compare the values to choose the most rewarding, modulo some exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold, at which point we start exploring other options. In order to determine which view better describes human decision-making, we developed a novel, foraging-inspired sequential decision-making model and used it to ask whether humans compare to threshold ("Forage") or compare alternatives ("Reinforcement-Learn" [RL]). We found that the foraging model was a better fit for participant behavior, better predicted the participants' tendency to repeat choices, and predicted the existence of held-out participants with a pattern of choice that was almost impossible under RL. Together, these results suggest that humans use foraging computations, rather than RL, even in classic reinforcement learning tasks.
Collapse
Affiliation(s)
- Meriam Zid
- Department of Neuroscience, University of Montreal, Montreal, QC , H3T 1J4, Canada
| | - Veldon-James Laurie
- Department of Neuroscience, University of Montreal, Montreal, QC , H3T 1J4, Canada
| | | | - Akram Shourkeshti
- Department of Neuroscience, University of Montreal, Montreal, QC , H3T 1J4, Canada
| | - Dameon Harrell
- Department of Psychiatry, University of Minnesota, Minneapolis, MN, 55455, USA
| | - Alexander B. Herman
- Department of Psychiatry, University of Minnesota, Minneapolis, MN, 55455, USA
| | - R. Becket Ebitz
- Department of Neuroscience, University of Montreal, Montreal, QC , H3T 1J4, Canada
| |
Collapse
|
40
|
Kirschner H, Molla HM, Nassar MR, de Wit H, Ullsperger M. Methamphetamine-induced adaptation of learning rate dynamics depend on baseline performance. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.04.602054. [PMID: 39026741 PMCID: PMC11257491 DOI: 10.1101/2024.07.04.602054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Abstract
The ability to calibrate learning according to new information is a fundamental component of an organism's ability to adapt to changing conditions. Yet, the exact neural mechanisms guiding dynamic learning rate adjustments remain unclear. Catecholamines appear to play a critical role in adjusting the degree to which we use new information over time, but individuals vary widely in the manner in which they adjust to changes. Here, we studied the effects of a low dose of methamphetamine (MA), and individual differences in these effects, on probabilistic reversal learning dynamics in a within-subject, double-blind, randomized design. Participants first completed a reversal learning task during a drug-free baseline session to provide a measure of baseline performance. Then they completed the task during two sessions, one with MA (20 mg oral) and one with placebo (PL). First, we showed that, relative to PL, MA modulates the ability to dynamically adjust learning from prediction errors. Second, this effect was more pronounced in participants who performed poorly at baseline. These results present novel evidence for the involvement of catecholaminergic transmission on learning flexibility and highlights that baseline performance modulates the effect of the drug.
Collapse
Affiliation(s)
- Hans Kirschner
- Institute of Psychology, Otto-von-Guericke University, D-39106 Magdeburg, Germany
| | - Hanna M Molla
- Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, Illinois, USA
| | - Matthew R Nassar
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence RI 02912-1821, USA
- Department of Neuroscience, Brown University, Providence RI 02912-1821, USA
| | - Harriet de Wit
- Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, Illinois, USA
| | - Markus Ullsperger
- Institute of Psychology, Otto-von-Guericke University, D-39106 Magdeburg, Germany
- Center for Behavioral Brain Sciences, D-39106 Magdeburg, Germany
- German Center for Mental Health (DZPG), Center for Intervention and Research on Adaptive and Maladaptive Brain Circuits Underlying Mental Health (C-I-R-C), Halle-Jena-Magdeburg, Germany
| |
Collapse
|
41
|
Zhukovskaya A, Christopher Z, Willmore L, Pan Vazquez A, Janarthanan S, Falkner A, Witten I. Heightened lateral habenula activity during stress produces brainwide and behavioral substrates of susceptibility. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.11.06.565681. [PMID: 39005438 PMCID: PMC11244933 DOI: 10.1101/2023.11.06.565681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/16/2024]
Abstract
Some individuals are susceptible to the experience of chronic stress and others are more resilient. While many brain regions implicated in learning are dysregulated after stress, little is known about whether and how neural teaching signals during stress differ between susceptible and resilient individuals. Here, we seek to determine if activity in the lateral habenula (LHb), which encodes a negative teaching signal, differs between susceptible and resilient mice during stress to produce different outcomes. After, but not before, chronic social defeat stress (CSDS), the LHb is active when susceptible mice are in the proximity of the aggressor strain. During stress itself, LHb activity is higher in susceptible mice during aggressor proximity, and activation of the LHb during stress biases mice towards susceptibility. This manipulation generates a persistent and widespread increase in the balance of subcortical versus cortical activity in susceptible mice. Taken together, our results indicate that heightened activity in the LHb during stress produces lasting brainwide and behavioral substrates of susceptibility.
Collapse
|
42
|
Martin DA, Delgado AM, Calu DJ. Effects of psychedelic, DOI, on nucleus accumbens dopamine signaling to predictable rewards and cues in rats. Neuropsychopharmacology 2024:10.1038/s41386-024-01912-4. [PMID: 38971932 DOI: 10.1038/s41386-024-01912-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/14/2024] [Revised: 06/10/2024] [Accepted: 06/24/2024] [Indexed: 07/08/2024]
Abstract
Psychedelics produce lasting therapeutic responses in neuropsychiatric diseases suggesting they may disrupt entrenched associations and catalyze learning. Here, we examine psychedelic 5-HT2A/2C agonist, DOI, effects on dopamine signaling in the nucleus accumbens (NAc) core, a region extensively linked to reward learning, motivation, and drug-seeking. We measure phasic dopamine transients following acute DOI administration in rats during well learned Pavlovian tasks in which sequential cues predict rewards. We find that DOI (0.0-1.2 mg/kg, i.p.) increases dopamine signals, photometrically measured using GRABDA optical sensor, to rewards and proximal reward cues, but not to the distal cues that predict these events. We determine that the elevated dopamine produced by DOI to reward cues occurs independently of DOI-induced changes in reward value. The increased dopamine associated with predictable reward cues and rewards supports DOI-induced increases in prediction error signaling. These findings lay a foundation for developing psychedelic strategies aimed at engaging error-driven learning mechanisms to disrupt entrenched associations or produce new associations.
Collapse
Affiliation(s)
- David A Martin
- Department of Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA.
| | - Angel M Delgado
- Department of Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Donna J Calu
- Department of Neurobiology, University of Maryland School of Medicine, Baltimore, MD, USA
| |
Collapse
|
43
|
Zhang M, Zhao C, Zhang M, Mao S, Yang M, Mao Z, Xing X. Influence of reinforcement learning on the inhibitory control of Internet gaming disorder. Psych J 2024. [PMID: 38965885 DOI: 10.1002/pchj.772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Accepted: 04/15/2024] [Indexed: 07/06/2024]
Abstract
Reward processing dysfunction and inhibition control deficiency have been observed in Internet gaming disorder (IGD). However, it is still unclear whether the previous reinforcement learning depends on reward/punishment feedback influences on the cognitive inhibitory control of IGD. This study compared the differences between an IGD group and healthy people without game experiences in the probability selection task and the subsequent stop signal task by the method of behavioral experiments, in order to explore whether the reward learning ability is impaired in the IGD group. We also discuss the influence of previous reward learning on subsequent inhibition control. The results showed that (1) during the reward learning phase, the IGD group's accuracy was significantly lower than that of the control group; (2) compared with the control group, the IGD group's reaction times were longer in the transfer phase; (3) for no-go trials of the inhibitory control phase after reward learning, the accuracy of the reward-related stimulation in the IGD group was lower than that of punishment-related or neutral stimulation, but there was no significant difference among the three conditions in the control group. These findings indicated that the reinforcement learning ability of the IGD group was impaired, which further caused the abnormal response to reinforcement stimuli.
Collapse
Affiliation(s)
- Mengyue Zhang
- Department of Psychology, Henan University, Kaifeng, China
| | - Chenyue Zhao
- Department of Psychology, Henan University, Kaifeng, China
| | - Meng Zhang
- Department of Psychology, Henan University, Kaifeng, China
| | - Shuangshuang Mao
- Department of Basic Education, Hunan Vocational College for Nationalities, Yueyang, China
| | - Mengyao Yang
- Department of Psychology, Henan University, Kaifeng, China
| | - Ziyu Mao
- Department of Psychology, Henan University, Kaifeng, China
| | - Xiaoli Xing
- Department of Psychology, Henan University, Kaifeng, China
| |
Collapse
|
44
|
Long C, Lee K, Yang L, Dafalias T, Wu AK, Masmanidis SC. Constraints on the subsecond modulation of striatal dynamics by physiological dopamine signaling. Nat Neurosci 2024:10.1038/s41593-024-01699-z. [PMID: 38961230 DOI: 10.1038/s41593-024-01699-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 06/07/2024] [Indexed: 07/05/2024]
Abstract
Dopaminergic neurons play a crucial role in associative learning, but their capacity to regulate behavior on subsecond timescales remains debated. It is thought that dopaminergic neurons drive certain behaviors by rapidly modulating striatal spiking activity; however, a view has emerged that only artificially high (that is, supra-physiological) dopamine signals alter behavior on fast timescales. This raises the possibility that moment-to-moment striatal spiking activity is not strongly shaped by dopamine signals in the physiological range. To test this, we transiently altered dopamine levels while monitoring spiking responses in the ventral striatum of behaving mice. These manipulations led to only weak changes in striatal activity, except when dopamine release exceeded reward-matched levels. These findings suggest that dopaminergic neurons normally play a minor role in the subsecond modulation of striatal dynamics in relation to other inputs and demonstrate the importance of discerning dopaminergic neuron contributions to brain function under physiological and potentially nonphysiological conditions.
Collapse
Affiliation(s)
- Charltien Long
- Department of Neurobiology, University of California, Los Angeles, CA, USA
- Medical Scientist Training Program, University of California, Los Angeles, CA, USA
| | - Kwang Lee
- Department of Neurobiology, University of California, Los Angeles, CA, USA
- Department of Brain Sciences, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, Republic of Korea
| | - Long Yang
- Department of Neurobiology, University of California, Los Angeles, CA, USA
| | - Theresia Dafalias
- Department of Neurobiology, University of California, Los Angeles, CA, USA
- Graduate Program for Neuroscience, Boston University, Boston, MA, USA
| | - Alexander K Wu
- Department of Neurobiology, University of California, Los Angeles, CA, USA
| | - Sotiris C Masmanidis
- Department of Neurobiology, University of California, Los Angeles, CA, USA.
- California Nanosystems Institute, University of California, Los Angeles, CA, USA.
| |
Collapse
|
45
|
Labouesse MA, Wilhelm M, Kagiampaki Z, Yee AG, Denis R, Harada M, Gresch A, Marinescu AM, Otomo K, Curreli S, Serratosa Capdevila L, Zhou X, Cola RB, Ravotto L, Glück C, Cherepanov S, Weber B, Zhou X, Katner J, Svensson KA, Fellin T, Trudeau LE, Ford CP, Sych Y, Patriarchi T. A chemogenetic approach for dopamine imaging with tunable sensitivity. Nat Commun 2024; 15:5551. [PMID: 38956067 PMCID: PMC11219860 DOI: 10.1038/s41467-024-49442-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Accepted: 06/05/2024] [Indexed: 07/04/2024] Open
Abstract
Genetically-encoded dopamine (DA) sensors enable high-resolution imaging of DA release, but their ability to detect a wide range of extracellular DA levels, especially tonic versus phasic DA release, is limited by their intrinsic affinity. Here we show that a human-selective dopamine receptor positive allosteric modulator (PAM) can be used to boost sensor affinity on-demand. The PAM enhances DA detection sensitivity across experimental preparations (in vitro, ex vivo and in vivo) via one-photon or two-photon imaging. In vivo photometry-based detection of optogenetically-evoked DA release revealed that DETQ administration produces a stable 31 minutes window of potentiation without effects on animal behavior. The use of the PAM revealed region-specific and metabolic state-dependent differences in tonic DA levels and enhanced single-trial detection of behavior-evoked phasic DA release in cortex and striatum. Our chemogenetic strategy can potently and flexibly tune DA imaging sensitivity and reveal multi-modal (tonic/phasic) DA signaling across preparations and imaging approaches.
Collapse
Affiliation(s)
- Marie A Labouesse
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
- Neuroscience Center Zurich, University and ETH Zürich, Zürich, Switzerland
- Department of Health Sciences and Technology, ETH Zürich, Zürich, Switzerland
| | - Maria Wilhelm
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
- Institute for Neuroscience, ETH Zurich, Zurich, Switzerland
| | | | - Andrew G Yee
- Department of Pharmacology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Raphaelle Denis
- Department of Pharmacology & Physiology, Faculty of Medicine, SNC and CIRCA Research groups, Université de Montréal, Montréal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, SNC and CIRCA Research groups, Université de Montréal, Montréal, QC, Canada
| | - Masaya Harada
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | - Andrea Gresch
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | | | - Kanako Otomo
- Department of Health Sciences and Technology, ETH Zürich, Zürich, Switzerland
| | - Sebastiano Curreli
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy
| | | | - Xuehan Zhou
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | - Reto B Cola
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | - Luca Ravotto
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | - Chaim Glück
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
| | - Stanislav Cherepanov
- Institute of Cellular and Integrative Neuroscience, University of Strasbourg, Strasbourg, France
| | - Bruno Weber
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland
- Neuroscience Center Zurich, University and ETH Zürich, Zürich, Switzerland
| | - Xin Zhou
- Eli Lilly and Company, Indianapolis, IN, USA
| | | | | | - Tommaso Fellin
- Optical Approaches to Brain Function Laboratory, Istituto Italiano di Tecnologia, Genova, Italy
| | - Louis-Eric Trudeau
- Department of Pharmacology & Physiology, Faculty of Medicine, SNC and CIRCA Research groups, Université de Montréal, Montréal, QC, Canada
- Department of Neurosciences, Faculty of Medicine, SNC and CIRCA Research groups, Université de Montréal, Montréal, QC, Canada
| | - Christopher P Ford
- Department of Pharmacology, University of Colorado School of Medicine, Aurora, CO, USA
| | - Yaroslav Sych
- Institute of Cellular and Integrative Neuroscience, University of Strasbourg, Strasbourg, France
| | - Tommaso Patriarchi
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland.
- Neuroscience Center Zurich, University and ETH Zürich, Zürich, Switzerland.
| |
Collapse
|
46
|
Elliott BL, Mohyee RA, Ballard IC, Olson IR, Ellman LM, Murty VP. In vivo structural connectivity of the reward system along the hippocampal long axis. Hippocampus 2024; 34:327-341. [PMID: 38700259 DOI: 10.1002/hipo.23608] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Revised: 03/11/2024] [Accepted: 04/18/2024] [Indexed: 05/05/2024]
Abstract
Recent work has identified a critical role for the hippocampus in reward-sensitive behaviors, including motivated memory, reinforcement learning, and decision-making. Animal histology and human functional neuroimaging have shown that brain regions involved in reward processing and motivation are more interconnected with the ventral/anterior hippocampus. However, direct evidence examining gradients of structural connectivity between reward regions and the hippocampus in humans is lacking. The present study used diffusion MRI (dMRI) and probabilistic tractography to quantify the structural connectivity of the hippocampus with key reward processing regions in vivo. Using a large sample of subjects (N = 628) from the human connectome dMRI data release, we found that connectivity profiles with the hippocampus varied widely between different regions of the reward circuit. While the dopaminergic midbrain (ventral tegmental area) showed stronger connectivity with the anterior versus posterior hippocampus, the ventromedial prefrontal cortex showed stronger connectivity with the posterior hippocampus. The limbic (ventral) striatum demonstrated a more homogeneous connectivity profile along the hippocampal long axis. This is the first study to generate a probabilistic atlas of the hippocampal structural connectivity with reward-related networks, which is essential to investigating how these circuits contribute to normative adaptive behavior and maladaptive behaviors in psychiatric illness. These findings describe nuanced structural connectivity that sets the foundation to better understand how the hippocampus influences reward-guided behavior in humans.
Collapse
Affiliation(s)
- Blake L Elliott
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania, USA
| | - Raana A Mohyee
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania, USA
| | - Ian C Ballard
- Department of Psychology, University of California, Riverside, California, USA
| | - Ingrid R Olson
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania, USA
| | - Lauren M Ellman
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania, USA
| | - Vishnu P Murty
- Department of Psychology and Neuroscience, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
47
|
Millard SJ, Hoang IB, Sherwood S, Taira M, Reyes V, Greer Z, O'Connor SL, Wassum KM, James MH, Barker DJ, Sharpe MJ. Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency. Nat Neurosci 2024; 27:1253-1259. [PMID: 38741021 PMCID: PMC11239488 DOI: 10.1038/s41593-024-01643-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 04/05/2024] [Indexed: 05/16/2024]
Abstract
Dopamine neurons in the ventral tegmental area support intracranial self-stimulation (ICSS), yet the cognitive representations underlying this phenomenon remain unclear. Here, 20-Hz stimulation of dopamine neurons, which approximates a physiologically relevant prediction error, was not sufficient to support ICSS beyond a continuously reinforced schedule and did not endow cues with a general or specific value. However, 50-Hz stimulation of dopamine neurons was sufficient to drive robust ICSS and was represented as a specific reward to motivate behavior. The frequency dependence of this effect is due to the rate (not the number) of action potentials produced by dopamine neurons, which differently modulates dopamine release downstream.
Collapse
Affiliation(s)
- Samuel J Millard
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Ivy B Hoang
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Savannah Sherwood
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Masakazu Taira
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
- Department of Psychology, University of Sydney, Camperdown, New South Wales, Australia
| | - Vanessa Reyes
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Zara Greer
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Shayna L O'Connor
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - Kate M Wassum
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Morgan H James
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - David J Barker
- Brain Health Institute, Rutgers Biomedical Health Sciences, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, USA
| | - Melissa J Sharpe
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA.
- Department of Psychology, University of Sydney, Camperdown, New South Wales, Australia.
| |
Collapse
|
48
|
Straus LD, ten Brink M, Sikka P, Srivastava R, Gross JJ, Colvonen PJ. The role of objective sleep in implicit and explicit affect regulation: A comprehensive review. Neurobiol Stress 2024; 31:100655. [PMID: 39036771 PMCID: PMC11260030 DOI: 10.1016/j.ynstr.2024.100655] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 06/10/2024] [Accepted: 06/11/2024] [Indexed: 07/23/2024] Open
Abstract
Impairments in sleep and affect regulation are evident across a wide range of mental disorders. Understanding the sleep factors that relate to affect regulatory difficulties will inform mechanistic understanding and aid in treatment. Despite rising interest, some research challenges in this area include integrating across different clinical and non-clinical literatures investigating the role of sleep architecture (measured with polysomnography) and experimentally manipulated sleep, as well as integrating more explicit versus implicit affect regulation processes. In this comprehensive review, we use a unifying framework to examine sleep's relationship with implicit-automatic regulation and explicit-controlled regulation, both of which are relevant to mental health (e.g., PTSD and depression). Many studies of implicit-automatic regulation (e.g., fear extinction and safety learning) demonstrate the importance of sleep, and REM sleep specifically. Studies of explicit-controlled regulation (e.g., cognitive reappraisal and expressive suppression) are less consistent in their findings, with results differing depending on the type of affect regulation and/or way that sleep was measured or manipulated. There is a clear relationship between objective sleep and affect regulation processes. However, there is a need for 1) more studies focusing on sleep and explicit-controlled affect regulation; 2) replication with the same types of regulation strategies; 3) more studies experimentally manipulating sleep to examine its impact on affect regulation and vice versa in order to infer cause and effect; and 4) more studies looking at sleep's impact on next-day affect regulation (not just overnight change in affect reactivity).
Collapse
Affiliation(s)
- Laura D. Straus
- San Francisco VA Health Care System, San Francisco, CA, USA
- Department of Psychiatry, University of California, San Francisco, CA, USA
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Maia ten Brink
- Department of Psychology, Stanford University, Stanford, CA, USA
- Center for Behavioral Cardiovascular Health, Columbia University Irving Medical Center, New York, NY, USA
| | - Pilleriin Sikka
- Department of Psychology, Stanford University, Stanford, CA, USA
- Department of Psychology and Speech-Language Pathology, University of Turku, Finland
- Department of Cognitive Neuroscience and Philosophy, University of Skövde, Sweden
| | | | - James J. Gross
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Peter J. Colvonen
- San Diego VA Health Care System, San Diego, CA, USA
- Department of Psychiatry, University of California, San Diego, CA, USA
| |
Collapse
|
49
|
Lee H, Kim HF, Hikosaka O. Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease. Neurosci Biobehav Rev 2024; 162:105719. [PMID: 38759470 PMCID: PMC11167649 DOI: 10.1016/j.neubiorev.2024.105719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 04/26/2024] [Accepted: 05/09/2024] [Indexed: 05/19/2024]
Abstract
To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.
Collapse
Affiliation(s)
- Hyunchan Lee
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892-4435, USA.
| | - Hyoung F Kim
- School of Biological Sciences, College of Natural Sciences, Seoul National University (SNU), Seoul 08826, Republic of Korea
| | - Okihide Hikosaka
- Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892-4435, USA
| |
Collapse
|
50
|
Wardle MC, Webber HE, Yoon JH, Heads AM, Stotts AL, Lane SD, Schmitz JM. Behavioral therapies targeting reward mechanisms in substance use disorders. Pharmacol Biochem Behav 2024; 240:173787. [PMID: 38705285 DOI: 10.1016/j.pbb.2024.173787] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 04/04/2024] [Accepted: 05/02/2024] [Indexed: 05/07/2024]
Abstract
Behavioral therapies are considered best practices in the treatment of substance use disorders (SUD) and used as first-line approaches for SUDs without FDA-approved pharmacotherapies. Decades of research on the neuroscience of drug reward and addiction have informed the development of current leading behavioral therapies that, while differing in focus and technique, have in common the overarching goal of shifting reward responding away from drug and toward natural non-drug rewards. This review begins by describing key neurobiological processes of reward in addiction, followed by a description of how various behavioral therapies address specific reward processes. Based on this review, a conceptual 'map' is crafted to pinpoint gaps and areas of overlap, serving as a guide for selecting and integrating behavioral therapies.
Collapse
Affiliation(s)
- Margaret C Wardle
- Department of Psychology, University of Illinois at Chicago, Chicago, IL, United States of America
| | - Heather E Webber
- Faillace Department of Psychiatry and Behavioral Sciences, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, United States of America
| | - Jin H Yoon
- Faillace Department of Psychiatry and Behavioral Sciences, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, United States of America
| | - Angela M Heads
- Faillace Department of Psychiatry and Behavioral Sciences, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, United States of America
| | - Angela L Stotts
- Department of Family and Community Medicine, McGovern Medical School, University of Texas Health Science Center at Houston, United States of America
| | - Scott D Lane
- Faillace Department of Psychiatry and Behavioral Sciences, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, United States of America
| | - Joy M Schmitz
- Faillace Department of Psychiatry and Behavioral Sciences, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX, United States of America.
| |
Collapse
|