1
|
Mintz Hemed N, Hwang FJ, Zhao ET, Ding JB, Melosh NA. Multiplexed neurochemical sensing with sub-nM sensitivity across 2.25 mm 2 area. Biosens Bioelectron 2024; 261:116474. [PMID: 38870827 DOI: 10.1016/j.bios.2024.116474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2024] [Revised: 05/20/2024] [Accepted: 06/05/2024] [Indexed: 06/15/2024]
Abstract
Multichannel arrays capable of real-time sensing of neuromodulators in the brain are crucial for gaining insights into new aspects of neural communication. However, measuring neurochemicals, such as dopamine, at low concentrations over large areas has proven challenging. In this research, we demonstrate a novel approach that leverages the scalability and processing power offered by microelectrode array devices integrated with a functionalized, high-density microwire bundle, enabling electrochemical sensing at an unprecedented scale and spatial resolution. The sensors demonstrate outstanding selective molecular recognition by incorporating a selective polymeric membrane. By combining cutting-edge commercial multiplexing, digitization, and data acquisition hardware with a bio-compatible and highly sensitive neurochemical interface array, we establish a powerful platform for neurochemical analysis. This multichannel array has been successfully utilized in vitro and ex vivo systems. Notably, our results show a sensing area of 2.25 mm2 with an impressive detection limit of 820 pM for dopamine. This new approach paves the way for investigating complex neurochemical processes and holds promise for advancing our understanding of brain function and neurological disorders.
Collapse
Affiliation(s)
- Nofar Mintz Hemed
- Department of Materials Science and Engineering, Stanford University, Stanford, CA, USA
| | - Fuu-Jiun Hwang
- Department of Neurosurgery, Stanford University, Stanford, CA, 94305, USA
| | - Eric T Zhao
- Department of Chemical Engineering, Stanford University, Stanford, CA, 94305, USA
| | - Jun B Ding
- Department of Neurosurgery, Stanford University, Stanford, CA, 94305, USA; Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA, 94305, USA
| | - Nicholas A Melosh
- Department of Materials Science and Engineering, Stanford University, Stanford, CA, USA.
| |
Collapse
|
2
|
Gershman SJ, Assad JA, Datta SR, Linderman SW, Sabatini BL, Uchida N, Wilbrecht L. Explaining dopamine through prediction errors and beyond. Nat Neurosci 2024; 27:1645-1655. [PMID: 39054370 DOI: 10.1038/s41593-024-01705-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 06/13/2024] [Indexed: 07/27/2024]
Abstract
The most influential account of phasic dopamine holds that it reports reward prediction errors (RPEs). The RPE-based interpretation of dopamine signaling is, in its original form, probably too simple and fails to explain all the properties of phasic dopamine observed in behaving animals. This Perspective helps to resolve some of the conflicting interpretations of dopamine that currently exist in the literature. We focus on the following three empirical challenges to the RPE theory of dopamine: why does dopamine (1) ramp up as animals approach rewards, (2) respond to sensory and motor features and (3) influence action selection? We argue that the prediction error concept, once it has been suitably modified and generalized based on an analysis of each computational problem, answers each challenge. Nonetheless, there are a number of additional empirical findings that appear to demand fundamentally different theoretical explanations beyond encoding RPE. Therefore, looking forward, we discuss the prospects for a unifying theory that respects the diversity of dopamine signaling and function as well as the complex circuitry that both underlies and responds to dopaminergic transmission.
Collapse
Affiliation(s)
- Samuel J Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA, USA.
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA.
| | - John A Assad
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Scott W Linderman
- Department of Statistics and Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
| | - Bernardo L Sabatini
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Linda Wilbrecht
- Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| |
Collapse
|
3
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
4
|
Algermissen J, den Ouden HEM. Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:720-739. [PMID: 38773022 PMCID: PMC11233311 DOI: 10.3758/s13415-024-01191-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 04/22/2024] [Indexed: 05/23/2024]
Abstract
"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.
Collapse
Affiliation(s)
- Johannes Algermissen
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Hanneke E M den Ouden
- Donders Institute for Brain, Radboud University, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6526 GD, Nijmegen, The Netherlands.
| |
Collapse
|
5
|
Lee RS, Sagiv Y, Engelhard B, Witten IB, Daw ND. A feature-specific prediction error model explains dopaminergic heterogeneity. Nat Neurosci 2024; 27:1574-1586. [PMID: 38961229 DOI: 10.1038/s41593-024-01689-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 05/22/2024] [Indexed: 07/05/2024]
Abstract
The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons. Instead, we introduce a complementary 'feature-specific RPE' model, positing that individual ventral tegmental area DA neurons report RPEs for different aspects of an animal's moment-to-moment situation. Further, we show how our framework can be extended to explain patterns of heterogeneity in action responses reported among substantia nigra pars compacta DA neurons. This theory reconciles new observations of DA heterogeneity with classic ideas about RPE coding while also providing a new perspective of how the brain performs reinforcement learning in high-dimensional environments.
Collapse
Affiliation(s)
- Rachel S Lee
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Yotam Sagiv
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | | | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
6
|
Engel L, Wolff AR, Blake M, Collins VL, Sinha S, Saunders BT. Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning. Curr Biol 2024; 34:3086-3101.e4. [PMID: 38925117 PMCID: PMC11279555 DOI: 10.1016/j.cub.2024.05.069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Revised: 04/25/2024] [Accepted: 05/29/2024] [Indexed: 06/28/2024]
Abstract
Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.
Collapse
Affiliation(s)
- Liv Engel
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Amy R Wolff
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Madelyn Blake
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Val L Collins
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA
| | - Sonal Sinha
- Krieger School of Arts & Sciences, Johns Hopkins University, 3400 N. Charles St, Baltimore, MD 21218, USA
| | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA; Medical Discovery Team on Addiction, University of Minnesota, 2001 6th St SE, Minneapolis, MN 55455, USA.
| |
Collapse
|
7
|
Hitchcock PF, Frank MJ. The challenge of learning adaptive mental behavior. JOURNAL OF PSYCHOPATHOLOGY AND CLINICAL SCIENCE 2024; 133:413-426. [PMID: 38815082 PMCID: PMC11229419 DOI: 10.1037/abn0000924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2024]
Abstract
Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Collapse
Affiliation(s)
- Peter F. Hitchcock
- Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI
- Department of Psychology, Emory University, Atlanta GA
| | - Michael J. Frank
- Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI
- Carney Institute for Brain Science, Brown University, Providence, RI
| |
Collapse
|
8
|
Xie L, Zhao J, Li Y, Bai J. PET brain imaging in neurological disorders. Phys Life Rev 2024; 49:100-111. [PMID: 38574584 DOI: 10.1016/j.plrev.2024.03.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Accepted: 03/20/2024] [Indexed: 04/06/2024]
Abstract
Brain disorders are a series of conditions with damage or loss of neurons, such as Parkinson's disease (PD), Alzheimer's disease (AD), or drug dependence. These individuals have gradual deterioration of cognitive, motor, and other central nervous system functions affected. This degenerative trajectory is intricately associated with dysregulations in neurotransmitter systems. Positron Emission Tomography (PET) imaging, employing radiopharmaceuticals and molecular imaging techniques, emerges as a crucial tool for detecting brain biomarkers. It offers invaluable insights for early diagnosis and distinguishing brain disorders. This article comprehensively reviews the application and progress of conventional and novel PET imaging agents in diagnosing brain disorders. Furthermore, it conducts a thorough analysis on merits and limitations. The article also provides a forward-looking perspective in the future development directions of PET imaging agents for diagnosing brain disorders and proposes potential innovative strategies. It aims to furnish clinicians and researchers with an all-encompassing overview of the latest advancements and forthcoming trends in the utilization of PET imaging for diagnosing brain disorders.
Collapse
Affiliation(s)
- Lijun Xie
- Faculty of Life science and Technology, Kunming University of Science and Technology, Kunming 650500, PR China; Laboratory of Molecular Neurobiology, Medical school, Kunming University of Science and Technology, Kunming 650500, PR China; Department of Nuclear Medicine, First Affiliated Hospital of Kunming Medical University, Kunming 650032, PR China
| | - Jihua Zhao
- Department of Nuclear Medicine, First Affiliated Hospital of Kunming Medical University, Kunming 650032, PR China
| | - Ye Li
- Laboratory of Molecular Neurobiology, Medical school, Kunming University of Science and Technology, Kunming 650500, PR China.
| | - Jie Bai
- Laboratory of Molecular Neurobiology, Medical school, Kunming University of Science and Technology, Kunming 650500, PR China.
| |
Collapse
|
9
|
Song MR, Lee SW. Rethinking dopamine-guided action sequence learning. Eur J Neurosci 2024; 60:3447-3465. [PMID: 38798086 DOI: 10.1111/ejn.16426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 04/21/2024] [Accepted: 05/08/2024] [Indexed: 05/29/2024]
Abstract
As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.
Collapse
Affiliation(s)
- Minryung R Song
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
| | - Sang Wan Lee
- Department of Brain and Cognitive Sciences, KAIST, Daejeon, South Korea
- Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea
- KI for Health Science and Technology, KAIST, Daejeon, South Korea
- Center for Neuroscience-inspired AI, KAIST, Daejeon, South Korea
| |
Collapse
|
10
|
Amjad U, Choi J, Gibson DJ, Murray R, Graybiel AM, Schwerdt HN. Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates. eNeuro 2024; 11:ENEURO.0001-24.2024. [PMID: 38918051 PMCID: PMC11232371 DOI: 10.1523/eneuro.0001-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 06/03/2024] [Accepted: 06/05/2024] [Indexed: 06/27/2024] Open
Abstract
Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. We developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) in monkeys. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys measurement sensors which may lead to misclassification of these signals as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. We identified and discriminated cell type-specific units in the monkey striatum that were shown to correlate to specific behavioral task parameters related to reward size and eye movement direction. Synchronously recorded spike and dopamine signals displayed contrasting relations to the task variables, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.
Collapse
Affiliation(s)
- Usamma Amjad
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213
| | - Jiwon Choi
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, Maryland 20815
| | - Daniel J Gibson
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
| | - Raymond Murray
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213
| | - Ann M Graybiel
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
| | - Helen N Schwerdt
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15213
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, Maryland 20815
| |
Collapse
|
11
|
Foster M, Scheinost D. Brain states as wave-like motifs. Trends Cogn Sci 2024; 28:492-503. [PMID: 38582654 DOI: 10.1016/j.tics.2024.03.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2023] [Revised: 02/29/2024] [Accepted: 03/11/2024] [Indexed: 04/08/2024]
Abstract
There is ample evidence of wave-like activity in the brain at multiple scales and levels. This emerging literature supports the broader adoption of a wave perspective of brain activity. Specifically, a brain state can be described as a set of recurring, sequential patterns of propagating brain activity, namely a wave. We examine a collective body of experimental work investigating wave-like properties. Based on these works, we consider brain states as waves using a scale-agnostic framework across time and space. Emphasis is placed on the sequentiality and periodicity associated with brain activity. We conclude by discussing the implications, prospects, and experimental opportunities of this framework.
Collapse
Affiliation(s)
- Maya Foster
- Department of Biomedical Engineering, Yale University, New Haven, CT, USA.
| | - Dustin Scheinost
- Department of Biomedical Engineering, Yale University, New Haven, CT, USA; Department of Radiology and Biomedical Engineering, Yale School of Medicine, New Haven, CT, USA
| |
Collapse
|
12
|
Mohan UR, Zhang H, Ermentrout B, Jacobs J. The direction of theta and alpha travelling waves modulates human memory processing. Nat Hum Behav 2024; 8:1124-1135. [PMID: 38459263 DOI: 10.1038/s41562-024-01838-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Accepted: 01/24/2024] [Indexed: 03/10/2024]
Abstract
To support a range of behaviours, the brain must flexibly coordinate neural activity across widespread brain regions. One potential mechanism for this coordination is a travelling wave, in which a neural oscillation propagates across the brain while organizing the order and timing of activity across regions. Although travelling waves are present across the brain in various species, their potential functional relevance has remained unknown. Here, using rare direct human brain recordings, we demonstrate a distinct functional role for travelling waves of theta- and alpha-band (2-13 Hz) oscillations in the cortex. Travelling waves propagate in different directions during separate cognitive processes. In episodic memory, travelling waves tended to propagate in a posterior-to-anterior direction during successful memory encoding and in an anterior-to-posterior direction during recall. Because travelling waves of oscillations correspond to local neuronal spiking, these patterns indicate that rhythmic pulses of activity move across the brain in different directions for separate behaviours. More broadly, our results suggest a fundamental role for travelling waves and oscillations in dynamically coordinating neural connectivity, by flexibly organizing the timing and directionality of network interactions across the cortex to support cognition and behaviour.
Collapse
Affiliation(s)
- Uma R Mohan
- Surgical Neurology Branch, NINDS, National Institutes of Health, Bethesda, MD, USA
| | | | - Bard Ermentrout
- Department of Mathematics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Joshua Jacobs
- Department of Biomedical Engineering, Columbia University, New York City, NY, USA.
- Department of Neurological Surgery, Columbia University, New York City, NY, USA.
| |
Collapse
|
13
|
Hart G, Burton TJ, Balleine BW. What Role Does Striatal Dopamine Play in Goal-directed Action? Neuroscience 2024; 546:20-32. [PMID: 38521480 DOI: 10.1016/j.neuroscience.2024.03.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 02/15/2024] [Accepted: 03/18/2024] [Indexed: 03/25/2024]
Abstract
Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.
Collapse
Affiliation(s)
- Genevra Hart
- Decision Neuroscience Lab, UNSW Sydney, Australia
| | | | | |
Collapse
|
14
|
Jang HJ, Ward RM, Golden CEM, Constantinople CM. Acetylcholine demixes heterogeneous dopamine signals for learning and moving. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.03.592444. [PMID: 38746300 PMCID: PMC11092744 DOI: 10.1101/2024.05.03.592444] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
Midbrain dopamine neurons promote reinforcement learning and movement vigor. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Here we characterized dopamine and acetylcholine release in the dorsomedial striatum (DMS) of rats performing a decision-making task. We found that dopamine acted as a reward prediction error (RPE), modulating behavior and DMS spiking on subsequent trials when coincident with pauses in cholinergic release. In contrast, at task events that elicited coincident bursts of acetylcholine and dopamine, dopamine preceded contralateral movements and predicted movement vigor without inducing plastic changes in DMS firing rates. Our findings provide a circuit-level mechanism by which cholinergic modulation allows the same dopamine signals to be used for either movement or learning depending on instantaneous behavioral context.
Collapse
|
15
|
Glaeser-Khan S, Savalia NK, Cressy J, Feng J, Li Y, Kwan AC, Kaye AP. Spatiotemporal Organization of Prefrontal Norepinephrine Influences Neuronal Activity. eNeuro 2024; 11:ENEURO.0252-23.2024. [PMID: 38702188 PMCID: PMC11134306 DOI: 10.1523/eneuro.0252-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 01/08/2024] [Accepted: 01/19/2024] [Indexed: 05/06/2024] Open
Abstract
Norepinephrine (NE), a neuromodulator released by locus ceruleus (LC) neurons throughout the cortex, influences arousal and learning through extrasynaptic vesicle exocytosis. While NE within cortical regions has been viewed as a homogenous field, recent studies have demonstrated heterogeneous axonal dynamics and advances in GPCR-based fluorescent sensors permit direct observation of the local dynamics of NE at cellular scale. To investigate how the spatiotemporal dynamics of NE release in the prefrontal cortex (PFC) affect neuronal firing, we employed in vivo two-photon imaging of layer 2/3 of the PFC in order to observe fine-scale neuronal calcium and NE dynamics concurrently. In this proof of principle study, we found that local and global NE fields can decouple from one another, providing a substrate for local NE spatiotemporal activity patterns. Optic flow analysis revealed putative release and reuptake events which can occur at the same location, albeit at different times, indicating the potential to create a heterogeneous NE field. Utilizing generalized linear models, we demonstrated that cellular Ca2+ fluctuations are influenced by both the local and global NE field. However, during periods of local/global NE field decoupling, the local field drives cell firing dynamics rather than the global field. These findings underscore the significance of localized, phasic NE fluctuations for structuring cell firing, which may provide local neuromodulatory control of cortical activity.
Collapse
Affiliation(s)
| | - Neil K Savalia
- Interdepartmental Neuroscience Program, Yale University, New Haven, Connecticut 06510
- Medical Scientist Training Program, Yale University School of Medicine, New Haven, Connecticut 06511
- Meinig School of Biomedical Engineering, Cornell University, Ithaca, New York 14853
| | - Jianna Cressy
- Department of Psychiatry, Yale University, New Haven, Connecticut 06511
- Clinical Neuroscience Division, VA National Center for PTSD, West Haven, Connecticut 06515
| | - Jiesi Feng
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China
- Chinese Institute for Brain Research, Beijing 102206, China
| | - Alex C Kwan
- Meinig School of Biomedical Engineering, Cornell University, Ithaca, New York 14853
| | - Alfred P Kaye
- Department of Psychiatry, Yale University, New Haven, Connecticut 06511
- Clinical Neuroscience Division, VA National Center for PTSD, West Haven, Connecticut 06515
| |
Collapse
|
16
|
Floeder JR, Jeong H, Mohebi A, Namboodiri VMK. Mesolimbic dopamine ramps reflect environmental timescales. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.03.27.587103. [PMID: 38659749 PMCID: PMC11042231 DOI: 10.1101/2024.03.27.587103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
Mesolimbic dopamine activity occasionally exhibits ramping dynamics, reigniting debate on theories of dopamine signaling. This debate is ongoing partly because the experimental conditions under which dopamine ramps emerge remain poorly understood. Here, we show that during Pavlovian and instrumental conditioning, mesolimbic dopamine ramps are only observed when the inter-trial interval is short relative to the trial period. These results constrain theories of dopamine signaling and identify a critical variable determining the emergence of dopamine ramps.
Collapse
Affiliation(s)
- Joseph R Floeder
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
| | - Huijeong Jeong
- Department of Neurology, University of California, San Francisco, CA, USA
| | - Ali Mohebi
- Department of Neurology, University of California, San Francisco, CA, USA
| | - Vijay Mohan K Namboodiri
- Neuroscience Graduate Program, University of California, San Francisco, CA, USA
- Department of Neurology, University of California, San Francisco, CA, USA
- Weill Institute for Neurosciences, Kavli Institute for Fundamental Neuroscience, Center for Integrative Neuroscience, University of California, San Francisco, CA, USA
| |
Collapse
|
17
|
Avvisati R, Kaufmann AK, Young CJ, Portlock GE, Cancemi S, Costa RP, Magill PJ, Dodson PD. Distributional coding of associative learning in discrete populations of midbrain dopamine neurons. Cell Rep 2024; 43:114080. [PMID: 38581677 PMCID: PMC7616095 DOI: 10.1016/j.celrep.2024.114080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 02/12/2024] [Accepted: 03/24/2024] [Indexed: 04/08/2024] Open
Abstract
Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.
Collapse
Affiliation(s)
- Riccardo Avvisati
- School of Physiology, Pharmacology, and Neuroscience, University of Bristol, Bristol BS8 1TD, UK; Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3TH, UK
| | - Anna-Kristin Kaufmann
- Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3TH, UK
| | - Callum J Young
- School of Physiology, Pharmacology, and Neuroscience, University of Bristol, Bristol BS8 1TD, UK; Computational Neuroscience Unit, Department of Computer Science, SCEEM, Faculty of Engineering, University of Bristol, Bristol BS8 1UB, UK
| | - Gabriella E Portlock
- School of Physiology, Pharmacology, and Neuroscience, University of Bristol, Bristol BS8 1TD, UK
| | - Sophie Cancemi
- School of Physiology, Pharmacology, and Neuroscience, University of Bristol, Bristol BS8 1TD, UK
| | - Rui Ponte Costa
- Computational Neuroscience Unit, Department of Computer Science, SCEEM, Faculty of Engineering, University of Bristol, Bristol BS8 1UB, UK
| | - Peter J Magill
- Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3TH, UK
| | - Paul D Dodson
- School of Physiology, Pharmacology, and Neuroscience, University of Bristol, Bristol BS8 1TD, UK; Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX1 3TH, UK.
| |
Collapse
|
18
|
Celinskis D, Black CJ, Murphy J, Barrios-Anderson A, Friedman NG, Shaner NC, Saab CY, Gomez-Ramirez M, Borton DA, Moore CI. Toward a brighter constellation: multiorgan neuroimaging of neural and vascular dynamics in the spinal cord and brain. NEUROPHOTONICS 2024; 11:024209. [PMID: 38725801 PMCID: PMC11079446 DOI: 10.1117/1.nph.11.2.024209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 03/18/2024] [Accepted: 03/19/2024] [Indexed: 05/12/2024]
Abstract
Significance Pain comprises a complex interaction between motor action and somatosensation that is dependent on dynamic interactions between the brain and spinal cord. This makes understanding pain particularly challenging as it involves rich interactions between many circuits (e.g., neural and vascular) and signaling cascades throughout the body. As such, experimentation on a single region may lead to an incomplete and potentially incorrect understanding of crucial underlying mechanisms. Aim We aimed to develop and validate tools to enable detailed and extended observation of neural and vascular activity in the brain and spinal cord. The first key set of innovations was targeted to developing novel imaging hardware that addresses the many challenges of multisite imaging. The second key set of innovations was targeted to enabling bioluminescent (BL) imaging, as this approach can address limitations of fluorescent microscopy including photobleaching, phototoxicity, and decreased resolution due to scattering of excitation signals. Approach We designed 3D-printed brain and spinal cord implants to enable effective surgical implantations and optical access with wearable miniscopes or an open window (e.g., for one- or two-photon microscopy or optogenetic stimulation). We also tested the viability for BL imaging and developed a novel modified miniscope optimized for these signals (BLmini). Results We describe "universal" implants for acute and chronic simultaneous brain-spinal cord imaging and optical stimulation. We further describe successful imaging of BL signals in both foci and a new miniscope, the "BLmini," which has reduced weight, cost, and form-factor relative to standard wearable miniscopes. Conclusions The combination of 3D-printed implants, advanced imaging tools, and bioluminescence imaging techniques offers a coalition of methods for understanding spinal cord-brain interactions. Our work has the potential for use in future research into neuropathic pain and other sensory disorders and motor behavior.
Collapse
Affiliation(s)
- Dmitrijs Celinskis
- Carney Institute for Brain Science, Providence, Rhode Island, United States
| | | | - Jeremy Murphy
- Carney Institute for Brain Science, Providence, Rhode Island, United States
| | | | - Nina G. Friedman
- Carney Institute for Brain Science, Providence, Rhode Island, United States
| | - Nathan C. Shaner
- University of California San Diego, School of Medicine, La Jolla, California, United States
| | - Carl Y. Saab
- Cleveland Clinic Lerner Research Institute, Neurological Institute, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Manuel Gomez-Ramirez
- University of Rochester, School of Arts and Sciences, Rochester, New York, United States
| | - David A. Borton
- Carney Institute for Brain Science, Providence, Rhode Island, United States
- Brown University, School of Engineering, Providence, Rhode Island, United States
- Center for Neurorestoration and Neurotechnology, Providence VA Medical Center, Providence, Rhode Island, United States
| | | |
Collapse
|
19
|
Klein E, Marsh S, Becker J, Andermann M, Lehtinen M, Moore CI. BioLuminescent OptoGenetics in the choroid plexus: integrated opto- and chemogenetic control in vivo. NEUROPHOTONICS 2024; 11:024210. [PMID: 38948888 PMCID: PMC11213259 DOI: 10.1117/1.nph.11.2.024210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 05/30/2024] [Accepted: 05/30/2024] [Indexed: 07/02/2024]
Abstract
Significance The choroid plexus (ChP) epithelial network displays diverse dynamics, including propagating calcium waves and individuated fluctuations in single cells. These rapid events underscore the possibility that ChP dynamics may reflect behaviorally relevant and clinically important changes in information processing and signaling. Optogenetic and chemogenetic tools provide spatiotemporally precise and sustained approaches for testing such dynamics in vivo. Here, we describe the feasibility of a novel combined opto- and chemogenetic tool, BioLuminescent-OptoGenetics (BL-OG), for the ChP in vivo. In the "LuMinOpsin" (LMO) BL-OG strategy, a luciferase is tethered to an adjacent optogenetic element. This molecule allows chemogenetic activation when the opsin is driven by light produced through luciferase binding a small molecule (luciferin) or by conventional optogenetic light sources and BL-OG report of activation through light production. Aim To test the viability of BL-OG/LMO for ChP control. Approach Using transgenic and Cre-directed targeting to the ChP, we expressed LMO3 (a Gaussia luciferase-VChR1 fusion), a highly effective construct in neural systems. In mice expressing LMO3 in ChP, we directly imaged BL light production following multiple routes of coelenterazine (CTZ: luciferin) administration using an implanted cannula system. We also used home-cage videography with Deep LabCut analysis to test for any impact of repeated CTZ administration on basic health and behavioral indices. Results Multiple routes of CTZ administration drove BL photon production, including intracerebroventricular, intravenous, and intraperitoneal injection. Intravenous administration resulted in fast "flash" kinetics that diminished in seconds to minutes, and intraperitoneal administration resulted in slow rising activity that sustained hours. Mice showed no consistent impact of 1 week of intraperitoneal CTZ administration on weight, drinking, motor behavior, or sleep/wake cycles. Conclusions BL-OG/LMO provides unique advantages for testing the role of ChP dynamics in biological processes.
Collapse
Affiliation(s)
- Eric Klein
- Brown University, Providence, Rhode Island, United States
| | - Sophie Marsh
- Brown University, Providence, Rhode Island, United States
| | - Jordan Becker
- Brown University, Providence, Rhode Island, United States
| | - Mark Andermann
- Beth Israel Deaconess Medical Center Harvard, Boston, Massachusetts, United States
| | - Maria Lehtinen
- Brown University, Providence, Rhode Island, United States
- Boston Children’s Hospital, Boston, Massachusetts, United States
| | | |
Collapse
|
20
|
Zhuo Y, Luo B, Yi X, Dong H, Miao X, Wan J, Williams JT, Campbell MG, Cai R, Qian T, Li F, Weber SJ, Wang L, Li B, Wei Y, Li G, Wang H, Zheng Y, Zhao Y, Wolf ME, Zhu Y, Watabe-Uchida M, Li Y. Improved green and red GRAB sensors for monitoring dopaminergic activity in vivo. Nat Methods 2024; 21:680-691. [PMID: 38036855 PMCID: PMC11009088 DOI: 10.1038/s41592-023-02100-w] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 10/23/2023] [Indexed: 12/02/2023]
Abstract
Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a large network of dopaminergic projections. To dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent G-protein-coupled receptor activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity and signal-to-noise ratio with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.
Collapse
Affiliation(s)
- Yizhou Zhuo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Bin Luo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking-Tsinghua Center for Life Sciences, New Cornerstone Science Laboratory, Academy for Advanced Interdisciplinary Studies, Beijing, China
| | - Xinyang Yi
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Hui Dong
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking-Tsinghua Center for Life Sciences, New Cornerstone Science Laboratory, Academy for Advanced Interdisciplinary Studies, Beijing, China
| | - Xiaolei Miao
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Department of Anesthesiology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Jinxia Wan
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking-Tsinghua Center for Life Sciences, New Cornerstone Science Laboratory, Academy for Advanced Interdisciplinary Studies, Beijing, China
| | - John T Williams
- Vollum Institute, Oregon Health & Science University, Portland, OR, USA
| | - Malcolm G Campbell
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Ruyi Cai
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Tongrui Qian
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Fengling Li
- Shenzhen Key Laboratory of Drug Addiction, Shenzhen Neher Neural Plasticity Laboratory, the Brain Cognition and Brain Disease Institute, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Sophia J Weber
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR, USA
| | - Lei Wang
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking University-Tsinghua University-National Institute of Biological Sciences Joint Graduate Program, Peking University, Beijing, China
| | - Bozhi Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Department of Neurology, The First Medical Center, Chinese PLA General Hospital, Beijing, China
| | - Yu Wei
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Guochuan Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Huan Wang
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Yu Zheng
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Yulin Zhao
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
| | - Marina E Wolf
- Department of Behavioral Neuroscience, Oregon Health & Science University, Portland, OR, USA
| | - Yingjie Zhu
- Shenzhen Key Laboratory of Drug Addiction, Shenzhen Neher Neural Plasticity Laboratory, the Brain Cognition and Brain Disease Institute, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Mitsuko Watabe-Uchida
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China.
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China.
- Peking-Tsinghua Center for Life Sciences, New Cornerstone Science Laboratory, Academy for Advanced Interdisciplinary Studies, Beijing, China.
- Chinese Institute for Brain Research, Beijing, China.
- Institute of Molecular Physiology, Shenzhen Bay Laboratory, Shenzhen, China.
- National Biomedical Imaging Center, Peking University, Beijing, China.
| |
Collapse
|
21
|
Engel L, Wolff AR, Blake M, Collins VL, Sinha S, Saunders BT. Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.07.01.547331. [PMID: 38585717 PMCID: PMC10996462 DOI: 10.1101/2023.07.01.547331] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.
Collapse
Affiliation(s)
- Liv Engel
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
- Current Address: Department of Psychology, University of Toronto
| | - Amy R Wolff
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | - Madelyn Blake
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | - Val L Collins
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| | | | - Benjamin T Saunders
- Department of Neuroscience, University of Minnesota
- Medical Discovery Team on Addiction, University of Minnesota
| |
Collapse
|
22
|
Hart G, Burton TJ, Nolan CR, Balleine BW. Striatal dopamine release tracks the relationship between actions and their consequences. Cell Rep 2024; 43:113828. [PMID: 38386550 DOI: 10.1016/j.celrep.2024.113828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 12/05/2023] [Accepted: 02/03/2024] [Indexed: 02/24/2024] Open
Abstract
The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.
Collapse
Affiliation(s)
- G Hart
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - T J Burton
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - C R Nolan
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia
| | - B W Balleine
- Decision Neuroscience Laboratory, School of Psychology, UNSW Sydney, Sydney, NSW, Australia.
| |
Collapse
|
23
|
Vu MAT, Brown EH, Wen MJ, Noggle CA, Zhang Z, Monk KJ, Bouabid S, Mroz L, Graham BM, Zhuo Y, Li Y, Otchy TM, Tian L, Davison IG, Boas DA, Howe MW. Targeted micro-fiber arrays for measuring and manipulating localized multi-scale neural dynamics over large, deep brain volumes during behavior. Neuron 2024; 112:909-923.e9. [PMID: 38242115 PMCID: PMC10957316 DOI: 10.1016/j.neuron.2023.12.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 11/11/2023] [Accepted: 12/15/2023] [Indexed: 01/21/2024]
Abstract
Neural population dynamics relevant to behavior vary over multiple spatial and temporal scales across three-dimensional volumes. Current optical approaches lack the spatial coverage and resolution necessary to measure and manipulate naturally occurring patterns of large-scale, distributed dynamics within and across deep brain regions such as the striatum. We designed a new micro-fiber array approach capable of chronically measuring and optogenetically manipulating local dynamics across over 100 targeted locations simultaneously in head-fixed and freely moving mice, enabling the investigation of cell-type- and neurotransmitter-specific signals over arbitrary 3D volumes at a spatial resolution and coverage previously inaccessible. We applied this method to resolve rapid dopamine release dynamics across the striatum, revealing distinct, modality-specific spatiotemporal patterns in response to salient sensory stimuli extending over millimeters of tissue. Targeted optogenetics enabled flexible control of neural signaling on multiple spatial scales, better matching endogenous signaling patterns, and the spatial localization of behavioral function across large circuits.
Collapse
Affiliation(s)
- Mai-Anh T Vu
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Eleanor H Brown
- Graduate Program for Neuroscience, Boston University, Boston, MA, USA
| | - Michelle J Wen
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Howard Hughes Medical Institute, Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Christian A Noggle
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Zicheng Zhang
- Department of Biology, Boston University, Boston, MA, USA
| | - Kevin J Monk
- Department of Biology, Boston University, Boston, MA, USA
| | - Safa Bouabid
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Lydia Mroz
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Northeastern University, Boston, MA, USA
| | - Benjamin M Graham
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Yizhou Zhuo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China; PKU-IDG/McGovern Institute for Brain Research, Beijing, China; Peking-Tsinghua Center for Life Sciences, Beijing, China
| | - Yulong Li
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA; State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China; PKU-IDG/McGovern Institute for Brain Research, Beijing, China; Peking-Tsinghua Center for Life Sciences, Beijing, China
| | | | - Lin Tian
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA; Max Planck Florida Institute of Neuroscience, Jupiter, FL, USA
| | - Ian G Davison
- Department of Biology, Boston University, Boston, MA, USA
| | - David A Boas
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Mark W Howe
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA.
| |
Collapse
|
24
|
Watanabe H, Dijkstra JM, Nagatsu T. Parkinson's Disease: Cells Succumbing to Lifelong Dopamine-Related Oxidative Stress and Other Bioenergetic Challenges. Int J Mol Sci 2024; 25:2009. [PMID: 38396687 PMCID: PMC10888576 DOI: 10.3390/ijms25042009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Revised: 01/31/2024] [Accepted: 02/02/2024] [Indexed: 02/25/2024] Open
Abstract
The core pathological event in Parkinson's disease (PD) is the specific dying of dopamine (DA) neurons of the substantia nigra pars compacta (SNc). The reasons why SNc DA neurons are especially vulnerable and why idiopathic PD has only been found in humans are still puzzling. The two main underlying factors of SNc DA neuron vulnerability appear related to high DA production, namely (i) the toxic effects of cytoplasmic DA metabolism and (ii) continuous cytosolic Ca2+ oscillations in the absence of the Ca2+-buffer protein calbindin. Both factors cause oxidative stress by producing highly reactive quinones and increasing intra-mitochondrial Ca2+ concentrations, respectively. High DA expression in human SNc DA neuron cell bodies is suggested by the abundant presence of the DA-derived pigment neuromelanin, which is not found in such abundance in other species and has been associated with toxicity at higher levels. The oxidative stress created by their DA production system, despite the fact that the SN does not use unusually high amounts of energy, explains why SNc DA neurons are sensitive to various genetic and environmental factors that create mitochondrial damage and thereby promote PD. Aging increases multiple risk factors for PD, and, to a large extent, PD is accelerated aging. To prevent PD neurodegeneration, possible approaches that are discussed here are (1) reducing cytoplasmic DA accumulation, (2) blocking cytoplasmic Ca2+ oscillations, and (3) providing bioenergetic support.
Collapse
Affiliation(s)
- Hirohisa Watanabe
- Department of Neurology, School of Medicine, Fujita Health University, Toyoake 470-1192, Aichi, Japan
| | - Johannes M. Dijkstra
- Center for Medical Science, Fujita Health University, Toyoake 470-1192, Aichi, Japan
| | - Toshiharu Nagatsu
- Center for Research Promotion and Support, Fujita Health University, Toyoake 470-1192, Aichi, Japan;
| |
Collapse
|
25
|
Costello H, Husain M, Roiser JP. Apathy and Motivation: Biological Basis and Drug Treatment. Annu Rev Pharmacol Toxicol 2024; 64:313-338. [PMID: 37585659 DOI: 10.1146/annurev-pharmtox-022423-014645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/18/2023]
Abstract
Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.
Collapse
Affiliation(s)
- Harry Costello
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom;
| | - Masud Husain
- Nuffield Department of Clinical Neurosciences and Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom;
| |
Collapse
|
26
|
Zimmerman CA, Pan-Vazquez A, Wu B, Keppler EF, Guthman EM, Fetcho RN, Bolkan SS, McMannon B, Lee J, Hoag AT, Lynch LA, Janarthanan SR, López Luna JF, Bondy AG, Falkner AL, Wang SSH, Witten IB. A neural mechanism for learning from delayed postingestive feedback. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.06.561214. [PMID: 37873112 PMCID: PMC10592633 DOI: 10.1101/2023.10.06.561214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Animals learn the value of foods based on their postingestive effects and thereby develop aversions to foods that are toxic1-6 and preferences to those that are nutritious7-14. However, it remains unclear how the brain is able to assign credit to flavors experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here, we reveal an unexpected role for postingestive reactivation of neural flavor representations in this temporal credit assignment process. To begin, we leverage the fact that mice learn to associate novel15-18, but not familiar, flavors with delayed gastric malaise signals to investigate how the brain represents flavors that support aversive postingestive learning. Surveying cellular resolution brainwide activation patterns reveals that a network of amygdala regions is unique in being preferentially activated by novel flavors across every stage of the learning process: the initial meal, delayed malaise, and memory retrieval. By combining high-density recordings in the amygdala with optogenetic stimulation of genetically defined hindbrain malaise cells, we find that postingestive malaise signals potently and specifically reactivate amygdalar novel flavor representations from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts strengthening of flavor responses upon memory retrieval, leading to stabilization of the population-level representation of the recently consumed flavor. In contrast, meals without postingestive consequences degrade neural flavor representations as flavors become familiar and safe. Thus, our findings demonstrate that interoceptive reactivation of amygdalar flavor representations provides a neural mechanism to resolve the temporal credit assignment problem inherent to postingestive learning.
Collapse
Affiliation(s)
| | | | - Bichan Wu
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Emma F Keppler
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Eartha Mae Guthman
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Robert N Fetcho
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Scott S Bolkan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Junuk Lee
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Austin T Hoag
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Laura A Lynch
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Juan F López Luna
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Adrian G Bondy
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Annegret L Falkner
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Samuel S-H Wang
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| |
Collapse
|
27
|
Grill F, Guitart-Masip M, Johansson J, Stiernman L, Axelsson J, Nyberg L, Rieckmann A. Dopamine release in human associative striatum during reversal learning. Nat Commun 2024; 15:59. [PMID: 38167691 PMCID: PMC10762220 DOI: 10.1038/s41467-023-44358-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Accepted: 12/11/2023] [Indexed: 01/05/2024] Open
Abstract
The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum. Individual differences in absolute reward prediction error and sensitivity to errors are associated with peak dopamine receptor occupancy. The fMRI response to perseverance errors at the onset of a reversal spatially overlap with the site of dopamine release. Trial-by-trial fMRI correlates of absolute prediction errors show a response in striatum and association cortices, closely overlapping with the location of dopamine release, and separable from a valence signal in ventral striatum. The results converge to implicate striatal dopamine release in associative striatum as a central component of reversal learning, possibly signifying the need for increased cognitive control when new stimuli-responses should be learned.
Collapse
Affiliation(s)
- Filip Grill
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden.
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden.
| | - Marc Guitart-Masip
- Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
- Center for Psychiatry Research, Region Stockholm, Stockholm, Sweden
- Center for Cognitive and Computational Neuropsychiatry (CCNP), Karolinska Institutet, Stockholm, Sweden
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Jarkko Johansson
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
| | - Lars Stiernman
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden
| | - Jan Axelsson
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Diagnostics and Intervention, Radiation Physics, Umeå University, Umeå, Sweden
| | - Lars Nyberg
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden
| | - Anna Rieckmann
- Department of Diagnostics and Intervention, Diagnostic Radiology, Umeå University, Umeå, Sweden.
- Umeå Center for Functional Brain Imaging, Umeå University, Umeå, Sweden.
- Department of Medical and Translational Biology, Umeå University, Umeå, Sweden.
- Institute for Psychology, University of the Bundeswehr Munich, Neubiberg, Germany.
| |
Collapse
|
28
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
29
|
Celinskis D, Black CJ, Murphy J, Barrios-Anderson A, Friedman N, Shaner NC, Saab C, Gomez-Ramirez M, Lipscombe D, Borton DA, Moore CI. Towards a Brighter Constellation: Multi-Organ Neuroimaging of Neural and Vascular Dynamics in the Spinal Cord and Brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.25.573323. [PMID: 38234789 PMCID: PMC10793404 DOI: 10.1101/2023.12.25.573323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
Significance Pain is comprised of a complex interaction between motor action and somatosensation that is dependent on dynamic interactions between the brain and spinal cord. This makes understanding pain particularly challenging as it involves rich interactions between many circuits (e.g., neural and vascular) and signaling cascades throughout the body. As such, experimentation on a single region may lead to an incomplete and potentially incorrect understanding of crucial underlying mechanisms. Aim Here, we aimed to develop and validate new tools to enable detailed and extended observation of neural and vascular activity in the brain and spinal cord. The first key set of innovations were targeted to developing novel imaging hardware that addresses the many challenges of multi-site imaging. The second key set of innovations were targeted to enabling bioluminescent imaging, as this approach can address limitations of fluorescent microscopy including photobleaching, phototoxicity and decreased resolution due to scattering of excitation signals. Approach We designed 3D-printed brain and spinal cord implants to enable effective surgical implantations and optical access with wearable miniscopes or an open window (e.g., for one- or two-photon microscopy or optogenetic stimulation). We also tested the viability for bioluminescent imaging, and developed a novel modified miniscope optimized for these signals (BLmini). Results Here, we describe novel 'universal' implants for acute and chronic simultaneous brain-spinal cord imaging and optical stimulation. We further describe successful imaging of bioluminescent signals in both foci, and a new miniscope, the 'BLmini,' which has reduced weight, cost and form-factor relative to standard wearable miniscopes. Conclusions The combination of 3D printed implants, advanced imaging tools, and bioluminescence imaging techniques offers a new coalition of methods for understanding spinal cord-brain interactions. This work has the potential for use in future research into neuropathic pain and other sensory disorders and motor behavior.
Collapse
Affiliation(s)
| | | | - Jeremy Murphy
- Carney Institute for Brain Science, Providence, RI, USA
| | | | - Nina Friedman
- Carney Institute for Brain Science, Providence, RI, USA
| | - Nathan C. Shaner
- University of California San Diego School of Medicine, La Jolla, CA, USA
| | - Carl Saab
- Cleveland Clinic Lerner Research Institute, Department of Biomedical Engineering and Neurological Institute, Cleveland, OH, USA
| | | | | | - David A. Borton
- Carney Institute for Brain Science, Providence, RI, USA
- School of Engineering, Brown University, RI, USA
- Center for Neurorestoration and Neurotechnology, Providence VA Medical Center, RI, USA
| | | |
Collapse
|
30
|
Amjad U, Choi J, Gibson DJ, Murray R, Graybiel AM, Schwerdt HN. Synchronous Measurements of Extracellular Action Potentials and Neurochemical Activity with Carbon Fiber Electrodes in Nonhuman Primates. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.23.573130. [PMID: 38187624 PMCID: PMC10769335 DOI: 10.1101/2023.12.23.573130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2024]
Abstract
Measuring the dynamic relationship between neuromodulators, such as dopamine, and neuronal action potentials is imperative to understand how these fundamental modes of neural signaling interact to mediate behavior. Here, we developed methods to measure concurrently dopamine and extracellular action potentials (i.e., spikes) and applied these in a monkey performing a behavioral task. Standard fast-scan cyclic voltammetric (FSCV) electrochemical (EChem) and electrophysiological (EPhys) recording systems are combined and used to collect spike and dopamine signals, respectively, from an array of carbon fiber (CF) sensors implanted in the monkey striatum. FSCV requires the application of small voltages at the implanted sensors to measure redox currents generated from target molecules, such as dopamine. These applied voltages create artifacts at neighboring EPhys-measurement sensors, producing signals that may falsely be classified as physiological spikes. Therefore, simple automated temporal interpolation algorithms were designed to remove these artifacts and enable accurate spike extraction. We validated these methods using simulated artifacts and demonstrated an average spike recovery rate of 84.5%. This spike extraction was performed on data collected from concurrent EChem and EPhys recordings made in a task-performing monkey to discriminate cell-type specific striatal units. These identified units were shown to correlate to specific behavioral task parameters related to reward size and eye-movement direction. Synchronous measures of spike and dopamine signals displayed contrasting relations to the behavioral task parameters, as taken from our small set of representative data, suggesting a complex relationship between these two modes of neural signaling. Future application of our methods will help advance our understanding of the interactions between neuromodulator signaling and neuronal activity, to elucidate more detailed mechanisms of neural circuitry and plasticity mediating behaviors in health and in disease.
Collapse
Affiliation(s)
- Usamma Amjad
- Department of Bioengineering, University of Pittsburgh, USA
| | - Jiwon Choi
- Department of Bioengineering, University of Pittsburgh, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Daniel J Gibson
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Raymond Murray
- Department of Bioengineering, University of Pittsburgh, USA
| | - Ann M Graybiel
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Helen N Schwerdt
- Department of Bioengineering, University of Pittsburgh, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| |
Collapse
|
31
|
Xia M, Agca BN, Yoshida T, Choi J, Amjad U, Bose K, Keren N, Zukerman S, Cima MJ, Graybiel AM, Schwerdt HN. Scalable, flexible carbon fiber electrode thread arrays for three-dimensional probing of neurochemical activity in deep brain structures of rodents. Biosens Bioelectron 2023; 241:115625. [PMID: 37708685 PMCID: PMC10591823 DOI: 10.1016/j.bios.2023.115625] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 08/17/2023] [Accepted: 08/19/2023] [Indexed: 09/16/2023]
Abstract
We developed a flexible "electrode-thread" array for recording dopamine neurochemicals from a lateral distribution of subcortical targets (up to 16) transverse to the axis of insertion. Ultrathin (∼10 μm diameter) carbon fiber (CF) electrode-threads (CFETs) are clustered into a tight bundle to introduce them into the brain from a single-entry point. The individual CFETs splay laterally in deep brain tissue during insertion due to their innate flexibility. This spatial redistribution allows navigation of the CFETs towards deep brain targets spreading horizontally from the axis of insertion. Commercial "linear" arrays provide single-entry insertion but only allow measurements along the axis of insertion. Horizontally configured arrays inflict separate penetrations for each individual channel. We tested functional performance of our CFET arrays in vivo for recording dopamine and for providing lateral spread to multiple distributed sites in the rat striatum. Spatial spread was further characterized in agar brain phantoms as a function of insertion depth. We also developed protocols to slice the embedded CFETs within fixed brain tissue using standard histology. This method allowed extraction of the precise spatial coordinates of the implanted CFETs and their recording sites as integrated with immunohistochemical staining for surrounding anatomical, cytological, and protein expression labels. Our CFET array has the potential to unlock a wide range of applications, from uncovering the role of neuromodulators in synaptic plasticity, to addressing critical safety barriers in clinical translation towards diagnostic and adaptive treatment in Parkinson's disease and major mood disorders.
Collapse
Affiliation(s)
- Mingyi Xia
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Busra Nur Agca
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Tomoko Yoshida
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Jiwon Choi
- Department of Bioengineering, University of Pittsburgh, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Usamma Amjad
- Department of Bioengineering, University of Pittsburgh, USA
| | - Kade Bose
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Nikol Keren
- Department of Bioengineering, University of Pittsburgh, USA
| | | | - Michael J Cima
- Koch Institute for Integrative Cancer Research and Department of Materials Science, Massachusetts Institute of Technology, USA
| | - Ann M Graybiel
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, USA
| | - Helen N Schwerdt
- Department of Bioengineering, University of Pittsburgh, USA; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA.
| |
Collapse
|
32
|
Vu MAT, Brown EH, Wen MJ, Noggle CA, Zhang Z, Monk KJ, Bouabid S, Mroz L, Graham BM, Zhuo Y, Li Y, Otchy TM, Tian L, Davison IG, Boas DA, Howe MW. Targeted micro-fiber arrays for measuring and manipulating localized multi-scale neural dynamics over large, deep brain volumes during behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.17.567425. [PMID: 38014018 PMCID: PMC10680831 DOI: 10.1101/2023.11.17.567425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2023]
Abstract
Neural population dynamics relevant for behavior vary over multiple spatial and temporal scales across 3-dimensional volumes. Current optical approaches lack the spatial coverage and resolution necessary to measure and manipulate naturally occurring patterns of large-scale, distributed dynamics within and across deep brain regions such as the striatum. We designed a new micro-fiber array and imaging approach capable of chronically measuring and optogenetically manipulating local dynamics across over 100 targeted locations simultaneously in head-fixed and freely moving mice. We developed a semi-automated micro-CT based strategy to precisely localize positions of each optical fiber. This highly-customizable approach enables investigation of multi-scale spatial and temporal patterns of cell-type and neurotransmitter specific signals over arbitrary 3-D volumes at a spatial resolution and coverage previously inaccessible. We applied this method to resolve rapid dopamine release dynamics across the striatum volume which revealed distinct, modality specific spatiotemporal patterns in response to salient sensory stimuli extending over millimeters of tissue. Targeted optogenetics through our fiber arrays enabled flexible control of neural signaling on multiple spatial scales, better matching endogenous signaling patterns, and spatial localization of behavioral function across large circuits.
Collapse
Affiliation(s)
- Mai-Anh T. Vu
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Eleanor H. Brown
- Graduate Program for Neuroscience, Boston University, Boston, MA, USA
| | - Michelle J. Wen
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
- Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School, Boston, MA, USA
| | - Christian A. Noggle
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
- Aligning Science Across Parkinson’s (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Zicheng Zhang
- Department of Biology, Boston University, Boston, MA, USA
| | - Kevin J. Monk
- Department of Biology, Boston University, Boston, MA, USA
| | - Safa Bouabid
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Lydia Mroz
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
- Northeastern University, Boston, MA, USA
| | - Benjamin M. Graham
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - Yizhou Zhuo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking-Tsinghua Center for Life Sciences, Beijing, China
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing, China
- Peking-Tsinghua Center for Life Sciences, Beijing, China
| | | | - Lin Tian
- Department of Biochemistry and Molecular Medicine, University of California, Davis, CA
| | - Ian G. Davison
- Department of Biology, Boston University, Boston, MA, USA
| | - David A. Boas
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Mark W. Howe
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| |
Collapse
|
33
|
Willmore L, Minerva AR, Engelhard B, Murugan M, McMannon B, Oak N, Thiberge SY, Peña CJ, Witten IB. Overlapping representations of food and social stimuli in mouse VTA dopamine neurons. Neuron 2023; 111:3541-3553.e8. [PMID: 37657441 DOI: 10.1016/j.neuron.2023.08.003] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Revised: 05/17/2023] [Accepted: 08/03/2023] [Indexed: 09/03/2023]
Abstract
Dopamine neurons of the ventral tegmental area (VTADA) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear whether the same or different VTADA neurons encode these different stimuli. To address this question, we performed two-photon calcium imaging in mice presented with food and conspecifics and found statistically significant overlap in the populations responsive to both stimuli. Both hunger and opposite-sex social experience further increased the proportion of neurons that respond to both stimuli, implying that increasing motivation for one stimulus increases overlap. In addition, single-nucleus RNA sequencing revealed significant co-expression of feeding- and social-hormone-related genes in individual VTADA neurons. Taken together, our functional and transcriptional data suggest overlapping VTADA populations underlie food and social motivation.
Collapse
Affiliation(s)
- Lindsay Willmore
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Adelaide R Minerva
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Faculty of Medicine, Technion, Haifa 3525433, Israel.
| | - Malavika Murugan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Nirja Oak
- Faculty of Medicine, Technion, Haifa 3525433, Israel
| | - Stephan Y Thiberge
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Catherine J Peña
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| |
Collapse
|
34
|
Krausz TA, Comrie AE, Kahn AE, Frank LM, Daw ND, Berke JD. Dual credit assignment processes underlie dopamine signals in a complex spatial environment. Neuron 2023; 111:3465-3478.e7. [PMID: 37611585 PMCID: PMC10841332 DOI: 10.1016/j.neuron.2023.07.017] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 06/23/2023] [Accepted: 07/25/2023] [Indexed: 08/25/2023]
Abstract
Animals frequently make decisions based on expectations of future reward ("values"). Values are updated by ongoing experience: places and choices that result in reward are assigned greater value. Yet, the specific algorithms used by the brain for such credit assignment remain unclear. We monitored accumbens dopamine as rats foraged for rewards in a complex, changing environment. We observed brief dopamine pulses both at reward receipt (scaling with prediction error) and at novel path opportunities. Dopamine also ramped up as rats ran toward reward ports, in proportion to the value at each location. By examining the evolution of these dopamine place-value signals, we found evidence for two distinct update processes: progressive propagation of value along taken paths, as in temporal difference learning, and inference of value throughout the maze, using internal models. Our results demonstrate that within rich, naturalistic environments dopamine conveys place values that are updated via multiple, complementary learning algorithms.
Collapse
Affiliation(s)
- Timothy A Krausz
- Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Alison E Comrie
- Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Ari E Kahn
- Department of Psychology, and Princeton Neuroscience Institute, Princeton University, Princeton, Princeton, NJ 08544, USA
| | - Loren M Frank
- Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, Chevy Chase, MD 20815, USA; Department of Physiology, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Nathaniel D Daw
- Department of Psychology, and Princeton Neuroscience Institute, Princeton University, Princeton, Princeton, NJ 08544, USA
| | - Joshua D Berke
- Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, and Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA 94158, USA; Department of Neurology and Department of Psychiatry and Behavioral Science, University of California, San Francisco, San Francisco, CA 94158, USA.
| |
Collapse
|
35
|
Matityahu L, Gilin N, Sarpong GA, Atamna Y, Tiroshi L, Tritsch NX, Wickens JR, Goldberg JA. Acetylcholine waves and dopamine release in the striatum. Nat Commun 2023; 14:6852. [PMID: 37891198 PMCID: PMC10611775 DOI: 10.1038/s41467-023-42311-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 10/06/2023] [Indexed: 10/29/2023] Open
Abstract
Striatal dopamine encodes reward, with recent work showing that dopamine release occurs in spatiotemporal waves. However, the mechanism of dopamine waves is unknown. Here we report that acetylcholine release in mouse striatum also exhibits wave activity, and that the spatial scale of striatal dopamine release is extended by nicotinic acetylcholine receptors. Based on these findings, and on our demonstration that single cholinergic interneurons can induce dopamine release, we hypothesized that the local reciprocal interaction between cholinergic interneurons and dopamine axons suffices to drive endogenous traveling waves. We show that the morphological and physiological properties of cholinergic interneuron - dopamine axon interactions can be modeled as a reaction-diffusion system that gives rise to traveling waves. Analytically-tractable versions of the model show that the structure and the nature of propagation of acetylcholine and dopamine traveling waves depend on their coupling, and that traveling waves can give rise to empirically observed correlations between these signals. Thus, our study provides evidence for striatal acetylcholine waves in vivo, and proposes a testable theoretical framework that predicts that the observed dopamine and acetylcholine waves are strongly coupled phenomena.
Collapse
Affiliation(s)
- Lior Matityahu
- Department of Medical Neurobiology, Institute of Medical Research Israel - Canada, The Faculty of Medicine, The Hebrew University of Jerusalem, 9112102, Jerusalem, Israel
| | - Naomi Gilin
- Department of Medical Neurobiology, Institute of Medical Research Israel - Canada, The Faculty of Medicine, The Hebrew University of Jerusalem, 9112102, Jerusalem, Israel
| | - Gideon A Sarpong
- Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
| | - Yara Atamna
- Department of Medical Neurobiology, Institute of Medical Research Israel - Canada, The Faculty of Medicine, The Hebrew University of Jerusalem, 9112102, Jerusalem, Israel
| | - Lior Tiroshi
- Department of Medical Neurobiology, Institute of Medical Research Israel - Canada, The Faculty of Medicine, The Hebrew University of Jerusalem, 9112102, Jerusalem, Israel
| | - Nicolas X Tritsch
- Neuroscience Institute, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Jeffery R Wickens
- Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
| | - Joshua A Goldberg
- Department of Medical Neurobiology, Institute of Medical Research Israel - Canada, The Faculty of Medicine, The Hebrew University of Jerusalem, 9112102, Jerusalem, Israel.
| |
Collapse
|
36
|
Bech P, Crochet S, Dard R, Ghaderi P, Liu Y, Malekzadeh M, Petersen CCH, Pulin M, Renard A, Sourmpis C. Striatal Dopamine Signals and Reward Learning. FUNCTION 2023; 4:zqad056. [PMID: 37841525 PMCID: PMC10572094 DOI: 10.1093/function/zqad056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 09/28/2023] [Accepted: 09/29/2023] [Indexed: 10/17/2023] Open
Abstract
We are constantly bombarded by sensory information and constantly making decisions on how to act. In order to optimally adapt behavior, we must judge which sequences of sensory inputs and actions lead to successful outcomes in specific circumstances. Neuronal circuits of the basal ganglia have been strongly implicated in action selection, as well as the learning and execution of goal-directed behaviors, with accumulating evidence supporting the hypothesis that midbrain dopamine neurons might encode a reward signal useful for learning. Here, we review evidence suggesting that midbrain dopaminergic neurons signal reward prediction error, driving synaptic plasticity in the striatum underlying learning. We focus on phasic increases in action potential firing of midbrain dopamine neurons in response to unexpected rewards. These dopamine neurons prominently innervate the dorsal and ventral striatum. In the striatum, the released dopamine binds to dopamine receptors, where it regulates the plasticity of glutamatergic synapses. The increase of striatal dopamine accompanying an unexpected reward activates dopamine type 1 receptors (D1Rs) initiating a signaling cascade that promotes long-term potentiation of recently active glutamatergic input onto striatonigral neurons. Sensorimotor-evoked glutamatergic input, which is active immediately before reward delivery will thus be strengthened onto neurons in the striatum expressing D1Rs. In turn, these neurons cause disinhibition of brainstem motor centers and disinhibition of the motor thalamus, thus promoting motor output to reinforce rewarded stimulus-action outcomes. Although many details of the hypothesis need further investigation, altogether, it seems likely that dopamine signals in the striatum might underlie important aspects of goal-directed reward-based learning.
Collapse
Affiliation(s)
- Pol Bech
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Sylvain Crochet
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Robin Dard
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Parviz Ghaderi
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Yanqi Liu
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Meriam Malekzadeh
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Carl C H Petersen
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Mauro Pulin
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Anthony Renard
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| | - Christos Sourmpis
- Laboratory of Sensory Processing, Brain Mind Institute, Faculty of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland
| |
Collapse
|
37
|
Azcorra M, Gaertner Z, Davidson C, He Q, Kim H, Nagappan S, Hayes CK, Ramakrishnan C, Fenno L, Kim YS, Deisseroth K, Longnecker R, Awatramani R, Dombeck DA. Unique functional responses differentially map onto genetic subtypes of dopamine neurons. Nat Neurosci 2023; 26:1762-1774. [PMID: 37537242 PMCID: PMC10545540 DOI: 10.1038/s41593-023-01401-9] [Citation(s) in RCA: 25] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 07/05/2023] [Indexed: 08/05/2023]
Abstract
Dopamine neurons are characterized by their response to unexpected rewards, but they also fire during movement and aversive stimuli. Dopamine neuron diversity has been observed based on molecular expression profiles; however, whether different functions map onto such genetic subtypes remains unclear. In this study, we established that three genetic dopamine neuron subtypes within the substantia nigra pars compacta, characterized by the expression of Slc17a6 (Vglut2), Calb1 and Anxa1, each have a unique set of responses to rewards, aversive stimuli and accelerations and decelerations, and these signaling patterns are highly correlated between somas and axons within subtypes. Remarkably, reward responses were almost entirely absent in the Anxa1+ subtype, which instead displayed acceleration-correlated signaling. Our findings establish a connection between functional and genetic dopamine neuron subtypes and demonstrate that molecular expression patterns can serve as a common framework to dissect dopaminergic functions.
Collapse
Affiliation(s)
- Maite Azcorra
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
- Department of Neurology, Northwestern University, Chicago, IL, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Zachary Gaertner
- Department of Neurology, Northwestern University, Chicago, IL, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Connor Davidson
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Qianzi He
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Hailey Kim
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
| | - Shivathmihai Nagappan
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Cooper K Hayes
- Department of Microbiology and Immunology, Northwestern University, Chicago, IL, USA
| | - Charu Ramakrishnan
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Lief Fenno
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
- Departments of Neuroscience & Psychiatry, The University of Texas at Austin, Austin, TX, USA
| | - Yoon Seok Kim
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University School of Medicine, Stanford, CA, USA
| | - Richard Longnecker
- Department of Microbiology and Immunology, Northwestern University, Chicago, IL, USA
| | - Rajeshwar Awatramani
- Department of Neurology, Northwestern University, Chicago, IL, USA.
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA.
| | - Daniel A Dombeck
- Department of Neurobiology, Northwestern University, Evanston, IL, USA.
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA.
| |
Collapse
|
38
|
Zabeh E, Foley NC, Jacobs J, Gottlieb JP. Beta traveling waves in monkey frontal and parietal areas encode recent reward history. Nat Commun 2023; 14:5428. [PMID: 37669966 PMCID: PMC10480436 DOI: 10.1038/s41467-023-41125-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Accepted: 08/22/2023] [Indexed: 09/07/2023] Open
Abstract
Brain function depends on neural communication, but the mechanisms of this communication are not well understood. Recent studies suggest that one form of neural communication is through traveling waves (TWs)-patterns of neural oscillations that propagate within and between brain areas. We show that TWs are robust in microarray recordings in frontal and parietal cortex and encode recent reward history. Two adult male monkeys made saccades to obtain probabilistic rewards and were sensitive to the (statistically irrelevant) reward on the previous trial. TWs in frontal and parietal areas were stronger in trials that followed a prior reward versus a lack of reward and, in the frontal lobe, correlated with the monkeys' behavioral sensitivity to the prior reward. The findings suggest that neural communication mediated by TWs within the frontal and parietal lobes contribute to maintaining information about recent reward history and mediating the impact of this history on the monkeys' expectations.
Collapse
Affiliation(s)
- Erfan Zabeh
- Department of Biomedical Engineering, Columbia University, New York, NY, USA
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Nicholas C Foley
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Joshua Jacobs
- Department of Biomedical Engineering, Columbia University, New York, NY, USA.
- Department of Neurological Surgery, Columbia University, New York, NY, USA.
| | - Jacqueline P Gottlieb
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA.
- Department of Neuroscience, Columbia University, New York, NY, USA.
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
39
|
Rouhani N, Niv Y, Frank MJ, Schwabe L. Multiple routes to enhanced memory for emotionally relevant events. Trends Cogn Sci 2023; 27:867-882. [PMID: 37479601 DOI: 10.1016/j.tics.2023.06.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 06/13/2023] [Accepted: 06/15/2023] [Indexed: 07/23/2023]
Abstract
Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.
Collapse
Affiliation(s)
- Nina Rouhani
- Division of Biology and Biological Engineering and Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Yael Niv
- Department of Psychology and Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Michael J Frank
- Department of Cognitive, Linguistic & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, RI, USA
| | - Lars Schwabe
- Department of Cognitive Psychology, Institute of Psychology, Universität Hamburg, Hamburg, Germany.
| |
Collapse
|
40
|
Zhuo Y, Luo B, Yi X, Dong H, Wan J, Cai R, Williams JT, Qian T, Campbell MG, Miao X, Li B, Wei Y, Li G, Wang H, Zheng Y, Watabe-Uchida M, Li Y. Improved dual-color GRAB sensors for monitoring dopaminergic activity in vivo. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.24.554559. [PMID: 37662187 PMCID: PMC10473776 DOI: 10.1101/2023.08.24.554559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a vast network of dopaminergic projections. To fully dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent GPCR activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity, and signal-to-noise properties with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in freely moving mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala, and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.
Collapse
Affiliation(s)
- Yizhou Zhuo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- These authors contributed equally
| | - Bin Luo
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
- These authors contributed equally
| | - Xinyang Yi
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Hui Dong
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
| | - Jinxia Wan
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
| | - Ruyi Cai
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - John T. Williams
- Vollum Institute, Oregon Health and Science University, Portland, OR 97239, USA
| | - Tongrui Qian
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Malcolm G. Campbell
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA
| | - Xiaolei Miao
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- Department of Anesthesiology, Beijing Chaoyang Hospital, Capital Medical University, Beijing 100020, China
| | - Bozhi Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- Department of Neurology, the First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Haidian District, Beijing 100853, China
| | - Yu Wei
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Guochuan Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Huan Wang
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Yu Zheng
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
| | - Mitsuko Watabe-Uchida
- Center for Brain Science, Department of Molecular and Cellular Biology, Harvard University, 16 Divinity Avenue, Cambridge, MA 02138, USA
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, Beijing 100871, China
- PKU-IDG/McGovern Institute for Brain Research, Beijing 100871, China
- Peking-Tsinghua Center for Life Sciences, Beijing 100871, China
- Chinese Institute for Brain Research, Beijing 102206, China
- Institute of Molecular Physiology, Shenzhen Bay Laboratory, Shenzhen, Guangdong 518055, China
- National Biomedical Imaging Center, Peking University, Beijing 100871, China
| |
Collapse
|
41
|
Kim MJ, Gibson DJ, Hu D, Mahar A, Schofield CJ, Sompolpong P, Yoshida T, Tran KT, Graybiel AM. Dopamine Release Plateau and Outcome Signals in Dorsal Striatum Contrast with Classic Reinforcement Learning Formulations. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.15.553421. [PMID: 37645888 PMCID: PMC10462077 DOI: 10.1101/2023.08.15.553421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
We recorded dopamine release signals in medial and lateral sectors of the striatum as mice learned consecutive visual cue-outcome conditioning tasks including cue association, cue discrimination, reversal, and probabilistic discrimination task versions. Dopamine release responses in medial and lateral sites exhibited learning-related changes within and across phases of acquisition. These were different for the medial and lateral sites. In neither sector could these be accounted for by classic reinforcement learning as applied to dopamine-containing neuron activity. Cue responses ranged from initial sharp peaks to modulated plateau responses. In the medial sector, outcome (reward) responses during cue conditioning were minimal or, initially, negative. By contrast, in lateral sites, strong, transient dopamine release responses occurred at both cue and outcome. Prolonged, plateau release responses to cues emerged in both regions when discriminative behavioral responses became required. In most sites, we found no evidence for a transition from outcome to cue signaling, a hallmark of temporal difference reinforcement learning as applied to midbrain dopamine activity. These findings delineate reshaping of dopamine release activity during learning and suggest that current views of reward prediction error encoding need review to accommodate distinct learning-related spatial and temporal patterns of striatal dopamine release in the dorsal striatum.
Collapse
|
42
|
Wärnberg E, Kumar A. Feasibility of dopamine as a vector-valued feedback signal in the basal ganglia. Proc Natl Acad Sci U S A 2023; 120:e2221994120. [PMID: 37527344 PMCID: PMC10410740 DOI: 10.1073/pnas.2221994120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Accepted: 06/08/2023] [Indexed: 08/03/2023] Open
Abstract
It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatum-wide RPE signal can support RL over a small discrete set of actions (e.g., no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of continuous outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). We implement a local, "three-factor" corticostriatal plasticity rule involving the presynaptic firing rate, a postsynaptic factor, and the unique dopamine concentration perceived by each striatal neuron. With this learning rule, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. Crucially, we demonstrate that this plasticity rule does not require precise nigrostriatal synapses but remains compatible with experimental observations of random placement of varicosities and diffuse volume transmission of dopamine.
Collapse
Affiliation(s)
- Emil Wärnberg
- Department of Neuroscience, Karolinska Institutet, 171 77Stockholm, Sweden
- Division of Computational Science and Technology, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, 114 28Stockholm, Sweden
| | - Arvind Kumar
- Division of Computational Science and Technology, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, 114 28Stockholm, Sweden
| |
Collapse
|
43
|
Glaeser-Khan S, Savalia NK, Cressy J, Feng J, Li Y, Kwan AC, Kaye AP. Spatiotemporal organization of prefrontal norepinephrine influences neuronal activity. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.06.09.544191. [PMID: 37502881 PMCID: PMC10370029 DOI: 10.1101/2023.06.09.544191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/29/2023]
Abstract
Norepinephrine (NE), a neuromodulator released by locus coeruleus neurons throughout cortex, influences arousal and learning through extra-synaptic vesicle exocytosis. While NE within cortical regions has been viewed as a homogenous field, recent studies have demonstrated heterogeneous axonal dynamics and advances in GPCR-based fluorescent sensors permit direct observation of the local dynamics of NE at cellular scale. To investigate how the spatiotemporal dynamics of NE release in the PFC affect neuronal firing, we employed in-vivo two-photon imaging of layer 2/3 of PFC in order to observe fine-scale neuronal calcium and NE dynamics concurrently. We found that local and global NE fields can decouple from one another, providing a substrate for local NE spatiotemporal activity patterns. Optic flow analysis revealed putative release and reuptake events which can occur at the same location, albeit at different times, indicating the potential to create a heterogeneous NE field. Utilizing generalized linear models, we demonstrated that cellular Ca2+ fluctuations are influenced by both the local and global NE field. However, during periods of local/global NE field decoupling, the local field drives cell firing dynamics rather than the global field. These findings underscore the significance of localized, phasic NE fluctuations for structuring cell firing, which may provide local neuromodulatory control of cortical activity.
Collapse
|
44
|
Dulberg Z, Dubey R, Berwian IM, Cohen JD. Having multiple selves helps learning agents explore and adapt in complex changing worlds. Proc Natl Acad Sci U S A 2023; 120:e2221180120. [PMID: 37399387 PMCID: PMC10334746 DOI: 10.1073/pnas.2221180120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 05/09/2023] [Indexed: 07/05/2023] Open
Abstract
Satisfying a variety of conflicting needs in a changing environment is a fundamental challenge for any adaptive agent. Here, we show that designing an agent in a modular fashion as a collection of subagents, each dedicated to a separate need, powerfully enhanced the agent's capacity to satisfy its overall needs. We used the formalism of deep reinforcement learning to investigate a biologically relevant multiobjective task: continually maintaining homeostasis of a set of physiologic variables. We then conducted simulations in a variety of environments and compared how modular agents performed relative to standard monolithic agents (i.e., agents that aimed to satisfy all needs in an integrated manner using a single aggregate measure of success). Simulations revealed that modular agents a) exhibited a form of exploration that was intrinsic and emergent rather than extrinsically imposed; b) were robust to changes in nonstationary environments, and c) scaled gracefully in their ability to maintain homeostasis as the number of conflicting objectives increased. Supporting analysis suggested that the robustness to changing environments and increasing numbers of needs were due to intrinsic exploration and efficiency of representation afforded by the modular architecture. These results suggest that the normative principles by which agents have adapted to complex changing environments may also explain why humans have long been described as consisting of "multiple selves."
Collapse
Affiliation(s)
- Zack Dulberg
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08544
| | - Rachit Dubey
- Department of Computer Science, Princeton University, Princeton, NJ08544
| | - Isabel M. Berwian
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08544
| | - Jonathan D. Cohen
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08544
| |
Collapse
|
45
|
Abstract
Striosomes form neurochemically specialized compartments of the striatum embedded in a large matrix made up of modules called matrisomes. Striosome-matrix architecture is multiplexed with the canonical direct-indirect organization of the striatum. Striosomal functions remain to be fully clarified, but key information is emerging. First, striosomes powerfully innervate nigral dopamine-containing neurons and can completely shut down their activity, with a following rebound excitation. Second, striosomes receive limbic and cognition-related corticostriatal afferents and are dynamically modulated in relation to value-based actions. Third, striosomes are spatially interspersed among matrisomes and interneurons and are influenced by local and global neuromodulatory and oscillatory activities. Fourth, striosomes tune engagement and the motivation to perform reinforcement learning, to manifest stereotypical behaviors, and to navigate valence conflicts and valence discriminations. We suggest that, at an algorithmic level, striosomes could serve as distributed scaffolds to provide formats of the striatal computations generated through development and refined through learning. We propose that striosomes affect subjective states. By transforming corticothalamic and other inputs to the functional formats of the striatum, they could implement state transitions in nigro-striato-nigral circuits to affect bodily and cognitive actions according to internal motives whose functions are compromised in neuropsychiatric conditions.
Collapse
Affiliation(s)
- Ann M Graybiel
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA;
| | - Ayano Matsushima
- McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA;
| |
Collapse
|
46
|
Zheng Y, Li Y. Past, present, and future of tools for dopamine detection. Neuroscience 2023:S0306-4522(23)00295-6. [PMID: 37419404 DOI: 10.1016/j.neuroscience.2023.06.025] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2023] [Revised: 06/26/2023] [Accepted: 06/29/2023] [Indexed: 07/09/2023]
Abstract
Dopamine (DA) is a critical neuromodulator involved in various brain functions. To understand how DA regulates neural circuits and behaviors in the physiological and pathological conditions, it is essential to have tools that enable the direct detection of DA dynamics in vivo. Recently, genetically encoded DA sensors based on G protein-coupled receptors revolutionized this field, as it allows us to track in vivo DA dynamic with unprecedented spatial-temporal resolution, high molecular specificity, and sub-second kinetics. In this review, we first summarize traditional DA detection methods. Then we focus on the development of genetically encoded DA sensors and feature its significance to understanding dopaminergic neuromodulation across diverse behaviors and species. Finally, we present our perspectives about the future direction of the next-generation DA sensors and extend their potential applications. Overall, this review offers a comprehensive perspective on the past, present, and future of DA detection tools, with important implications for the study of DA functions in health and disease.
Collapse
Affiliation(s)
- Yu Zheng
- Peking-Tsinghua Center for Life Sciences, 100871 Beijing, China
| | - Yulong Li
- Peking-Tsinghua Center for Life Sciences, 100871 Beijing, China; State Key Laboratory of Membrane Biology, Peking University School of Life Sciences, 100871 Beijing, China; PKU-IDG/McGovern Institute for Brain Research, 100871 Beijing, China; National Biomedical Imaging Center, Peking University, 100871 Beijing, China.
| |
Collapse
|
47
|
Lamba A, Nassar MR, FeldmanHall O. Prefrontal cortex state representations shape human credit assignment. eLife 2023; 12:e84888. [PMID: 37399050 PMCID: PMC10351919 DOI: 10.7554/elife.84888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Accepted: 06/16/2023] [Indexed: 07/04/2023] Open
Abstract
People learn adaptively from feedback, but the rate of such learning differs drastically across individuals and contexts. Here, we examine whether this variability reflects differences in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative reward learning task, we link the specificity of credit assignment-how well people are able to appropriately attribute outcomes to their causes-to the precision of neural codes in the prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared vto nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in concert to match the neural codes from feedback to those at choice, and the strength of these common neural codes predicts credit assignment precision. Together this work provides a window into how neural representations drive adaptive learning.
Collapse
Affiliation(s)
- Amrita Lamba
- Department of Cognitive Linguistic & Psychological Sciences, Brown UniversityProvidenceUnited States
| | - Matthew R Nassar
- Department of Neuroscience, Brown UniversityProvidenceUnited States
- Carney Institute of Brain Sciences, Brown UniversityProvidenceUnited States
| | - Oriel FeldmanHall
- Department of Cognitive Linguistic & Psychological Sciences, Brown UniversityProvidenceUnited States
- Carney Institute of Brain Sciences, Brown UniversityProvidenceUnited States
| |
Collapse
|
48
|
Pang JC, Aquino KM, Oldehinkel M, Robinson PA, Fulcher BD, Breakspear M, Fornito A. Geometric constraints on human brain function. Nature 2023; 618:566-574. [PMID: 37258669 PMCID: PMC10266981 DOI: 10.1038/s41586-023-06098-1] [Citation(s) in RCA: 71] [Impact Index Per Article: 71.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 04/18/2023] [Indexed: 06/02/2023]
Abstract
The anatomy of the brain necessarily constrains its function, but precisely how remains unclear. The classical and dominant paradigm in neuroscience is that neuronal dynamics are driven by interactions between discrete, functionally specialized cell populations connected by a complex array of axonal fibres1-3. However, predictions from neural field theory, an established mathematical framework for modelling large-scale brain activity4-6, suggest that the geometry of the brain may represent a more fundamental constraint on dynamics than complex interregional connectivity7,8. Here, we confirm these theoretical predictions by analysing human magnetic resonance imaging data acquired under spontaneous and diverse task-evoked conditions. Specifically, we show that cortical and subcortical activity can be parsimoniously understood as resulting from excitations of fundamental, resonant modes of the brain's geometry (that is, its shape) rather than from modes of complex interregional connectivity, as classically assumed. We then use these geometric modes to show that task-evoked activations across over 10,000 brain maps are not confined to focal areas, as widely believed, but instead excite brain-wide modes with wavelengths spanning over 60 mm. Finally, we confirm predictions that the close link between geometry and function is explained by a dominant role for wave-like activity, showing that wave dynamics can reproduce numerous canonical spatiotemporal properties of spontaneous and evoked recordings. Our findings challenge prevailing views and identify a previously underappreciated role of geometry in shaping function, as predicted by a unifying and physically principled model of brain-wide dynamics.
Collapse
Affiliation(s)
- James C Pang
- The Turner Institute for Brain and Mental Health, School of Psychological Sciences and Monash Biomedical Imaging, Monash University, Clayton, Victoria, Australia.
| | - Kevin M Aquino
- School of Physics, University of Sydney, Camperdown, New South Wales, Australia
- BrainKey Inc., San Francisco, CA, USA
| | - Marianne Oldehinkel
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University Medical Centre, Nijmegen, the Netherlands
| | - Peter A Robinson
- School of Physics, University of Sydney, Camperdown, New South Wales, Australia
| | - Ben D Fulcher
- School of Physics, University of Sydney, Camperdown, New South Wales, Australia
| | - Michael Breakspear
- School of Psychological Sciences, College of Engineering, Science and the Environment, University of Newcastle, Callaghan, New South Wales, Australia
- School of Medicine and Public Health, College of Health, Medicine and Wellbeing, University of Newcastle, Callaghan, New South Wales, Australia
| | - Alex Fornito
- The Turner Institute for Brain and Mental Health, School of Psychological Sciences and Monash Biomedical Imaging, Monash University, Clayton, Victoria, Australia
| |
Collapse
|
49
|
Willmore L, Minerva AR, Engelhard B, Murugan M, McMannon B, Oak N, Thiberge SY, Peña CJ, Witten IB. Overlapping representations of food and social stimuli in VTA dopamine neurons. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.17.541104. [PMID: 37293057 PMCID: PMC10245666 DOI: 10.1101/2023.05.17.541104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Dopamine neurons of the ventral tegmental area (VTA DA ) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear if the same or different VTA DA neurons encode these different stimuli. To address this question, we performed 2-photon calcium imaging in mice presented with food and conspecifics, and found statistically significant overlap in the populations responsive to both stimuli. Both hunger and opposite-sex social experience further increased the proportion of neurons that respond to both stimuli, implying that modifying motivation for one stimulus affects responses to both stimuli. In addition, single-nucleus RNA sequencing revealed significant co-expression of feeding- and social-hormone related genes in individual VTA DA neurons. Taken together, our functional and transcriptional data suggest overlapping VTA DA populations underlie food and social motivation.
Collapse
Affiliation(s)
- Lindsay Willmore
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Adelaide R. Minerva
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Ben Engelhard
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
- Department of Medicine, Technion, Haifa, 3525433, Israel
| | - Malavika Murugan
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Nirja Oak
- Department of Medicine, Technion, Haifa, 3525433, Israel
| | - Stephan Y. Thiberge
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Catherine J. Peña
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| | - Ilana B. Witten
- Princeton Neuroscience Institute, Princeton University, Princeton NJ 08544 USA
| |
Collapse
|
50
|
Xia M, Agca BN, Yoshida T, Choi J, Amjad U, Bose K, Keren N, Zukerman S, Cima MJ, Graybiel AM, Schwerdt HN. Scalable, flexible carbon fiber electrode thread arrays for three-dimensional spatial profiling of neurochemical activity in deep brain structures of rodents. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.15.537033. [PMID: 37131810 PMCID: PMC10153108 DOI: 10.1101/2023.04.15.537033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
We developed a flexible "electrode-thread" array for recording dopamine neurochemical activity from a lateral distribution of subcortical targets (up to 16) transverse to the axis of insertion. Ultrathin (∼ 10 µm diameter) carbon fiber (CF) electrode-threads (CFETs) are clustered into a tight bundle to introduce them into the brain from a single entry point. The individual CFETs splay laterally in deep brain tissue during insertion due to their innate flexibility. This spatial redistribution allows navigation of the CFETs towards deep brain targets spreading horizontally from the axis of insertion. Commercial "linear" arrays provide single entry insertion but only allow measurements along the axis of insertion. Horizontally configured neurochemical recording arrays inflict separate penetrations for each individual channel (i.e., electrode). We tested functional performance of our CFET arrays in vivo for recording dopamine neurochemical dynamics and for providing lateral spread to multiple distributed sites in the striatum of rats. Spatial spread was further characterized using agar brain phantoms to measure electrode deflection as a function of insertion depth. We also developed protocols to slice the embedded CFETs within fixed brain tissue using standard histology techniques. This method allowed extraction of the precise spatial coordinates of the implanted CFETs and their recording sites as integrated with immunohistochemical staining for surrounding anatomical, cytological, and protein expression labels. Neurochemical recording operations tested here can be integrated with already widely established capabilities of CF-based electrodes to record single neuron activity and local field potentials, to enable multi-modal recording functions. Our CFET array has the potential to unlock a wide range of applications, from uncovering the role of neuromodulators in synaptic plasticity, to addressing critical safety barriers in clinical translation towards diagnostic and adaptive treatment in Parkinson's disease and major mood disorders.
Collapse
Affiliation(s)
- Mingyi Xia
- McGovern Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | - Busra Nur Agca
- McGovern Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | - Tomoko Yoshida
- McGovern Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | - Jiwon Choi
- Department of Bioengineering, University of Pittsburgh
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD
| | - Usamma Amjad
- Department of Bioengineering, University of Pittsburgh
| | - Kade Bose
- McGovern Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | - Nikol Keren
- Department of Bioengineering, University of Pittsburgh
| | | | - Michael J Cima
- Koch Institute for Integrative Cancer Research and Department of Materials Science, Massachusetts Institute of Technology
| | - Ann M Graybiel
- McGovern Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | - Helen N Schwerdt
- Department of Bioengineering, University of Pittsburgh
- Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD
| |
Collapse
|