1
|
Farrell M, Pehlevan C. Recall tempo of Hebbian sequences depends on the interplay of Hebbian kernel with tutor signal timing. Proc Natl Acad Sci U S A 2024; 121:e2309876121. [PMID: 39078676 DOI: 10.1073/pnas.2309876121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 06/04/2024] [Indexed: 07/31/2024] Open
Abstract
Understanding how neural circuits generate sequential activity is a longstanding challenge. While foundational theoretical models have shown how sequences can be stored as memories in neural networks with Hebbian plasticity rules, these models considered only a narrow range of Hebbian rules. Here, we introduce a model for arbitrary Hebbian plasticity rules, capturing the diversity of spike-timing-dependent synaptic plasticity seen in experiments, and show how the choice of these rules and of neural activity patterns influences sequence memory formation and retrieval. In particular, we derive a general theory that predicts the tempo of sequence replay. This theory lays a foundation for explaining how cortical tutor signals might give rise to motor actions that eventually become "automatic." Our theory also captures the impact of changing the tempo of the tutor signal. Beyond shedding light on biological circuits, this theory has relevance in artificial intelligence by laying a foundation for frameworks whereby slow and computationally expensive deliberation can be stored as memories and eventually replaced by inexpensive recall.
Collapse
Affiliation(s)
- Matthew Farrell
- John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138
- Center for Brain Science, Harvard University, Cambridge, MA 02138
| | - Cengiz Pehlevan
- John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138
- Center for Brain Science, Harvard University, Cambridge, MA 02138
- Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA 02138
| |
Collapse
|
2
|
Yang Z, Inagaki M, Gerfen CR, Fontolan L, Inagaki HK. Integrator dynamics in the cortico-basal ganglia loop underlie flexible motor timing. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.29.601348. [PMID: 39005437 PMCID: PMC11244898 DOI: 10.1101/2024.06.29.601348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/16/2024]
Abstract
Flexible control of motor timing is crucial for behavior. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviors, distinguishing their individual roles in this timer function remains challenging. To address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings suggest the striatum is a part of the network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.
Collapse
|
3
|
Lohnas LJ, Howard MW. The influence of emotion on temporal context models. Cogn Emot 2024:1-29. [PMID: 39007902 DOI: 10.1080/02699931.2024.2371075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Accepted: 06/17/2024] [Indexed: 07/16/2024]
Abstract
Temporal context models (TCMs) have been influential in understanding episodic memory and its neural underpinnings. Recently, TCMs have been extended to explain emotional memory effects, one of the most clinically important findings in the field of memory research. This review covers recent advances in hypotheses for the neural representation of spatiotemporal context through the lens of TCMs, including their ability to explain the influence of emotion on episodic and temporal memory. In recent years, simplifying assumptions of "classical" TCMs - with exponential trace decay and the mechanism by which temporal context is recovered - have become increasingly clear. The review also outlines how recent advances could be incorporated into a future TCM, beyond classical assumptions, to integrate emotional modulation.
Collapse
Affiliation(s)
- Lynn J Lohnas
- Department of Psychology, Syracuse University, Syracuse, NY, USA
| | - Marc W Howard
- Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA
| |
Collapse
|
4
|
Cone I, Clopath C, Shouval HZ. Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time. Nat Commun 2024; 15:5856. [PMID: 38997276 PMCID: PMC11245539 DOI: 10.1038/s41467-024-50205-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 07/02/2024] [Indexed: 07/14/2024] Open
Abstract
The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference learning (TD) learning, whereby certain units signal reward prediction errors (RPE). The TD algorithm has been traditionally mapped onto the dopaminergic system, as firing properties of dopamine neurons can resemble RPEs. However, certain predictions of TD learning are inconsistent with experimental results, and previous implementations of the algorithm have made unscalable assumptions regarding stimulus-specific fixed temporal bases. We propose an alternate framework to describe dopamine signaling in the brain, FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, dopamine release is similar, but not identical to RPE, leading to predictions that contrast to those of TD. While FLEX itself is a general theoretical framework, we describe a specific, biophysically plausible implementation, the results of which are consistent with a preponderance of both existing and reanalyzed experimental data.
Collapse
Affiliation(s)
- Ian Cone
- Department of Bioengineering, Imperial College London, London, UK
- Department of Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, USA
- Applied Physics Program, Rice University, Houston, TX, USA
| | - Claudia Clopath
- Department of Bioengineering, Imperial College London, London, UK
| | - Harel Z Shouval
- Department of Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, USA.
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA.
| |
Collapse
|
5
|
Redinbaugh MJ, Saalmann YB. Contributions of Basal Ganglia Circuits to Perception, Attention, and Consciousness. J Cogn Neurosci 2024; 36:1620-1642. [PMID: 38695762 PMCID: PMC11223727 DOI: 10.1162/jocn_a_02177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/04/2024]
Abstract
Research into ascending sensory pathways and cortical networks has generated detailed models of perception. These same cortical regions are strongly connected to subcortical structures, such as the basal ganglia (BG), which have been conceptualized as playing key roles in reinforcement learning and action selection. However, because the BG amasses experiential evidence from higher and lower levels of cortical hierarchies, as well as higher-order thalamus, it is well positioned to dynamically influence perception. Here, we review anatomical, functional, and clinical evidence to demonstrate how the BG can influence perceptual processing and conscious states. This depends on the integrative relationship between cortex, BG, and thalamus, which allows contributions to sensory gating, predictive processing, selective attention, and representation of the temporal structure of events.
Collapse
Affiliation(s)
| | - Yuri B Saalmann
- University of Wisconsin-Madison
- Wisconsin National Primate Research Center
| |
Collapse
|
6
|
Bruce RA, Weber MA, Bova AS, Volkman RA, Jacobs CE, Sivakumar K, Stutt HR, Kim YC, Curtu R, Narayanan NS. Complementary opposing D2-MSNs and D1-MSNs dynamics during interval timing. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.07.25.550569. [PMID: 37546735 PMCID: PMC10402049 DOI: 10.1101/2023.07.25.550569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
The role of striatal pathways in cognitive processing is unclear. We studied dorsomedial striatal cognitive processing during interval timing, an elementary cognitive task that requires mice to estimate intervals of several seconds and involves working memory for temporal rules as well as attention to the passage of time. We harnessed optogenetic tagging to record from striatal D2-dopamine receptor-expressing medium spiny neurons (D2-MSNs) in the indirect pathway and from D1-dopamine receptor-expressing MSNs (D1-MSNs) in the direct pathway. We found that D2-MSNs and D1-MSNs exhibited distinct dynamics over temporal intervals as quantified by principal component analyses and trial-by-trial generalized linear models. MSN recordings helped construct and constrain a four-parameter drift-diffusion computational model. This model predicted that disrupting either D2-MSNs or D1-MSNs would increase interval timing response times and alter MSN firing. In line with this prediction, we found that optogenetic inhibition or pharmacological disruption of either D2-MSNs or D1-MSNs increased interval timing response times. Pharmacologically disrupting D2-MSNs or D1-MSNs also changed MSN dynamics and degraded trial-by-trial temporal decoding. Together, our findings demonstrate that D2-MSNs and D1-MSNs make complementary contributions to interval timing despite opposing dynamics, implying that striatal direct and indirect pathways work together to shape temporal control of action. These data provide novel insight into basal ganglia cognitive operations beyond movement and have implications for human striatal diseases and therapies targeting striatal pathways.
Collapse
|
7
|
Bigus ER, Lee HW, Bowler JC, Shi J, Heys JG. Medial entorhinal cortex mediates learning of context-dependent interval timing behavior. Nat Neurosci 2024:10.1038/s41593-024-01683-7. [PMID: 38877306 DOI: 10.1038/s41593-024-01683-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 05/14/2024] [Indexed: 06/16/2024]
Abstract
Episodic memory requires encoding the temporal structure of experience and relies on brain circuits in the medial temporal lobe, including the medial entorhinal cortex (MEC). Recent studies have identified MEC 'time cells', which fire at specific moments during interval timing tasks, collectively tiling the entire timing period. It has been hypothesized that MEC time cells could provide temporal information necessary for episodic memories, yet it remains unknown whether they display learning dynamics required for encoding different temporal contexts. To explore this, we developed a new behavioral paradigm requiring mice to distinguish temporal contexts. Combined with methods for cellular resolution calcium imaging, we found that MEC time cells display context-dependent neural activity that emerges with task learning. Through chemogenetic inactivation we found that MEC activity is necessary for learning of context-dependent interval timing behavior. Finally, we found evidence of a common circuit mechanism that could drive sequential activity of both time cells and spatially selective neurons in MEC. Our work suggests that the clock-like firing of MEC time cells can be modulated by learning, allowing the tracking of various temporal structures that emerge through experience.
Collapse
Affiliation(s)
- Erin R Bigus
- Interdepartmental PhD Program in Neuroscience, University of Utah, Salt Lake City, UT, USA
| | - Hyun-Woo Lee
- Department of Neurobiology, University of Utah, Salt Lake City, UT, USA
| | - John C Bowler
- Department of Neurobiology, University of Utah, Salt Lake City, UT, USA
| | - Jiani Shi
- Department of Neurobiology, University of Utah, Salt Lake City, UT, USA
| | - James G Heys
- Department of Neurobiology, University of Utah, Salt Lake City, UT, USA.
| |
Collapse
|
8
|
Nishioka M, Hata T. Cholinergic interneurons in the dorsal striatum play an important role in the acquisition of duration memory. Eur J Neurosci 2024; 59:3061-3073. [PMID: 38576223 DOI: 10.1111/ejn.16337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 02/15/2024] [Accepted: 03/09/2024] [Indexed: 04/06/2024]
Abstract
The present study aimed to examine the effect of cholinergic interneuron lesions in the dorsal striatum on duration-memory formation. Cholinergic interneurons in the dorsal striatum may be involved in the formation of duration memory since they are among the main inputs to the dorsal striatal muscarinic acetylcholine-1 receptors, which play a role in the consolidation of duration memory. Rats were sufficiently trained using a peak-interval 20 s procedure and then infused with anti-choline acetyltransferase-saporin into the dorsal striatum to cause selective ablation of cholinergic interneurons. To make the rats acquire new duration-memories, we trained them with a peak interval 40 s after lesion. Before lesion, the peak times (an index of duration memory) for sham-lesioned and lesioned groups were similar at approximately 20 s. In the peak interval 40 s session, the peak times for the sham-lesioned and lesioned groups were approximately 30 and 20 s, respectively. After additional peak interval 40 s sessions, the peak times of both groups were shifted to approximately 40 s. Those results suggest that the cholinergic interneuron lesion delayed new duration-memory acquisition. Subsequent experiments showed that cholinergic interneuron lesions did not retard the shift of peak time to the original target time (20 s). Following experiment without changing the target time after lesion showed that cholinergic interneuron lesions did not change their peak times. Our findings suggest that cholinergic interneurons in the dorsal striatum are involved in new duration-memory acquisition but not in the utilization of already acquired duration memory and interval timing.
Collapse
Affiliation(s)
- Masahiko Nishioka
- Graduate School of Psychology, Doshisha University, Kyotanabe, Kyoto, 610-0394, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
| | - Toshimichi Hata
- Faculty of Psychology, Doshisha University, Kyotanabe, Kyoto, 610-0394, Japan
| |
Collapse
|
9
|
Wang X, Shi S, Bao Y. Parallel processes of temporal control in the supplementary motor area and the frontoparietal circuit. Psych J 2024; 13:355-368. [PMID: 38105556 PMCID: PMC11169752 DOI: 10.1002/pchj.701] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 10/04/2023] [Indexed: 12/19/2023]
Abstract
Durations in the several seconds' range are cognitively accessible during active timing. Functional neuroimaging studies suggest the engagement of the basal ganglia (BG) and supplementary motor area (SMA). However, their functional relevance and arrangement remain unclear because non-timing cognitive processes temporally coincide with the active timing. To examine the potential contamination by parallel processes, we introduced a sensory control and a motor control to the duration-reproduction task. By comparing their hemodynamic functions, we decomposed the neural activities in multiple brain loci linked to different cognitive processes. Our results show a dissociation of two cortical neural circuits: the SMA for both active timing and motor preparation, followed by a prefrontal-parietal circuit related to duration working memory. We argue that these cortical processes represent duration as the content but at different levels of abstraction, while the subcortical structures, including the BG and thalamus, provide the logistic basis of timing by coordinating the temporal framework across brain structures.
Collapse
Affiliation(s)
- Xuanyu Wang
- School of Psychological and Cognitive SciencesPeking UniversityBeijingChina
- Graduate School of Systemic NeurosciencesLudwig‐Maximilians‐Universität MünchenMunichGermany
| | - Shunyu Shi
- School of Psychological and Cognitive SciencesPeking UniversityBeijingChina
| | - Yan Bao
- School of Psychological and Cognitive SciencesPeking UniversityBeijingChina
- Institute of Medical Psychology, Ludwig‐Maximilians‐Universität MünchenMunichGermany
- Beijing Key Laboratory of Behavior and Mental HealthPeking UniversityBeijingChina
| |
Collapse
|
10
|
Vignoud G, Venance L, Touboul JD. Anti-Hebbian plasticity drives sequence learning in striatum. Commun Biol 2024; 7:555. [PMID: 38724614 PMCID: PMC11082161 DOI: 10.1038/s42003-024-06203-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Accepted: 04/17/2024] [Indexed: 05/12/2024] Open
Abstract
Spatio-temporal activity patterns have been observed in a variety of brain areas in spontaneous activity, prior to or during action, or in response to stimuli. Biological mechanisms endowing neurons with the ability to distinguish between different sequences remain largely unknown. Learning sequences of spikes raises multiple challenges, such as maintaining in memory spike history and discriminating partially overlapping sequences. Here, we show that anti-Hebbian spike-timing dependent plasticity (STDP), as observed at cortico-striatal synapses, can naturally lead to learning spike sequences. We design a spiking model of the striatal output neuron receiving spike patterns defined as sequential input from a fixed set of cortical neurons. We use a simple synaptic plasticity rule that combines anti-Hebbian STDP and non-associative potentiation for a subset of the presented patterns called rewarded patterns. We study the ability of striatal output neurons to discriminate rewarded from non-rewarded patterns by firing only after the presentation of a rewarded pattern. In particular, we show that two biological properties of striatal networks, spiking latency and collateral inhibition, contribute to an increase in accuracy, by allowing a better discrimination of partially overlapping sequences. These results suggest that anti-Hebbian STDP may serve as a biological substrate for learning sequences of spikes.
Collapse
Affiliation(s)
- Gaëtan Vignoud
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France
| | - Laurent Venance
- Center for Interdisciplinary Research in Biology (CIRB), College de France, CNRS, INSERM, Université PSL, Paris, France.
| | - Jonathan D Touboul
- Department of Mathematics and Volen National Center for Complex Systems, Brandeis University, Waltham, MA, USA.
| |
Collapse
|
11
|
Zhou S, Buonomano DV. Unified control of temporal and spatial scales of sensorimotor behavior through neuromodulation of short-term synaptic plasticity. SCIENCE ADVANCES 2024; 10:eadk7257. [PMID: 38701208 DOI: 10.1126/sciadv.adk7257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 04/03/2024] [Indexed: 05/05/2024]
Abstract
Neuromodulators have been shown to alter the temporal profile of short-term synaptic plasticity (STP); however, the computational function of this neuromodulation remains unexplored. Here, we propose that the neuromodulation of STP provides a general mechanism to scale neural dynamics and motor outputs in time and space. We trained recurrent neural networks that incorporated STP to produce complex motor trajectories-handwritten digits-with different temporal (speed) and spatial (size) scales. Neuromodulation of STP produced temporal and spatial scaling of the learned dynamics and enhanced temporal or spatial generalization compared to standard training of the synaptic weights in the absence of STP. The model also accounted for the results of two experimental studies involving flexible sensorimotor timing. Neuromodulation of STP provides a unified and biologically plausible mechanism to control the temporal and spatial scales of neural dynamics and sensorimotor behaviors.
Collapse
Affiliation(s)
- Shanglin Zhou
- Institute for Translational Brain Research, Fudan University, Shanghai, China
- State Key Laboratory of Medical Neurobiology, Fudan University, Shanghai, China
- MOE Frontiers Center for Brain Science, Fudan University, Shanghai, China
- Zhongshan Hospital, Fudan University, Shanghai, China
| | - Dean V Buonomano
- Department of Neurobiology, University of California, Los Angeles, Los Angeles, CA, USA
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| |
Collapse
|
12
|
Ferguson LA, Matamales M, Nolan C, Balleine BW, Bertran-Gonzalez J. Adaptation of sequential action benefits from timing variability related to lateral basal ganglia circuitry. iScience 2024; 27:109274. [PMID: 38496293 PMCID: PMC10943431 DOI: 10.1016/j.isci.2024.109274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 10/11/2023] [Accepted: 02/15/2024] [Indexed: 03/19/2024] Open
Abstract
Streamlined action sequences must remain flexible should stable contingencies in the environment change. By combining analyses of behavioral structure with a circuit-specific manipulation in mice, we report on a relationship between action timing variability and successful adaptation that relates to post-synaptic targets of primary motor cortical (M1) projections to dorsolateral striatum (DLS). In a two-lever instrumental task, mice formed successful action sequences by, first, establishing action scaffolds and, second, smoothly extending action duration to adapt to increased task requirements. Interruption of DLS neurons in M1 projection territories altered this process, evoking higher-rate actions that were more stereotyped in their timing, reducing opportunities for success. Based on evidence from neuronal tracing experiments, we propose that DLS neurons in M1 projection territories supply action timing variability to facilitate adaptation, a function that may involve additional downstream subcortical processing relating to collateralization of descending motor pathways to multiple basal ganglia centers.
Collapse
Affiliation(s)
- Lachlan A. Ferguson
- Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, NSW, Australia
| | - Miriam Matamales
- Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, NSW, Australia
| | - Christopher Nolan
- Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, NSW, Australia
| | - Bernard W. Balleine
- Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, NSW, Australia
| | - Jesus Bertran-Gonzalez
- Decision Neuroscience Laboratory, School of Psychology, University of New South Wales, Sydney, NSW, Australia
| |
Collapse
|
13
|
Subramanian DL, Smith DM. Time Cells in the Retrosplenial Cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.03.01.583039. [PMID: 38464235 PMCID: PMC10925311 DOI: 10.1101/2024.03.01.583039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
The retrosplenial cortex (RSC) is a key component of the brain's memory systems, with anatomical connections to the hippocampus, anterior thalamus, and entorhinal cortex. This circuit has been implicated in episodic memory and many of these structures have been shown to encode temporal information, which is critical for episodic memory. For example, hippocampal time cells reliably fire during specific segments of time during a delay period. Although RSC lesions are known to disrupt temporal memory, time cells have not been observed there. In the present study, we examined the firing patterns of RSC neurons during the intertrial delay period of two behavioral tasks, a blocked alternation task and a cued T-maze task. For the blocked alternation task, rats were required to approach the east or west arm of a plus maze for reward during different blocks of trials. Because the reward locations were not cued, the rat had to remember the goal location for each trial. In the cued T-maze task, the reward location was explicitly cued with a light and the rats simply had to approach the light for reward, so there was no requirement to hold a memory during the intertrial delay. Time cells were prevalent in the blocked alternation task, and most time cells clearly differentiated the east and west trials. We also found that RSC neurons could exhibit off-response time fields, periods of reliably inhibited firing. Time cells were also observed in the cued T-maze, but they were less prevalent and they did not differentiate left and right trials as well as in the blocked alternation task, suggesting that RSC time cells are sensitive to the memory demands of the task. These results suggest that temporal coding is a prominent feature of RSC firing patterns, consistent with an RSC role in episodic memory.
Collapse
Affiliation(s)
| | - David M. Smith
- Department of Psychology, Cornell University, Ithaca, NY 14853
| |
Collapse
|
14
|
Banerjee A, Chen F, Druckmann S, Long MA. Temporal scaling of motor cortical dynamics reveals hierarchical control of vocal production. Nat Neurosci 2024; 27:527-535. [PMID: 38291282 DOI: 10.1038/s41593-023-01556-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 12/13/2023] [Indexed: 02/01/2024]
Abstract
Neocortical activity is thought to mediate voluntary control over vocal production, but the underlying neural mechanisms remain unclear. In a highly vocal rodent, the male Alston's singing mouse, we investigate neural dynamics in the orofacial motor cortex (OMC), a structure critical for vocal behavior. We first describe neural activity that is modulated by component notes (~100 ms), probably representing sensory feedback. At longer timescales, however, OMC neurons exhibit diverse and often persistent premotor firing patterns that stretch or compress with song duration (~10 s). Using computational modeling, we demonstrate that such temporal scaling, acting through downstream motor production circuits, can enable vocal flexibility. These results provide a framework for studying hierarchical control circuits, a common design principle across many natural and artificial systems.
Collapse
Affiliation(s)
- Arkarup Banerjee
- NYU Neuroscience Institute, New York University Langone Health, New York, NY, USA.
- Department of Otolaryngology, New York University Langone Health, New York, NY, USA.
- Center for Neural Science, New York University, New York, NY, USA.
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA.
| | - Feng Chen
- Department of Applied Physics, Stanford University, Stanford, CA, USA
| | - Shaul Druckmann
- Department of Neurobiology, Stanford University, Stanford, CA, USA
| | - Michael A Long
- NYU Neuroscience Institute, New York University Langone Health, New York, NY, USA.
- Department of Otolaryngology, New York University Langone Health, New York, NY, USA.
- Center for Neural Science, New York University, New York, NY, USA.
| |
Collapse
|
15
|
Cao R, Bright IM, Howard MW. Ramping cells in rodent mPFC encode time to past and future events via real Laplace transform. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.13.580170. [PMID: 38405896 PMCID: PMC10888827 DOI: 10.1101/2024.02.13.580170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/27/2024]
Abstract
In interval reproduction tasks, animals must remember the event starting the interval and anticipate the time of the planned response to terminate the interval. The interval reproduction task thus allows for studying both memory for the past and anticipation of the future. We analyzed previously published recordings from rodent mPFC (Henke et al., 2021) during an interval reproduction task and identified two cell groups by modeling their temporal receptive fields using hierarchical Bayesian models. The firing in the "past cells" group peaked at the start of the interval and relaxed exponentially back to baseline. The firing in the "future cells" group increased exponentially and peaked right before the planned action at the end of the interval. Contrary to the previous assumption that timing information in the brain has one or two time scales for a given interval, we found strong evidence for a continuous distribution of the exponential rate constants for both past and future cell populations. The real Laplace transformation of time predicts exponential firing with a continuous distribution of rate constants across the population. Therefore, the firing pattern of the past cells can be identified with the Laplace transform of time since the past event while the firing pattern of the future cells can be identified with the Laplace transform of time until the planned future event.
Collapse
Affiliation(s)
- Rui Cao
- Department of Psychological and Brain Sciences, Boston University
| | - Ian M Bright
- Department of Psychological and Brain Sciences, Boston University
| | - Marc W Howard
- Department of Psychological and Brain Sciences, Boston University
| |
Collapse
|
16
|
Bigus ER, Lee HW, Bowler JC, Shi J, Heys JG. Medial entorhinal cortex plays a specialized role in learning of flexible, context-dependent interval timing behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.01.18.524598. [PMID: 38260332 PMCID: PMC10802491 DOI: 10.1101/2023.01.18.524598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Episodic memory requires encoding the temporal structure of experience and relies on brain circuits in the medial temporal lobe, including the medial entorhinal cortex (MEC). Recent studies have identified MEC 'time cells', which fire at specific moments during interval timing tasks, collectively tiling the entire timing period. It has been hypothesized that MEC time cells could provide temporal information necessary for episodic memories, yet it remains unknown whether MEC time cells display learning dynamics required for encoding different temporal contexts. To explore this, we developed a novel behavioral paradigm that requires distinguishing temporal contexts. Combined with methods for cellular resolution calcium imaging, we find that MEC time cells display context-dependent neural activity that emerges with task learning. Through chemogenetic inactivation we find that MEC activity is necessary for learning of context-dependent interval timing behavior. Finally, we find evidence of a common circuit mechanism that could drive sequential activity of both time cells and spatially selective neurons in MEC. Our work suggests that the clock-like firing of MEC time cells can be modulated by learning, allowing the tracking of various temporal structures that emerge through experience.
Collapse
|
17
|
de Lafuente V, Jazayeri M, Merchant H, García-Garibay O, Cadena-Valencia J, Malagón AM. Keeping time and rhythm by internal simulation of sensory stimuli and behavioral actions. SCIENCE ADVANCES 2024; 10:eadh8185. [PMID: 38198556 PMCID: PMC10780886 DOI: 10.1126/sciadv.adh8185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 12/11/2023] [Indexed: 01/12/2024]
Abstract
Effective behavior often requires synchronizing our actions with changes in the environment. Rhythmic changes in the environment are easy to predict, and we can readily time our actions to them. Yet, how the brain encodes and maintains rhythms is not known. Here, we trained primates to internally maintain rhythms of different tempos and performed large-scale recordings of neuronal activity across the sensory-motor hierarchy. Results show that maintaining rhythms engages multiple brain areas, including visual, parietal, premotor, prefrontal, and hippocampal regions. Each recorded area displayed oscillations in firing rates and oscillations in broadband local field potential power that reflected the temporal and spatial characteristics of an internal metronome, which flexibly encoded fast, medium, and slow tempos. The presence of widespread metronome-related activity, in the absence of stimuli and motor activity, suggests that internal simulation of stimuli and actions underlies timekeeping and rhythm maintenance.
Collapse
Affiliation(s)
- Victor de Lafuente
- Institute of Neurobiology, National Autonomous University of Mexico, Boulevard Juriquilla 3001, Querétaro, QRO 76230, México
| | - Mehrdad Jazayeri
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Hugo Merchant
- Institute of Neurobiology, National Autonomous University of Mexico, Boulevard Juriquilla 3001, Querétaro, QRO 76230, México
| | - Otto García-Garibay
- Institute of Neurobiology, National Autonomous University of Mexico, Boulevard Juriquilla 3001, Querétaro, QRO 76230, México
| | - Jaime Cadena-Valencia
- Institute of Neurobiology, National Autonomous University of Mexico, Boulevard Juriquilla 3001, Querétaro, QRO 76230, México
- Faculty of Science and Medicine, Department of Neurosciences and Movement Sciences, University of Fribourg, Fribourg 1700, Switzerland
- Cognitive Neuroscience Laboratory, German Primate Center—Leibniz Institute for Primate Research, Göttingen 37077, Germany
| | - Ana M. Malagón
- Institute of Neurobiology, National Autonomous University of Mexico, Boulevard Juriquilla 3001, Querétaro, QRO 76230, México
| |
Collapse
|
18
|
Rolando F, Kononowicz TW, Duhamel JR, Doyère V, Wirth S. Distinct neural adaptations to time demand in the striatum and the hippocampus. Curr Biol 2024; 34:156-170.e7. [PMID: 38141617 DOI: 10.1016/j.cub.2023.11.066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 10/18/2023] [Accepted: 11/30/2023] [Indexed: 12/25/2023]
Abstract
How do neural codes adjust to track time across a range of resolutions, from milliseconds to multi-seconds, as a function of the temporal frequency at which events occur? To address this question, we studied time-modulated cells in the striatum and the hippocampus, while macaques categorized three nested intervals within the sub-second or the supra-second range (up to 1, 2, 4, or 8 s), thereby modifying the temporal resolution needed to solve the task. Time-modulated cells carried more information for intervals with explicit timing demand, than for any other interval. The striatum, particularly the caudate, supported the most accurate temporal prediction throughout all time ranges. Strikingly, its temporal readout adjusted non-linearly to the time range, suggesting that the striatal resolution shifted from a precise millisecond to a coarse multi-second range as a function of demand. This is in line with monkey's behavioral latencies, which indicated that they tracked time until 2 s but employed a coarse categorization strategy for durations beyond. By contrast, the hippocampus discriminated only the beginning from the end of intervals, regardless of the range. We propose that the hippocampus may provide an overall poor signal marking an event's beginning, whereas the striatum optimizes neural resources to process time throughout an interval adapting to the ongoing timing necessity.
Collapse
Affiliation(s)
- Felipe Rolando
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, Université Lyon 1, 67 boulevard Pinel, 69500 Bron, France
| | - Tadeusz W Kononowicz
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, Université Lyon 1, 67 boulevard Pinel, 69500 Bron, France; Université Paris-Saclay, CNRS, Institut des Neurosciences Paris-Saclay (NeuroPSI), 91400 Saclay, France; Institute of Psychology, The Polish Academy of Sciences, ul. Jaracza 1, 00-378 Warsaw, Poland
| | - Jean-René Duhamel
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, Université Lyon 1, 67 boulevard Pinel, 69500 Bron, France
| | - Valérie Doyère
- Université Paris-Saclay, CNRS, Institut des Neurosciences Paris-Saclay (NeuroPSI), 91400 Saclay, France
| | - Sylvia Wirth
- Institut des Sciences Cognitives Marc Jeannerod, CNRS, Université Lyon 1, 67 boulevard Pinel, 69500 Bron, France.
| |
Collapse
|
19
|
Soldado-Magraner S, Buonomano DV. Neural Sequences and the Encoding of Time. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1455:81-93. [PMID: 38918347 DOI: 10.1007/978-3-031-60183-5_5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Abstract
Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.
Collapse
Affiliation(s)
| | - Dean V Buonomano
- Department of Neurobiology, University of California, Los Angeles, Los Angeles, CA, USA.
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
20
|
Rueda-Orozco PE, Hidalgo-Balbuena AE, González-Pereyra P, Martinez-Montalvo MG, Báez-Cordero AS. The Interactions of Temporal and Sensory Representations in the Basal Ganglia. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1455:141-158. [PMID: 38918350 DOI: 10.1007/978-3-031-60183-5_8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Abstract
In rodents and primates, interval estimation has been associated with a complex network of cortical and subcortical structures where the dorsal striatum plays a paramount role. Diverse evidence ranging from individual neurons to population activity has demonstrated that this area hosts temporal-related neural representations that may be instrumental for the perception and production of time intervals. However, little is known about how temporal representations interact with other well-known striatal representations, such as kinematic parameters of movements or somatosensory representations. An attractive hypothesis suggests that somatosensory representations may serve as the scaffold for complex representations such as elapsed time. Alternatively, these representations may coexist as independent streams of information that could be integrated into downstream nuclei, such as the substantia nigra or the globus pallidus. In this review, we will revise the available information suggesting an instrumental role of sensory representations in the construction of temporal representations at population and single-neuron levels throughout the basal ganglia.
Collapse
Affiliation(s)
- Pavel E Rueda-Orozco
- Institute of Neurobiology, National Autonomous University of México, Querétaro, Mexico.
| | | | | | | | - Ana S Báez-Cordero
- Institute of Neurobiology, National Autonomous University of México, Querétaro, Mexico
| |
Collapse
|
21
|
Balcı F, Simen P. Neurocomputational Models of Interval Timing: Seeing the Forest for the Trees. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1455:51-78. [PMID: 38918346 DOI: 10.1007/978-3-031-60183-5_4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Abstract
Extracting temporal regularities and relations from experience/observation is critical for organisms' adaptiveness (communication, foraging, predation, prediction) in their ecological niches. Therefore, it is not surprising that the internal clock that enables the perception of seconds-to-minutes-long intervals (interval timing) is evolutionarily well-preserved across many species of animals. This comparative claim is primarily supported by the fact that the timing behavior of many vertebrates exhibits common statistical signatures (e.g., on-average accuracy, scalar variability, positive skew). These ubiquitous statistical features of timing behaviors serve as empirical benchmarks for modelers in their efforts to unravel the processing dynamics of the internal clock (namely answering how internal clock "ticks"). In this chapter, we introduce prominent (neuro)computational approaches to modeling interval timing at a level that can be understood by general audience. These models include Treisman's pacemaker accumulator model, the information processing variant of scalar expectancy theory, the striatal beat frequency model, behavioral expectancy theory, the learning to time model, the time-adaptive opponent Poisson drift-diffusion model, time cell models, and neural trajectory models. Crucially, we discuss these models within an overarching conceptual framework that categorizes different models as threshold vs. clock-adaptive models and as dedicated clock/ramping vs. emergent time/population code models.
Collapse
Affiliation(s)
- Fuat Balcı
- Department of Biological Sciences, University of Manitoba, Winnipeg, MB, Canada.
| | - Patrick Simen
- Department of Neuroscience, Oberlin College, Oberlin, OH, USA
| |
Collapse
|
22
|
Ordás CM, Alonso-Frech F. The neural basis of somatosensory temporal discrimination threshold as a paradigm for time processing in the sub-second range: An updated review. Neurosci Biobehav Rev 2024; 156:105486. [PMID: 38040074 DOI: 10.1016/j.neubiorev.2023.105486] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 11/20/2023] [Accepted: 11/27/2023] [Indexed: 12/03/2023]
Abstract
BACKGROUND AND OBJECTIVE The temporal aspect of somesthesia is a feature of any somatosensory process and a pre-requisite for the elaboration of proper behavior. Time processing in the milliseconds range is crucial for most of behaviors in everyday life. The somatosensory temporal discrimination threshold (STDT) is the ability to perceive two successive stimuli as separate in time, and deals with time processing in this temporal range. Herein, we focus on the physiology of STDT, on a background of the anatomophysiology of somesthesia and the neurobiological substrates of timing. METHODS A review of the literature through PubMed & Cochrane databases until March 2023 was performed with inclusion and exclusion criteria following PRISMA recommendations. RESULTS 1151 abstracts were identified. 4 duplicate records were discarded before screening. 957 abstracts were excluded because of redundancy, less relevant content or not English-written. 4 were added after revision. Eventually, 194 articles were included. CONCLUSIONS STDT encoding relies on intracortical inhibitory S1 function and is modulated by the basal ganglia-thalamic-cortical interplay through circuits involving the nigrostriatal dopaminergic pathway and probably the superior colliculus.
Collapse
Affiliation(s)
- Carlos M Ordás
- Universidad Rey Juan Carlos, Móstoles, Madrid, Spain; Department of Neurology, Hospital Rey Juan Carlos, Móstoles, Madrid, Spain.
| | - Fernando Alonso-Frech
- Department of Neurology, Hospital Clínico San Carlos, Universidad Complutense de Madrid, Spain
| |
Collapse
|
23
|
Merchant H, Mendoza G, Pérez O, Betancourt A, García-Saldivar P, Prado L. Diverse Time Encoding Strategies Within the Medial Premotor Areas of the Primate. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1455:117-140. [PMID: 38918349 DOI: 10.1007/978-3-031-60183-5_7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Abstract
The measurement of time in the subsecond scale is critical for many sophisticated behaviors, yet its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the neural activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing. During single interval categorization, we found that preSMA encodes a subjective category limit by reaching a peak of activity at a time that divides the set of test intervals into short and long. We also observed neural signals associated with the category selected by the subjects and the reward outcomes of the perceptual decision. On the other hand, we have studied the behavioral and neurophysiological basis of rhythmic timing. First, we have shown in different tapping tasks that macaques are able to produce predictively and accurately intervals that are cued by auditory or visual metronomes or when intervals are produced internally without sensory guidance. In addition, we found that the rhythmic timing mechanism in MPC is governed by different layers of neural clocks. Next, the instantaneous activity of single cells shows ramping activity that encodes the elapsed or remaining time for a tapping movement. In addition, we found MPC neurons that build neural sequences, forming dynamic patterns of activation that flexibly cover all the produced interval depending on the tapping tempo. This rhythmic neural clock resets on every interval providing an internal representation of pulse. Furthermore, the MPC cells show mixed selectivity, encoding not only elapsed time, but also the tempo of the tapping and the serial order element in the rhythmic sequence. Hence, MPC can map different task parameters, including the passage of time, using different cell populations. Finally, the projection of the time varying activity of MPC hundreds of cells into a low dimensional state space showed circular neural trajectories whose geometry represented the internal pulse and the tapping tempo. Overall, these findings support the notion that MPC is part of the core timing mechanism for both single interval and rhythmic timing, using neural clocks with different encoding principles, probably to flexibly encode and mix the timing representation with other task parameters.
Collapse
Affiliation(s)
- Hugo Merchant
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro, Mexico.
| | - Germán Mendoza
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro, Mexico
| | - Oswaldo Pérez
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro, Mexico
| | | | | | - Luis Prado
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Querétaro, Mexico
| |
Collapse
|
24
|
Tanaka M, Kameda M, Okada KI. Temporal Information Processing in the Cerebellum and Basal Ganglia. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2024; 1455:95-116. [PMID: 38918348 DOI: 10.1007/978-3-031-60183-5_6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/27/2024]
Abstract
Temporal information processing in the range of a few hundred milliseconds to seconds involves the cerebellum and basal ganglia. In this chapter, we present recent studies on nonhuman primates. In the studies presented in the first half of the chapter, monkeys were trained to make eye movements when a certain amount of time had elapsed since the onset of the visual cue (time production task). The animals had to report time lapses ranging from several hundred milliseconds to a few seconds based on the color of the fixation point. In this task, the saccade latency varied with the time length to be measured and showed stochastic variability from one trial to the other. Trial-to-trial variability under the same conditions correlated well with pupil diameter and the preparatory activity in the deep cerebellar nuclei and the motor thalamus. Inactivation of these brain regions delayed saccades when asked to report subsecond intervals. These results suggest that the internal state, which changes with each trial, may cause fluctuations in cerebellar neuronal activity, thereby producing variations in self-timing. When measuring different time intervals, the preparatory activity in the cerebellum always begins approximately 500 ms before movements, regardless of the length of the time interval being measured. However, the preparatory activity in the striatum persists throughout the mandatory delay period, which can be up to 2 s, with different rate of increasing activity. Furthermore, in the striatum, the visual response and low-frequency oscillatory activity immediately before time measurement were altered by the length of the intended time interval. These results indicate that the state of the network, including the striatum, changes with the intended timing, which lead to different time courses of preparatory activity. Thus, the basal ganglia appear to be responsible for measuring time in the range of several hundred milliseconds to seconds, whereas the cerebellum is responsible for regulating self-timing variability in the subsecond range. The second half of this chapter presents studies related to periodic timing. During eye movements synchronized with alternating targets at regular intervals, different neurons in the cerebellar nuclei exhibit activity related to movement timing, predicted stimulus timing, and the temporal error of synchronization. Among these, the activity associated with target appearance is particularly enhanced during synchronized movements and may represent an internal model of the temporal structure of stimulus sequence. We also considered neural mechanism underlying the perception of periodic timing in the absence of movement. During perception of rhythm, we predict the timing of the next stimulus and focus our attention on that moment. In the missing oddball paradigm, the subjects had to detect the omission of a regularly repeated stimulus. When employed in humans, the results show that the fastest temporal limit for predicting each stimulus timing is about 0.25 s (4 Hz). In monkeys performing this task, neurons in the cerebellar nuclei, striatum, and motor thalamus exhibit periodic activity, with different time courses depending on the brain region. Since electrical stimulation or inactivation of recording sites changes the reaction time to stimulus omission, these neuronal activities must be involved in periodic temporal processing. Future research is needed to elucidate the mechanism of rhythm perception, which appears to be processed by both cortico-cerebellar and cortico-basal ganglia pathways.
Collapse
Affiliation(s)
- Masaki Tanaka
- Department of Physiology, Hokkaido University School of Medicine, Sapporo, Japan.
| | - Masashi Kameda
- Department of Physiology, Hokkaido University School of Medicine, Sapporo, Japan
| | - Ken-Ichi Okada
- Department of Physiology, Hokkaido University School of Medicine, Sapporo, Japan
| |
Collapse
|
25
|
Lin D, Huang AZ, Richards BA. Temporal encoding in deep reinforcement learning agents. Sci Rep 2023; 13:22335. [PMID: 38102369 PMCID: PMC10724179 DOI: 10.1038/s41598-023-49847-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Accepted: 12/12/2023] [Indexed: 12/17/2023] Open
Abstract
Neuroscientists have observed both cells in the brain that fire at specific points in time, known as "time cells", and cells whose activity steadily increases or decreases over time, known as "ramping cells". It is speculated that time and ramping cells support temporal computations in the brain and carry mnemonic information. However, due to the limitations in animal experiments, it is difficult to determine how these cells really contribute to behavior. Here, we show that time cells and ramping cells naturally emerge in the recurrent neural networks of deep reinforcement learning models performing simulated interval timing and working memory tasks, which have learned to estimate expected rewards in the future. We show that these cells do indeed carry information about time and items stored in working memory, but they contribute to behavior in large part by providing a dynamic representation on which policy can be computed. Moreover, the information that they do carry depends on both the task demands and the variables provided to the models. Our results suggest that time cells and ramping cells could contribute to temporal and mnemonic calculations, but the way in which they do so may be complex and unintuitive to human observers.
Collapse
Affiliation(s)
- Dongyan Lin
- Integrated Program in Neuroscience, McGill University, Montreal, QC, H3A 0G4, Canada.
- Mila, Montreal, QC, H2S 3H1, Canada.
| | - Ann Zixiang Huang
- Mila, Montreal, QC, H2S 3H1, Canada
- School of Computer Science, McGill University, Montreal, QC, H3A 0G4, Canada
| | - Blake Aaron Richards
- Integrated Program in Neuroscience, McGill University, Montreal, QC, H3A 0G4, Canada
- Mila, Montreal, QC, H2S 3H1, Canada
- School of Computer Science, McGill University, Montreal, QC, H3A 0G4, Canada
- Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC, H3A 0G4, Canada
- Learning in Machines and Brains Program, Canadian Institute for Advanced Research, Toronto, ON, M5G 1M1, Canada
| |
Collapse
|
26
|
Young ME, Spencer-Salmon C, Mosher C, Tamang S, Rajan K, Rudebeck PH. Temporally specific patterns of neural activity in interconnected corticolimbic structures during reward anticipation. Neuron 2023; 111:3668-3682.e5. [PMID: 37586366 PMCID: PMC10840822 DOI: 10.1016/j.neuron.2023.07.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Revised: 04/25/2023] [Accepted: 07/20/2023] [Indexed: 08/18/2023]
Abstract
Functional neuroimaging studies indicate that interconnected parts of the subcallosal anterior cingulate cortex (ACC), striatum, and amygdala play a fundamental role in affect in health and disease. Yet, although the patterns of neural activity engaged in the striatum and amygdala during affective processing are well established, especially during reward anticipation, less is known about subcallosal ACC. Here, we recorded neural activity in non-human primate subcallosal ACC and compared this with interconnected parts of the basolateral amygdala and rostromedial striatum while macaque monkeys performed reward-based tasks. Applying multiple analysis approaches, we found that neurons in subcallosal ACC and rostromedial striatum preferentially signal anticipated reward using short bursts of activity that form temporally specific patterns. By contrast, the basolateral amygdala uses a mixture of both temporally specific and more sustained patterns of activity to signal anticipated reward. Thus, dynamic patterns of neural activity across populations of neurons are engaged in affect, especially in subcallosal ACC.
Collapse
Affiliation(s)
- Megan E Young
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA
| | - Camille Spencer-Salmon
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA
| | - Clayton Mosher
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA
| | - Sarita Tamang
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA
| | - Kanaka Rajan
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA
| | - Peter H Rudebeck
- Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10029, USA.
| |
Collapse
|
27
|
Masset P, Tano P, Kim HR, Malik AN, Pouget A, Uchida N. Multi-timescale reinforcement learning in the brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.12.566754. [PMID: 38014166 PMCID: PMC10680596 DOI: 10.1101/2023.11.12.566754] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/29/2023]
Abstract
To thrive in complex environments, animals and artificial agents must learn to act adaptively to maximize fitness and rewards. Such adaptive behavior can be learned through reinforcement learning1, a class of algorithms that has been successful at training artificial agents2-6 and at characterizing the firing of dopamine neurons in the midbrain7-9. In classical reinforcement learning, agents discount future rewards exponentially according to a single time scale, controlled by the discount factor. Here, we explore the presence of multiple timescales in biological reinforcement learning. We first show that reinforcement agents learning at a multitude of timescales possess distinct computational benefits. Next, we report that dopamine neurons in mice performing two behavioral tasks encode reward prediction error with a diversity of discount time constants. Our model explains the heterogeneity of temporal discounting in both cue-evoked transient responses and slower timescale fluctuations known as dopamine ramps. Crucially, the measured discount factor of individual neurons is correlated across the two tasks suggesting that it is a cell-specific property. Together, our results provide a new paradigm to understand functional heterogeneity in dopamine neurons, a mechanistic basis for the empirical observation that humans and animals use non-exponential discounts in many situations10-14, and open new avenues for the design of more efficient reinforcement learning algorithms.
Collapse
Affiliation(s)
- Paul Masset
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
| | - Pablo Tano
- Department of Basic Neuroscience, University of Geneva, Switzerland
| | - HyungGoo R. Kim
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
- Center for Neuroscience Imaging Research, Institute for Basic Science (IBS), Suwon 16419, Republic of Korea
| | - Athar N. Malik
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
- Department of Neurosurgery, Warren Alpert Medical School of Brown University, USA
- Norman Prince Neurosciences Institute, Rhode Island Hospital, USA
| | - Alexandre Pouget
- Department of Basic Neuroscience, University of Geneva, Switzerland
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Harvard University, USA
- Center for Brain Science, Harvard University, USA
| |
Collapse
|
28
|
Buonomano DV, Buzsáki G, Davachi L, Nobre AC. Time for Memories. J Neurosci 2023; 43:7565-7574. [PMID: 37940593 PMCID: PMC10634580 DOI: 10.1523/jneurosci.1430-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 08/22/2023] [Accepted: 08/25/2023] [Indexed: 11/10/2023] Open
Abstract
The ability to store information about the past to dynamically predict and prepare for the future is among the most fundamental tasks the brain performs. To date, the problems of understanding how the brain stores and organizes information about the past (memory) and how the brain represents and processes temporal information for adaptive behavior have generally been studied as distinct cognitive functions. This Symposium explores the inherent link between memory and temporal cognition, as well as the potential shared neural mechanisms between them. We suggest that working memory and implicit timing are interconnected and may share overlapping neural mechanisms. Additionally, we explore how temporal structure is encoded in associative and episodic memory and, conversely, the influences of episodic memory on subsequent temporal anticipation and the perception of time. We suggest that neural sequences provide a general computational motif that contributes to timing and working memory, as well as the spatiotemporal coding and recall of episodes.
Collapse
Affiliation(s)
- Dean V Buonomano
- Department of Neurobiology, University of California, Los Angeles, California 90095
- Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095
- Integrative Center for Learning and Memory, UCLA, Los Angeles, California 90025
| | - György Buzsáki
- Neuroscience Institute and Department of Neurology, NYU Grossman School of Medicine, New York University, New York, New York 10016
- Center for Neural Science, New York University, New York, New York 10003
| | - Lila Davachi
- Department of Psychology, Columbia University, New York, New York 10027
- Center for Clinical Research, Nathan S. Kline Institute for Psychiatric Research, Orangeburg, New York 10962
| | - Anna C Nobre
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
- Oxford Centre for Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford OX3 7JX, United Kingdom
- Department of Psychology, Yale University, New Haven, Connecticut 06510
- Wu Tsai Center for Neurocognition and Behavior, Wu Tsai Institute, Yale University, New Haven, Connecticut 06510
| |
Collapse
|
29
|
Wilhelm M, Sych Y, Fomins A, Alatorre Warren JL, Lewis C, Serratosa Capdevila L, Boehringer R, Amadei EA, Grewe B, O'Connor EC, Hall BJ, Helmchen F. Striatum-projecting prefrontal cortex neurons support working memory maintenance. Nat Commun 2023; 14:7016. [PMID: 37919287 PMCID: PMC10622437 DOI: 10.1038/s41467-023-42777-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Accepted: 10/20/2023] [Indexed: 11/04/2023] Open
Abstract
Neurons in the medial prefrontal cortex (mPFC) are functionally linked to working memory (WM) but how distinct projection pathways contribute to WM remains unclear. Based on optical recordings, optogenetic perturbations, and pharmacological interventions in male mice, we report here that dorsomedial striatum (dmStr)-projecting mPFC neurons are essential for WM maintenance, but not encoding or retrieval, in a T-maze spatial memory task. Fiber photometry of GCaMP6m-labeled mPFC→dmStr neurons revealed strongest activity during the maintenance period, and optogenetic inhibition of these neurons impaired performance only when applied during this period. Conversely, enhancing mPFC→dmStr pathway activity-via pharmacological suppression of HCN1 or by optogenetic activation during the maintenance period-alleviated WM impairment induced by NMDA receptor blockade. Moreover, cellular-resolution miniscope imaging revealed that >50% of mPFC→dmStr neurons are active during WM maintenance and that this subpopulation is distinct from neurons active during encoding and retrieval. In all task periods, neuronal sequences were evident. Striatum-projecting mPFC neurons thus critically contribute to spatial WM maintenance.
Collapse
Affiliation(s)
- Maria Wilhelm
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
- Institute for Neuroscience, ETH Zurich, 8057, Zurich, Switzerland
| | - Yaroslav Sych
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland
- Institute of Cellular and Integrative Neuroscience, CNRS, University of Strasbourg, Strasbourg, France
| | - Aleksejs Fomins
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
| | - José Luis Alatorre Warren
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland
- Center for Lifespan Changes in Brain and Cognition, University of Oslo, Oslo, 0317, Norway
| | - Christopher Lewis
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland
| | | | - Roman Boehringer
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
| | - Elizabeth A Amadei
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
| | - Benjamin Grewe
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland
- University Research Priority Program (URPP) Adaptive Brain Circuits in Development and Learning (AdaBD), University of Zurich, Zurich, Switzerland
| | - Eoin C O'Connor
- Neuroscience & Rare Diseases, Roche Pharma Research and Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | - Benjamin J Hall
- Neuroscience & Rare Diseases, Roche Pharma Research and Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland
- Circuit Biology Department, H. Lundbeck A/S, Valby, Denmark
| | - Fritjof Helmchen
- Brain Research Institute, University of Zurich, 8057, Zurich, Switzerland.
- Neuroscience Center Zurich, University of Zurich and ETH Zurich, 8057, Zurich, Switzerland.
- University Research Priority Program (URPP) Adaptive Brain Circuits in Development and Learning (AdaBD), University of Zurich, Zurich, Switzerland.
| |
Collapse
|
30
|
Betancourt A, Pérez O, Gámez J, Mendoza G, Merchant H. Amodal population clock in the primate medial premotor system for rhythmic tapping. Cell Rep 2023; 42:113234. [PMID: 37838944 DOI: 10.1016/j.celrep.2023.113234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 08/09/2023] [Accepted: 09/24/2023] [Indexed: 10/17/2023] Open
Abstract
The neural substrate for beat extraction and response entrainment to rhythms is not fully understood. Here we analyze the activity of medial premotor neurons in monkeys performing isochronous tapping guided by brief flashing stimuli or auditory tones. The population dynamics shared the following properties across modalities: the circular dynamics of the neural trajectories form a regenerating loop for every produced interval; the trajectories converge in similar state space at tapping times resetting the clock; and the tempo of the synchronized tapping is encoded in the trajectories by a combination of amplitude modulation and temporal scaling. Notably, the modality induces displacement in the neural trajectories in the auditory and visual subspaces without greatly altering the time-keeping mechanism. These results suggest that the interaction between the medial premotor cortex's amodal internal representation of pulse and a modality-specific external input generates a neural rhythmic clock whose dynamics govern rhythmic tapping execution across senses.
Collapse
Affiliation(s)
- Abraham Betancourt
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Boulevard Juriquilla No. 3001, Querétaro, Qro 76230, México
| | - Oswaldo Pérez
- Escuela Nacional de Estudios Superiores, Unidad Juriquilla, UNAM, Boulevard Juriquilla No. 3001, Querétaro, Qro 76230, México
| | - Jorge Gámez
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Boulevard Juriquilla No. 3001, Querétaro, Qro 76230, México
| | - Germán Mendoza
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Boulevard Juriquilla No. 3001, Querétaro, Qro 76230, México
| | - Hugo Merchant
- Instituto de Neurobiología, UNAM, Campus Juriquilla, Boulevard Juriquilla No. 3001, Querétaro, Qro 76230, México.
| |
Collapse
|
31
|
Mizes KGC, Lindsey J, Escola GS, Ölveczky BP. Dissociating the contributions of sensorimotor striatum to automatic and visually guided motor sequences. Nat Neurosci 2023; 26:1791-1804. [PMID: 37667040 PMCID: PMC11187818 DOI: 10.1038/s41593-023-01431-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 08/14/2023] [Indexed: 09/06/2023]
Abstract
The ability to sequence movements in response to new task demands enables rich and adaptive behavior. However, such flexibility is computationally costly and can result in halting performances. Practicing the same motor sequence repeatedly can render its execution precise, fast and effortless, that is, 'automatic'. The basal ganglia are thought to underlie both types of sequence execution, yet whether and how their contributions differ is unclear. We parse this in rats trained to perform the same motor sequence instructed by cues and in a self-initiated overtrained, or 'automatic,' condition. Neural recordings in the sensorimotor striatum revealed a kinematic code independent of the execution mode. Although lesions reduced the movement speed and affected detailed kinematics similarly, they disrupted high-level sequence structure for automatic, but not visually guided, behaviors. These results suggest that the basal ganglia are essential for 'automatic' motor skills that are defined in terms of continuous kinematics, but can be dispensable for discrete motor sequences guided by sensory cues.
Collapse
Affiliation(s)
- Kevin G C Mizes
- Program in Biophysics, Harvard University, Cambridge, MA, USA
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Jack Lindsey
- Zuckerman Mind Brain and Behavior Institute, Columbia University, New York City, NY, USA
| | - G Sean Escola
- Zuckerman Mind Brain and Behavior Institute, Columbia University, New York City, NY, USA
- Department of Psychiatry, Columbia University, New York City, NY, USA
| | - Bence P Ölveczky
- Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
32
|
Robbe D. Lost in time: Relocating the perception of duration outside the brain. Neurosci Biobehav Rev 2023; 153:105312. [PMID: 37467906 DOI: 10.1016/j.neubiorev.2023.105312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 07/08/2023] [Indexed: 07/21/2023]
Abstract
It is well-accepted in neuroscience that animals process time internally to estimate the duration of intervals lasting between one and several seconds. More than 100 years ago, Henri Bergson nevertheless remarked that, because animals have memory, their inner experience of time is ever-changing, making duration impossible to measure internally and time a source of change. Bergson proposed that quantifying the inner experience of time requires its externalization in movements (observed or self-generated), as their unfolding leaves measurable traces in space. Here, studies across species are reviewed and collectively suggest that, in line with Bergson's ideas, animals spontaneously solve time estimation tasks through a movement-based spatialization of time. Moreover, the well-known scalable anticipatory responses of animals to regularly spaced rewards can be explained by the variable pressure of time on reward-oriented actions. Finally, the brain regions linked with time perception overlap with those implicated in motor control, spatial navigation and motivation. Thus, instead of considering time as static information processed by the brain, it might be fruitful to conceptualize it as a kind of force to which animals are more or less sensitive depending on their internal state and environment.
Collapse
Affiliation(s)
- David Robbe
- Institut de Neurobiologie de la Méditerranée (INMED), INSERM, Marseille, France; Aix-Marseille Université, Marseille, France.
| |
Collapse
|
33
|
Hennig JA, Romero Pinto SA, Yamaguchi T, Linderman SW, Uchida N, Gershman SJ. Emergence of belief-like representations through reinforcement learning. PLoS Comput Biol 2023; 19:e1011067. [PMID: 37695776 PMCID: PMC10513382 DOI: 10.1371/journal.pcbi.1011067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 09/21/2023] [Accepted: 08/27/2023] [Indexed: 09/13/2023] Open
Abstract
To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.
Collapse
Affiliation(s)
- Jay A. Hennig
- Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America
- Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
| | - Sandra A. Romero Pinto
- Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, Massachusetts, United States of America
- Program in Speech and Hearing Bioscience and Technology, Harvard Medical School, Boston, Massachusetts, USA
| | - Takahiro Yamaguchi
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, Massachusetts, United States of America
- Future Research Department, Toyota Research Institute of North America, Toyota Motor North America, Ann Arbor, Michigan, United States of America
| | - Scott W. Linderman
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, California, United States of America
- Department of Statistics, Stanford University, Stanford, California, United States of America
| | - Naoshige Uchida
- Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, Massachusetts, United States of America
| | - Samuel J. Gershman
- Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America
- Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
| |
Collapse
|
34
|
Zhang Y, Shi K, Luo X, Chen Y, Wang Y, Qu H. A biologically inspired auto-associative network with sparse temporal population coding. Neural Netw 2023; 166:670-682. [PMID: 37604076 DOI: 10.1016/j.neunet.2023.07.040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 06/25/2023] [Accepted: 07/26/2023] [Indexed: 08/23/2023]
Abstract
Associative system has attracted increasing attention for it can store basic information and then infer details to match perception with an efficient self-organization algorithm. However, the implementation of the associative system with the application of real-world data is relatively difficult. To address this issue, we propose a novel biologically inspired auto-associative (BIAA) network to explore the structure, encoding and formation of associative memory as well as to extend the ability to real-world application. Our network is constructed by imitating the organization of the cortical minicolumns where each minicolumn contains plenty of parallel biological spiking neurons. To allow the network to learn and predict one symbol per theta cycle, we incorporate synaptic delay and theta oscillation into the neuron dynamic process. Subsequently, we design a sparse temporal population (STP) coding scheme that allows each input symbol to be represented as stable, unique, and easily recallable sparsely distributed representations. By combining associative learning dynamics with the STP coding, our network realizes efficient storage and inference in an ordered manner. Experimental results indicate that the proposed network successfully performs sequence retrieval from partial text and sequence recovery from distorted information. BIAA network provides new insight into introducing biologically inspired mechanisms into associative system and has enormous potential for hardware and software applications.
Collapse
Affiliation(s)
- Ya Zhang
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China
| | - Kexin Shi
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China
| | - Xiaoling Luo
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China
| | - Yi Chen
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China
| | - Yucheng Wang
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China
| | - Hong Qu
- Department of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, PR China.
| |
Collapse
|
35
|
Monteiro T, Rodrigues FS, Pexirra M, Cruz BF, Gonçalves AI, Rueda-Orozco PE, Paton JJ. Using temperature to analyze the neural basis of a time-based decision. Nat Neurosci 2023; 26:1407-1416. [PMID: 37443279 DOI: 10.1038/s41593-023-01378-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 06/12/2023] [Indexed: 07/15/2023]
Abstract
The basal ganglia are thought to contribute to decision-making and motor control. These functions are critically dependent on timing information, which can be extracted from the evolving state of neural populations in their main input structure, the striatum. However, it is debated whether striatal activity underlies latent, dynamic decision processes or kinematics of overt movement. Here, we measured the impact of temperature on striatal population activity and the behavior of rats, and compared the observed effects with neural activity and behavior collected in multiple versions of a temporal categorization task. Cooling caused dilation, and warming contraction, of both neural activity and patterns of judgment in time, mimicking endogenous decision-related variability in striatal activity. However, temperature did not similarly affect movement kinematics. These data provide compelling evidence that the timecourse of evolving striatal activity dictates the speed of a latent process that is used to guide choices, but not continuous motor control. More broadly, they establish temporal scaling of population activity as a likely neural basis for variability in timing behavior.
Collapse
Affiliation(s)
- Tiago Monteiro
- Neuroscience Programme, Champalimaud Foundation, Lisbon, Portugal
- Department of Biology, University of Oxford, Oxford, UK
| | | | - Margarida Pexirra
- Neuroscience Programme, Champalimaud Foundation, Lisbon, Portugal
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK
| | - Bruno F Cruz
- Neuroscience Programme, Champalimaud Foundation, Lisbon, Portugal
- NeuroGEARS Ltd., London, UK
| | - Ana I Gonçalves
- Neuroscience Programme, Champalimaud Foundation, Lisbon, Portugal
| | | | - Joseph J Paton
- Neuroscience Programme, Champalimaud Foundation, Lisbon, Portugal.
| |
Collapse
|
36
|
Levcik D, Sugi AH, Aguilar-Rivera M, Pochapski JA, Baltazar G, Pulido LN, Villas-Boas CA, Fuentes-Flores R, Nicola SM, Da Cunha C. Nucleus Accumbens Shell Neurons Encode the Kinematics of Reward Approach Locomotion. Neuroscience 2023; 524:181-196. [PMID: 37330195 PMCID: PMC10527230 DOI: 10.1016/j.neuroscience.2023.06.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/30/2023] [Accepted: 06/02/2023] [Indexed: 06/19/2023]
Abstract
The nucleus accumbens (NAc) is considered an interface between motivation and action, with NAc neurons playing an important role in promoting reward approach. However, the encoding by NAc neurons that contributes to this role remains unknown. We recorded 62 NAc neurons in male Wistar rats (n = 5) running towards rewarded locations in an 8-arm radial maze. Variables related to locomotor approach kinematics were the best predictors of the firing rate for most NAc neurons. Nearly 18% of the recorded neurons were inhibited during the entire approach run (locomotion-off cells), suggesting that reduction in firing of these neurons promotes initiation of locomotor approach. 27% of the neurons presented a peak of activity during acceleration followed by a valley during deceleration (acceleration-on cells). Together, these neurons accounted for most of the speed and acceleration encoding identified in our analysis. In contrast, a further 16% of neurons presented a valley during acceleration followed by a peak just prior to or after reaching reward (deceleration-on cells). These findings suggest that these three classes of NAc neurons influence the time course of speed changes during locomotor approach to reward.
Collapse
Affiliation(s)
- David Levcik
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Institute of Physiology of the Czech Academy of Sciences, Videnska 1083, 142 20 Prague, Czech Republic
| | - Adam H Sugi
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Marcelo Aguilar-Rivera
- Department of Bioengineering, University of California, 9500 Gilman Drive MC 0412, La Jolla, San Diego 92093, USA
| | - José A Pochapski
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Gabriel Baltazar
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil
| | - Laura N Pulido
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil
| | - Cyrus A Villas-Boas
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil
| | - Romulo Fuentes-Flores
- Departamento de Neurociencia, Facultad de Medicina, Universidad de Chile, Av. Independencia 1027, Independencia 8380453, Santiago, Chile
| | - Saleem M Nicola
- Department of Neuroscience, Albert Einstein College of Medicine, 1300 Morris Park Ave, Bronx, NY 10461, USA; Department of Psychiatry, Albert Einstein College of Medicine, New York, USA
| | - Claudio Da Cunha
- Laboratório de Fisiologia e Farmacologia do Sistema Nervoso Central, Universidade Federal do Paraná, 81531-980 Curitiba, Brazil; Department of Pharmacology, Universidade Federal do Paraná, Curitiba, Brazil; Department of Biochemistry, Universidade Federal do Paraná, Curitiba, Brazil.
| |
Collapse
|
37
|
Petroccione MA, D'Brant LY, Affinnih N, Wehrle PH, Todd GC, Zahid S, Chesbro HE, Tschang IL, Scimemi A. Neuronal glutamate transporters control reciprocal inhibition and gain modulation in D1 medium spiny neurons. eLife 2023; 12:e81830. [PMID: 37435808 PMCID: PMC10411972 DOI: 10.7554/elife.81830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 07/09/2023] [Indexed: 07/13/2023] Open
Abstract
Understanding the function of glutamate transporters has broad implications for explaining how neurons integrate information and relay it through complex neuronal circuits. Most of what is currently known about glutamate transporters, specifically their ability to maintain glutamate homeostasis and limit glutamate diffusion away from the synaptic cleft, is based on studies of glial glutamate transporters. By contrast, little is known about the functional implications of neuronal glutamate transporters. The neuronal glutamate transporter EAAC1 is widely expressed throughout the brain, particularly in the striatum, the primary input nucleus of the basal ganglia, a region implicated with movement execution and reward. Here, we show that EAAC1 limits synaptic excitation onto a population of striatal medium spiny neurons identified for their expression of D1 dopamine receptors (D1-MSNs). In these cells, EAAC1 also contributes to strengthen lateral inhibition from other D1-MSNs. Together, these effects contribute to reduce the gain of the input-output relationship and increase the offset at increasing levels of synaptic inhibition in D1-MSNs. By reducing the sensitivity and dynamic range of action potential firing in D1-MSNs, EAAC1 limits the propensity of mice to rapidly switch between behaviors associated with different reward probabilities. Together, these findings shed light on some important molecular and cellular mechanisms implicated with behavior flexibility in mice.
Collapse
Affiliation(s)
| | | | | | | | | | - Shergil Zahid
- SUNY Albany, Department of BiologyAlbanyUnited States
| | | | - Ian L Tschang
- SUNY Albany, Department of BiologyAlbanyUnited States
| | | |
Collapse
|
38
|
Cazettes F, Mazzucato L, Murakami M, Morais JP, Augusto E, Renart A, Mainen ZF. A reservoir of foraging decision variables in the mouse brain. Nat Neurosci 2023; 26:840-849. [PMID: 37055628 PMCID: PMC10280691 DOI: 10.1038/s41593-023-01305-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 03/15/2023] [Indexed: 04/15/2023]
Abstract
In any given situation, the environment can be parsed in different ways to yield decision variables (DVs) defining strategies useful for different tasks. It is generally presumed that the brain only computes a single DV defining the current behavioral strategy. Here to test this assumption, we recorded neural ensembles in the frontal cortex of mice performing a foraging task admitting multiple DVs. Methods developed to uncover the currently employed DV revealed the use of multiple strategies and occasional switches in strategy within sessions. Optogenetic manipulations showed that the secondary motor cortex (M2) is needed for mice to use the different DVs in the task. Surprisingly, we found that regardless of which DV best explained the current behavior, M2 activity concurrently encoded a full basis set of computations defining a reservoir of DVs appropriate for alternative tasks. This form of neural multiplexing may confer considerable advantages for learning and adaptive behavior.
Collapse
Affiliation(s)
| | - Luca Mazzucato
- Departments of Biology, Mathematics & Physics, Institute of Neuroscience, University of Oregon, Eugene, OR, USA
| | - Masayoshi Murakami
- Champalimaud Foundation, Lisbon, Portugal
- Department of Neurophysiology, University of Yamanashi, Yamanashi, Japan
| | | | | | | | | |
Collapse
|
39
|
Disse GD, Nandakumar B, Pauzin FP, Blumenthal GH, Kong Z, Ditterich J, Moxon KA. Neural ensemble dynamics in trunk and hindlimb sensorimotor cortex encode for the control of postural stability. Cell Rep 2023; 42:112347. [PMID: 37027302 DOI: 10.1016/j.celrep.2023.112347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Revised: 02/09/2023] [Accepted: 03/21/2023] [Indexed: 04/08/2023] Open
Abstract
The cortex has a disputed role in monitoring postural equilibrium and intervening in cases of major postural disturbances. Here, we investigate the patterns of neural activity in the cortex that underlie neural dynamics during unexpected perturbations. In both the primary sensory (S1) and motor (M1) cortices of the rat, unique neuronal classes differentially covary their responses to distinguish different characteristics of applied postural perturbations; however, there is substantial information gain in M1, demonstrating a role for higher-order computations in motor control. A dynamical systems model of M1 activity and forces generated by the limbs reveals that these neuronal classes contribute to a low-dimensional manifold comprised of separate subspaces enabled by congruent and incongruent neural firing patterns that define different computations depending on the postural responses. These results inform how the cortex engages in postural control, directing work aiming to understand postural instability after neurological disease.
Collapse
Affiliation(s)
- Gregory D Disse
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA
| | | | - Francois P Pauzin
- Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA
| | - Gary H Blumenthal
- School of Biomedical Engineering Science and Health Systems, Drexel University, Philadelphia, PA 19104, USA
| | - Zhaodan Kong
- Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA 95616, USA
| | - Jochen Ditterich
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Neurobiology, Physiology and Behavior, University of California, Davis, Davis, CA 95616, USA
| | - Karen A Moxon
- Neuroscience Graduate Group, University of California, Davis, Davis, CA 95616, USA; Biomedical Engineering, University of California, Davis, Davis, CA 95616, USA.
| |
Collapse
|
40
|
Hennig JA, Pinto SAR, Yamaguchi T, Linderman SW, Uchida N, Gershman SJ. Emergence of belief-like representations through reinforcement learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.04.535512. [PMID: 37066383 PMCID: PMC10104054 DOI: 10.1101/2023.04.04.535512] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/18/2023]
Abstract
To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity. Author Summary Natural environments are full of uncertainty. For example, just because my fridge had food in it yesterday does not mean it will have food today. Despite such uncertainty, animals can estimate which states and actions are the most valuable. Previous work suggests that animals estimate value using a brain area called the basal ganglia, using a process resembling a reinforcement learning algorithm called TD learning. However, traditional reinforcement learning algorithms cannot accurately estimate value in environments with state uncertainty (e.g., when my fridge's contents are unknown). One way around this problem is if agents form "beliefs," a probabilistic estimate of how likely each state is, given any observations so far. However, estimating beliefs is a demanding process that may not be possible for animals in more complex environments. Here we show that an artificial recurrent neural network (RNN) trained with TD learning can estimate value from observations, without explicitly estimating beliefs. The trained RNN's error signals resembled the neural activity of dopamine neurons measured during the same task. Importantly, the RNN's activity resembled beliefs, but only when the RNN had enough capacity. This work illustrates how animals could estimate value in uncertain environments without needing to first form beliefs, which may be useful in environments where computing the true beliefs is too costly.
Collapse
Affiliation(s)
- Jay A. Hennig
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Sandra A. Romero Pinto
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Takahiro Yamaguchi
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Future Vehicle Research Department, Toyota Research Institute North America, Toyota Motor North America Inc., Ann Arbor, MI, USA
| | - Scott W. Linderman
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Statistics, Stanford University, Stanford, CA, USA
| | - Naoshige Uchida
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Samuel J. Gershman
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Center for Brain Science, Harvard University, Cambridge, MA, USA
| |
Collapse
|
41
|
Xie T, Huang C, Zhang Y, Liu J, Yao H. Influence of Recent Trial History on Interval Timing. Neurosci Bull 2023; 39:559-575. [PMID: 36209314 PMCID: PMC10073370 DOI: 10.1007/s12264-022-00954-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2022] [Accepted: 07/10/2022] [Indexed: 11/30/2022] Open
Abstract
Interval timing is involved in a variety of cognitive behaviors such as associative learning and decision-making. While it has been shown that time estimation is adaptive to the temporal context, it remains unclear how interval timing behavior is influenced by recent trial history. Here we found that, in mice trained to perform a licking-based interval timing task, a decrease of inter-reinforcement interval in the previous trial rapidly shifted the time of anticipatory licking earlier. Optogenetic inactivation of the anterior lateral motor cortex (ALM), but not the medial prefrontal cortex, for a short time before reward delivery caused a decrease in the peak time of anticipatory licking in the next trial. Electrophysiological recordings from the ALM showed that the response profiles preceded by short and long inter-reinforcement intervals exhibited task-engagement-dependent temporal scaling. Thus, interval timing is adaptive to recent experience of the temporal interval, and ALM activity during time estimation reflects recent experience of interval.
Collapse
Affiliation(s)
- Taorong Xie
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
| | - Can Huang
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Yijie Zhang
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Jing Liu
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China
- University of Chinese Academy of Sciences, Beijing, 100049, China
| | - Haishan Yao
- Institute of Neuroscience, State Key Laboratory of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China.
- Shanghai Center for Brain Science and Brain-Inspired Intelligence Technology, Shanghai, 201210, China.
| |
Collapse
|
42
|
Howard C, Simen P. A Time to Remember: Neural Insights into Rapid Updating of Timed Behaviors. Neurosci Bull 2023; 39:699-702. [PMID: 36525232 PMCID: PMC10073378 DOI: 10.1007/s12264-022-00999-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 11/06/2022] [Indexed: 12/23/2022] Open
Affiliation(s)
| | - Patrick Simen
- Neuroscience Department, Oberlin College, Oberlin, OH, 44074, USA
| |
Collapse
|
43
|
Sawatani F, Ide K, Takahashi S. The neural representation of time distributed across multiple brain regions differs between implicit and explicit time demands. Neurobiol Learn Mem 2023; 199:107731. [PMID: 36764645 DOI: 10.1016/j.nlm.2023.107731] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Revised: 01/19/2023] [Accepted: 02/04/2023] [Indexed: 02/10/2023]
Abstract
Animals appear to possess an internal timer during action, based on the passage of time. However, the neural underpinnings of the perception of time, ranging from seconds to minutes, remain unclear. Herein, we considered the neural representation of time based on mounting evidence on the neural correlates of time perception. The passage of time in the brain is represented by two types of neural encoding as follows: (i) the modulation of firing rates in single neurons and (ii) the sequential activity in neural ensembles. Time-dependent neural activity reflects the relative time rather than the absolute time, similar to a clock. They emerge in multiple regions, including the hippocampus, medial and lateral entorhinal cortices, medial prefrontal cortex, and dorsal striatum. Moreover, they involve different brain regions, depending on an implicit or explicit event duration. Thus, the two types of internal timers distributed across multiple brain regions simultaneously engage in time perception, in response to implicit or explicit time demands.
Collapse
Affiliation(s)
- Fumiya Sawatani
- Laboratory of Cognitive and Behavioral Neuroscience, Graduate School of Brain Science, Doshisha University, Kyotanabe City, Kyoto 610-0394, Japan.
| | - Kaoru Ide
- Laboratory of Cognitive and Behavioral Neuroscience, Graduate School of Brain Science, Doshisha University, Kyotanabe City, Kyoto 610-0394, Japan
| | - Susumu Takahashi
- Laboratory of Cognitive and Behavioral Neuroscience, Graduate School of Brain Science, Doshisha University, Kyotanabe City, Kyoto 610-0394, Japan.
| |
Collapse
|
44
|
Beiran M, Meirhaeghe N, Sohn H, Jazayeri M, Ostojic S. Parametric control of flexible timing through low-dimensional neural manifolds. Neuron 2023; 111:739-753.e8. [PMID: 36640766 PMCID: PMC9992137 DOI: 10.1016/j.neuron.2022.12.016] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2021] [Revised: 09/23/2022] [Accepted: 12/08/2022] [Indexed: 01/15/2023]
Abstract
Biological brains possess an unparalleled ability to adapt behavioral responses to changing stimuli and environments. How neural processes enable this capacity is a fundamental open question. Previous works have identified two candidate mechanisms: a low-dimensional organization of neural activity and a modulation by contextual inputs. We hypothesized that combining the two might facilitate generalization and adaptation in complex tasks. We tested this hypothesis in flexible timing tasks where dynamics play a key role. Examining trained recurrent neural networks, we found that confining the dynamics to a low-dimensional subspace allowed tonic inputs to parametrically control the overall input-output transform, enabling generalization to novel inputs and adaptation to changing conditions. Reverse-engineering and theoretical analyses demonstrated that this parametric control relies on a mechanism where tonic inputs modulate the dynamics along non-linear manifolds while preserving their geometry. Comparisons with data from behaving monkeys confirmed the behavioral and neural signatures of this mechanism.
Collapse
Affiliation(s)
- Manuel Beiran
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Ecole Normale Superieure - PSL University, 75005 Paris, France; Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA
| | - Nicolas Meirhaeghe
- Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Institut de Neurosciences de la Timone (INT), UMR 7289, CNRS, Aix-Marseille Université, Marseille 13005, France
| | - Hansem Sohn
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Mehrdad Jazayeri
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Srdjan Ostojic
- Laboratoire de Neurosciences Cognitives et Computationnelles, INSERM U960, Ecole Normale Superieure - PSL University, 75005 Paris, France.
| |
Collapse
|
45
|
Banerjee A, Chen F, Druckmann S, Long MA. Neural dynamics in the rodent motor cortex enables flexible control of vocal timing. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.23.525252. [PMID: 36747850 PMCID: PMC9900850 DOI: 10.1101/2023.01.23.525252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Neocortical activity is thought to mediate voluntary control over vocal production, but the underlying neural mechanisms remain unclear. In a highly vocal rodent, the Alston's singing mouse, we investigate neural dynamics in the orofacial motor cortex (OMC), a structure critical for vocal behavior. We first describe neural activity that is modulated by component notes (approx. 100 ms), likely representing sensory feedback. At longer timescales, however, OMC neurons exhibit diverse and often persistent premotor firing patterns that stretch or compress with song duration (approx. 10 s). Using computational modeling, we demonstrate that such temporal scaling, acting via downstream motor production circuits, can enable vocal flexibility. These results provide a framework for studying hierarchical control circuits, a common design principle across many natural and artificial systems.
Collapse
Affiliation(s)
- Arkarup Banerjee
- NYU Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA
- Department of Otolaryngology, New York University Langone Health, New York, NY 10016, USA
- Center for Neural Science, New York University, New York, NY 10003, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
| | - Feng Chen
- Department of Applied Physics, Stanford University, Stanford, CA 94305, USA
| | - Shaul Druckmann
- Department of Neuroscience, Stanford University, Stanford, CA 94304, USA
| | - Michael A Long
- NYU Neuroscience Institute, New York University Langone Health, New York, NY 10016, USA
- Department of Otolaryngology, New York University Langone Health, New York, NY 10016, USA
- Center for Neural Science, New York University, New York, NY 10003, USA
- Cold Spring Harbor Laboratory, Cold Spring Harbor, NY 11724, USA
| |
Collapse
|
46
|
Alhassen W, Alhassen S, Chen J, Monfared RV, Alachkar A. Cilia in the Striatum Mediate Timing-Dependent Functions. Mol Neurobiol 2023; 60:545-565. [PMID: 36322337 PMCID: PMC9849326 DOI: 10.1007/s12035-022-03095-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Accepted: 10/16/2022] [Indexed: 11/07/2022]
Abstract
Almost all brain cells contain cilia, antennae-like microtubule-based organelles. Yet, the significance of cilia, once considered vestigial organelles, in the higher-order brain functions is unknown. Cilia act as a hub that senses and transduces environmental sensory stimuli to generate an appropriate cellular response. Similarly, the striatum, a brain structure enriched in cilia, functions as a hub that receives and integrates various types of environmental information to drive appropriate motor response. To understand cilia's role in the striatum functions, we used loxP/Cre technology to ablate cilia from the dorsal striatum of male mice and monitored the behavioral consequences. Our results revealed an essential role for striatal cilia in the acquisition and brief storage of information, including learning new motor skills, but not in long-term consolidation of information or maintaining habitual/learned motor skills. A fundamental aspect of all disrupted functions was the "time perception/judgment deficit." Furthermore, the observed behavioral deficits form a cluster pertaining to clinical manifestations overlapping across psychiatric disorders that involve the striatum functions and are known to exhibit timing deficits. Thus, striatal cilia may act as a calibrator of the timing functions of the basal ganglia-cortical circuit by maintaining proper timing perception. Our findings suggest that dysfunctional cilia may contribute to the pathophysiology of neuro-psychiatric disorders, as related to deficits in timing perception.
Collapse
Affiliation(s)
- Wedad Alhassen
- Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, University of California-Irvine, 356A Med Surge II, Irvine, CA 92697-4625 USA
| | - Sammy Alhassen
- Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, University of California-Irvine, 356A Med Surge II, Irvine, CA 92697-4625 USA
| | - Jiaqi Chen
- Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, University of California-Irvine, 356A Med Surge II, Irvine, CA 92697-4625 USA
| | - Roudabeh Vakil Monfared
- Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, University of California-Irvine, 356A Med Surge II, Irvine, CA 92697-4625 USA
| | - Amal Alachkar
- Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, University of California-Irvine, 356A Med Surge II, Irvine, CA 92697-4625 USA ,UC Irvine Center for the Neurobiology of Learning and Memory, University of California-Irvine, Irvine, CA 92697 USA ,Institute for Genomics and Bioinformatics, School of Information and Computer Sciences, University of California-Irvine, Irvine, CA 92697 USA
| |
Collapse
|
47
|
De Corte BJ, Akdoğan B, Balsam PD. Temporal scaling and computing time in neural circuits: Should we stop watching the clock and look for its gears? Front Behav Neurosci 2022; 16:1022713. [PMID: 36570701 PMCID: PMC9773401 DOI: 10.3389/fnbeh.2022.1022713] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Accepted: 10/31/2022] [Indexed: 12/13/2022] Open
Abstract
Timing underlies a variety of functions, from walking to perceiving causality. Neural timing models typically fall into one of two categories-"ramping" and "population-clock" theories. According to ramping models, individual neurons track time by gradually increasing or decreasing their activity as an event approaches. To time different intervals, ramping neurons adjust their slopes, ramping steeply for short intervals and vice versa. In contrast, according to "population-clock" models, multiple neurons track time as a group, and each neuron can fire nonlinearly. As each neuron changes its rate at each point in time, a distinct pattern of activity emerges across the population. To time different intervals, the brain learns the population patterns that coincide with key events. Both model categories have empirical support. However, they often differ in plausibility when applied to certain behavioral effects. Specifically, behavioral data indicate that the timing system has a rich computational capacity, allowing observers to spontaneously compute novel intervals from previously learned ones. In population-clock theories, population patterns map to time arbitrarily, making it difficult to explain how different patterns can be computationally combined. Ramping models are viewed as more plausible, assuming upstream circuits can set the slope of ramping neurons according to a given computation. Critically, recent studies suggest that neurons with nonlinear firing profiles often scale to time different intervals-compressing for shorter intervals and stretching for longer ones. This "temporal scaling" effect has led to a hybrid-theory where, like a population-clock model, population patterns encode time, yet like a ramping neuron adjusting its slope, the speed of each neuron's firing adapts to different intervals. Here, we argue that these "relative" population-clock models are as computationally plausible as ramping theories, viewing population-speed and ramp-slope adjustments as equivalent. Therefore, we view identifying these "speed-control" circuits as a key direction for evaluating how the timing system performs computations. Furthermore, temporal scaling highlights that a key distinction between different neural models is whether they propose an absolute or relative time-representation. However, we note that several behavioral studies suggest the brain processes both scales, cautioning against a dichotomy.
Collapse
Affiliation(s)
- Benjamin J. De Corte
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
| | - Başak Akdoğan
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
| | - Peter D. Balsam
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
- Department of Neuroscience and Behavior, Barnard College, New York, NY, United States
| |
Collapse
|
48
|
De Corte BJ, Farley SJ, Heslin KA, Parker KL, Freeman JH. The dorsal hippocampus' role in context-based timing in rodents. Neurobiol Learn Mem 2022; 194:107673. [PMID: 35985617 DOI: 10.1016/j.nlm.2022.107673] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Revised: 08/05/2022] [Accepted: 08/11/2022] [Indexed: 01/13/2023]
Abstract
To act proactively, we must predict when future events will occur. Individuals generate temporal predictions using cues that indicate an event will happen after a certain duration elapses. Neural models of timing focus on how the brain represents these cue-duration associations. However, these models often overlook the fact that situational factors frequently modulate temporal expectations. For example, in realistic environments, the intervals associated with different cues will often covary due to a common underlying cause. According to the 'common cause hypothesis,' observers anticipate this covariance such that, when one cue's interval changes, temporal expectations for other cues shift in the same direction. Furthermore, as conditions will often differ across environments, the same cue can mean different things in different contexts. Therefore, updates to temporal expectations should be context-specific. Behavioral work supports these predictions, yet their underlying neural mechanisms are unclear. Here, we asked whether the dorsal hippocampus mediates context-based timing, given its broad role in context-conditioning. Specifically, we trained rats with either hippocampal or sham lesions that two cues predicted reward after either a short or long duration elapsed (e.g., tone-8 s/light-16 s). Then, we moved rats to a new context and extended the long cue's interval (e.g., light-32 s). This caused rats to respond later to the short cue, despite never being trained to do so. Importantly, when returned to the initial training context, sham rats shifted back toward both cues' original intervals. In contrast, lesion rats continued to respond at the long cue's newer interval. Surprisingly, they still showed contextual modulation for the short cue, responding earlier like shams. These data suggest the hippocampus only mediates context-based timing if a cue is explicitly paired and/or rewarded across distinct contexts. Furthermore, as lesions did not impact timing measures at baseline or acquisition for the long cue's new interval, our data suggests that the hippocampus only modulates timing when context is relevant.
Collapse
Affiliation(s)
- Benjamin J De Corte
- Department of Psychiatry, Columbia University, New York, NY, USA; New York State Psychiatric Institute, New York, NY, USA
| | - Sean J Farley
- Department of Psychological and Brain Sciences, The University of Iowa, Iowa City, IA, USA
| | - Kelsey A Heslin
- Department of Neuroscience and the Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Krystal L Parker
- Department of Psychiatry, The University of Iowa, Iowa City, IA, USA
| | - John H Freeman
- Department of Psychological and Brain Sciences, The University of Iowa, Iowa City, IA, USA.
| |
Collapse
|
49
|
Tunes GC, Fermino de Oliveira E, Vieira EUP, Caetano MS, Cravo AM, Bussotti Reyes M. Time encoding migrates from prefrontal cortex to dorsal striatum during learning of a self-timed response duration task. eLife 2022; 11:65495. [PMID: 36169996 PMCID: PMC9519146 DOI: 10.7554/elife.65495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2020] [Accepted: 09/01/2022] [Indexed: 11/13/2022] Open
Abstract
Although time is a fundamental dimension of life, we do not know how brain areas cooperate to keep track and process time intervals. Notably, analyses of neural activity during learning are rare, mainly because timing tasks usually require training over many days. We investigated how the time encoding evolves when animals learn to time a 1.5 s interval. We designed a novel training protocol where rats go from naive- to proficient-level timing performance within a single session, allowing us to investigate neuronal activity from very early learning stages. We used pharmacological experiments and machine-learning algorithms to evaluate the level of time encoding in the medial prefrontal cortex and the dorsal striatum. Our results show a double dissociation between the medial prefrontal cortex and the dorsal striatum during temporal learning, where the former commits to early learning stages while the latter engages as animals become proficient in the task.
Collapse
Affiliation(s)
- Gabriela C Tunes
- Center for Mathematics, Computing, and Cognition, Universidade Federal do ABC, Sao Bernardo do Campo, Brazil
| | | | - Estevão U P Vieira
- Center for Mathematics, Computing, and Cognition, Universidade Federal do ABC, Sao Bernardo do Campo, Brazil
| | - Marcelo S Caetano
- Center for Mathematics, Computing, and Cognition, Universidade Federal do ABC, Sao Bernardo do Campo, Brazil.,Instituto Nacional de Ciência e Tecnologia sobre Comportamento, Cognição e Ensino, Brazil
| | - André M Cravo
- Center for Mathematics, Computing, and Cognition, Universidade Federal do ABC, Sao Bernardo do Campo, Brazil
| | - Marcelo Bussotti Reyes
- Center for Mathematics, Computing, and Cognition, Universidade Federal do ABC, Sao Bernardo do Campo, Brazil
| |
Collapse
|
50
|
Tsao A, Yousefzadeh SA, Meck WH, Moser MB, Moser EI. The neural bases for timing of durations. Nat Rev Neurosci 2022; 23:646-665. [PMID: 36097049 DOI: 10.1038/s41583-022-00623-3] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/19/2022] [Indexed: 11/10/2022]
Abstract
Durations are defined by a beginning and an end, and a major distinction is drawn between durations that start in the present and end in the future ('prospective timing') and durations that start in the past and end either in the past or the present ('retrospective timing'). Different psychological processes are thought to be engaged in each of these cases. The former is thought to engage a clock-like mechanism that accurately tracks the continuing passage of time, whereas the latter is thought to engage a reconstructive process that utilizes both temporal and non-temporal information from the memory of past events. We propose that, from a biological perspective, these two forms of duration 'estimation' are supported by computational processes that are both reliant on population state dynamics but are nevertheless distinct. Prospective timing is effectively carried out in a single step where the ongoing dynamics of population activity directly serve as the computation of duration, whereas retrospective timing is carried out in two steps: the initial generation of population state dynamics through the process of event segmentation and the subsequent computation of duration utilizing the memory of those dynamics.
Collapse
Affiliation(s)
- Albert Tsao
- Department of Biology, Stanford University, Stanford, CA, USA.
| | | | - Warren H Meck
- Department of Psychology and Neuroscience, Duke University, Durham, NC, USA
| | - May-Britt Moser
- Centre for Neural Computation, Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway
| | - Edvard I Moser
- Centre for Neural Computation, Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway.
| |
Collapse
|