1
|
Zajzon B, Duarte R, Morrison A. Toward reproducible models of sequence learning: replication and analysis of a modular spiking network with reward-based learning. Front Integr Neurosci 2023; 17:935177. [PMID: 37396571 PMCID: PMC10310927 DOI: 10.3389/fnint.2023.935177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 05/15/2023] [Indexed: 07/04/2023] Open
Abstract
To acquire statistical regularities from the world, the brain must reliably process, and learn from, spatio-temporally structured information. Although an increasing number of computational models have attempted to explain how such sequence learning may be implemented in the neural hardware, many remain limited in functionality or lack biophysical plausibility. If we are to harvest the knowledge within these models and arrive at a deeper mechanistic understanding of sequential processing in cortical circuits, it is critical that the models and their findings are accessible, reproducible, and quantitatively comparable. Here we illustrate the importance of these aspects by providing a thorough investigation of a recently proposed sequence learning model. We re-implement the modular columnar architecture and reward-based learning rule in the open-source NEST simulator, and successfully replicate the main findings of the original study. Building on these, we perform an in-depth analysis of the model's robustness to parameter settings and underlying assumptions, highlighting its strengths and weaknesses. We demonstrate a limitation of the model consisting in the hard-wiring of the sequence order in the connectivity patterns, and suggest possible solutions. Finally, we show that the core functionality of the model is retained under more biologically-plausible constraints.
Collapse
Affiliation(s)
- Barna Zajzon
- Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-BRAIN Institute I, Jülich Research Centre, Jülich, Germany
- Department of Computer Science 3—Software Engineering, RWTH Aachen University, Aachen, Germany
| | - Renato Duarte
- Donders Institute for Brain, Cognition and Behavior, Radboud University, Nijmegen, Netherlands
| | - Abigail Morrison
- Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-BRAIN Institute I, Jülich Research Centre, Jülich, Germany
- Department of Computer Science 3—Software Engineering, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
2
|
Hong SZ, Mesik L, Grossman CD, Cohen JY, Lee B, Severin D, Lee HK, Hell JW, Kirkwood A. Norepinephrine potentiates and serotonin depresses visual cortical responses by transforming eligibility traces. Nat Commun 2022; 13:3202. [PMID: 35680879 PMCID: PMC9184610 DOI: 10.1038/s41467-022-30827-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Accepted: 05/19/2022] [Indexed: 11/18/2022] Open
Abstract
Reinforcement allows organisms to learn which stimuli predict subsequent biological relevance. Hebbian mechanisms of synaptic plasticity are insufficient to account for reinforced learning because neuromodulators signaling biological relevance are delayed with respect to the neural activity associated with the stimulus. A theoretical solution is the concept of eligibility traces (eTraces), silent synaptic processes elicited by activity which upon arrival of a neuromodulator are converted into a lasting change in synaptic strength. Previously we demonstrated in visual cortical slices the Hebbian induction of eTraces and their conversion into LTP and LTD by the retroactive action of norepinephrine and serotonin Here we show in vivo in mouse V1 that the induction of eTraces and their conversion to LTP/D by norepinephrine and serotonin respectively potentiates and depresses visual responses. We also show that the integrity of this process is crucial for ocular dominance plasticity, a canonical model of experience-dependent plasticity.
Collapse
Affiliation(s)
- Su Z Hong
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Lukas Mesik
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Cooper D Grossman
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Jeremiah Y Cohen
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Boram Lee
- Department of Pharmacology, University of California at Davis, Davis, CA, 95616, USA
| | - Daniel Severin
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Hey-Kyoung Lee
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Johannes W Hell
- Department of Pharmacology, University of California at Davis, Davis, CA, 95616, USA
| | - Alfredo Kirkwood
- Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, 21218, USA.
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, 21205, USA.
| |
Collapse
|
3
|
Wert-Carvajal C, Reneaux M, Tchumatchenko T, Clopath C. Dopamine and serotonin interplay for valence-based spatial learning. Cell Rep 2022; 39:110645. [PMID: 35417691 DOI: 10.1016/j.celrep.2022.110645] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Revised: 10/31/2021] [Accepted: 03/17/2022] [Indexed: 11/17/2022] Open
Abstract
Dopamine (DA) and serotonin (5-HT) are important neuromodulators of synaptic plasticity that have been linked to learning from positive or negative outcomes or valence-based learning. In the hippocampus, both affect long-term plasticity but play different roles in encoding uncertainty or predicted reward. DA has been related to positive valence, from reward consumption or avoidance behavior, and 5-HT to aversive encoding. We propose DA produces overall LTP while 5-HT elicits LTD. Here, we compare two reward-modulated spike timing-dependent plasticity (R-STDP) rules to describe the action of these neuromodulators. We examined their role in cognitive performance and flexibility for computational models of the Morris water maze task and reversal learning. Our results show that the interplay of DA and 5-HT improves learning performance and can explain experimental evidence. This study reinforces the importance of neuromodulation in determining the direction of plasticity.
Collapse
Affiliation(s)
- Carlos Wert-Carvajal
- Bioengineering Department, Imperial College London, London SW7 2AZ, UK; Theory of Neural Dynamics Group, Max Planck Institute for Brain Research, 60438 Frankfurt, Germany; Institute of Experimental Epileptology and Cognition Research, Life and Brain Center, University of Bonn Medical Center, 53127 Bonn, Germany
| | - Melissa Reneaux
- Bioengineering Department, Imperial College London, London SW7 2AZ, UK
| | - Tatjana Tchumatchenko
- Theory of Neural Dynamics Group, Max Planck Institute for Brain Research, 60438 Frankfurt, Germany; Institute of Experimental Epileptology and Cognition Research, Life and Brain Center, University of Bonn Medical Center, 53127 Bonn, Germany; Institute of Physiological Chemistry, University of Mainz Medical Center, 55131 Mainz, Germany.
| | - Claudia Clopath
- Bioengineering Department, Imperial College London, London SW7 2AZ, UK.
| |
Collapse
|
4
|
Cone I, Shouval HZ. Learning precise spatiotemporal sequences via biophysically realistic learning rules in a modular, spiking network. eLife 2021; 10:63751. [PMID: 33734085 PMCID: PMC7972481 DOI: 10.7554/elife.63751] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Accepted: 02/16/2021] [Indexed: 11/13/2022] Open
Abstract
Multiple brain regions are able to learn and express temporal sequences, and this functionality is an essential component of learning and memory. We propose a substrate for such representations via a network model that learns and recalls discrete sequences of variable order and duration. The model consists of a network of spiking neurons placed in a modular microcolumn based architecture. Learning is performed via a biophysically realistic learning rule that depends on synaptic 'eligibility traces'. Before training, the network contains no memory of any particular sequence. After training, presentation of only the first element in that sequence is sufficient for the network to recall an entire learned representation of the sequence. An extended version of the model also demonstrates the ability to successfully learn and recall non-Markovian sequences. This model provides a possible framework for biologically plausible sequence learning and memory, in agreement with recent experimental results.
Collapse
Affiliation(s)
- Ian Cone
- Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, United States.,Applied Physics, Rice University, Houston, TX, United States
| | - Harel Z Shouval
- Neurobiology and Anatomy, University of Texas Medical School at Houston, Houston, TX, United States
| |
Collapse
|
5
|
Cone I, Shouval HZ. Behavioral Time Scale Plasticity of Place Fields: Mathematical Analysis. Front Comput Neurosci 2021; 15:640235. [PMID: 33732128 PMCID: PMC7959845 DOI: 10.3389/fncom.2021.640235] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Accepted: 02/08/2021] [Indexed: 11/17/2022] Open
Abstract
Traditional synaptic plasticity experiments and models depend on tight temporal correlations between pre- and postsynaptic activity. These tight temporal correlations, on the order of tens of milliseconds, are incompatible with significantly longer behavioral time scales, and as such might not be able to account for plasticity induced by behavior. Indeed, recent findings in hippocampus suggest that rapid, bidirectional synaptic plasticity which modifies place fields in CA1 operates at behavioral time scales. These experimental results suggest that presynaptic activity generates synaptic eligibility traces both for potentiation and depression, which last on the order of seconds. These traces can be converted to changes in synaptic efficacies by the activation of an instructive signal that depends on naturally occurring or experimentally induced plateau potentials. We have developed a simple mathematical model that is consistent with these observations. This model can be fully analyzed to find the fixed points of induced place fields and how these fixed points depend on system parameters such as the size and shape of presynaptic place fields, the animal's velocity during induction, and the parameters of the plasticity rule. We also make predictions about the convergence time to these fixed points, both for induced and pre-existing place fields.
Collapse
Affiliation(s)
- Ian Cone
- Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX, United States
- Applied Physics Program, Rice University, Houston, TX, United States
| | - Harel Z. Shouval
- Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX, United States
| |
Collapse
|
6
|
Beyond STDP-towards diverse and functionally relevant plasticity rules. Curr Opin Neurobiol 2018; 54:12-19. [PMID: 30056261 DOI: 10.1016/j.conb.2018.06.011] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Revised: 06/06/2018] [Accepted: 06/18/2018] [Indexed: 01/08/2023]
Abstract
Synaptic plasticity, induced by the close temporal association of two neural signals, supports associative forms of learning. However, the millisecond timescales for association often do not match the much longer delays for behaviorally relevant signals that supervise learning. In particular, information about the behavioral outcome of neural activity can be delayed, leading to a problem of temporal credit assignment. Recent studies suggest that synaptic plasticity can have temporal rules that not only accommodate the delays relevant to the circuit, but also be precisely tuned to the behavior the circuit supports. These discoveries highlight the diversity of plasticity rules, whose temporal requirements may depend on circuit delays and the contingencies of behavior.
Collapse
|