1
|
Simpson EH, Akam T, Patriarchi T, Blanco-Pozo M, Burgeno LM, Mohebi A, Cragg SJ, Walton ME. Lights, fiber, action! A primer on in vivo fiber photometry. Neuron 2024; 112:718-739. [PMID: 38103545 PMCID: PMC10939905 DOI: 10.1016/j.neuron.2023.11.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 10/16/2023] [Accepted: 11/15/2023] [Indexed: 12/19/2023]
Abstract
Fiber photometry is a key technique for characterizing brain-behavior relationships in vivo. Initially, it was primarily used to report calcium dynamics as a proxy for neural activity via genetically encoded indicators. This generated new insights into brain functions including movement, memory, and motivation at the level of defined circuits and cell types. Recently, the opportunity for discovery with fiber photometry has exploded with the development of an extensive range of fluorescent sensors for biomolecules including neuromodulators and peptides that were previously inaccessible in vivo. This critical advance, combined with the new availability of affordable "plug-and-play" recording systems, has made monitoring molecules with high spatiotemporal precision during behavior highly accessible. However, while opening exciting new avenues for research, the rapid expansion in fiber photometry applications has occurred without coordination or consensus on best practices. Here, we provide a comprehensive guide to help end-users execute, analyze, and suitably interpret fiber photometry studies.
Collapse
Affiliation(s)
- Eleanor H Simpson
- Department of Psychiatry, Columbia University Medical Center, New York, NY, USA; New York State Psychiatric Institute, New York, NY, USA.
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - Tommaso Patriarchi
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland; Neuroscience Center Zürich, University and ETH Zürich, Zürich, Switzerland.
| | - Marta Blanco-Pozo
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Lauren M Burgeno
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK
| | - Ali Mohebi
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| | - Stephanie J Cragg
- Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Mark E Walton
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| |
Collapse
|
2
|
Sagiv Y, Akam T, Witten IB, Daw ND. Prioritizing replay when future goals are unknown. bioRxiv 2024:2024.02.29.582822. [PMID: 38496674 PMCID: PMC10942393 DOI: 10.1101/2024.02.29.582822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, and it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Our work extends the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses.
Collapse
Affiliation(s)
- Yotam Sagiv
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Thomas Akam
- Department of Experimental Psychology, Oxford University, Oxford, UK
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| |
Collapse
|
3
|
Blanco-Pozo M, Akam T, Walton ME. Dopamine-independent effect of rewards on choices through hidden-state inference. Nat Neurosci 2024; 27:286-297. [PMID: 38216649 PMCID: PMC10849965 DOI: 10.1038/s41593-023-01542-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 12/01/2023] [Indexed: 01/14/2024]
Abstract
Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.
Collapse
Affiliation(s)
- Marta Blanco-Pozo
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| | - Thomas Akam
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| | - Mark E Walton
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| |
Collapse
|
4
|
Rowland JM, van der Plas TL, Loidolt M, Lees RM, Keeling J, Dehning J, Akam T, Priesemann V, Packer AM. Propagation of activity through the cortical hierarchy and perception are determined by neural variability. Nat Neurosci 2023; 26:1584-1594. [PMID: 37640911 PMCID: PMC10471496 DOI: 10.1038/s41593-023-01413-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Accepted: 07/18/2023] [Indexed: 08/31/2023]
Abstract
Brains are composed of anatomically and functionally distinct regions performing specialized tasks, but regions do not operate in isolation. Orchestration of complex behaviors requires communication between brain regions, but how neural dynamics are organized to facilitate reliable transmission is not well understood. Here we studied this process directly by generating neural activity that propagates between brain regions and drives behavior, assessing how neural populations in sensory cortex cooperate to transmit information. We achieved this by imaging two densely interconnected regions-the primary and secondary somatosensory cortex (S1 and S2)-in mice while performing two-photon photostimulation of S1 neurons and assigning behavioral salience to the photostimulation. We found that the probability of perception is determined not only by the strength of the photostimulation but also by the variability of S1 neural activity. Therefore, maximizing the signal-to-noise ratio of the stimulus representation in cortex relative to the noise or variability is critical to facilitate activity propagation and perception.
Collapse
Affiliation(s)
- James M Rowland
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK
| | - Thijs L van der Plas
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK
| | - Matthias Loidolt
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK
- Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany
- Laboratory for Molecular Cell Biology, University College London, London, UK
| | - Robert M Lees
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK
- Science and Technology Facilities Council, Octopus Imaging Facility, Research Complex at Harwell, Harwell Campus, Oxfordshire, UK
| | - Joshua Keeling
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK
| | - Jonas Dehning
- Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Viola Priesemann
- Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany
- Institute for the Dynamics of Complex Systems, University of Göttingen, Göttingen, Germany
| | - Adam M Packer
- Department of Physiology, Anatomy, and Genetics, University of Oxford, Oxford, UK.
| |
Collapse
|
5
|
Jendryka MM, Lewin U, van der Veen B, Kapanaiah SKT, Prex V, Strahnen D, Akam T, Liss B, Pekcec A, Nissen W, Kätzel D. Control of sustained attention and impulsivity by G q-protein signalling in parvalbumin interneurons of the anterior cingulate cortex. Transl Psychiatry 2023; 13:243. [PMID: 37407615 DOI: 10.1038/s41398-023-02541-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 06/19/2023] [Accepted: 06/23/2023] [Indexed: 07/07/2023] Open
Abstract
The anterior cingulate cortex (ACC) has been implicated in attention deficit hyperactivity disorder (ADHD). More specifically, an appropriate balance of excitatory and inhibitory activity in the ACC may be critical for the control of impulsivity, hyperactivity, and sustained attention which are centrally affected in ADHD. Hence, pharmacological augmentation of parvalbumin- (PV) or somatostatin-positive (Sst) inhibitory ACC interneurons could be a potential treatment strategy. We, therefore, tested whether stimulation of Gq-protein-coupled receptors (GqPCRs) in these interneurons could improve attention or impulsivity assessed with the 5-choice-serial reaction-time task in male mice. When challenging impulse control behaviourally or pharmacologically, activation of the chemogenetic GqPCR hM3Dq in ACC PV-cells caused a selective decrease of active erroneous-i.e. incorrect and premature-responses, indicating improved attentional and impulse control. When challenging attention, in contrast, omissions were increased, albeit without extension of reward latencies or decreases of attentional accuracy. These effects largely resembled those of the ADHD medication atomoxetine. Additionally, they were mostly independent of each other within individual animals. GqPCR activation in ACC PV-cells also reduced hyperactivity. In contrast, if hM3Dq was activated in Sst-interneurons, no improvement of impulse control was observed, and a reduction of incorrect responses was only induced at high agonist levels and accompanied by reduced motivational drive. These results suggest that the activation of GqPCRs expressed specifically in PV-cells of the ACC may be a viable strategy to improve certain aspects of sustained attention, impulsivity and hyperactivity in ADHD.
Collapse
Affiliation(s)
- Martin M Jendryka
- Institute of Applied Physiology, Ulm University, Ulm, Germany
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Uwe Lewin
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | | | | | - Vivien Prex
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Daniel Strahnen
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Thomas Akam
- Department of Experimental Psychology and Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Birgit Liss
- Institute of Applied Physiology, Ulm University, Ulm, Germany
- Linacre College and New College, University of Oxford, Oxford, UK
| | - Anton Pekcec
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Wiebke Nissen
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm University, Ulm, Germany.
| |
Collapse
|
6
|
Samborska V, Butler JL, Walton ME, Behrens TEJ, Akam T. Complementary task representations in hippocampus and prefrontal cortex for generalizing the structure of problems. Nat Neurosci 2022; 25:1314-1326. [PMID: 36171429 PMCID: PMC9534768 DOI: 10.1038/s41593-022-01149-8] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Accepted: 07/19/2022] [Indexed: 11/16/2022]
Abstract
Humans and other animals effortlessly generalize prior knowledge to solve novel problems, by abstracting common structure and mapping it onto new sensorimotor specifics. To investigate how the brain achieves this, in this study, we trained mice on a series of reversal learning problems that shared the same structure but had different physical implementations. Performance improved across problems, indicating transfer of knowledge. Neurons in medial prefrontal cortex (mPFC) maintained similar representations across problems despite their different sensorimotor correlates, whereas hippocampal (dCA1) representations were more strongly influenced by the specifics of each problem. This was true for both representations of the events that comprised each trial and those that integrated choices and outcomes over multiple trials to guide an animal’s decisions. These data suggest that prefrontal cortex and hippocampus play complementary roles in generalization of knowledge: PFC abstracts the common structure among related problems, and hippocampus maps this structure onto the specifics of the current situation. Samborska et al. trained mice on a set of problems with the same structure but different physical layouts to study generalization. Neurons in prefrontal cortex generalized across problems, whereas those in hippocampus were more problem specific.
Collapse
Affiliation(s)
- Veronika Samborska
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - James L Butler
- Department of Clinical and Movement Neurosciences, University College London, London, UK.,Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK
| | - Mark E Walton
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Timothy E J Behrens
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK. .,Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK. .,Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Thomas Akam
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
7
|
Castro-Rodrigues P, Akam T, Snorasson I, Camacho M, Paixão V, Maia A, Barahona-Corrêa JB, Dayan P, Simpson HB, Costa RM, Oliveira-Maia AJ. Explicit knowledge of task structure is a primary determinant of human model-based action. Nat Hum Behav 2022; 6:1126-1141. [PMID: 35589826 DOI: 10.1038/s41562-022-01346-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Revised: 03/19/2022] [Accepted: 03/31/2022] [Indexed: 11/09/2022]
Abstract
Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.
Collapse
Affiliation(s)
- Pedro Castro-Rodrigues
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Centro Hospitalar Psiquiátrico de Lisboa, Lisbon, Portugal
| | - Thomas Akam
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Ivar Snorasson
- Center for Obsessive-Compulsive & Related Disorders, New York State Psychiatric Institute, New York, NY, USA
| | - Marta Camacho
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,John Van Geest Center for Brain Repair, University of Cambridge, Cambridge, UK
| | - Vitor Paixão
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
| | - Ana Maia
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Department of Psychiatry and Mental Health, Centro Hospitalar de Lisboa Ocidental, Lisbon, Portugal
| | - J Bernardo Barahona-Corrêa
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany.,The University of Tübingen, Tübingen, Germany
| | - H Blair Simpson
- Center for Obsessive-Compulsive & Related Disorders, New York State Psychiatric Institute, New York, NY, USA.,Department of Psychiatry, Columbia University, New York, NY, USA
| | - Rui M Costa
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Albino J Oliveira-Maia
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal. .,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal. .,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.
| |
Collapse
|
8
|
Akam T, Lustig A, Rowland JM, Kapanaiah SKT, Esteve-Agraz J, Panniello M, Márquez C, Kohl MM, Kätzel D, Costa RM, Walton ME. Open-source, Python-based, hardware and software for controlling behavioural neuroscience experiments. eLife 2022; 11:e67846. [PMID: 35043782 PMCID: PMC8769647 DOI: 10.7554/elife.67846] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 01/03/2022] [Indexed: 01/05/2023] Open
Abstract
Laboratory behavioural tasks are an essential research tool. As questions asked of behaviour and brain activity become more sophisticated, the ability to specify and run richly structured tasks becomes more important. An increasing focus on reproducibility also necessitates accurate communication of task logic to other researchers. To these ends, we developed pyControl, a system of open-source hardware and software for controlling behavioural experiments comprising a simple yet flexible Python-based syntax for specifying tasks as extended state machines, hardware modules for building behavioural setups, and a graphical user interface designed for efficiently running high-throughput experiments on many setups in parallel, all with extensive online documentation. These tools make it quicker, easier, and cheaper to implement rich behavioural tasks at scale. As important, pyControl facilitates communication and reproducibility of behavioural experiments through a highly readable task definition syntax and self-documenting features. Here, we outline the system's design and rationale, present validation experiments characterising system performance, and demonstrate example applications in freely moving and head-fixed mouse behaviour.
Collapse
Affiliation(s)
- Thomas Akam
- Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Champalimaud Neuroscience Program, Champalimaud Centre for the UnknownLisbonPortugal
| | - Andy Lustig
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - James M Rowland
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
| | | | - Joan Esteve-Agraz
- Instituto de Neurociencias (Universidad Miguel Hernández-Consejo Superior de Investigaciones Científicas)Sant Joan d’AlacantSpain
| | - Mariangela Panniello
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
| | - Cristina Márquez
- Instituto de Neurociencias (Universidad Miguel Hernández-Consejo Superior de Investigaciones Científicas)Sant Joan d’AlacantSpain
| | - Michael M Kohl
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm UniversityUlmGermany
| | - Rui M Costa
- Champalimaud Neuroscience Program, Champalimaud Centre for the UnknownLisbonPortugal
- Department of Neuroscience and Neurology, Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Mark E Walton
- Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of OxfordOxfordUnited Kingdom
| |
Collapse
|
9
|
Kapanaiah SKT, van der Veen B, Strahnen D, Akam T, Kätzel D. A low-cost open-source 5-choice operant box system optimized for electrophysiology and optophysiology in mice. Sci Rep 2021; 11:22279. [PMID: 34782697 PMCID: PMC8593009 DOI: 10.1038/s41598-021-01717-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 11/01/2021] [Indexed: 11/16/2022] Open
Abstract
Operant boxes enable the application of complex behavioural paradigms to support circuit neuroscience and drug discovery research. However, commercial operant box systems are expensive and often not optimised for combining behaviour with neurophysiology. Here we introduce a fully open-source Python-based operant-box system in a 5-choice design (pyOS-5) that enables assessment of multiple cognitive and affective functions. It is optimized for fast turn-over between animals, and for testing of tethered mice for simultaneous physiological recordings or optogenetic manipulation. For reward delivery, we developed peristaltic and syringe pumps based on a stepper motor and 3D-printed parts. Tasks are specified using a Python-based syntax implemented on custom-designed printed circuit boards that are commercially available at low cost. We developed an open-source graphical user interface (GUI) and task definition scripts to conduct assays assessing operant learning, attention, impulsivity, working memory, or cognitive flexibility, alleviating the need for programming skills of the end user. All behavioural events are recorded with millisecond resolution, and TTL-outputs and -inputs allow straightforward integration with physiological recordings and closed-loop manipulations. This combination of features realizes a cost-effective, nose-poke-based operant box system that allows reliable circuit-neuroscience experiments investigating correlates of cognition and emotion in large cohorts of subjects.
Collapse
Affiliation(s)
| | | | - Daniel Strahnen
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm University, Ulm, Germany.
| |
Collapse
|
10
|
van der Veen B, Kapanaiah SKT, Kilonzo K, Steele-Perkins P, Jendryka MM, Schulz S, Tasic B, Yao Z, Zeng H, Akam T, Nicholson JR, Liss B, Nissen W, Pekcec A, Kätzel D. Control of impulsivity by G i-protein signalling in layer-5 pyramidal neurons of the anterior cingulate cortex. Commun Biol 2021; 4:662. [PMID: 34079054 PMCID: PMC8172539 DOI: 10.1038/s42003-021-02188-w] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2020] [Accepted: 05/06/2021] [Indexed: 12/19/2022] Open
Abstract
Pathological impulsivity is a debilitating symptom of multiple psychiatric diseases with few effective treatment options. To identify druggable receptors with anti-impulsive action we developed a systematic target discovery approach combining behavioural chemogenetics and gene expression analysis. Spatially restricted inhibition of three subdivisions of the prefrontal cortex of mice revealed that the anterior cingulate cortex (ACC) regulates premature responding, a form of motor impulsivity. Probing three G-protein cascades with designer receptors, we found that the activation of Gi-signalling in layer-5 pyramidal cells (L5-PCs) of the ACC strongly, reproducibly, and selectively decreased challenge-induced impulsivity. Differential gene expression analysis across murine ACC cell-types and 402 GPCRs revealed that - among Gi-coupled receptor-encoding genes - Grm2 is the most selectively expressed in L5-PCs while alternative targets were scarce. Validating our approach, we confirmed that mGluR2 activation reduced premature responding. These results suggest Gi-coupled receptors in ACC L5-PCs as therapeutic targets for impulse control disorders.
Collapse
Affiliation(s)
| | | | - Kasyoka Kilonzo
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | | | | | - Stefanie Schulz
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | | | - Zizhen Yao
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Hongkui Zeng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Janet R Nicholson
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Birgit Liss
- Institute of Applied Physiology, Ulm University, Ulm, Germany
- Linacre College and New College, University of Oxford, Oxford, UK
| | - Wiebke Nissen
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Anton Pekcec
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm University, Ulm, Germany.
| |
Collapse
|
11
|
Abstract
Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine's involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine's involvement in model-based RL.
Collapse
|
12
|
Akam T, Rodrigues-Vaz I, Marcelo I, Zhang X, Pereira M, Oliveira RF, Dayan P, Costa RM. The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection. Neuron 2021; 109:149-163.e7. [PMID: 33152266 PMCID: PMC7837117 DOI: 10.1016/j.neuron.2020.10.013] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 09/01/2020] [Accepted: 10/09/2020] [Indexed: 01/19/2023]
Abstract
Behavioral control is not unitary. It comprises parallel systems, model based and model free, that respectively generate flexible and habitual behaviors. Model-based decisions use predictions of the specific consequences of actions, but how these are implemented in the brain is poorly understood. We used calcium imaging and optogenetics in a sequential decision task for mice to show that the anterior cingulate cortex (ACC) predicts the state that actions will lead to, not simply whether they are good or bad, and monitors whether outcomes match these predictions. ACC represents the complete state space of the task, with reward signals that depend strongly on the state where reward is obtained but minimally on the preceding choice. Accordingly, ACC is necessary only for updating model-based strategies, not for basic reward-driven action reinforcement. These results reveal that ACC is a critical node in model-based control, with a specific role in predicting future states given chosen actions.
Collapse
Affiliation(s)
- Thomas Akam
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal; Department of Experimental Psychology, Oxford University, Oxford, UK.
| | - Ines Rodrigues-Vaz
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal; Department of Neuroscience and Neurology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Ivo Marcelo
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal; Department of Psychiatry, Erasmus MC University Medical Center, 3015 GD Rotterdam, the Netherlands
| | - Xiangyu Zhang
- RIKEN-MIT Center for Neural Circuit Genetics at the Picower Institute for Learning and Memory, Department of Biology and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Michael Pereira
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal
| | | | - Peter Dayan
- Gatsby Computational Neuroscience Unit, University College London, London, UK; Max Planck Institute for Biological Cybernetics, Tübingen, Germany; University of Tübingen, Tübingen, Germany
| | - Rui M Costa
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal; Department of Neuroscience and Neurology, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| |
Collapse
|
13
|
Kolling N, Akam T. (Reinforcement?) Learning to forage optimally. Curr Opin Neurobiol 2017; 46:162-169. [PMID: 28918312 DOI: 10.1016/j.conb.2017.08.008] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Revised: 08/06/2017] [Accepted: 08/17/2017] [Indexed: 11/24/2022]
Abstract
Foraging effectively is critical to the survival of all animals and this imperative is thought to have profoundly shaped brain evolution. Decisions made by foraging animals often approximate optimal strategies, but the learning and decision mechanisms generating these choices remain poorly understood. Recent work with laboratory foraging tasks in humans suggest their behaviour is poorly explained by model-free reinforcement learning, with simple heuristic strategies better describing behaviour in some tasks, and in others evidence of prospective prediction of the future state of the environment. We suggest that model-based average reward reinforcement learning may provide a common framework for understanding these apparently divergent foraging strategies.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, United Kingdom
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, United Kingdom; Champalimaud Neuroscience Program, Champalimaud Center for the Unknown, Portugal.
| |
Collapse
|
14
|
Manohar SG, Akam T. Cortical areas needed for choosing actions based on desires. Brain 2017; 140:1539-1542. [PMID: 28549135 PMCID: PMC5445251 DOI: 10.1093/brain/awx119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Affiliation(s)
- Sanjay G Manohar
- Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford; Fundação Champalimaud, Lisbon, Portugal
| |
Collapse
|
15
|
Akam T, Costa R, Dayan P. Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task. PLoS Comput Biol 2015; 11:e1004648. [PMID: 26657806 PMCID: PMC4686094 DOI: 10.1371/journal.pcbi.1004648] [Citation(s) in RCA: 61] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2015] [Accepted: 11/09/2015] [Indexed: 11/28/2022] Open
Abstract
The recently developed ‘two-step’ behavioural task promises to differentiate model-based from model-free reinforcement learning, while generating neurophysiologically-friendly decision datasets with parametric variation of decision variables. These desirable features have prompted its widespread adoption. Here, we analyse the interactions between a range of different strategies and the structure of transitions and outcomes in order to examine constraints on what can be learned from behavioural performance. The task involves a trade-off between the need for stochasticity, to allow strategies to be discriminated, and a need for determinism, so that it is worth subjects’ investment of effort to exploit the contingencies optimally. We show through simulation that under certain conditions model-free strategies can masquerade as being model-based. We first show that seemingly innocuous modifications to the task structure can induce correlations between action values at the start of the trial and the subsequent trial events in such a way that analysis based on comparing successive trials can lead to erroneous conclusions. We confirm the power of a suggested correction to the analysis that can alleviate this problem. We then consider model-free reinforcement learning strategies that exploit correlations between where rewards are obtained and which actions have high expected value. These generate behaviour that appears model-based under these, and also more sophisticated, analyses. Exploiting the full potential of the two-step task as a tool for behavioural neuroscience requires an understanding of these issues. Planning is the use of a predictive model of the consequences of actions to guide decision making. Planning plays a critical role in human behaviour, but isolating its contribution is challenging because it is complemented by control systems which learn values of actions directly from the history of reinforcement, resulting in automatized mappings from states to actions often termed habits. Our study examined a recently developed behavioural task which uses choices in a multi-step decision tree to differentiate planning from value-based control. We compared various strategies using simulations, showing a range that produce behaviour that resembles planning but in fact arises as a fixed mapping from particular sorts of states to action. These results show that when a planning problem is faced repeatedly, sophisticated automatization strategies may be developed which identify that there are in fact a limited number of relevant states of the world each with an appropriate fixed or habitual response. Understanding such strategies is important for the design and interpretation of tasks which aim to isolate the contribution of planning to behaviour. Such strategies are also of independent scientific interest as they may contribute to automatization of behaviour in complex environments.
Collapse
Affiliation(s)
- Thomas Akam
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- * E-mail:
| | - Rui Costa
- Champalimaud Neuroscience Program, Champalimaud Centre for the Unknown, Lisbon, Portugal
| | - Peter Dayan
- Gatsby Computational Neuroscience Unit, UCL, London, United Kingdom
| |
Collapse
|
16
|
Abstract
In a recent study in the journal Cell, Tervo et al. (2014) show that animals can implement stochastic choice policies in environments unfavorable to predictive strategies. The shift toward stochastic behavior was driven by noradrenergic signaling in the anterior cingulate cortex.
Collapse
Affiliation(s)
- Thomas Akam
- Champalimaud Neuroscience Programme, Champalimaud Center for the Unknown, Av. De Brasília, 1400-038 Lisbon, Portugal.
| | - Rui M Costa
- Champalimaud Neuroscience Programme, Champalimaud Center for the Unknown, Av. De Brasília, 1400-038 Lisbon, Portugal.
| |
Collapse
|
17
|
Abstract
Mammalian brains exhibit population oscillations, the structures of which vary in time and space according to behavioural state. A proposed function of these oscillations is to control the flow of signals among anatomically connected networks. However, the nature of neural coding that may support selective communication that depends on oscillations has received relatively little attention. Here, we consider the role of multiplexing, whereby multiple information streams share a common neural substrate. We suggest that multiplexing implemented through periodic modulation of firing-rate population codes enables flexible reconfiguration of effective connectivity among brain areas.
Collapse
Affiliation(s)
- Thomas Akam
- Champalimaud Centre, Av. Brasília, Doca de Pedrouços, Lisbon 1400-038, Portugal
| | - Dimitri M Kullmann
- UCL Institute of Neurology, University College London, Queen Square, London WC1N 3BG, UK
| |
Collapse
|
18
|
Akam T, Oren I, Mantoan L, Ferenczi E, Kullmann DM. Oscillatory dynamics in the hippocampus support dentate gyrus–CA3 coupling. Nat Neurosci 2012; 15:763-8. [PMID: 22466505 DOI: 10.1038/nn.3081] [Citation(s) in RCA: 63] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2011] [Accepted: 03/05/2012] [Indexed: 11/09/2022]
Abstract
Gamma oscillations in the dentate gyrus and hippocampal CA3 show variable coherence in vivo, but the mechanisms and relevance for information flow are unknown. We found that carbachol-induced oscillations in rat CA3 have biphasic phase-response curves, consistent with the ability to couple with oscillations in afferent projections. Differences in response to stimulation of either the intrinsic feedback circuit or the dentate gyrus were well described by varying an impulse vector in a two-dimensional dynamical system, representing the relative input to excitatory and inhibitory neurons. Responses to sinusoidally modulated optogenetic stimulation confirmed that the CA3 network oscillation can entrain to periodic inputs, with a steep dependence of entrainment phase on input frequency. CA3 oscillations are therefore suited to coupling with oscillations in the dentate gyrus over a broad range of frequencies.
Collapse
Affiliation(s)
- Thomas Akam
- University College London Institute of Neurology, University College London, London, UK.
| | | | | | | | | |
Collapse
|
19
|
Bauer M, Akam T, Joseph S, Freeman E, Driver J. Does visual flicker phase at gamma frequency modulate neural signal propagation and stimulus selection? J Vis 2012; 12:12.4.5. [PMID: 22505620 DOI: 10.1167/12.4.5] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Oscillatory synchronization of neuronal populations has been proposed to play a role in perceptual integration and attentional processing. However, some conflicting evidence has been found with respect to its causal relevance for sensory processing, particularly when using flickering visual stimuli with the aim of driving oscillations. We tested psychophysically whether the relative phase of gamma frequency flicker (60 Hz) between stimuli modulates well-known facilitatory lateral interactions between collinear Gabor patches (Experiment 1) or crowding of a peripheral target by irrelevant distractors (Experiment 2). Experiment 1 assessed the impact of suprathreshold Gabor flankers on detection of a near-threshold central Gabor target ("Lateral interactions paradigm"). The flanking stimuli could flicker either in phase or in anti-phase with each other. The typical facilitation of target detection was found with collinear flankers, but this was unaffected by flicker phase. Experiment 2 employed a "crowding" paradigm, where orientation discrimination of a peripheral target Gabor patch is disrupted when surrounded by irrelevant distractors. We found the usual crowding effect, which declined with spatial separation, but this was unaffected by relative flicker phase between target and distractors at all separations. These results imply that externally driven manipulations of gamma frequency phase cannot modulate perceptual integration in vision.
Collapse
Affiliation(s)
- Markus Bauer
- UCL Institute of Cognitive Neuroscience, Wellcome Trust Centre for Neuroimaging, University College London, London, UK.
| | | | | | | | | |
Collapse
|