1
|
Oesch LT, Ryan MB, Churchland AK. From innate to instructed: A new look at perceptual decision-making. Curr Opin Neurobiol 2024; 86:102871. [PMID: 38569230 PMCID: PMC11162954 DOI: 10.1016/j.conb.2024.102871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 03/07/2024] [Accepted: 03/08/2024] [Indexed: 04/05/2024]
Abstract
Understanding how subjects perceive sensory stimuli in their environment and use this information to guide appropriate actions is a major challenge in neuroscience. To study perceptual decision-making in animals, researchers use tasks that either probe spontaneous responses to stimuli (often described as "naturalistic") or train animals to associate stimuli with experimenter-defined responses. Spontaneous decisions rely on animals' pre-existing knowledge, while trained tasks offer greater versatility, albeit often at the cost of extensive training. Here, we review emerging approaches to investigate perceptual decision-making using both spontaneous and trained behaviors, highlighting their strengths and limitations. Additionally, we propose how trained decision-making tasks could be improved to achieve faster learning and a more generalizable understanding of task rules.
Collapse
Affiliation(s)
- Lukas T Oesch
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, United States
| | - Michael B Ryan
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, United States. https://twitter.com/NeuroMikeRyan
| | - Anne K Churchland
- Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, United States.
| |
Collapse
|
2
|
Marrero K, Aruljothi K, Delgadillo C, Kabbara S, Swatch L, Zagha E. Goal-Directed Learning is Multidimensional and Accompanied by Diverse and Widespread Changes in Neocortical Signaling. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.02.13.528412. [PMID: 36824924 PMCID: PMC9948952 DOI: 10.1101/2023.02.13.528412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/17/2023]
Abstract
New tasks are often learned in stages with each stage reflecting a different learning challenge. Accordingly, each learning stage is likely mediated by distinct neuronal processes. And yet, most rodent studies of the neuronal correlates of goal-directed learning focus on individual outcome measures and individual brain regions. Here, we longitudinally studied mice from naïve to expert performance in a head-fixed, operant conditioning whisker discrimination task. In addition to tracking the primary behavioral outcome of stimulus discrimination, we tracked and compared an array of object-based and temporal-based behavioral measures. These behavioral analyses identify multiple, partially overlapping learning stages in this task, consistent with initial response implementation, early stimulus-response generalization, and late response inhibition. To begin to understand the neuronal foundations of these learning processes, we performed widefield Ca2+ imaging of dorsal neocortex throughout learning and correlated behavioral measures with neuronal activity. We found distinct and widespread correlations between neocortical activation patterns and various behavioral measures. For example, improvements in sensory discrimination correlated with target stimulus evoked activations of licking-related cortices along with distractor stimulus evoked global cortical suppression. Our study reveals multidimensional learning for a simple goal-directed learning task and generates hypotheses for the neuronal modulations underlying these various learning processes.
Collapse
Affiliation(s)
- Krista Marrero
- Neuroscience Graduate Program, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| | - Krithiga Aruljothi
- Department of Psychology, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| | - Christian Delgadillo
- Division of Biomedical Sciences, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| | - Sarah Kabbara
- Neuroscience Graduate Program, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| | - Lovleen Swatch
- College of Natural & Agricultural Sciences, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| | - Edward Zagha
- Neuroscience Graduate Program, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
- Department of Psychology, University of California, Riverside 900 University Avenue, Riverside CA 92521 USA
| |
Collapse
|
3
|
Ishizu K, Nishimoto S, Ueoka Y, Funamizu A. Localized and global representation of prior value, sensory evidence, and choice in male mouse cerebral cortex. Nat Commun 2024; 15:4071. [PMID: 38778078 PMCID: PMC11111702 DOI: 10.1038/s41467-024-48338-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 04/26/2024] [Indexed: 05/25/2024] Open
Abstract
Adaptive behavior requires integrating prior knowledge of action outcomes and sensory evidence for making decisions while maintaining prior knowledge for future actions. As outcome- and sensory-based decisions are often tested separately, it is unclear how these processes are integrated in the brain. In a tone frequency discrimination task with two sound durations and asymmetric reward blocks, we found that neurons in the medial prefrontal cortex of male mice represented the additive combination of prior reward expectations and choices. The sensory inputs and choices were selectively decoded from the auditory cortex irrespective of reward priors and the secondary motor cortex, respectively, suggesting localized computations of task variables are required within single trials. In contrast, all the recorded regions represented prior values that needed to be maintained across trials. We propose localized and global computations of task variables in different time scales in the cerebral cortex.
Collapse
Affiliation(s)
- Kotaro Ishizu
- Institute for Quantitative Biosciences, University of Tokyo, Laboratory of Neural Computation, 1-1-1 Yayoi, Bunkyo-ku, Tokyo, 113-0032, Japan
| | - Shosuke Nishimoto
- Institute for Quantitative Biosciences, University of Tokyo, Laboratory of Neural Computation, 1-1-1 Yayoi, Bunkyo-ku, Tokyo, 113-0032, Japan
- Department of Life Sciences, Graduate School of Arts and Sciences, University of Tokyo, 3-8-2, Komaba, Meguro-ku, Tokyo, 153-8902, Japan
| | - Yutaro Ueoka
- Institute for Quantitative Biosciences, University of Tokyo, Laboratory of Neural Computation, 1-1-1 Yayoi, Bunkyo-ku, Tokyo, 113-0032, Japan
| | - Akihiro Funamizu
- Institute for Quantitative Biosciences, University of Tokyo, Laboratory of Neural Computation, 1-1-1 Yayoi, Bunkyo-ku, Tokyo, 113-0032, Japan.
- Department of Life Sciences, Graduate School of Arts and Sciences, University of Tokyo, 3-8-2, Komaba, Meguro-ku, Tokyo, 153-8902, Japan.
| |
Collapse
|
4
|
Zhu Z, Kuchibhotla KV. Performance errors during rodent learning reflect a dynamic choice strategy. Curr Biol 2024; 34:2107-2117.e5. [PMID: 38677279 DOI: 10.1016/j.cub.2024.04.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 02/10/2024] [Accepted: 04/08/2024] [Indexed: 04/29/2024]
Abstract
Humans, even as infants, use cognitive strategies, such as exploration and hypothesis testing, to learn about causal interactions in the environment. In animal learning studies, however, it is challenging to disentangle higher-order behavioral strategies from errors arising from imperfect task knowledge or inherent biases. Here, we trained head-fixed mice on a wheel-based auditory two-choice task and exploited the intra- and inter-animal variability to understand the drivers of errors during learning. During learning, performance errors are dominated by a choice bias, which, despite appearing maladaptive, reflects a dynamic strategy. Early in learning, mice develop an internal model of the task contingencies such that violating their expectation of reward on correct trials (by using short blocks of non-rewarded "probe" trials) leads to an abrupt shift in strategy. During the probe block, mice behave more accurately with less bias, thereby using their learned stimulus-action knowledge to test whether the outcome contingencies have changed. Despite having this knowledge, mice continued to exhibit a strong choice bias during reinforced trials. This choice bias operates on a timescale of tens to hundreds of trials with a dynamic structure, shifting between left, right, and unbiased epochs. Biased epochs also coincided with faster motor kinematics. Although bias decreased across learning, expert mice continued to exhibit short bouts of biased choices interspersed with longer bouts of unbiased choices and higher performance. These findings collectively suggest that during learning, rodents actively probe their environment in a structured manner to refine their decision-making and maintain long-term flexibility.
Collapse
Affiliation(s)
- Ziyi Zhu
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA; Johns Hopkins Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD 21218, USA; The Solomon Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA
| | - Kishore V Kuchibhotla
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA; Johns Hopkins Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD 21218, USA; The Solomon Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA; Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.
| |
Collapse
|
5
|
Stüttgen MC, Dietl A, Stoilova Eckert VV, de la Cuesta-Ferrer L, Blanke JH, Koß C, Jäkel F. Influence of reinforcement and its omission on trial-by-trial changes of response bias in perceptual decision making. J Exp Anal Behav 2024; 121:294-313. [PMID: 38426657 DOI: 10.1002/jeab.908] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 02/09/2024] [Indexed: 03/02/2024]
Abstract
Discrimination performance in perceptual choice tasks is known to reflect both sensory discriminability and nonsensory response bias. In the framework of signal detection theory, these aspects of discrimination performance are quantified through separate measures, sensitivity (d') for sensory discriminability and decision criterion (c) for response bias. However, it is unknown how response bias (i.e., criterion) changes at the single-trial level as a consequence of reinforcement history. We subjected rats to a two-stimulus two-response conditional discrimination task with auditory stimuli and induced response bias through unequal reinforcement probabilities for the two responses. We compared three signal-detection-theory-based criterion learning models with respect to their ability to fit experimentally observed fluctuations of response bias on a trial-by-trial level. These models shift the criterion by a fixed step (1) after each reinforced response or (2) after each nonreinforced response or (3) after both. We find that all three models fail to capture essential aspects of the data. Prompted by the observation that steady-state criterion values conformed well to a behavioral model of signal detection based on the generalized matching law, we constructed a trial-based version of this model and find that it provides a superior account of response bias fluctuations under changing reinforcement contingencies.
Collapse
Affiliation(s)
- Maik C Stüttgen
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
| | - Andrea Dietl
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
| | - Vanya V Stoilova Eckert
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
| | - Luis de la Cuesta-Ferrer
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
| | - Jan-Hendrik Blanke
- Institute of Pathophysiology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
| | - Christina Koß
- Centre for Cognitive Science, Institute of Psychology, Technical University of Darmstadt, Germany
| | - Frank Jäkel
- Centre for Cognitive Science, Institute of Psychology, Technical University of Darmstadt, Germany
| |
Collapse
|
6
|
Mayne P, Das J, Zou S, Sullivan RKP, Burne THJ. Perineuronal nets are associated with decision making under conditions of uncertainty in female but not male mice. Behav Brain Res 2024; 461:114845. [PMID: 38184206 DOI: 10.1016/j.bbr.2024.114845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 12/21/2023] [Accepted: 01/02/2024] [Indexed: 01/08/2024]
Abstract
Biological sex influences decision-making processes in significant ways, differentiating the responses animals choose when faced with a range of stimuli. The neurobiological underpinnings that dictate sex differences in decision-making tasks remains an important open question, yet single-sex studies of males form most studies in behavioural neuroscience. Here we used female and male BALB/c mice on two spatial learning and memory tasks and examined the expression of perineuronal nets (PNNs) and parvalbumin interneurons (PV) in regions correlated with spatial memory. Mice underwent the aversive active place avoidance (APA) task or the appetitive trial-unique nonmatching-to-location (TUNL) touchscreen task. Mice in the APA cohort learnt to avoid the foot-shock and no differences were observed on key measures of the task nor in the number and intensity of PNNs and PV. On the delay but not separation manipulation in the TUNL task, females received more incorrect trials and less correct trials compared to males. Furthermore, females in this cohort exhibited higher intensity PNNs and PV cells in the agranular and granular retrosplenial cortex, compared to males. These data show that female and male mice perform similarly on spatial learning tasks. However, sex differences in neural circuitry may underly differences in making decisions under conditions of uncertainty on an appetitive task. These data emphasise the importance of using mice of both sexes in studies of decision-making neuroscience.
Collapse
Affiliation(s)
- Phoebe Mayne
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Joyosmita Das
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Simin Zou
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Robert K P Sullivan
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia
| | - Thomas H J Burne
- Queensland Brain Institute, The University of Queensland, St Lucia, QLD 4072, Australia; Queensland Centre for Mental Health Research, Wacol, QLD 4076, Australia.
| |
Collapse
|
7
|
Gupta D, DePasquale B, Kopec CD, Brody CD. Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making. Nat Commun 2024; 15:662. [PMID: 38253526 PMCID: PMC10803295 DOI: 10.1038/s41467-024-44880-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 01/04/2024] [Indexed: 01/24/2024] Open
Abstract
Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.
Collapse
Affiliation(s)
- Diksha Gupta
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Sainsbury Wellcome Centre, University College London, London, UK.
| | - Brian DePasquale
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Charles D Kopec
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Carlos D Brody
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Howard Hughes Medical Institute, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
8
|
Tumkaya S, Yücens B, Gündüz M, Maheu M, Berkovitch L. Disruption of consciousness depends on insight in OCD and on positive symptoms in schizophrenia. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.02.571832. [PMID: 38293050 PMCID: PMC10827121 DOI: 10.1101/2024.01.02.571832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2024]
Abstract
Disruption of conscious access contributes to the advent of psychotic symptoms in schizophrenia but could also explain lack of insight in other psychiatric disorders. In this study, we explored how insight and psychotic symptoms related to disruption of consciousness. We explored consciousness in patients with schizophrenia, patients with obsessive-compulsive disorder (OCD) with good vs. poor insight and matched controls. Participants underwent clinical assessments and performed a visual masking task allowing us to measure individual consciousness threshold. We used a principal component analysis to reduce symptom dimensionality and explored how consciousness measures related to symptomatology. We found that clinical dimensions could be well summarized by a restricted set of principal components which also correlated with the extent of consciousness disruption. More specifically, positive symptoms were associated with impaired conscious access in patients with schizophrenia whereas the level of insight delineated two subtypes of OCD patients, those with poor insight who had consciousness impairments similar to patients with schizophrenia, and those with good insight who resemble healthy controls. Our study provides new insights about consciousness disruption in psychiatric disorders, showing that it relates to positive symptoms in schizophrenia and with insight in OCD. In OCD, it revealed a distinct subgroup sharing neuropathological features with schizophrenia. Our findings refine the mapping between symptoms and cognition, paving the way for a better treatment selection.
Collapse
Affiliation(s)
- Selim Tumkaya
- Department of Psychiatry, Pamukkale University School of Medicine, Denizli, Turkey
- Department of Neuroscience, Pamukkale University School of Medicine, Denizli, Turkey
| | - Bengü Yücens
- Department of Psychiatry, Pamukkale University School of Medicine, Denizli, Turkey
| | - Muhammet Gündüz
- Department of Psychiatry, Government Hospital of Bolvadin, Bolvadin, Turkey
| | - Maxime Maheu
- Department of Neurophysiology and Pathophysiology, Center for Experimental Medicine, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
- Department of Synaptic Physiology, Centre for Molecular Neurobiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Lucie Berkovitch
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, United States
- University Department of Psychiatry, Pôle Hospitalo-Universitaire Psychiatrie Paris 15, Groupe Hospitalier Universitaire Paris, Paris, France
- Saclay CEA Centre, Neurospin, Gif-Sur-Yvette Cedex, France
- Paris Cité University, Paris, France
| |
Collapse
|
9
|
Reinhold K, Iadarola M, Tang S, Kuwamoto W, Sun S, Hakim R, Zimmer J, Wang W, Sabatini BL. Striatum supports fast learning but not memory recall. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.08.566333. [PMID: 37986941 PMCID: PMC10659398 DOI: 10.1101/2023.11.08.566333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
Animals learn to carry out motor actions in specific sensory contexts to achieve goals. The striatum has been implicated in producing sensory-motor associations, yet its contribution to memory formation or recall is not clear. To investigate the contribution of striatum to these processes, mice were taught to associate a cue, consisting of optogenetic activation of striatum-projecting neurons in visual cortex, with forelimb reaches to access food pellets. As necessary to direct learning, striatal neural activity encoded both the sensory context and outcome of reaching. With training, the rate of cued reaching increased, but brief optogenetic inhibition of striatal activity arrested learning and prevented trial-to-trial improvements in performance. However, the same manipulation did not affect performance improvements already consolidated into short- (within an hour) or long-term (across days) memories. Hence, striatal activity is necessary for trial-to-trial improvements in task performance, leading to plasticity in other brain areas that mediate memory recall.
Collapse
|
10
|
Funamizu A, Marbach F, Zador AM. Stable sound decoding despite modulated sound representation in the auditory cortex. Curr Biol 2023; 33:4470-4483.e7. [PMID: 37802051 PMCID: PMC10665086 DOI: 10.1016/j.cub.2023.09.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 07/27/2023] [Accepted: 09/13/2023] [Indexed: 10/08/2023]
Abstract
The activity of neurons in the auditory cortex is driven by both sounds and non-sensory context. To investigate the neuronal correlates of non-sensory context, we trained head-fixed mice to perform a two-alternative-choice auditory task in which either reward or stimulus expectation (prior) was manipulated in blocks. Using two-photon calcium imaging to record populations of single neurons in the auditory cortex, we found that both stimulus and reward expectation modulated the activity of these neurons. A linear decoder trained on this population activity could decode stimuli as well or better than predicted by the animal's performance. Interestingly, the optimal decoder was stable even in the face of variable sensory representations. Neither the context nor the mouse's choice could be reliably decoded from the recorded neural activity. Our findings suggest that, in spite of modulation of auditory cortical activity by task priors, the auditory cortex does not represent sufficient information about these priors to exploit them optimally. Thus, the combination of rapidly changing sensory information with more slowly varying task information required for decisions in this task might be represented in brain regions other than the auditory cortex.
Collapse
Affiliation(s)
- Akihiro Funamizu
- Cold Spring Harbor Laboratory, 1 Bungtown Rd, Cold Spring Harbor, NY 11724, USA.
| | - Fred Marbach
- Cold Spring Harbor Laboratory, 1 Bungtown Rd, Cold Spring Harbor, NY 11724, USA
| | - Anthony M Zador
- Cold Spring Harbor Laboratory, 1 Bungtown Rd, Cold Spring Harbor, NY 11724, USA
| |
Collapse
|
11
|
Le NM, Yildirim M, Wang Y, Sugihara H, Jazayeri M, Sur M. Mixtures of strategies underlie rodent behavior during reversal learning. PLoS Comput Biol 2023; 19:e1011430. [PMID: 37708113 PMCID: PMC10501641 DOI: 10.1371/journal.pcbi.1011430] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 08/09/2023] [Indexed: 09/16/2023] Open
Abstract
In reversal learning tasks, the behavior of humans and animals is often assumed to be uniform within single experimental sessions to facilitate data analysis and model fitting. However, behavior of agents can display substantial variability in single experimental sessions, as they execute different blocks of trials with different transition dynamics. Here, we observed that in a deterministic reversal learning task, mice display noisy and sub-optimal choice transitions even at the expert stages of learning. We investigated two sources of the sub-optimality in the behavior. First, we found that mice exhibit a high lapse rate during task execution, as they reverted to unrewarded directions after choice transitions. Second, we unexpectedly found that a majority of mice did not execute a uniform strategy, but rather mixed between several behavioral modes with different transition dynamics. We quantified the use of such mixtures with a state-space model, block Hidden Markov Model (block HMM), to dissociate the mixtures of dynamic choice transitions in individual blocks of trials. Additionally, we found that blockHMM transition modes in rodent behavior can be accounted for by two different types of behavioral algorithms, model-free or inference-based learning, that might be used to solve the task. Combining these approaches, we found that mice used a mixture of both exploratory, model-free strategies and deterministic, inference-based behavior in the task, explaining their overall noisy choice sequences. Together, our combined computational approach highlights intrinsic sources of noise in rodent reversal learning behavior and provides a richer description of behavior than conventional techniques, while uncovering the hidden states that underlie the block-by-block transitions.
Collapse
Affiliation(s)
- Nhat Minh Le
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Murat Yildirim
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- Department of Neurosciences, Cleveland Clinic Lerner Research Institute, Cleveland, Ohio, United States of America
| | - Yizhi Wang
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Hiroki Sugihara
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Mehrdad Jazayeri
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| | - Mriganka Sur
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America
| |
Collapse
|
12
|
Coen P, Sit TPH, Wells MJ, Carandini M, Harris KD. Mouse frontal cortex mediates additive multisensory decisions. Neuron 2023; 111:2432-2447.e13. [PMID: 37295419 PMCID: PMC10957398 DOI: 10.1016/j.neuron.2023.05.008] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 12/02/2022] [Accepted: 05/10/2023] [Indexed: 06/12/2023]
Abstract
The brain can combine auditory and visual information to localize objects. However, the cortical substrates underlying audiovisual integration remain uncertain. Here, we show that mouse frontal cortex combines auditory and visual evidence; that this combination is additive, mirroring behavior; and that it evolves with learning. We trained mice in an audiovisual localization task. Inactivating frontal cortex impaired responses to either sensory modality, while inactivating visual or parietal cortex affected only visual stimuli. Recordings from >14,000 neurons indicated that after task learning, activity in the anterior part of frontal area MOs (secondary motor cortex) additively encodes visual and auditory signals, consistent with the mice's behavioral strategy. An accumulator model applied to these sensory representations reproduced the observed choices and reaction times. These results suggest that frontal cortex adapts through learning to combine evidence across sensory cortices, providing a signal that is transformed into a binary decision by a downstream accumulator.
Collapse
Affiliation(s)
- Philip Coen
- UCL Queen Square Institute of Neurology, University College London, London, UK; UCL Institute of Ophthalmology, University College London, London, UK.
| | - Timothy P H Sit
- Sainsbury-Wellcome Center, University College London, London, UK
| | - Miles J Wells
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Matteo Carandini
- UCL Institute of Ophthalmology, University College London, London, UK
| | - Kenneth D Harris
- UCL Queen Square Institute of Neurology, University College London, London, UK
| |
Collapse
|
13
|
Shourkeshti A, Marrocco G, Jurewicz K, Moore T, Ebitz RB. Pupil size predicts the onset of exploration in brain and behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.24.541981. [PMID: 37292773 PMCID: PMC10245915 DOI: 10.1101/2023.05.24.541981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible.
Collapse
Affiliation(s)
- Akram Shourkeshti
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| | - Gabriel Marrocco
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| | - Katarzyna Jurewicz
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
- Department of Physiology, McGill University, Montréal, QC, Canada
| | - Tirin Moore
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| | - R. Becket Ebitz
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| |
Collapse
|
14
|
Cazettes F, Mazzucato L, Murakami M, Morais JP, Augusto E, Renart A, Mainen ZF. A reservoir of foraging decision variables in the mouse brain. Nat Neurosci 2023; 26:840-849. [PMID: 37055628 PMCID: PMC10280691 DOI: 10.1038/s41593-023-01305-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Accepted: 03/15/2023] [Indexed: 04/15/2023]
Abstract
In any given situation, the environment can be parsed in different ways to yield decision variables (DVs) defining strategies useful for different tasks. It is generally presumed that the brain only computes a single DV defining the current behavioral strategy. Here to test this assumption, we recorded neural ensembles in the frontal cortex of mice performing a foraging task admitting multiple DVs. Methods developed to uncover the currently employed DV revealed the use of multiple strategies and occasional switches in strategy within sessions. Optogenetic manipulations showed that the secondary motor cortex (M2) is needed for mice to use the different DVs in the task. Surprisingly, we found that regardless of which DV best explained the current behavior, M2 activity concurrently encoded a full basis set of computations defining a reservoir of DVs appropriate for alternative tasks. This form of neural multiplexing may confer considerable advantages for learning and adaptive behavior.
Collapse
Affiliation(s)
| | - Luca Mazzucato
- Departments of Biology, Mathematics & Physics, Institute of Neuroscience, University of Oregon, Eugene, OR, USA
| | - Masayoshi Murakami
- Champalimaud Foundation, Lisbon, Portugal
- Department of Neurophysiology, University of Yamanashi, Yamanashi, Japan
| | | | | | | | | |
Collapse
|
15
|
Barnett WH, Kuznetsov A, Lapish CC. Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior. PLoS One 2023; 18:e0279841. [PMID: 36943842 PMCID: PMC10030038 DOI: 10.1371/journal.pone.0279841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 12/15/2022] [Indexed: 03/23/2023] Open
Abstract
Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior.
Collapse
Affiliation(s)
- William H. Barnett
- Department of Psychology, Indiana University—Purdue University Indianapolis, Indianapolis, Indiana, United States of America
| | - Alexey Kuznetsov
- Department of Mathematics, Indiana University—Purdue University Indianapolis, Indianapolis, Indiana, United States of America
| | - Christopher C. Lapish
- Department of Psychology, Indiana University—Purdue University Indianapolis, Indianapolis, Indiana, United States of America
- Stark Neurosciences Research Institute, Indiana University—Purdue University Indianapolis, Indianapolis, Indiana, United States of America
| |
Collapse
|
16
|
Gupta D, DePasquale B, Kopec CD, Brody CD. Trial-history biases in evidence accumulation can give rise to apparent lapses. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.18.524599. [PMID: 36778392 PMCID: PMC9915493 DOI: 10.1101/2023.01.18.524599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.
Collapse
Affiliation(s)
- Diksha Gupta
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Brian DePasquale
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Charles D Kopec
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Carlos D Brody
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
- Howard Hughes Medical Institute, Princeton University, Princeton, United States
| |
Collapse
|
17
|
Mafi F, Tang MF, Afarinesh MR, Ghasemian S, Sheibani V, Arabzadeh E. Temporal order judgment of multisensory stimuli in rat and human. Front Behav Neurosci 2023; 16:1070452. [PMID: 36710957 PMCID: PMC9879721 DOI: 10.3389/fnbeh.2022.1070452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 12/16/2022] [Indexed: 01/13/2023] Open
Abstract
We do not fully understand the resolution at which temporal information is processed by different species. Here we employed a temporal order judgment (TOJ) task in rats and humans to test the temporal precision with which these species can detect the order of presentation of simple stimuli across two modalities of vision and audition. Both species reported the order of audiovisual stimuli when they were presented from a central location at a range of stimulus onset asynchronies (SOA)s. While both species could reliably distinguish the temporal order of stimuli based on their sensory content (i.e., the modality label), rats outperformed humans at short SOAs (less than 100 ms) whereas humans outperformed rats at long SOAs (greater than 100 ms). Moreover, rats produced faster responses compared to humans. The reaction time data further revealed key differences in decision process across the two species: at longer SOAs, reaction times increased in rats but decreased in humans. Finally, drift-diffusion modeling allowed us to isolate the contribution of various parameters including evidence accumulation rates, lapse and bias to the sensory decision. Consistent with the psychophysical findings, the model revealed higher temporal sensitivity and a higher lapse rate in rats compared to humans. These findings suggest that these species applied different strategies for making perceptual decisions in the context of a multimodal TOJ task.
Collapse
Affiliation(s)
- Fatemeh Mafi
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Cognitive Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran
| | - Matthew F. Tang
- Eccles Institute of Neuroscience, John Curtin School of Medical Research, The Australian National University, Canberra, ACT, Australia
| | - Mohammad Reza Afarinesh
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Cognitive Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran
| | - Sadegh Ghasemian
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Cognitive Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran
| | - Vahid Sheibani
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Cognitive Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran
| | - Ehsan Arabzadeh
- Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Cognitive Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran,Eccles Institute of Neuroscience, John Curtin School of Medical Research, The Australian National University, Canberra, ACT, Australia,*Correspondence: Ehsan Arabzadeh,
| |
Collapse
|
18
|
Drevet J, Drugowitsch J, Wyart V. Efficient stabilization of imprecise statistical inference through conditional belief updating. Nat Hum Behav 2022; 6:1691-1704. [PMID: 36138224 DOI: 10.1038/s41562-022-01445-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Accepted: 08/11/2022] [Indexed: 01/14/2023]
Abstract
Statistical inference is the optimal process for forming and maintaining accurate beliefs about uncertain environments. However, human inference comes with costs due to its associated biases and limited precision. Indeed, biased or imprecise inference can trigger variable beliefs and unwarranted changes in behaviour. Here, by studying decisions in a sequential categorization task based on noisy visual stimuli, we obtained converging evidence that humans reduce the variability of their beliefs by updating them only when the reliability of incoming sensory information is judged as sufficiently strong. Instead of integrating the evidence provided by all stimuli, participants actively discarded as much as a third of stimuli. This conditional belief updating strategy shows good test-retest reliability, correlates with perceptual confidence and explains human behaviour better than previously described strategies. This seemingly suboptimal strategy not only reduces the costs of imprecise computations but also, counterintuitively, increases the accuracy of resulting decisions.
Collapse
Affiliation(s)
- Julie Drevet
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France.
- Département d'Études Cognitives, École Normale Supérieure, Université PSL, Paris, France.
| | - Jan Drugowitsch
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Valentin Wyart
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale (Inserm), Paris, France.
- Département d'Études Cognitives, École Normale Supérieure, Université PSL, Paris, France.
| |
Collapse
|
19
|
Pisupati S, Niv Y. The challenges of lifelong learning in biological and artificial systems. Trends Cogn Sci 2022; 26:1051-1053. [PMID: 36335012 PMCID: PMC9676180 DOI: 10.1016/j.tics.2022.09.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 09/28/2022] [Indexed: 11/11/2022]
Abstract
How do biological systems learn continuously throughout their lifespans, adapting to change while retaining old knowledge, and how can these principles be applied to artificial learning systems? In this Forum article we outline challenges and strategies of 'lifelong learning' in biological and artificial systems, and argue that a collaborative study of each system's failure modes can benefit both.
Collapse
|
20
|
Manneschi L, Gigante G, Vasilaki E, Del Giudice P. Signal neutrality, scalar property, and collapsing boundaries as consequences of a learned multi-timescale strategy. PLoS Comput Biol 2022; 18:e1009393. [PMID: 35930590 PMCID: PMC9462745 DOI: 10.1371/journal.pcbi.1009393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 09/09/2022] [Accepted: 06/08/2022] [Indexed: 11/18/2022] Open
Abstract
We postulate that three fundamental elements underlie a decision making process: perception of time passing, information processing in multiple timescales and reward maximisation. We build a simple reinforcement learning agent upon these principles that we train on a random dot-like task. Our results, similar to the experimental data, demonstrate three emerging signatures. (1) signal neutrality: insensitivity to the signal coherence in the interval preceding the decision. (2) Scalar property: the mean of the response times varies widely for different signal coherences, yet the shape of the distributions stays almost unchanged. (3) Collapsing boundaries: the “effective” decision-making boundary changes over time in a manner reminiscent of the theoretical optimal. Removing the perception of time or the multiple timescales from the model does not preserve the distinguishing signatures. Our results suggest an alternative explanation for signal neutrality. We propose that it is not part of motor planning. It is part of the decision-making process and emerges from information processing on multiple timescales.
Collapse
Affiliation(s)
- Luca Manneschi
- Department of Computer Science, University of Sheffield, Sheffield, United Kingdom
- * E-mail:
| | - Guido Gigante
- Istituto Superiore di Sanità, Rome, Italy
- INFN, Sezione di Roma, Rome, Italy
| | - Eleni Vasilaki
- Department of Computer Science, University of Sheffield, Sheffield, United Kingdom
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland
| | - Paolo Del Giudice
- Istituto Superiore di Sanità, Rome, Italy
- INFN, Sezione di Roma, Rome, Italy
| |
Collapse
|
21
|
Bellucci G, Münte TF, Park SQ. Influences of social uncertainty and serotonin on gambling decisions. Sci Rep 2022; 12:10220. [PMID: 35715450 PMCID: PMC9205937 DOI: 10.1038/s41598-022-13778-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Accepted: 05/12/2022] [Indexed: 11/24/2022] Open
Abstract
In many instances in life, our decisions’ outcomes hinge on someone else’s choices (i.e., under social uncertainty). Behavioral and pharmacological work has previously focused on different types of uncertainty, such as risk and ambiguity, but not so much on risk behaviors under social uncertainty. Here, in two different studies using a double-blind, placebo-controlled, within-subject design, we administrated citalopram (a selective-serotonin-reuptake inhibitor) to male participants and investigated decisions in a gambling task under social and nonsocial uncertainty. In the social condition, gamble outcomes were determined by another participant. In the nonsocial condition, gamble outcomes were determined by a coin toss. We observed increased gamble acceptance under social uncertainty, especially for gambles with lower gains and higher losses, which might be indicative of a positivity bias in social expectations in conditions of high uncertainty about others’ behaviors. A similar effect was found for citalopram, which increased overall acceptance behavior for gambles irrespective of the source of uncertainty (social/nonsocial). These results provide insights into the cognitive and neurochemical processes underlying decisions under social uncertainty, with implications for research in risk-taking behaviors in healthy and clinical populations.
Collapse
Affiliation(s)
- Gabriele Bellucci
- Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, 72076, Tübingen, Germany. .,Department of Psychology I, University of Lübeck, Lübeck, Germany. .,Decision Neuroscience and Nutrition, German Institute of Human Nutrition (DIfE), Potsdam-Rehbruecke, Nuthetal, Germany.
| | - Thomas F Münte
- Department of Neurology, Universitätsklinikum Schleswig-Holstein, Lübeck, Germany.,Department of Psychology II, University of Lübeck, Lübeck, Germany
| | - Soyoung Q Park
- Department of Psychology I, University of Lübeck, Lübeck, Germany. .,Decision Neuroscience and Nutrition, German Institute of Human Nutrition (DIfE), Potsdam-Rehbruecke, Nuthetal, Germany. .,Neuroscience Research Center, Berlin Institute of Health, Corporate Member of Freie Universität Berlin, Charité-Universitätsmedizin Berlin, Humboldt-Universität zu Berlin, Berlin, Germany. .,Deutsches Zentrum für Diabetes, Neuherberg, Germany.
| |
Collapse
|
22
|
Miller CT, Gire D, Hoke K, Huk AC, Kelley D, Leopold DA, Smear MC, Theunissen F, Yartsev M, Niell CM. Natural behavior is the language of the brain. Curr Biol 2022; 32:R482-R493. [PMID: 35609550 PMCID: PMC10082559 DOI: 10.1016/j.cub.2022.03.031] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
The breadth and complexity of natural behaviors inspires awe. Understanding how our perceptions, actions, and internal thoughts arise from evolved circuits in the brain has motivated neuroscientists for generations. Researchers have traditionally approached this question by focusing on stereotyped behaviors, either natural or trained, in a limited number of model species. This approach has allowed for the isolation and systematic study of specific brain operations, which has greatly advanced our understanding of the circuits involved. At the same time, the emphasis on experimental reductionism has left most aspects of the natural behaviors that have shaped the evolution of the brain largely unexplored. However, emerging technologies and analytical tools make it possible to comprehensively link natural behaviors to neural activity across a broad range of ethological contexts and timescales, heralding new modes of neuroscience focused on natural behaviors. Here we describe a three-part roadmap that aims to leverage the wealth of behaviors in their naturally occurring distributions, linking their variance with that of underlying neural processes to understand how the brain is able to successfully navigate the everyday challenges of animals' social and ecological landscapes. To achieve this aim, experimenters must harness one challenge faced by all neurobiological systems, namely variability, in order to gain new insights into the language of the brain.
Collapse
Affiliation(s)
- Cory T Miller
- Cortical Systems and Behavior Laboratory, University of California San Diego, 9500 Gilman Drive, La Jolla, CA 92039, USA.
| | - David Gire
- Department of Psychology, University of Washington, Guthrie Hall, Seattle, WA 98105, USA
| | - Kim Hoke
- Department of Biology, Colorado State University, 1878 Campus Delivery, Fort Collins, CO 80523, USA
| | - Alexander C Huk
- Center for Perceptual Systems, Departments of Neuroscience and Psychology, University of Texas at Austin, 116 Inner Campus Drive, Austin, TX 78712, USA
| | - Darcy Kelley
- Department of Biological Sciences, Columbia University, 1212 Amsterdam Avenue, New York, NY 10027, USA
| | - David A Leopold
- Section of Cognitive Neurophysiology and Imaging, National Institute of Mental Health, 49 Convent Drive, Bethesda, MD 20892, USA
| | - Matthew C Smear
- Department of Psychology and Institute of Neuroscience, University of Oregon, 1227 University Street, Eugene, OR 97403, USA
| | - Frederic Theunissen
- Department of Psychology, University of California Berkeley, 2121 Berkeley Way, Berkeley, CA 94720, USA
| | - Michael Yartsev
- Department of Bioengineering, University of California Berkeley, 306 Stanley Hall, Berkeley, CA 94720, USA
| | - Cristopher M Niell
- Department of Biology and Institute of Neuroscience, University of Oregon, 222 Huestis Hall, Eugene, OR 97403, USA.
| |
Collapse
|
23
|
Chen G, Gong P. A spatiotemporal mechanism of visual attention: Superdiffusive motion and theta oscillations of neural population activity patterns. SCIENCE ADVANCES 2022; 8:eabl4995. [PMID: 35452293 PMCID: PMC9032965 DOI: 10.1126/sciadv.abl4995] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
Recent evidence has demonstrated that during visual spatial attention sampling, neural activity and behavioral performance exhibit large fluctuations. To understand the origin of these fluctuations and their functional role, here, we introduce a mechanism based on the dynamical activity pattern (attention spotlight) emerging from neural circuit models in the transition regime between different dynamical states. This attention activity pattern with rich spatiotemporal dynamics flexibly samples from different stimulus locations, explaining many key aspects of temporal fluctuations such as variable theta oscillations of visual spatial attention. Moreover, the mechanism expands our understanding of how visual attention exploits spatially complex fluctuations characterized by superdiffusive motion in space and makes experimentally testable predictions. We further illustrate that attention sampling based on such spatiotemporal fluctuations provides profound functional advantages such as adaptive switching between exploitation and exploration activities and is particularly efficient at sampling natural scenes with multiple salient objects.
Collapse
Affiliation(s)
- Guozhang Chen
- School of Physics, University of Sydney, NSW 2006, Australia
- ARC Center of Excellence for Integrative Brain Function, University of Sydney, NSW 2006, Australia
- Institute of Theoretical Computer Science, Graz University of Technology, Graz, Austria
| | - Pulin Gong
- School of Physics, University of Sydney, NSW 2006, Australia
- ARC Center of Excellence for Integrative Brain Function, University of Sydney, NSW 2006, Australia
- Corresponding author.
| |
Collapse
|
24
|
Schreiner DC, Cazares C, Renteria R, Gremel CM. Information normally considered task-irrelevant drives decision-making and affects premotor circuit recruitment. Nat Commun 2022; 13:2134. [PMID: 35440120 PMCID: PMC9018678 DOI: 10.1038/s41467-022-29807-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Accepted: 03/24/2022] [Indexed: 02/02/2023] Open
Abstract
Decision-making is a continuous and dynamic process with prior experience reflected in and used by the brain to guide adaptive behavior. However, most neurobiological studies constrain behavior and/or analyses to task-related variables, not accounting for the continuous internal and temporal space in which they occur. We show mice rely on information learned through recent and longer-term experience beyond just prior actions and reward - including checking behavior and the passage of time - to guide self-initiated, self-paced, and self-generated actions. These experiences are represented in secondary motor cortex (M2) activity and its projections into dorsal medial striatum (DMS). M2 integrates this information to bias strategy-level decision-making, and DMS projections reflect specific aspects of this recent experience to guide actions. This suggests diverse aspects of experience drive decision-making and its neural representation, and shows premotor corticostriatal circuits are crucial for using selective aspects of experiential information to guide adaptive behavior.
Collapse
Affiliation(s)
- Drew C Schreiner
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA
| | - Christian Cazares
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, 92093, USA
| | - Rafael Renteria
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA
| | - Christina M Gremel
- Department of Psychology, University of California San Diego, La Jolla, CA, 92093, USA.
- The Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, 92093, USA.
| |
Collapse
|
25
|
Mice exhibit stochastic and efficient action switching during probabilistic decision making. Proc Natl Acad Sci U S A 2022; 119:e2113961119. [PMID: 35385355 PMCID: PMC9169659 DOI: 10.1073/pnas.2113961119] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/05/2022] Open
Abstract
To obtain rewards in changing and uncertain environments, animals must adapt their behavior. We found that mouse choice and trial-to-trial switching behavior in a dynamic and probabilistic two-choice task could be modeled by equivalent theoretical, algorithmic, and descriptive models. These models capture components of evidence accumulation, choice history bias, and stochasticity in mouse behavior. Furthermore, they reveal that mice adapt their behavior in different environmental contexts by modulating their level of stickiness to their previous choice. Despite deviating from the behavior of a theoretically ideal observer, the empirical models achieve comparable levels of near-maximal reward. These results make predictions to guide interrogation of the neural mechanisms underlying flexible decision-making strategies. In probabilistic and nonstationary environments, individuals must use internal and external cues to flexibly make decisions that lead to desirable outcomes. To gain insight into the process by which animals choose between actions, we trained mice in a task with time-varying reward probabilities. In our implementation of such a two-armed bandit task, thirsty mice use information about recent action and action–outcome histories to choose between two ports that deliver water probabilistically. Here we comprehensively modeled choice behavior in this task, including the trial-to-trial changes in port selection, i.e., action switching behavior. We find that mouse behavior is, at times, deterministic and, at others, apparently stochastic. The behavior deviates from that of a theoretically optimal agent performing Bayesian inference in a hidden Markov model (HMM). We formulate a set of models based on logistic regression, reinforcement learning, and sticky Bayesian inference that we demonstrate are mathematically equivalent and that accurately describe mouse behavior. The switching behavior of mice in the task is captured in each model by a stochastic action policy, a history-dependent representation of action value, and a tendency to repeat actions despite incoming evidence. The models parsimoniously capture behavior across different environmental conditionals by varying the stickiness parameter, and like the mice, they achieve nearly maximal reward rates. These results indicate that mouse behavior reaches near-maximal performance with reduced action switching and can be described by a set of equivalent models with a small number of relatively fixed parameters.
Collapse
|
26
|
Logistic analysis of choice data: A primer. Neuron 2022; 110:1615-1630. [PMID: 35334232 DOI: 10.1016/j.neuron.2022.03.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2022] [Revised: 02/25/2022] [Accepted: 02/28/2022] [Indexed: 11/21/2022]
Abstract
Logistic regressions were developed in economics to model individual choice behavior. In recent years, they have become an important tool in decision neuroscience. Here, I describe and discuss different logistic models, emphasizing the underlying assumptions and possible interpretations. Logistic models may be used to quantify a variety of behavioral traits, including the relative subjective value of different goods, the choice accuracy, risk attitudes, and choice biases. More complex logistic models can be used for choices between good bundles, in cases of nonlinear value functions, and for choices between multiple options. Finally, logistic models can quantify the explanatory power of neuronal activity on choices, thus providing a valid alternative to receiver operating characteristic (ROC) analyses.
Collapse
|
27
|
Histed MH, O’Rawe JF. From choices to internal states. Nat Neurosci 2022; 25:138-139. [DOI: 10.1038/s41593-021-01008-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
28
|
Ashwood ZC, Roy NA, Stone IR, Urai AE, Churchland AK, Pouget A, Pillow JW. Mice alternate between discrete strategies during perceptual decision-making. Nat Neurosci 2022; 25:201-212. [PMID: 35132235 PMCID: PMC8890994 DOI: 10.1038/s41593-021-01007-z] [Citation(s) in RCA: 52] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2021] [Accepted: 12/17/2021] [Indexed: 12/21/2022]
Abstract
Classical models of perceptual decision-making assume that subjects use a single, consistent strategy to form decisions, or that decision-making strategies evolve slowly over time. Here we present new analyses suggesting that this common view is incorrect. We analyzed data from mouse and human decision-making experiments and found that choice behavior relies on an interplay among multiple interleaved strategies. These strategies, characterized by states in a hidden Markov model, persist for tens to hundreds of trials before switching, and often switch multiple times within a session. The identified decision-making strategies were highly consistent across mice and comprised a single 'engaged' state, in which decisions relied heavily on the sensory stimulus, and several biased states in which errors frequently occurred. These results provide a powerful alternate explanation for 'lapses' often observed in rodent behavioral experiments, and suggest that standard measures of performance mask the presence of major changes in strategy across trials.
Collapse
Affiliation(s)
- Zoe C Ashwood
- Deptartment of Computer Science, Princeton University, Princeton, NJ, USA.
- Princeton Neuroscience Institute, Princeton, NJ, USA.
| | | | - Iris R Stone
- Princeton Neuroscience Institute, Princeton, NJ, USA
| | - Anne E Urai
- Cognitive Psychology Unit, Leiden University, Leiden, Netherlands
| | - Anne K Churchland
- David Geffen School of Medicine, The University of California, Los Angeles, Los Angeles, CA, USA
| | - Alexandre Pouget
- Faculty of Medicine & Deptartment of Basic Neurosciences, University of Geneva, Geneva, Switzerland
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
29
|
Ebitz RB, Hayden BY. The population doctrine in cognitive neuroscience. Neuron 2021; 109:3055-3068. [PMID: 34416170 PMCID: PMC8725976 DOI: 10.1016/j.neuron.2021.07.011] [Citation(s) in RCA: 60] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 07/02/2021] [Accepted: 07/13/2021] [Indexed: 01/08/2023]
Abstract
A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.
Collapse
Affiliation(s)
- R Becket Ebitz
- Department of Neurosciences, Faculté de médecine, Université de Montréal, Montréal, QC, Canada.
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
30
|
Choice history effects in mice and humans improve reward harvesting efficiency. PLoS Comput Biol 2021; 17:e1009452. [PMID: 34606493 PMCID: PMC8516315 DOI: 10.1371/journal.pcbi.1009452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Revised: 10/14/2021] [Accepted: 09/15/2021] [Indexed: 12/04/2022] Open
Abstract
Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias. Animals foraging for food in natural habitats compete to obtain better quality food patches. To achieve this goal, animals can rely on memory and choose the same patches that have provided higher quality of food in the past. However, in natural habitats simply identifying better food patches may not be sufficient to successfully compete with their conspecifics, as food resources can grow over time. Therefore, it makes sense to visit from time to time those patches that were associated with lower food quality in the past. This demands optimal foraging animals to keep in memory not only which food patches provided the best food quality, but also which food patches they visited recently. To see if animals track their history of visits and use it to maximize the food harvesting efficiency, we subjected them to experimental conditions that mimicked natural foraging behavior. In our behavioral tasks, we replaced food foraging behavior with a two choice task that provided rewards to mice and humans. By developing a new computational model and subjecting animals to various behavioral manipulations, we demonstrate that keeping a memory of past visits helps the animals to optimize the efficiency with which they can harvest rewards.
Collapse
|
31
|
Funamizu A. Integration of sensory evidence and reward expectation in mouse perceptual decision-making task with various sensory uncertainties. iScience 2021; 24:102826. [PMID: 34355152 PMCID: PMC8319806 DOI: 10.1016/j.isci.2021.102826] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2021] [Revised: 06/07/2021] [Accepted: 07/05/2021] [Indexed: 11/16/2022] Open
Abstract
In perceptual decision-making, prior knowledge of action outcomes is essential, especially when sensory inputs are insufficient for proper choices. Signal detection theory (SDT) shows that optimal choice bias depends not only on the prior but also the sensory uncertainty; however, it is unclear how animals integrate sensory inputs with various uncertainties and reward expectations to optimize choices. We developed a tone-frequency discrimination task for head-fixed mice in which we randomly presented either a long or short sound stimulus and biased the choice outcomes. The choice was less accurate and more biased toward the large-reward side in short- than in long-stimulus trials. Analysis with SDT found that mice did not use a separate, optimal choice threshold in different sound durations. Instead, mice updated one threshold for short and long stimuli with a simple reinforcement-learning rule. Our task in head-fixed mice helps understanding how the brain integrates sensory inputs and prior.
Collapse
Affiliation(s)
- Akihiro Funamizu
- Institute for Quantitative Biosciences, University of Tokyo, Laboratory of Neural Computation, 1-1-1 Yayoi, Bunkyo-ku, Tokyo 113-0032, Japan
- Department of Life Sciences, Graduate School of Arts and Sciences, University of Tokyo, Meguro-ku, Tokyo 153-8902, Japan
| |
Collapse
|
32
|
Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias. Neurosci Res 2021; 173:1-13. [PMID: 34274406 DOI: 10.1016/j.neures.2021.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 06/15/2021] [Accepted: 07/13/2021] [Indexed: 11/24/2022]
Abstract
The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.
Collapse
|
33
|
Rosenberg M, Zhang T, Perona P, Meister M. Mice in a labyrinth show rapid learning, sudden insight, and efficient exploration. eLife 2021; 10:66175. [PMID: 34196271 PMCID: PMC8294850 DOI: 10.7554/elife.66175] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Accepted: 06/30/2021] [Indexed: 12/20/2022] Open
Abstract
Animals learn certain complex tasks remarkably fast, sometimes after a single experience. What behavioral algorithms support this efficiency? Many contemporary studies based on two-alternative-forced-choice (2AFC) tasks observe only slow or incomplete learning. As an alternative, we study the unconstrained behavior of mice in a complex labyrinth and measure the dynamics of learning and the behaviors that enable it. A mouse in the labyrinth makes ~2000 navigation decisions per hour. The animal explores the maze, quickly discovers the location of a reward, and executes correct 10-bit choices after only 10 reward experiences — a learning rate 1000-fold higher than in 2AFC experiments. Many mice improve discontinuously from one minute to the next, suggesting moments of sudden insight about the structure of the labyrinth. The underlying search algorithm does not require a global memory of places visited and is largely explained by purely local turning rules.
Collapse
Affiliation(s)
- Matthew Rosenberg
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, United States
| | - Tony Zhang
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, United States
| | - Pietro Perona
- Division of Engineering and Applied Science, California Institute of Technology, Pasadena, United States
| | - Markus Meister
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, United States
| |
Collapse
|
34
|
Jia X, Hong H, DiCarlo JJ. Unsupervised changes in core object recognition behavior are predicted by neural plasticity in inferior temporal cortex. eLife 2021; 10:e60830. [PMID: 34114566 PMCID: PMC8324291 DOI: 10.7554/elife.60830] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2020] [Accepted: 06/10/2021] [Indexed: 11/13/2022] Open
Abstract
Temporal continuity of object identity is a feature of natural visual input and is potentially exploited - in an unsupervised manner - by the ventral visual stream to build the neural representation in inferior temporal (IT) cortex. Here, we investigated whether plasticity of individual IT neurons underlies human core object recognition behavioral changes induced with unsupervised visual experience. We built a single-neuron plasticity model combined with a previously established IT population-to-recognition-behavior-linking model to predict human learning effects. We found that our model, after constrained by neurophysiological data, largely predicted the mean direction, magnitude, and time course of human performance changes. We also found a previously unreported dependency of the observed human performance change on the initial task difficulty. This result adds support to the hypothesis that tolerant core object recognition in human and non-human primates is instructed - at least in part - by naturally occurring unsupervised temporal contiguity experience.
Collapse
Affiliation(s)
- Xiaoxuan Jia
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain ResearchCambridgeUnited States
| | - Ha Hong
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain ResearchCambridgeUnited States
- Harvard-MIT Division of Health Sciences and TechnologyCambridgeUnited States
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, Massachusetts Institute of TechnologyCambridgeUnited States
- McGovern Institute for Brain ResearchCambridgeUnited States
- Center for Brains, Minds and MachinesCambridgeUnited States
| |
Collapse
|
35
|
Aguillon-Rodriguez V, Angelaki D, Bayer H, Bonacchi N, Carandini M, Cazettes F, Chapuis G, Churchland AK, Dan Y, Dewitt E, Faulkner M, Forrest H, Haetzel L, Häusser M, Hofer SB, Hu F, Khanal A, Krasniak C, Laranjeira I, Mainen ZF, Meijer G, Miska NJ, Mrsic-Flogel TD, Murakami M, Noel JP, Pan-Vazquez A, Rossant C, Sanders J, Socha K, Terry R, Urai AE, Vergara H, Wells M, Wilson CJ, Witten IB, Wool LE, Zador AM. Standardized and reproducible measurement of decision-making in mice. eLife 2021; 10:63711. [PMID: 34011433 DOI: 10.1101/2020.01.17.909838] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2020] [Accepted: 04/08/2021] [Indexed: 05/25/2023] Open
Abstract
Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.
Collapse
Affiliation(s)
| | - Dora Angelaki
- Center for Neural Science, New York University, New York, United States
| | - Hannah Bayer
- Zuckerman Institute, Columbia University, New York, United States
| | | | - Matteo Carandini
- UCL Institute of Ophthalmology, University College London, London, United Kingdom
| | | | - Gaelle Chapuis
- Wolfson Institute for Biomedical Research, University College London, London, United Kingdom
| | | | - Yang Dan
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, United States
| | - Eric Dewitt
- Champalimaud Centre for the Unknown, Lisbon, Portugal
| | - Mayo Faulkner
- Wolfson Institute for Biomedical Research, University College London, London, United Kingdom
| | - Hamish Forrest
- UCL Institute of Ophthalmology, University College London, London, United Kingdom
| | - Laura Haetzel
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Michael Häusser
- Wolfson Institute for Biomedical Research, University College London, London, United Kingdom
| | - Sonja B Hofer
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom
| | - Fei Hu
- Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, United States
| | - Anup Khanal
- Cold Spring Harbor Laboratory, New York, United States
| | - Christopher Krasniak
- Cold Spring Harbor Laboratory, New York, United States
- Watson School of Biological Sciences, New York, United States
| | | | | | - Guido Meijer
- Champalimaud Centre for the Unknown, Lisbon, Portugal
| | - Nathaniel J Miska
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom
| | - Thomas D Mrsic-Flogel
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom
| | | | - Jean-Paul Noel
- Center for Neural Science, New York University, New York, United States
| | | | - Cyrille Rossant
- UCL Queen Square Institute of Neurology, University College London, London, United Kingdom
| | | | - Karolina Socha
- UCL Institute of Ophthalmology, University College London, London, United Kingdom
| | - Rebecca Terry
- UCL Queen Square Institute of Neurology, University College London, London, United Kingdom
| | - Anne E Urai
- Cold Spring Harbor Laboratory, New York, United States
- Cognitive Psychology Unit, Leiden University, Leiden, Netherlands
| | - Hernando Vergara
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College London, London, United Kingdom
| | - Miles Wells
- UCL Queen Square Institute of Neurology, University College London, London, United Kingdom
| | | | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| | - Lauren E Wool
- UCL Queen Square Institute of Neurology, University College London, London, United Kingdom
| | | |
Collapse
|
36
|
Aguillon-Rodriguez V, Angelaki D, Bayer H, Bonacchi N, Carandini M, Cazettes F, Chapuis G, Churchland AK, Dan Y, Dewitt E, Faulkner M, Forrest H, Haetzel L, Häusser M, Hofer SB, Hu F, Khanal A, Krasniak C, Laranjeira I, Mainen ZF, Meijer G, Miska NJ, Mrsic-Flogel TD, Murakami M, Noel JP, Pan-Vazquez A, Rossant C, Sanders J, Socha K, Terry R, Urai AE, Vergara H, Wells M, Wilson CJ, Witten IB, Wool LE, Zador AM. Standardized and reproducible measurement of decision-making in mice. eLife 2021; 10:e63711. [PMID: 34011433 PMCID: PMC8137147 DOI: 10.7554/elife.63711] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2020] [Accepted: 04/08/2021] [Indexed: 12/20/2022] Open
Abstract
Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.
Collapse
Affiliation(s)
- The International Brain Laboratory
- Cold Spring Harbor LaboratoryNew YorkUnited States
- Center for Neural Science, New York UniversityNew YorkUnited States
- Zuckerman Institute, Columbia UniversityNew YorkUnited States
- Champalimaud Centre for the UnknownLisbonPortugal
- UCL Institute of Ophthalmology, University College LondonLondonUnited Kingdom
- Wolfson Institute for Biomedical Research, University College LondonLondonUnited Kingdom
- Department of Molecular and Cell Biology, University of California, BerkeleyBerkeleyUnited States
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
- Watson School of Biological SciencesNew YorkUnited States
- UCL Queen Square Institute of Neurology, University College LondonLondonUnited Kingdom
- Sanworks LLCNew YorkUnited States
- Cognitive Psychology Unit, Leiden UniversityLeidenNetherlands
| | | | - Dora Angelaki
- Center for Neural Science, New York UniversityNew YorkUnited States
| | - Hannah Bayer
- Zuckerman Institute, Columbia UniversityNew YorkUnited States
| | | | - Matteo Carandini
- UCL Institute of Ophthalmology, University College LondonLondonUnited Kingdom
| | | | - Gaelle Chapuis
- Wolfson Institute for Biomedical Research, University College LondonLondonUnited Kingdom
| | | | - Yang Dan
- Department of Molecular and Cell Biology, University of California, BerkeleyBerkeleyUnited States
| | - Eric Dewitt
- Champalimaud Centre for the UnknownLisbonPortugal
| | - Mayo Faulkner
- Wolfson Institute for Biomedical Research, University College LondonLondonUnited Kingdom
| | - Hamish Forrest
- UCL Institute of Ophthalmology, University College LondonLondonUnited Kingdom
| | - Laura Haetzel
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Michael Häusser
- Wolfson Institute for Biomedical Research, University College LondonLondonUnited Kingdom
| | - Sonja B Hofer
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
| | - Fei Hu
- Department of Molecular and Cell Biology, University of California, BerkeleyBerkeleyUnited States
| | - Anup Khanal
- Cold Spring Harbor LaboratoryNew YorkUnited States
| | - Christopher Krasniak
- Cold Spring Harbor LaboratoryNew YorkUnited States
- Watson School of Biological SciencesNew YorkUnited States
| | | | | | - Guido Meijer
- Champalimaud Centre for the UnknownLisbonPortugal
| | - Nathaniel J Miska
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
| | - Thomas D Mrsic-Flogel
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
| | | | - Jean-Paul Noel
- Center for Neural Science, New York UniversityNew YorkUnited States
| | | | - Cyrille Rossant
- UCL Queen Square Institute of Neurology, University College LondonLondonUnited Kingdom
| | | | - Karolina Socha
- UCL Institute of Ophthalmology, University College LondonLondonUnited Kingdom
| | - Rebecca Terry
- UCL Queen Square Institute of Neurology, University College LondonLondonUnited Kingdom
| | - Anne E Urai
- Cold Spring Harbor LaboratoryNew YorkUnited States
- Cognitive Psychology Unit, Leiden UniversityLeidenNetherlands
| | - Hernando Vergara
- Sainsbury-Wellcome Centre for Neural Circuits and Behaviour, University College LondonLondonUnited Kingdom
| | - Miles Wells
- UCL Queen Square Institute of Neurology, University College LondonLondonUnited Kingdom
| | | | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Lauren E Wool
- UCL Queen Square Institute of Neurology, University College LondonLondonUnited Kingdom
| | | |
Collapse
|
37
|
Roy NA, Bak JH, Akrami A, Brody CD, Pillow JW. Extracting the dynamics of behavior in sensory decision-making experiments. Neuron 2021; 109:597-610.e6. [PMID: 33412101 PMCID: PMC7897255 DOI: 10.1016/j.neuron.2020.12.004] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Revised: 10/23/2020] [Accepted: 12/03/2020] [Indexed: 11/21/2022]
Abstract
Decision-making strategies evolve during training and can continue to vary even in well-trained animals. However, studies of sensory decision-making tend to characterize behavior in terms of a fixed psychometric function that is fit only after training is complete. Here, we present PsyTrack, a flexible method for inferring the trajectory of sensory decision-making strategies from choice data. We apply PsyTrack to training data from mice, rats, and human subjects learning to perform auditory and visual decision-making tasks. We show that it successfully captures trial-to-trial fluctuations in the weighting of sensory stimuli, bias, and task-irrelevant covariates such as choice and stimulus history. This analysis reveals dramatic differences in learning across mice and rapid adaptation to changes in task statistics. PsyTrack scales easily to large datasets and offers a powerful tool for quantifying time-varying behavior in a wide variety of animals and tasks.
Collapse
Affiliation(s)
- Nicholas A Roy
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| | - Ji Hyun Bak
- Korea Institute for Advanced Study, Seoul 02455, South Korea; Redwood Center for Theoretical Neuroscience, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Athena Akrami
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Sainsbury Wellcome Centre, University College London, London W1T 4JG, UK
| | - Carlos D Brody
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Howard Hughes Medical Institute, Princeton University, Princeton, NJ 08544, USA
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Department of Psychology, Princeton University, Princeton, NJ 08544, USA.
| |
Collapse
|
38
|
Pisupati S, Chartarifsky-Lynn L, Khanal A, Churchland AK. Lapses in perceptual decisions reflect exploration. eLife 2021; 10:55490. [PMID: 33427198 PMCID: PMC7846276 DOI: 10.7554/elife.55490] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2020] [Accepted: 01/10/2021] [Indexed: 12/17/2022] Open
Abstract
Perceptual decision-makers often display a constant rate of errors independent of evidence strength. These ‘lapses’ are treated as a nuisance arising from noise tangential to the decision, e.g. inattention or motor errors. Here, we use a multisensory decision task in rats to demonstrate that these explanations cannot account for lapses’ stimulus dependence. We propose a novel explanation: lapses reflect a strategic trade-off between exploiting known rewarding actions and exploring uncertain ones. We tested this model’s predictions by selectively manipulating one action’s reward magnitude or probability. As uniquely predicted by this model, changes were restricted to lapses associated with that action. Finally, we show that lapses are a powerful tool for assigning decision-related computations to neural structures based on disruption experiments (here, posterior striatum and secondary motor cortex). These results suggest that lapses reflect an integral component of decision-making and are informative about action values in normal and disrupted brain states.
Collapse
Affiliation(s)
- Sashank Pisupati
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States.,CSHL School of Biological Sciences, Cold Spring Harbor, New York, United States
| | - Lital Chartarifsky-Lynn
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States.,CSHL School of Biological Sciences, Cold Spring Harbor, New York, United States
| | - Anup Khanal
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, United States
| | | |
Collapse
|
39
|
Cowley BR, Snyder AC, Acar K, Williamson RC, Yu BM, Smith MA. Slow Drift of Neural Activity as a Signature of Impulsivity in Macaque Visual and Prefrontal Cortex. Neuron 2020; 108:551-567.e8. [PMID: 32810433 PMCID: PMC7822647 DOI: 10.1016/j.neuron.2020.07.021] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Revised: 06/15/2020] [Accepted: 07/17/2020] [Indexed: 12/22/2022]
Abstract
An animal's decision depends not only on incoming sensory evidence but also on its fluctuating internal state. This state embodies multiple cognitive factors, such as arousal and fatigue, but it is unclear how these factors influence the neural processes that encode sensory stimuli and form a decision. We discovered that, unprompted by task conditions, animals slowly shifted their likelihood of detecting stimulus changes over the timescale of tens of minutes. Neural population activity from visual area V4, as well as from prefrontal cortex, slowly drifted together with these behavioral fluctuations. We found that this slow drift, rather than altering the encoding of the sensory stimulus, acted as an impulsivity signal, overriding sensory evidence to dictate the final decision. Overall, this work uncovers an internal state embedded in population activity across multiple brain areas and sheds further light on how internal states contribute to the decision-making process.
Collapse
Affiliation(s)
- Benjamin R Cowley
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA; Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Adam C Snyder
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14642, USA; Department of Neuroscience, University of Rochester, Rochester, NY 14642, USA; Center for Visual Science, University of Rochester, Rochester, NY 14642, USA
| | - Katerina Acar
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Center for Neuroscience, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| | - Ryan C Williamson
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Department of Machine Learning, Carnegie Mellon University, Pittsburgh, PA 15213, USA; University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| | - Byron M Yu
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Matthew A Smith
- Center for the Neural Basis of Cognition, Pittsburgh, PA 15213, USA; Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA; Department of Ophthalmology, University of Pittsburgh, Pittsburgh, PA 15213, USA.
| |
Collapse
|
40
|
Ashwood ZC, Roy NA, Bak JH, Pillow JW. Inferring learning rules from animal decision-making. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 2020; 33:3442-3453. [PMID: 36177341 PMCID: PMC9518972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
How do animals learn? This remains an elusive question in neuroscience. Whereas reinforcement learning often focuses on the design of algorithms that enable artificial agents to efficiently learn new tasks, here we develop a modeling framework to directly infer the empirical learning rules that animals use to acquire new behaviors. Our method efficiently infers the trial-to-trial changes in an animal's policy, and decomposes those changes into a learning component and a noise component. Specifically, this allows us to: (i) compare different learning rules and objective functions that an animal may be using to update its policy; (ii) estimate distinct learning rates for different parameters of an animal's policy; (iii) identify variations in learning across cohorts of animals; and (iv) uncover trial-to-trial changes that are not captured by normative learning rules. After validating our framework on simulated choice data, we applied our model to data from rats and mice learning perceptual decision-making tasks. We found that certain learning rules were far more capable of explaining trial-to-trial changes in an animal's policy. Whereas the average contribution of the conventional REINFORCE learning rule to the policy update for mice learning the International Brain Laboratory's task was just 30%, we found that adding baseline parameters allowed the learning rule to explain 92% of the animals' policy updates under our model. Intriguingly, the best-fitting learning rates and baseline values indicate that an animal's policy update, at each trial, does not occur in the direction that maximizes expected reward. Understanding how an animal transitions from chance-level to high-accuracy performance when learning a new task not only provides neuroscientists with insight into their animals, but also provides concrete examples of biological learning algorithms to the machine learning community.
Collapse
Affiliation(s)
- Zoe C Ashwood
- Princeton Neuroscience Institute, Princeton University
- Dept. of Computer Science, Princeton University
| | | | - Ji Hyun Bak
- Redwood Center for Theoretical Neuroscience, UC Berkeley
| | - Jonathan W Pillow
- Princeton Neuroscience Institute, Princeton University
- Dept. of Psychology, Princeton University
| |
Collapse
|