1
|
Nick Q, Gale DJ, Areshenkoff C, De Brouwer A, Nashed J, Wammes J, Zhu T, Flanagan R, Smallwood J, Gallivan J. Reconfigurations of cortical manifold structure during reward-based motor learning. eLife 2024; 12:RP91928. [PMID: 38916598 PMCID: PMC11198988 DOI: 10.7554/elife.91928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/26/2024] Open
Abstract
Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.
Collapse
Affiliation(s)
- Qasem Nick
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
| | - Daniel J Gale
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
| | - Corson Areshenkoff
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
| | - Anouk De Brouwer
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
| | - Joseph Nashed
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Medicine, Queen's UniversityKingstonCanada
| | - Jeffrey Wammes
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
| | - Tianyao Zhu
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
| | - Randy Flanagan
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
| | - Jonny Smallwood
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
| | - Jason Gallivan
- Centre for Neuroscience Studies, Queen’s UniversityKingstonCanada
- Department of Psychology, Queen’s UniversityKingstonCanada
- Department of Biomedical and Molecular Sciences, Queen’s UniversityKingstonCanada
| |
Collapse
|
2
|
Kobayashi K, Kable JW. Neural mechanisms of information seeking. Neuron 2024; 112:1741-1756. [PMID: 38703774 DOI: 10.1016/j.neuron.2024.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/30/2024] [Accepted: 04/08/2024] [Indexed: 05/06/2024]
Abstract
We ubiquitously seek information to make better decisions. Particularly in the modern age, when more information is available at our fingertips than ever, the information we choose to collect determines the quality of our decisions. Decision neuroscience has long adopted empirical approaches where the information available to decision-makers is fully controlled by the researchers, leaving neural mechanisms of information seeking less understood. Although information seeking has long been studied in the context of the exploration-exploitation trade-off, recent studies have widened the scope to investigate more overt information seeking in a way distinct from other decision processes. Insights gained from these studies, accumulated over the last few years, raise the possibility that information seeking is driven by the reward system signaling the subjective value of information. In this piece, we review findings from the recent studies, highlighting the conceptual and empirical relationships between distinct literatures, and discuss future research directions necessary to establish a more comprehensive understanding of how individuals seek information as a part of value-based decision-making.
Collapse
Affiliation(s)
- Kenji Kobayashi
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| | - Joseph W Kable
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
3
|
Shintaki R, Tanaka D, Suzuki S, Yoshimoto T, Sadato N, Chikazoe J, Jimura K. Continuous decision to wait for a future reward is guided by fronto-hippocampal anticipatory dynamics. Cereb Cortex 2024; 34:bhae217. [PMID: 38798003 DOI: 10.1093/cercor/bhae217] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2023] [Revised: 05/02/2024] [Accepted: 05/08/2024] [Indexed: 05/29/2024] Open
Abstract
Deciding whether to wait for a future reward is crucial for surviving in an uncertain world. While seeking rewards, agents anticipate a reward in the present environment and constantly face a trade-off between staying in their environment or leaving it. It remains unclear, however, how humans make continuous decisions in such situations. Here, we show that anticipatory activity in the anterior prefrontal cortex, ventrolateral prefrontal cortex, and hippocampus underpins continuous stay-leave decision-making. Participants awaited real liquid rewards available after tens of seconds, and their continuous decision was tracked by dynamic brain activity associated with the anticipation of a reward. Participants stopped waiting more frequently and sooner after they experienced longer delays and received smaller rewards. When the dynamic anticipatory brain activity was enhanced in the anterior prefrontal cortex, participants remained in their current environment, but when this activity diminished, they left the environment. Moreover, while experiencing a delayed reward in a novel environment, the ventrolateral prefrontal cortex and hippocampus showed anticipatory activity. Finally, the activity in the anterior prefrontal cortex and ventrolateral prefrontal cortex was enhanced in participants adopting a leave strategy, whereas those remaining stationary showed enhanced hippocampal activity. Our results suggest that fronto-hippocampal anticipatory dynamics underlie continuous decision-making while anticipating a future reward.
Collapse
Affiliation(s)
- Reiko Shintaki
- Department of Biosciences and Informatics, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama, 223-8522, Japan
| | - Daiki Tanaka
- Department of Biosciences and Informatics, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama, 223-8522, Japan
| | - Shinsuke Suzuki
- Centre for Brain, Mind and Markets, The University of Melbourne, Grattan Street, Parkville, Victoria, 3010, Australia
- Faculty of Social Data Science and HIAS Brain Research Center, Hitotsubashi University, 2-1 Naka, Kunitachi, 186-8601, Japan
| | - Takaaki Yoshimoto
- Research Organization of Science and Technology, Ritsumeikan University, 1-1-1, Nojihigashi, Kusatsu, 525-8577, Japan
- Section of Brain Function Information, Supportive Center for Brain Research, National Institute for Physiological Sciences, 38 Nishigonaka, Myodaiji, Okazaki, 444-8585, Japan
| | - Norihiro Sadato
- Research Organization of Science and Technology, Ritsumeikan University, 1-1-1, Nojihigashi, Kusatsu, 525-8577, Japan
- Section of Brain Function Information, Supportive Center for Brain Research, National Institute for Physiological Sciences, 38 Nishigonaka, Myodaiji, Okazaki, 444-8585, Japan
| | - Junichi Chikazoe
- Section of Brain Function Information, Supportive Center for Brain Research, National Institute for Physiological Sciences, 38 Nishigonaka, Myodaiji, Okazaki, 444-8585, Japan
- Araya, Inc., 1-11 Kanda Sakuma-cho, Chiyoda, Tokyo, 101-0025, Japan
| | - Koji Jimura
- Department of Informatics, Gunma University, 4-2 Aramaki-machi, Maebashi, 371-8510, Japan
| |
Collapse
|
4
|
Basile BM, Waters SJ, Murray EA. What does preferential viewing tell us about the neurobiology of recognition memory? Trends Neurosci 2024; 47:326-337. [PMID: 38582659 PMCID: PMC11096050 DOI: 10.1016/j.tins.2024.03.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 02/09/2024] [Accepted: 03/14/2024] [Indexed: 04/08/2024]
Abstract
The two tests most widely used in nonhuman primates to assess the neurobiology of recognition memory produce conflicting results. Preferential viewing tests (e.g., visual paired comparison) produce robust impairments following hippocampal lesions, whereas matching tests (e.g., delayed nonmatching-to-sample) often show complete sparing. Here, we review the data, the proposed explanations for this discrepancy, and then critically evaluate those explanations. The most likely explanation is that preferential viewing tests are not a process-pure assessment of recognition memory, but also test elements of novelty-seeking, habituation, and motivation. These confounds likely explain the conflicting results. Thus, we propose that memory researchers should prefer explicit matching tests and readers interested in the neural substrates of recognition memory should give explicit matching tests greater interpretive weight.
Collapse
Affiliation(s)
| | - Spencer J Waters
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, USA; Section on the Neurobiology of Learning and Memory, Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD, USA
| | - Elisabeth A Murray
- Section on the Neurobiology of Learning and Memory, Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD, USA.
| |
Collapse
|
5
|
Conn K, Milton LK, Huang K, Munguba H, Ruuska J, Lemus MB, Greaves E, Homman-Ludiye J, Oldfield BJ, Foldi CJ. Psilocybin restrains activity-based anorexia in female rats by enhancing cognitive flexibility: contributions from 5-HT1A and 5-HT2A receptor mechanisms. Mol Psychiatry 2024:10.1038/s41380-024-02575-9. [PMID: 38678087 DOI: 10.1038/s41380-024-02575-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Revised: 04/16/2024] [Accepted: 04/19/2024] [Indexed: 04/29/2024]
Abstract
Psilocybin has shown promise for alleviating symptoms of depression and is currently in clinical trials for the treatment of anorexia nervosa (AN), a condition that is characterised by persistent cognitive inflexibility. Considering that enhanced cognitive flexibility after psilocybin treatment is reported to occur in individuals with depression, it is plausible that psilocybin could improve symptoms of AN by breaking down cognitive inflexibility. A mechanistic understanding of the actions of psilocybin is required to tailor the clinical application of psilocybin to individuals most likely to respond with positive outcomes. This can only be achieved using incisive neurobiological approaches in animal models. Here, we use the activity-based anorexia (ABA) rat model and comprehensively assess aspects of reinforcement learning to show that psilocybin (post-acutely) improves body weight maintenance in female rats and facilitates cognitive flexibility, specifically via improved adaptation to the initial reversal of reward contingencies. Further, we reveal the involvement of signalling through the serotonin (5-HT) 1 A and 5-HT2A receptor subtypes in specific aspects of learning, demonstrating that 5-HT1A antagonism negates the cognitive enhancing effects of psilocybin. Moreover, we show that psilocybin elicits a transient increase and decrease in cortical transcription of these receptors (Htr2a and Htr1a, respectively), and a further reduction in the abundance of Htr2a transcripts in rats exposed to the ABA model. Together, these findings support the hypothesis that psilocybin could ameliorate cognitive inflexibility in the context of AN and highlight a need to better understand the therapeutic mechanisms independent of 5-HT2A receptor binding.
Collapse
Affiliation(s)
- K Conn
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - L K Milton
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - K Huang
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - H Munguba
- Department of Biochemistry, Weill Cornell Medicine, New York, NY, 10065, USA
| | - J Ruuska
- University of Helsinki, Yliopistonkatu 4, 00100, Helsinki, Finland
| | - M B Lemus
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - E Greaves
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - J Homman-Ludiye
- Monash Micro Imaging, Monash University, 15 Innovation Walk, Clayton, VIC, 3800, Australia
| | - B J Oldfield
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia
| | - C J Foldi
- Monash University, Department of Physiology, 26 Innovation Walk, Clayton, VIC, 3800, Australia.
- Monash Biomedicine Discovery Institute, 23 Innovation Walk, Clayton, VIC, 3800, Australia.
| |
Collapse
|
6
|
Lloyd A, Roiser JP, Skeen S, Freeman Z, Badalova A, Agunbiade A, Busakhwe C, DeFlorio C, Marcu A, Pirie H, Saleh R, Snyder T, Fearon P, Viding E. Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024:10.3758/s13415-024-01186-9. [PMID: 38653937 DOI: 10.3758/s13415-024-01186-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/27/2024] [Indexed: 04/25/2024]
Abstract
In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.
Collapse
Affiliation(s)
- Alex Lloyd
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK.
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London, UK
| | - Sarah Skeen
- Institute for Life Course Health Research, Stellenbosch University, Stellenbosch, South Africa
| | - Ze Freeman
- Department of Psychology, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Aygun Badalova
- Institute of Neurology, University College London, London, UK
| | | | | | | | - Anna Marcu
- Young People's Advisor Group, London, UK
| | | | | | | | - Pasco Fearon
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK
- Centre for Family Research, Department of Psychology, University of Cambridge, Cambridge, UK
| | - Essi Viding
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK
| |
Collapse
|
7
|
Aberg KC, Paz R. The neurobehavioral correlates of exploration without learning: Trading off value for explicit, prospective, and variable information gains. Cell Rep 2024; 43:113880. [PMID: 38416639 DOI: 10.1016/j.celrep.2024.113880] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 01/10/2024] [Accepted: 02/13/2024] [Indexed: 03/01/2024] Open
Abstract
Exploration is typically motivated by gaining information, with previous research showing that potential information gains drive a "directed" type of exploration. Yet, this research usually studies exploration in the context of learning paradigms and does not directly manipulate multiple levels of information gain. Here, we present a task that isolates learning from decision-making and controls the magnitude of prospective information gains. As predicted, participants explore more with larger future information gains. Both value gains and information gains, at a trial-by-trial level, engage the ventromedial prefrontal cortex (vmPFC), the ventral striatum (VStr), the amygdala, the dorsal anterior cingulate cortex (dACC), and the anterior insula (aINS). Moreover, individual sensitivities to value gains and information gains modulate the vmPFC, dACC, and aINS, but the amygdala and VStr are modulated only by individual sensitivities to information gains. Overall, we identify the neural circuitry of information-based exploration and its relationship with inter-individual exploration biases.
Collapse
Affiliation(s)
- Kristoffer C Aberg
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel.
| | - Rony Paz
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot 76100, Israel
| |
Collapse
|
8
|
Venditto SJC, Miller KJ, Brody CD, Daw ND. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.28.582617. [PMID: 38464244 PMCID: PMC10925334 DOI: 10.1101/2024.02.28.582617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step,reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.
Collapse
|
9
|
Hallquist MN, Hwang K, Luna B, Dombrovski AY. Reward-based option competition in human dorsal stream and transition from stochastic exploration to exploitation in continuous space. SCIENCE ADVANCES 2024; 10:eadj2219. [PMID: 38394198 PMCID: PMC10889364 DOI: 10.1126/sciadv.adj2219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Accepted: 01/23/2024] [Indexed: 02/25/2024]
Abstract
Primates exploring and exploiting a continuous sensorimotor space rely on dynamic maps in the dorsal stream. Two complementary perspectives exist on how these maps encode rewards. Reinforcement learning models integrate rewards incrementally over time, efficiently resolving the exploration/exploitation dilemma. Working memory buffer models explain rapid plasticity of parietal maps but lack a plausible exploration/exploitation policy. The reinforcement learning model presented here unifies both accounts, enabling rapid, information-compressing map updates and efficient transition from exploration to exploitation. As predicted by our model, activity in human frontoparietal dorsal stream regions, but not in MT+, tracks the number of competing options, as preferred options are selectively maintained on the map, while spatiotemporally distant alternatives are compressed out. When valuable new options are uncovered, posterior β1/α oscillations desynchronize within 0.4 to 0.7 s, consistent with option encoding by competing β1-stabilized subpopulations. Together, outcomes matching locally cached reward representations rapidly update parietal maps, biasing choices toward often-sampled, rewarded options.
Collapse
Affiliation(s)
| | - Kai Hwang
- Department of Psychological and Brain Sciences, Iowa Neuroscience Institute, University of Iowa, Iowa City, IA, USA
| | - Beatriz Luna
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA
| | | |
Collapse
|
10
|
Nougaret S, Ferrucci L, Ceccarelli F, Sacchetti S, Benozzo D, Fascianelli V, Saunders RC, Renaud L, Genovesio A. Neurons in the monkey frontopolar cortex encode learning stage and goal during a fast learning task. PLoS Biol 2024; 22:e3002500. [PMID: 38363801 PMCID: PMC10903959 DOI: 10.1371/journal.pbio.3002500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 02/29/2024] [Accepted: 01/17/2024] [Indexed: 02/18/2024] Open
Abstract
The frontopolar cortex (FPC) is, to date, one of the least understood regions of the prefrontal cortex. The current understanding of its function suggests that it plays a role in the control of exploratory behaviors by coordinating the activities of other prefrontal cortex areas involved in decision-making and exploiting actions based on their outcomes. Based on this hypothesis, FPC would drive fast-learning processes through a valuation of the different alternatives. In our study, we used a modified version of a well-known paradigm, the object-in-place (OIP) task, to test this hypothesis in electrophysiology. This paradigm is designed to maximize learning, enabling monkeys to learn in one trial, which is an ability specifically impaired after a lesion of the FPC. We showed that FPC neurons presented an extremely specific pattern of activity by representing the learning stage, exploration versus exploitation, and the goal of the action. However, our results do not support the hypothesis that neurons in the frontal pole compute an evaluation of different alternatives. Indeed, the position of the chosen target was strongly encoded at its acquisition, but the position of the unchosen target was not. Once learned, this representation was also found at the problem presentation, suggesting a monitoring activity of the synthetic goal preceding its acquisition. Our results highlight important features of FPC neurons in fast-learning processes without confirming their role in the disengagement of cognitive control from the current goals.
Collapse
Affiliation(s)
- Simon Nougaret
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Lorenzo Ferrucci
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Francesco Ceccarelli
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
- PhD program in Behavioral Neuroscience, Sapienza University of Rome, Rome, Italy
| | - Stefano Sacchetti
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Danilo Benozzo
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Valeria Fascianelli
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Richard C. Saunders
- Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, Maryland, United States of America
| | - Luc Renaud
- Institut de Neurosciences de la Timone, UMR7289, Centre National de la Recherche Scientifique and Aix-Marseille Université, Marseille, France
| | - Aldo Genovesio
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
11
|
Giarrocco F, Costa VD, Basile BM, Pujara MS, Murray EA, Averbeck BB. Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors. J Neurosci 2024; 44:e1206232023. [PMID: 38296647 PMCID: PMC10860650 DOI: 10.1523/jneurosci.1206-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 11/17/2023] [Accepted: 11/21/2023] [Indexed: 02/02/2024] Open
Abstract
Deciding whether to forego immediate rewards or explore new opportunities is a key component of flexible behavior and is critical for the survival of the species. Although previous studies have shown that different cortical and subcortical areas, including the amygdala and ventral striatum (VS), are implicated in representing the immediate (exploitative) and future (explorative) value of choices, the effect of the motor system used to make choices has not been examined. Here, we tested male rhesus macaques with amygdala or VS lesions on two versions of a three-arm bandit task where choices were registered with either a saccade or an arm movement. In both tasks we presented the monkeys with explore-exploit tradeoffs by periodically replacing familiar options with novel options that had unknown reward probabilities. We found that monkeys explored more with saccades but showed better learning with arm movements. VS lesions caused the monkeys to be more explorative with arm movements and less explorative with saccades, although this may have been due to an overall decrease in performance. VS lesions affected the monkeys' ability to learn novel stimulus-reward associations in both tasks, while after amygdala lesions this effect was stronger when choices were made with saccades. Further, on average, VS and amygdala lesions reduced the monkeys' ability to choose better options only when choices were made with a saccade. These results show that learning reward value associations to manage explore-exploit behaviors is motor system dependent and they further define the contributions of amygdala and VS to reinforcement learning.
Collapse
Affiliation(s)
- Franco Giarrocco
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
| | - Vincent D Costa
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
- Division of Neuroscience, Oregon National Primate Research Center, Beaverton 97006, OR
| | - Benjamin M Basile
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
- Department of Psychology, Dickinson College, Carlisle 17013, PA
| | - Maia S Pujara
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
| | - Elisabeth A Murray
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
| | - Bruno B Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda 20892-4415, MD
| |
Collapse
|
12
|
Rolls ET, Deco G, Huang CC, Feng J. The connectivity of the human frontal pole cortex, and a theory of its involvement in exploit versus explore. Cereb Cortex 2024; 34:bhad416. [PMID: 37991264 DOI: 10.1093/cercor/bhad416] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 10/16/2023] [Accepted: 10/17/2023] [Indexed: 11/23/2023] Open
Abstract
The frontal pole is implicated in humans in whether to exploit resources versus explore alternatives. Effective connectivity, functional connectivity, and tractography were measured between six human frontal pole regions and for comparison 13 dorsolateral and dorsal prefrontal cortex regions, and the 360 cortical regions in the Human Connectome Project Multi-modal-parcellation atlas in 171 HCP participants. The frontal pole regions have effective connectivity with Dorsolateral Prefrontal Cortex regions, the Dorsal Prefrontal Cortex, both implicated in working memory; and with the orbitofrontal and anterior cingulate cortex reward/non-reward system. There is also connectivity with temporal lobe, inferior parietal, and posterior cingulate regions. Given this new connectivity evidence, and evidence from activations and damage, it is proposed that the frontal pole cortex contains autoassociation attractor networks that are normally stable in a short-term memory state, and maintain stability in the other prefrontal networks during stable exploitation of goals and strategies. However, if an input from the orbitofrontal or anterior cingulate cortex that expected reward, non-reward, or punishment is received, this destabilizes the frontal pole and thereby other prefrontal networks to enable exploration of competing alternative goals and strategies. The frontal pole connectivity with reward systems may be key in exploit versus explore.
Collapse
Affiliation(s)
- Edmund T Rolls
- Oxford Centre for Computational Neuroscience, Oxford, United Kingdom
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200403, China
| | - Gustavo Deco
- Center for Brain and Cognition, Computational Neuroscience Group, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Roc Boronat 138, Barcelona 08018, Spain
- Brain and Cognition, Pompeu Fabra University, Barcelona 08018, Spain
- Institució Catalana de la Recerca i Estudis Avançats (ICREA), Universitat Pompeu Fabra, Passeig Lluís Companys 23, Barcelona 08010, Spain
| | - Chu-Chung Huang
- Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), Institute of Brain and Education Innovation, School of Psychology and Cognitive Science, East China Normal University, Shanghai 200602, China
- Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200602, China
| | - Jianfeng Feng
- Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom
- Institute of Science and Technology for Brain Inspired Intelligence, Fudan University, Shanghai 200403, China
| |
Collapse
|
13
|
Wyatt LE, Hewan PA, Hogeveen J, Spreng RN, Turner GR. Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies. Neuropsychologia 2024; 192:108740. [PMID: 38036246 DOI: 10.1016/j.neuropsychologia.2023.108740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Revised: 10/15/2023] [Accepted: 11/21/2023] [Indexed: 12/02/2023]
Abstract
Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.
Collapse
Affiliation(s)
- Lindsay E Wyatt
- Department of Psychology, York University, Toronto, ON, Canada
| | - Patrick A Hewan
- Department of Psychology, York University, Toronto, ON, Canada
| | - Jeremy Hogeveen
- Department of Psychology, The University of New Mexico, Albuquerque, NM, USA
| | - R Nathan Spreng
- Montréal Neurological Institute, Department of Neurology and Neurosurgery, McGill University, Montréal, QC, H3A 2B4, Canada; Department of Psychology, McGill University, Montréal, QC, Canada; Department of Psychiatry, McGill University, Montréal, QC, Canada; McConnell Brain Imaging Centre, Montréal Neurological Institute, McGill University, Montréal, QC, Canada.
| | - Gary R Turner
- Department of Psychology, York University, Toronto, ON, Canada.
| |
Collapse
|
14
|
Basile BM, Costa VD, Schafroth JL, Karaskiewicz CL, Lucas DR, Murray EA. The amygdala is not necessary for the familiarity aspect of recognition memory. Nat Commun 2023; 14:8109. [PMID: 38062014 PMCID: PMC10703781 DOI: 10.1038/s41467-023-43906-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Accepted: 11/23/2023] [Indexed: 12/18/2023] Open
Abstract
Dual-process accounts of item recognition posit two memory processes: slow but detailed recollection, and quick but vague familiarity. It has been proposed, based on prior rodent work, that the amygdala is critical for the familiarity aspect of item recognition. Here, we evaluated this proposal in male rhesus monkeys (Macaca mulatta) with selective bilateral excitotoxic amygdala damage. We used four established visual memory tests designed to assess different aspects of familiarity, all administered on touchscreen computers. Specifically, we assessed monkeys' tendencies to make low-latency false alarms, to make false alarms to recently seen lures, to produce curvilinear ROC curves, and to discriminate stimuli based on repetition across days. Three of the four tests showed no familiarity impairment and the fourth was explained by a deficit in reward processing. Consistent with this, amygdala damage did produce an anticipated deficit in reward processing in a three-arm-bandit gambling task, verifying the effectiveness of the lesions. Together, these results contradict prior rodent work and suggest that the amygdala is not critical for the familiarity aspect of item recognition.
Collapse
Affiliation(s)
- Benjamin M Basile
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA.
- Department of Psychology, Dickinson College, Carlisle, PA, USA.
| | - Vincent D Costa
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA
- Division of Neuroscience, Oregon National Primate Research Center, Portland, OR, USA
| | - Jamie L Schafroth
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA
- School of Anthropology, University of Arizona, Tucson, AZ, USA
| | - Chloe L Karaskiewicz
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA
- Department of Psychology, UC Davis, Davis, CA, USA
| | - Daniel R Lucas
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Elisabeth A Murray
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, 20892, USA
| |
Collapse
|
15
|
Campbell EM, Singh G, Claus ED, Witkiewitz K, Costa VD, Hogeveen J, Cavanagh JF. Electrophysiological Markers of Aberrant Cue-Specific Exploration in Hazardous Drinkers. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2023; 7:47-59. [PMID: 38774639 PMCID: PMC11104413 DOI: 10.5334/cpsy.96] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Accepted: 06/28/2023] [Indexed: 05/24/2024]
Abstract
Background Hazardous drinking is associated with maladaptive alcohol-related decision-making. Existing studies have often focused on how participants learn to exploit familiar cues based on prior reinforcement, but little is known about the mechanisms that drive hazardous drinkers to explore novel alcohol cues when their value is not known. Methods We investigated exploration of novel alcohol and non-alcohol cues in hazardous drinkers (N = 27) and control participants (N = 26) during electroencephalography (EEG). A normative computational model with two free parameters was fit to estimate participants' weighting of the future value of exploration and immediate value of exploitation. Results Hazardous drinkers demonstrated increased exploration of novel alcohol cues, and conversely, increased probability of exploiting familiar alternatives instead of exploring novel non-alcohol cues. The motivation to explore novel alcohol stimuli in hazardous drinkers was driven by an elevated relative future valuation of uncertain alcohol cues. P3a predicted more exploratory decision policies driven by an enhanced relative future valuation of novel alcohol cues. P3b did not predict choice behavior, but computational parameter estimates suggested that hazardous drinkers with enhanced P3b to alcohol cues were likely to learn to exploit their immediate expected value. Conclusions Hazardous drinkers did not display atypical choice behavior, different P3a/P3b amplitudes, or computational estimates to novel non-alcohol cues-diverging from previous studies in addiction showing atypical generalized explore-exploit decisions with non-drug-related cues. These findings reveal that cue-specific neural computations may drive aberrant alcohol-related decision-making in hazardous drinkers-highlighting the importance of drug-relevant cues in studies of decision-making in addiction.
Collapse
Affiliation(s)
- Ethan M. Campbell
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Garima Singh
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Eric D. Claus
- Department of Biobehavioral Health, Pennsylvania State University, US
| | - Katie Witkiewitz
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Vincent D. Costa
- Division of Neuroscience, Oregon National Primate Research Center, US
| | - Jeremy Hogeveen
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - James F. Cavanagh
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| |
Collapse
|
16
|
Brown VM, Price R, Dombrovski AY. Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:844-868. [PMID: 36869259 PMCID: PMC10475148 DOI: 10.3758/s13415-023-01080-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/14/2023] [Indexed: 03/05/2023]
Abstract
In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.
Collapse
Affiliation(s)
- Vanessa M Brown
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Rebecca Price
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, USA
| | | |
Collapse
|
17
|
Speers LJ, Bilkey DK. Maladaptive explore/exploit trade-offs in schizophrenia. Trends Neurosci 2023; 46:341-354. [PMID: 36878821 DOI: 10.1016/j.tins.2023.02.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 01/30/2023] [Accepted: 02/08/2023] [Indexed: 03/07/2023]
Abstract
Schizophrenia is a complex disorder that remains poorly understood, particularly at the systems level. In this opinion article we argue that the explore/exploit trade-off concept provides a holistic and ecologically valid framework to resolve some of the apparent paradoxes that have emerged within schizophrenia research. We review recent evidence suggesting that fundamental explore/exploit behaviors may be maladaptive in schizophrenia during physical, visual, and cognitive foraging. We also describe how theories from the broader optimal foraging literature, such as the marginal value theorem (MVT), could provide valuable insight into how aberrant processing of reward, context, and cost/effort evaluations interact to produce maladaptive responses.
Collapse
Affiliation(s)
- Lucinda J Speers
- Department of Psychology, University of Otago, Dunedin 9016, New Zealand
| | - David K Bilkey
- Department of Psychology, University of Otago, Dunedin 9016, New Zealand.
| |
Collapse
|
18
|
Miyamoto K, Rushworth MFS, Shea N. Imagining the future self through thought experiments. Trends Cogn Sci 2023; 27:446-455. [PMID: 36801162 DOI: 10.1016/j.tics.2023.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Revised: 01/23/2023] [Accepted: 01/23/2023] [Indexed: 02/19/2023]
Abstract
The ability of the mind to conceptualize what is not present is essential. It allows us to reason counterfactually about what might have happened had events unfolded differently or had another course of action been taken. It allows us to think about what might happen - to perform 'Gedankenexperimente' (thought experiments) - before we act. However, the cognitive and neural mechanisms mediating this ability are poorly understood. We suggest that the frontopolar cortex (FPC) keeps track of and evaluates alternative choices (what we might have done), whereas the anterior lateral prefrontal cortex (alPFC) compares simulations of possible future scenarios (what we might do) and evaluates their reward values. Together, these brain regions support the construction of suppositional scenarios.
Collapse
Affiliation(s)
- Kentaro Miyamoto
- Laboratory for Imagination and Executive Functions, RIKEN Center for Brain Science, Wako, Japan.
| | - Matthew F S Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Nicholas Shea
- Institute of Philosophy, School of Advanced Study, University of London, London, UK; Faculty of Philosophy, University of Oxford, Oxford, UK
| |
Collapse
|
19
|
Goupil L, Proust J. Curiosity as a metacognitive feeling. Cognition 2023; 231:105325. [PMID: 36434942 DOI: 10.1016/j.cognition.2022.105325] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 10/28/2022] [Accepted: 11/07/2022] [Indexed: 11/24/2022]
Abstract
Curious information-seeking is known to be a key driver for learning, but characterizing this important psychological phenomenon remains a challenge. In this article, we argue that solving this challenge requires qualifying the relationships between metacognition and curiosity. The idea that curiosity is a metacognitive competence has been resisted: researchers have assumed both that young children and non-human animals can be genuinely curious, and that metacognition requires conceptual and culturally situated resources that are unavailable to young children and non-human animals. Here, we argue that this resistance is unwarranted given accumulating evidence that metacognition can be deployed procedurally, and we defend the view that curiosity is a metacognitive feeling. Our metacognitive view singles out two monitoring steps as a triggering condition for curiosity: evaluating one's own informational needs, and predicting the likelihood that explorations of the proximate environment afford significant information gains. We review empirical evidence and computational models of curiosity, and show that they fit well with this metacognitive account, while on the contrary, they remain difficult to explain by a competing account according to which curiosity is a basic attitude of questioning. Finally, we propose a new way to construe the relationships between curiosity and the human-specific communicative practice of questioning, discuss the issue of how children may learn to express their curiosity through interactions with others, and conclude by briefly exploring the implications of our proposal for educational practices.
Collapse
Affiliation(s)
- Louise Goupil
- Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, LPNC, 38000 Grenoble, France.
| | - Joëlle Proust
- Institut Jean Nicod, Ecole Normale Supérieure, Paris, France
| |
Collapse
|
20
|
Jahn CI, Grohn J, Cuell S, Emberton A, Bouret S, Walton ME, Kolling N, Sallet J. Neural responses in macaque prefrontal cortex are linked to strategic exploration. PLoS Biol 2023; 21:e3001985. [PMID: 36716348 PMCID: PMC9910800 DOI: 10.1371/journal.pbio.3001985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Revised: 02/09/2023] [Accepted: 01/03/2023] [Indexed: 02/01/2023] Open
Abstract
Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.
Collapse
Affiliation(s)
- Caroline I. Jahn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
- Sorbonne Paris Cité universités, Université Paris Descartes, Frontières du Vivant, Paris, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jan Grohn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Steven Cuell
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Andrew Emberton
- Biomedical Science Services, University of Oxford, Oxford, United Kingdom
| | - Sebastien Bouret
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
| | - Mark E. Walton
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging, OBHA, University of Oxford, Headington, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jérôme Sallet
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| |
Collapse
|
21
|
Hogeveen J, Medalla M, Ainsworth M, Galeazzi JM, Hanlon CA, Mansouri FA, Costa VD. What Does the Frontopolar Cortex Contribute to Goal-Directed Cognition and Action? J Neurosci 2022; 42:8508-8513. [PMID: 36351824 PMCID: PMC9665930 DOI: 10.1523/jneurosci.1143-22.2022] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 08/30/2022] [Accepted: 08/31/2022] [Indexed: 11/17/2022] Open
Abstract
Understanding the unique functions of different subregions of primate prefrontal cortex has been a longstanding goal in cognitive neuroscience. Yet, the anatomy and function of one of its largest subregions (the frontopolar cortex) remain enigmatic and underspecified. Our Society for Neuroscience minisymposium Primate Frontopolar Cortex: From Circuits to Complex Behaviors will comprise a range of new anatomic and functional approaches that have helped to clarify the basic circuit anatomy of the frontal pole, its functional involvement during performance of cognitively demanding behavioral paradigms in monkeys and humans, and its clinical potential as a target for noninvasive brain stimulation in patients with brain disorders. This review consolidates knowledge about the anatomy and connectivity of frontopolar cortex and provides an integrative summary of its function in primates. We aim to answer the question: what, if anything, does frontopolar cortex contribute to goal-directed cognition and action?
Collapse
Affiliation(s)
- Jeremy Hogeveen
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM 87131
| | - Maria Medalla
- Department of Anatomy & Neurobiology, Boston University, Boston, MA 02118
| | - Matthew Ainsworth
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom, OX2 6GG
| | - Juan M Galeazzi
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom, OX2 6GG
| | - Colleen A Hanlon
- Department of Cancer Biology
- Department of Physiology and Pharmacology, Wake Forest School of Medicine, Winston-Salem, NC 27101
| | - Farshad Alizadeh Mansouri
- Department of Physiology, Monash Biomedicine Discovery Institute, Clayton Victoria, 3800, Australia
- ARC Centre for Integrative Brain Function, Monash University, Clayton Victoria, 3800, Australia
| | - Vincent D Costa
- Division of Neuroscience, Oregon National Primate Research Center, Beaverton, OR 97006
| |
Collapse
|
22
|
de-Sousa MR, Aguiar TRXD. Dedução, Indução e a Arte do Raciocínio Clínico na Educação Médica: Revisão Sistemática e Proposta Bayesiana. Arq Bras Cardiol 2022; 119:27-34. [DOI: 10.36660/abc.20220405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 08/09/2022] [Indexed: 11/13/2022] Open
|
23
|
Cazares C, Schreiner DC, Valencia ML, Gremel CM. Orbitofrontal cortex populations are differentially recruited to support actions. Curr Biol 2022; 32:4675-4687.e5. [PMID: 36195096 PMCID: PMC9643660 DOI: 10.1016/j.cub.2022.09.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 08/03/2022] [Accepted: 09/09/2022] [Indexed: 11/09/2022]
Abstract
The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.
Collapse
Affiliation(s)
- Christian Cazares
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Drew C Schreiner
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Mariela Lopez Valencia
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Christina M Gremel
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA; Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA.
| |
Collapse
|
24
|
Shared mechanisms mediate the explore-exploit tradeoff in macaques and humans. Neuron 2022; 110:1751-1753. [PMID: 35654023 DOI: 10.1016/j.neuron.2022.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Ancestors of macaques and humans separated into distinct lineages 25 million years ago. Despite this long separation, Hogeveen et al. (2022) show, in this issue of Neuron, that they mediate the explore-exploit tradeoff, which must be managed by any agent adapting to a dynamic environment, using similar computational and neural mechanisms.
Collapse
|