1
|
Witkowski PP, Rondot L, Kurth-Nelson Z, Garvert MM, Dolan RJ, Behrens TEJ, Boorman ED. Neural mechanisms of credit assignment for delayed outcomes during contingent learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.08.06.606895. [PMID: 39149338 PMCID: PMC11326259 DOI: 10.1101/2024.08.06.606895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/17/2024]
Abstract
Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (FPl). Our results indicate that FPl holds previous causal choices in a "pending" state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.
Collapse
Affiliation(s)
- Phillip P. Witkowski
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD, USA
| | - Lindsay Rondot
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Google DeepMind, London, UK
| | - Mona M. Garvert
- Faculty of Human Sciences, Julius-Maximilians-Universität Würzburg, Würzburg, Germany
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Timothy E. J. Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford, UK
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK
| | - Erie D. Boorman
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
| |
Collapse
|
2
|
Tang H, Bartolo-Orozco R, Averbeck BB. Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.03.587097. [PMID: 38617219 PMCID: PMC11014508 DOI: 10.1101/2024.04.03.587097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and the mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, OFC leads the process of symbolic reinforcement learning in the ventral frontostriatal circuitry.
Collapse
|
3
|
Brockett AT, Kumar N, Sharalla P, Roesch MR. Optogenetic Inhibition of the Orbitofrontal Cortex Disrupts Inhibitory Control during Stop-Change Performance in Male Rats. eNeuro 2024; 11:ENEURO.0015-24.2024. [PMID: 38697842 PMCID: PMC11097625 DOI: 10.1523/eneuro.0015-24.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2024] [Revised: 03/27/2024] [Accepted: 04/22/2024] [Indexed: 05/05/2024] Open
Abstract
Historically, the orbitofrontal cortex (OFC) has been implicated in a variety of behaviors ranging from reversal learning and inhibitory control to more complex representations of reward value and task space. While modern interpretations of the OFC's function have focused on a role in outcome evaluation, these cognitive processes often require an organism to inhibit a maladaptive response or strategy. Single-unit recordings from the OFC in rats performing a stop-change task show that the OFC responds strongly to STOP trials. To investigate the role that the OFC plays in stop-change performance, we expressed halorhodopsin (eNpHR3.0) in excitatory neurons in the OFC and tested rats on the stop-change task. Previous work suggests that the OFC differentiates between STOP trials based on trial sequence (i.e., gS trials: STOP trials preceded by a GO vs sS trials: STOP trials preceded by a STOP). We found that yellow light activation of the eNpHR3.0-expressing neurons significantly decreased accuracy only on STOP trials that followed GO trials (gS trials). Further, optogenetic inhibition of the OFC speeded reaction times on error trials. This suggests that the OFC plays a role in inhibitory control processes and that this role needs to be accounted for in modern interpretations of OFC function.
Collapse
Affiliation(s)
- Adam T Brockett
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
- Department of Biological Sciences, University of New Hampshire, Durham, New Hampshire 03824
| | - Neeraj Kumar
- Department of Psychology, University of Maryland, College Park, Maryland 20742
| | - Paul Sharalla
- Department of Psychology, University of Maryland, College Park, Maryland 20742
| | - Matthew R Roesch
- Department of Psychology, University of Maryland, College Park, Maryland 20742
- Program in Neuroscience and Cognitive Science, University of Maryland, College Park, Maryland 20742
| |
Collapse
|
4
|
Gabriel DB, Havugimana F, Liley AE, Aguilar I, Yeasin M, Simon NW. Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.08.588332. [PMID: 38645204 PMCID: PMC11030364 DOI: 10.1101/2024.04.08.588332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/23/2024]
Abstract
Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.
Collapse
Affiliation(s)
- Daniel B.K. Gabriel
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN 46202
| | - Felix Havugimana
- Department of Computer Engineering, University of Memphis, Memphis, TN, 38152
| | - Anna E. Liley
- Institut du Cerveau/Paris Brain Institute, Paris, France, 75013
| | - Ivan Aguilar
- Department of Psychology, University of Memphis, Memphis, TN, 38152
| | - Mohammed Yeasin
- Department of Computer Engineering, University of Memphis, Memphis, TN, 38152
| | | |
Collapse
|
5
|
Sharpe MJ. The cognitive (lateral) hypothalamus. Trends Cogn Sci 2024; 28:18-29. [PMID: 37758590 PMCID: PMC10841673 DOI: 10.1016/j.tics.2023.08.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Revised: 08/23/2023] [Accepted: 08/30/2023] [Indexed: 09/29/2023]
Abstract
Despite the physiological complexity of the hypothalamus, its role is typically restricted to initiation or cessation of innate behaviors. For example, theories of lateral hypothalamus argue that it is a switch to turn feeding 'on' and 'off' as dictated by higher-order structures that render when feeding is appropriate. However, recent data demonstrate that the lateral hypothalamus is critical for learning about food-related cues. Furthermore, the lateral hypothalamus opposes learning about information that is neutral or distal to food. This reveals the lateral hypothalamus as a unique arbitrator of learning capable of shifting behavior toward or away from important events. This has relevance for disorders characterized by changes in this balance, including addiction and schizophrenia. Generally, this suggests that hypothalamic function is more complex than increasing or decreasing innate behaviors.
Collapse
Affiliation(s)
- Melissa J Sharpe
- Department of Psychology, University of Sydney, Camperdown, NSW 2006, Australia; Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
6
|
Shi W, Meisner OC, Blackmore S, Jadi MP, Nandy AS, Chang SWC. The orbitofrontal cortex: A goal-directed cognitive map framework for social and non-social behaviors. Neurobiol Learn Mem 2023; 203:107793. [PMID: 37353191 PMCID: PMC10527225 DOI: 10.1016/j.nlm.2023.107793] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 05/28/2023] [Accepted: 06/19/2023] [Indexed: 06/25/2023]
Abstract
The orbitofrontal cortex (OFC) is regarded as one of the core brain areas in a variety of value-based behaviors. Over the past two decades, tremendous knowledge about the OFC function was gained from studying the behaviors of single subjects. As a result, our previous understanding of the OFC's function of encoding decision variables, such as the value and identity of choices, has evolved to the idea that the OFC encodes a more complex representation of the task space as a cognitive map. Accumulating evidence also indicates that the OFC importantly contributes to behaviors in social contexts, especially those involved in cooperative interactions. However, it remains elusive how exactly OFC neurons contribute to social functions and how non-social and social behaviors are related to one another in the computations performed by OFC neurons. In this review, we aim to provide an integrated view of the OFC function across both social and non-social behavioral contexts. We propose that seemingly complex functions of the OFC may be explained by its role in providing a goal-directed cognitive map to guide a wide array of adaptive reward-based behaviors in both social and non-social domains.
Collapse
Affiliation(s)
- Weikang Shi
- Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Psychology, Yale University, New Haven, CT 06510, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA
| | - Olivia C Meisner
- Department of Psychology, Yale University, New Haven, CT 06510, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA
| | - Sylvia Blackmore
- Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Psychology, Yale University, New Haven, CT 06510, USA
| | - Monika P Jadi
- Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA; Department of Psychiatry, Yale University School of Medicine, New Haven, CT 06510, USA
| | - Anirvan S Nandy
- Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA; Kavli Institute for Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA
| | - Steve W C Chang
- Wu Tsai Institute, Yale University, New Haven, CT 06510, USA; Department of Psychology, Yale University, New Haven, CT 06510, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA; Kavli Institute for Neuroscience, Yale University School of Medicine, New Haven, CT 06510, USA.
| |
Collapse
|
7
|
Casula A, Milazzo BM, Martino G, Sergi A, Lucifora C, Tomaiuolo F, Quartarone A, Nitsche MA, Vicario CM. Non-Invasive Brain Stimulation for the Modulation of Aggressive Behavior-A Systematic Review of Randomized Sham-Controlled Studies. Life (Basel) 2023; 13:life13051220. [PMID: 37240865 DOI: 10.3390/life13051220] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 05/10/2023] [Accepted: 05/13/2023] [Indexed: 05/28/2023] Open
Abstract
INTRO Aggressive behavior represents a significant public health issue, with relevant social, political, and security implications. Non-invasive brain stimulation (NIBS) techniques may modulate aggressive behavior through stimulation of the prefrontal cortex. AIMS To review research on the effectiveness of NIBS to alter aggression, discuss the main findings and potential limitations, consider the specifics of the techniques and protocols employed, and discuss clinical implications. METHODS A systematic review of the literature available in the PubMed database was carried out, and 17 randomized sham-controlled studies investigating the effectiveness of NIBS techniques on aggression were included. Exclusion criteria included reviews, meta-analyses, and articles not referring to the subject of interest or not addressing cognitive and emotional modulation aims. CONCLUSIONS The reviewed data provide promising evidence for the beneficial effects of tDCS, conventional rTMS, and cTBS on aggression in healthy adults, forensic, and clinical samples. The specific stimulation target is a key factor for the success of stimulation on aggression modulation. rTMS and cTBS showed opposite effects on aggression compared with tDCS. However, due to the heterogeneity of stimulation protocols, experimental designs, and samples, we cannot exclude other factors that may play a confounding role.
Collapse
Affiliation(s)
- Antony Casula
- Dipartimento di Scienze Cognitive, Psicologiche, Pedagogiche e Degli Studi Culturali, Università di Messina, 98121 Messina, Italy
| | - Bianca M Milazzo
- Dipartimento di Scienze Cognitive, Psicologiche, Pedagogiche e Degli Studi Culturali, Università di Messina, 98121 Messina, Italy
| | - Gabriella Martino
- Dipartimento di Medicina e Clinica Sperimentale, Università degli Studi di Messina, A.O.U. "G. Martino", Via Consolare Valeria, 98125 Messina, Italy
| | - Alessandro Sergi
- Dipartimento di Scienze Matematiche e Informatiche, Scienze Fisiche e Scienze della Terra, Università degli Studi di Messina, Viale F. Stagno d'Alcontres 31, 98166 Messina, Italy
| | - Chiara Lucifora
- Dipartimento di Filosofia e Comunicazione, Università di Bologna, 40131 Bologna, Italy
| | - Francesco Tomaiuolo
- Dipartimento di Medicina e Clinica Sperimentale, Università degli Studi di Messina, A.O.U. "G. Martino", Via Consolare Valeria, 98125 Messina, Italy
| | | | - Michael A Nitsche
- Department of Psychology and Neurosciences, Leibniz Research Centre for Working Environment and Human Factors, 44139 Dortmund, Germany
- University Clinic of Psychiatry and Psychotherapy and University Clinic of Child and Adolescent Psychiatry and Psychotherapy, Protestant Hospital of Bethel Foundation, University Hospital OWL, Bielefeld University, 33615 Bielefeld, Germany
| | - Carmelo M Vicario
- Dipartimento di Scienze Cognitive, Psicologiche, Pedagogiche e Degli Studi Culturali, Università di Messina, 98121 Messina, Italy
| |
Collapse
|
8
|
Costa KM, Scholz R, Lloyd K, Moreno-Castilla P, Gardner MPH, Dayan P, Schoenbaum G. The role of the lateral orbitofrontal cortex in creating cognitive maps. Nat Neurosci 2023; 26:107-115. [PMID: 36550290 PMCID: PMC9839657 DOI: 10.1038/s41593-022-01216-0] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Accepted: 10/26/2022] [Indexed: 12/24/2022]
Abstract
We use mental models of the world-cognitive maps-to guide behavior. The lateral orbitofrontal cortex (lOFC) is typically thought to support behavior by deploying these maps to simulate outcomes, but recent evidence suggests that it may instead support behavior by underlying map creation. We tested between these two alternatives using outcome-specific devaluation and a high-potency chemogenetic approach. Selectively inactivating lOFC principal neurons when male rats learned distinct cue-outcome associations, but before outcome devaluation, disrupted subsequent inference, confirming a role for the lOFC in creating new maps. However, lOFC inactivation surprisingly led to generalized devaluation, a result that is inconsistent with a complete mapping failure. Using a reinforcement learning framework, we show that this effect is best explained by a circumscribed deficit in credit assignment precision during map construction, suggesting that the lOFC has a selective role in defining the specificity of associations that comprise cognitive maps.
Collapse
Affiliation(s)
- Kauê Machado Costa
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD, USA.
| | - Robert Scholz
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany
- Max Planck School of Cognition, Leipzig, Germany
| | - Kevin Lloyd
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| | - Perla Moreno-Castilla
- National Institute on Aging Intramural Research Program, National Institutes of Health, Baltimore, MD, USA
| | | | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany
- University of Tübingen, Tübingen, Germany
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD, USA.
| |
Collapse
|
9
|
Hart EE, Gardner MPH, Panayi MC, Kahnt T, Schoenbaum G. Calcium activity is a degraded estimate of spikes. Curr Biol 2022; 32:5364-5373.e4. [PMID: 36368324 PMCID: PMC9772124 DOI: 10.1016/j.cub.2022.10.037] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 09/20/2022] [Accepted: 10/18/2022] [Indexed: 11/12/2022]
Abstract
Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.
Collapse
Affiliation(s)
- Evan E Hart
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- National Institute of General Medical Sciences, 45 Center Drive, Bethesda, MD 20892, USA
| | - Matthew PH Gardner
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Psychology, Concordia University, 7141 Sherbrooke West, Montreal, QC H4B 1R6, CA
| | - Marios C Panayi
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, 110 S Paca Street, Baltimore, MD 21201, USA
- Department of Neuroscience, Johns Hopkins School of Medicine, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Psychiatry, University of Maryland School of Medicine, 110 S Paca Street, Baltimore, MD 21201, USA
| |
Collapse
|
10
|
Cazares C, Schreiner DC, Valencia ML, Gremel CM. Orbitofrontal cortex populations are differentially recruited to support actions. Curr Biol 2022; 32:4675-4687.e5. [PMID: 36195096 PMCID: PMC9643660 DOI: 10.1016/j.cub.2022.09.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 08/03/2022] [Accepted: 09/09/2022] [Indexed: 11/09/2022]
Abstract
The ability to use information from one's prior actions is necessary for decision-making. While orbitofrontal cortex (OFC) has been hypothesized as key for inferences made using cue and value-related information, whether OFC populations contribute to the use of information from volitional actions to guide behavior is not clear. Here, we used a self-paced lever-press hold-down task in which mice infer prior lever-press durations to guide subsequent action performance. We show that the activity of genetically identified lateral OFC (lOFC) subpopulations differentially instantiate current and prior action information during ongoing action execution. Transient state-dependent lOFC circuit disruptions of specified subpopulations reduced the encoding of ongoing press durations but did not disrupt the use of prior action information to guide future action performance. In contrast, a chronic functional loss of lOFC circuit activity resulted in increased reliance on recently executed lever-press durations and impaired contingency reversal, suggesting the recruitment of compensatory mechanisms that resulted in repetitive action control. Our results identify a novel role for lOFC in the integration of action information to guide adaptive behavior.
Collapse
Affiliation(s)
- Christian Cazares
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Drew C Schreiner
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Mariela Lopez Valencia
- Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA
| | - Christina M Gremel
- The Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA; Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093, USA.
| |
Collapse
|
11
|
Riceberg JS, Srinivasan A, Guise KG, Shapiro ML. Hippocampal signals modify orbitofrontal representations to learn new paths. Curr Biol 2022; 32:3407-3413.e6. [PMID: 35764092 PMCID: PMC11073633 DOI: 10.1016/j.cub.2022.06.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Revised: 04/17/2022] [Accepted: 06/07/2022] [Indexed: 10/17/2022]
Abstract
We often remember the consequences of past choices to adapt to changing circumstances. Recalling past events requires the hippocampus (HPC), and using stimuli to anticipate outcome values requires the orbitofrontal cortex (OFC).1-3 Spatial reversal tasks require both structures to navigate newly rewarded paths.4,5 Both HPC place6 and OFC value cells7,8 fire in phase with theta (4-12 Hz) oscillations. Both structures are described as cognitive maps: HPC maps space9 and OFC maps task states.10 These similarities imply that OFC-HPC interactions are crucial for using memory to predict outcomes when circumstances change, but the mechanisms remain largely unknown. To investigate possible interactions, we simultaneously recorded ensembles in OFC and CA1 as rats learned spatial reversals in a plus maze. Striking interactions occurred only while rats learned their first reversal: CA1 population vectors predicted changes in OFC activity but not vice versa, OFC spikes phase locked to hippocampal theta oscillations, mixed pairs of CA1 and OFC neurons fired together within single theta cycles, and CA1 led OFC spikes by ∼30 ms. After the new contingency became familiar, CA1 ensembles stably represented distinct spatial paths, whereas OFC ensembles developed more generalized goal arm representations in different paths to identical rewards. These frontotemporal interactions, engaged selectively when new task features inform decision-making, suggest a mechanism for linking novel episodes with expected outcomes, when HPC signals trigger "cognitive remapping" by OFC.11.
Collapse
Affiliation(s)
- Justin S Riceberg
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA; Department of Psychiatry, Icahn School of Medicine at Mount Sinai, Hess Center for Science and Medicine, 1470 Madison Avenue, New York, NY 10029, USA.
| | - Aditya Srinivasan
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA
| | - Kevin G Guise
- Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, Hess Center for Science and Medicine, 1470 Madison Avenue, New York, NY 10029, USA
| | - Matthew L Shapiro
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA
| |
Collapse
|
12
|
Musa A, Khan S, Mujahid M, El-Gaby M. The shallow cognitive map hypothesis: A hippocampal framework for thought disorder in schizophrenia. SCHIZOPHRENIA (HEIDELBERG, GERMANY) 2022; 8:34. [PMID: 35853896 PMCID: PMC9261089 DOI: 10.1038/s41537-022-00247-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 03/11/2022] [Indexed: 12/31/2022]
Abstract
Memories are not formed in isolation. They are associated and organized into relational knowledge structures that allow coherent thought. Failure to express such coherent thought is a key hallmark of Schizophrenia. Here we explore the hypothesis that thought disorder arises from disorganized Hippocampal cognitive maps. In doing so, we combine insights from two key lines of investigation, one concerning the neural signatures of cognitive mapping, and another that seeks to understand lower-level cellular mechanisms of cognition within a dynamical systems framework. Specifically, we propose that multiple distinct pathological pathways converge on the shallowing of Hippocampal attractors, giving rise to disorganized Hippocampal cognitive maps and driving conceptual disorganization. We discuss the available evidence at the computational, behavioural, network, and cellular levels. We also outline testable predictions from this framework, including how it could unify major chemical and psychological theories of schizophrenia and how it can provide a rationale for understanding the aetiology and treatment of the disease.
Collapse
Affiliation(s)
- Ayesha Musa
- Green Templeton College, University of Oxford, Oxford, OX2 6HG, UK
| | - Safia Khan
- Green Templeton College, University of Oxford, Oxford, OX2 6HG, UK
| | - Minahil Mujahid
- St Anne's college, University of Oxford, Oxford, OX2 6HS, UK
| | - Mohamady El-Gaby
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, OX1 3SR, UK.
| |
Collapse
|
13
|
Fine JM, Hayden BY. The whole prefrontal cortex is premotor cortex. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200524. [PMID: 34957853 PMCID: PMC8710885 DOI: 10.1098/rstb.2020.0524] [Citation(s) in RCA: 38] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Accepted: 10/01/2021] [Indexed: 11/12/2022] Open
Abstract
We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- Justin M. Fine
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Y. Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
14
|
K Namboodiri VM, Stuber GD. The learning of prospective and retrospective cognitive maps within neural circuits. Neuron 2021; 109:3552-3575. [PMID: 34678148 PMCID: PMC8809184 DOI: 10.1016/j.neuron.2021.09.034] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 08/26/2021] [Accepted: 09/16/2021] [Indexed: 11/18/2022]
Abstract
Brain circuits are thought to form a "cognitive map" to process and store statistical relationships in the environment. A cognitive map is commonly defined as a mental representation that describes environmental states (i.e., variables or events) and the relationship between these states. This process is commonly conceptualized as a prospective process, as it is based on the relationships between states in chronological order (e.g., does reward follow a given state?). In this perspective, we expand this concept on the basis of recent findings to postulate that in addition to a prospective map, the brain forms and uses a retrospective cognitive map (e.g., does a given state precede reward?). In doing so, we demonstrate that many neural signals and behaviors (e.g., habits) that seem inflexible and non-cognitive can result from retrospective cognitive maps. Together, we present a significant conceptual reframing of the neurobiological study of associative learning, memory, and decision making.
Collapse
Affiliation(s)
- Vijay Mohan K Namboodiri
- Department of Neurology, Center for Integrative Neuroscience, Kavli Institute for Fundamental Neuroscience, Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA 94158, USA.
| | - Garret D Stuber
- Center for the Neurobiology of Addiction, Pain, and Emotion, Department of Anesthesiology and Pain Medicine, Department of Pharmacology, Neuroscience Graduate Program, University of Washington, Seattle, WA 98195, USA.
| |
Collapse
|
15
|
Fournier DI, Cheng HY, Robinson S, Todd TP. Cortical Contributions to Higher-Order Conditioning: A Review of Retrosplenial Cortex Function. Front Behav Neurosci 2021; 15:682426. [PMID: 34093148 PMCID: PMC8170078 DOI: 10.3389/fnbeh.2021.682426] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 04/20/2021] [Indexed: 12/18/2022] Open
Abstract
In higher-order conditioning paradigms, such as sensory preconditioning or second-order conditioning, discrete (e.g., phasic) or contextual (e.g., static) stimuli can gain the ability to elicit learned responses despite never being directly paired with reinforcement. The purpose of this mini-review is to examine the neuroanatomical basis of high-order conditioning, by selectively reviewing research that has examined the role of the retrosplenial cortex (RSC) in sensory preconditioning and second-order conditioning. For both forms of higher-order conditioning, we first discuss the types of associations that may occur and then review findings from RSC lesion/inactivation experiments. These experiments demonstrate a role for the RSC in sensory preconditioning, suggesting that this cortical region might contribute to higher-order conditioning via the encoding of neutral stimulus-stimulus associations. In addition, we address knowledge gaps, avenues for future research, and consider the contribution of the RSC to higher-order conditioning in relation to related brain structures.
Collapse
Affiliation(s)
- Danielle I. Fournier
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Han Yin Cheng
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Siobhan Robinson
- Program in Neuroscience, Psychology Department, Hamilton College, Clinton, NY, United States
| | - Travis P. Todd
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| |
Collapse
|