1
|
Selvan RN, Cheng M, Siestrup S, Mecklenbrauck F, Jainta B, Pomp J, Zahedi A, Tamosiunaite M, Wörgötter F, Schubotz RI. Updating predictions in a complex repertoire of actions and its neural representation. Neuroimage 2024; 296:120687. [PMID: 38871038 DOI: 10.1016/j.neuroimage.2024.120687] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Revised: 05/03/2024] [Accepted: 06/11/2024] [Indexed: 06/15/2024] Open
Abstract
Even though actions we observe in everyday life seem to unfold in a continuous manner, they are automatically divided into meaningful chunks, that are single actions or segments, which provide information for the formation and updating of internal predictive models. Specifically, boundaries between actions constitute a hub for predictive processing since the prediction of the current action comes to an end and calls for updating of predictions for the next action. In the current study, we investigated neural processes which characterize such boundaries using a repertoire of complex action sequences with a predefined probabilistic structure. Action sequences consisted of actions that started with the hand touching an object (T) and ended with the hand releasing the object (U). These action boundaries were determined using an automatic computer vision algorithm. Participants trained all action sequences by imitating demo videos. Subsequently, they returned for an fMRI session during which the original action sequences were presented in addition to slightly modified versions thereof. Participants completed a post-fMRI memory test to assess the retention of original action sequences. The exchange of individual actions, and thus a violation of action prediction, resulted in increased activation of the action observation network and the anterior insula. At U events, marking the end of an action, increased brain activation in supplementary motor area, striatum, and lingual gyrus was indicative of the retrieval of the previously encoded action repertoire. As expected, brain activation at U events also reflected the predefined probabilistic branching structure of the action repertoire. At T events, marking the beginning of the next action, midline and hippocampal regions were recruited, reflecting the selected prediction of the unfolding action segment. In conclusion, our findings contribute to a better understanding of the various cerebral processes characterizing prediction during the observation of complex action repertoires.
Collapse
Affiliation(s)
- Rosari Naveena Selvan
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany; Department for Computational Neuroscience, Third Institute of Physics - Biophysics, University of Göttingen, Göttingen, Germany.
| | - Minghao Cheng
- Department for Computational Neuroscience, Third Institute of Physics - Biophysics, University of Göttingen, Göttingen, Germany
| | - Sophie Siestrup
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Falko Mecklenbrauck
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Benjamin Jainta
- Department of Psychology, University of Münster, Münster, Germany
| | - Jennifer Pomp
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Anoushiravan Zahedi
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| | - Minija Tamosiunaite
- Department for Computational Neuroscience, Third Institute of Physics - Biophysics, University of Göttingen, Göttingen, Germany; Faculty of Informatics, Vytautas Magnus University, Kaunas, Lithuania
| | - Florentin Wörgötter
- Department for Computational Neuroscience, Third Institute of Physics - Biophysics, University of Göttingen, Göttingen, Germany
| | - Ricarda I Schubotz
- Department of Psychology, University of Münster, Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Münster, Germany
| |
Collapse
|
2
|
Regalado JM, Corredera Asensio A, Haunold T, Toader AC, Li YR, Neal LA, Rajasethupathy P. Neural activity ramps in frontal cortex signal extended motivation during learning. eLife 2024; 13:RP93983. [PMID: 39037775 PMCID: PMC11262795 DOI: 10.7554/elife.93983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024] Open
Abstract
Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward-associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection-specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward-associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.
Collapse
Affiliation(s)
- Josue M Regalado
- Laboratory of Neural Dynamics & Cognition, The Rockefeller UniversityNew YorkUnited States
| | | | - Theresa Haunold
- Laboratory of Neural Dynamics & Cognition, The Rockefeller UniversityNew YorkUnited States
| | - Andrew C Toader
- Laboratory of Neural Dynamics & Cognition, The Rockefeller UniversityNew YorkUnited States
| | - Yan Ran Li
- Laboratory of Neural Dynamics & Cognition, The Rockefeller UniversityNew YorkUnited States
| | - Lauren A Neal
- Laboratory of Neural Dynamics & Cognition, The Rockefeller UniversityNew YorkUnited States
| | | |
Collapse
|
3
|
Giannone F, Ebrahimi C, Endrass T, Hansson AC, Schlagenhauf F, Sommer WH. Bad habits-good goals? Meta-analysis and translation of the habit construct to alcoholism. Transl Psychiatry 2024; 14:298. [PMID: 39030169 PMCID: PMC11271507 DOI: 10.1038/s41398-024-02965-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 05/19/2024] [Accepted: 05/24/2024] [Indexed: 07/21/2024] Open
Abstract
Excessive alcohol consumption remains a global public health crisis, with millions suffering from alcohol use disorder (AUD, or simply "alcoholism"), leading to significantly reduced life expectancy. This review examines the interplay between habitual and goal-directed behaviors and the associated neurobiological changes induced by chronic alcohol exposure. Contrary to a strict habit-goal dichotomy, our meta-analysis of the published animal experiments combined with a review of human studies reveals a nuanced transition between these behavioral control systems, emphasizing the need for refined terminology to capture the probabilistic nature of decision biases in individuals with a history of chronic alcohol exposure. Furthermore, we distinguish habitual responding from compulsivity, viewing them as separate entities with diverse roles throughout the stages of the addiction cycle. By addressing species-specific differences and translational challenges in habit research, we provide insights to enhance future investigations and inform strategies for combatting AUD.
Collapse
Affiliation(s)
- F Giannone
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, 68159, Mannheim, Germany
| | - C Ebrahimi
- Faculty of Psychology, Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden, 01062, Dresden, Germany
| | - T Endrass
- Faculty of Psychology, Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden, 01062, Dresden, Germany
| | - A C Hansson
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, 68159, Mannheim, Germany
| | - F Schlagenhauf
- Department of Psychotherapy, Campus Charité Mitte, Charité Universitätsmedizin Berlin & St. Hedwig Hospital, 10117, Berlin, Germany
| | - W H Sommer
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, 68159, Mannheim, Germany.
- Bethania Hospital for Psychiatry, Psychosomatics and Psychotherapy, Greifswald, Germany.
- German Center for Mental Health (DZPG), Partner Site Mannheim-Heidelberg-Ulm, 68159, Mannheim, Germany.
| |
Collapse
|
4
|
Chen W, Liang J, Wu Q, Han Y. Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation. Nat Commun 2024; 15:6020. [PMID: 39019943 PMCID: PMC11255269 DOI: 10.1038/s41467-024-50388-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Accepted: 07/05/2024] [Indexed: 07/19/2024] Open
Abstract
Adjusting decision-making under uncertain and dynamic situations is the hallmark of intelligence. It requires a system capable of converting feedback information to renew the internal value. The anterior cingulate cortex (ACC) involves in error and reward events that prompt switching or maintenance of current decision strategies. However, it is unclear whether and how the changes of stimulus-action mapping during behavioral adaptation are encoded, nor how such computation drives decision adaptation. Here, we tracked ACC activity in male mice performing go/no-go auditory discrimination tasks with manipulated stimulus-reward contingencies. Individual ACC neurons integrate the outcome information to the value representation in the next-run trials. Dynamic recruitment of them determines the learning rate of error-guided value iteration and decision adaptation, forming a non-linear feedback-driven updating system to secure the appropriate decision switch. Optogenetically suppressing ACC significantly slowed down feedback-driven decision switching without interfering with the execution of the established strategy.
Collapse
Affiliation(s)
- Wenqi Chen
- Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430030, China
| | - Jiejunyi Liang
- State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Qiyun Wu
- State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Yunyun Han
- Department of Neurobiology, School of Basic Medicine, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430030, China.
| |
Collapse
|
5
|
Yang WF, Sparby T, Wright M, Kim E, Sacchet MD. Volitional mental absorption in meditation: Toward a scientific understanding of advanced concentrative absorption meditation and the case of jhana. Heliyon 2024; 10:e31223. [PMID: 38803854 PMCID: PMC11129010 DOI: 10.1016/j.heliyon.2024.e31223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 04/04/2024] [Accepted: 05/13/2024] [Indexed: 05/29/2024] Open
Abstract
Meditation has been integral to human culture for millennia, deeply rooted in various spiritual and contemplative traditions. While the field of contemplative science has made significant steps toward understanding the effects of meditation on health and well-being, there has been little study of advanced meditative states, including those achieved through intense concentration and absorption. We refer to these types of states as advanced concentrative absorption meditation (ACAM), characterized by absorption with the meditation object leading to states of heightened attention, clarity, energy, effortlessness, and bliss. This review focuses on a type of ACAM known as jhana (ACAM-J) due to its well-documented history, systematic practice approach, recurring phenomenological themes, and growing popularity among contemplative scientists and more generally in media and society. ACAM-J encompasses eight layers of deep concentration, awareness, and internal experiences. Here, we describe the phenomenology of ACAM-J and present evidence from phenomenological and neuroscientific studies that highlight their potential applications in contemplative practices, psychological sciences, and therapeutics. We additionally propose theoretical ACAM-J frameworks grounded in current cognitive neuroscientific understanding of meditation and ancient contemplative traditions. We aim to stimulate further research on ACAM more broadly, encompassing advanced meditation including meditative development and meditative endpoints. Studying advanced meditation including ACAM, and specific practices such as ACAM-J, can potentially revolutionize our understanding of consciousness and applications for mental health.
Collapse
Affiliation(s)
- Winson F.Z. Yang
- Meditation Research Program, Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Terje Sparby
- Steiner University College, 0260, Oslo, Norway
- Department of Psychology and Psychotherapy, Witten/Herdecke University, 58448, Witten, Germany
- Integrated Curriculum for Anthroposophic Psychology, Witten/Herdecke University, 58448, Witten, Germany
| | - Malcolm Wright
- School of Communication, Journalism and Marketing, Massey University, Albany, New Zealand
| | - Eunmi Kim
- Center for Contemplative Science, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Matthew D. Sacchet
- Meditation Research Program, Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
6
|
Alejandro RJ, Holroyd CB. Hierarchical control over foraging behavior by anterior cingulate cortex. Neurosci Biobehav Rev 2024; 160:105623. [PMID: 38490499 DOI: 10.1016/j.neubiorev.2024.105623] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 02/14/2024] [Accepted: 03/13/2024] [Indexed: 03/17/2024]
Abstract
Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).
Collapse
Affiliation(s)
| | - Clay B Holroyd
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
7
|
Sonneborn A, Bartlett L, Olson RJ, Milton R, Abbas AI. Divergent Subregional Information Processing in Mouse Prefrontal Cortex During Working Memory. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.25.591167. [PMID: 38712304 PMCID: PMC11071486 DOI: 10.1101/2024.04.25.591167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Working memory (WM) is a critical cognitive function allowing recent information to be temporarily held in mind to inform future action. This process depends on coordination between key subregions in prefrontal cortex (PFC) and other connected brain areas. However, few studies have examined the degree of functional specialization between these subregions throughout the phases of WM using electrophysiological recordings in freely-moving animals, particularly mice. To this end, we recorded single-units in three neighboring medial PFC (mPFC) subregions in mouse - supplementary motor area (MOs), dorsomedial PFC (dmPFC), and ventromedial (vmPFC) - during a freely-behaving non-match-to-position WM task. We found divergent patterns of task-related activity across the phases of WM. The MOs is most active around task phase transitions and encodes the starting sample location most selectively. Dorsomedial PFC contains a more stable population code, including persistent sample-location-specific firing during a five second delay period. Finally, the vmPFC responds most strongly to reward-related information during the choice phase. Our results reveal anatomically and temporally segregated computation of WM task information in mPFC and motivate more precise consideration of the dynamic neural activity required for WM.
Collapse
Affiliation(s)
- Alex Sonneborn
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR
| | - Lowell Bartlett
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR
| | - Randall J. Olson
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR
| | - Russell Milton
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR
| | - Atheir I. Abbas
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR
- Department of Psychiatry, Oregon Health and Science University, Portland, OR
- VA Portland Health Care System, Portland, OR
| |
Collapse
|
8
|
Regalado JM, Asensio AC, Haunold T, Toader AC, Li YR, Neal LA, Rajasethupathy P. Neural activity ramps in frontal cortex signal extended motivation during learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.15.562395. [PMID: 37905153 PMCID: PMC10614791 DOI: 10.1101/2023.10.15.562395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2023]
Abstract
Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.
Collapse
Affiliation(s)
- Josue M. Regalado
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
| | | | - Theresa Haunold
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
| | - Andrew C. Toader
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
| | - Yan Ran Li
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
| | - Lauren A. Neal
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
| | - Priya Rajasethupathy
- Laboratory of Neural Dynamics & Cognition, The Rockefeller University, New York, NY 10065 USA
- Lead contact
| |
Collapse
|
9
|
Brilliant, Yaar-Soffer Y, Herrmann CS, Henkin Y, Kral A. Theta and alpha oscillatory signatures of auditory sensory and cognitive loads during complex listening. Neuroimage 2024; 289:120546. [PMID: 38387743 DOI: 10.1016/j.neuroimage.2024.120546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Revised: 02/07/2024] [Accepted: 02/15/2024] [Indexed: 02/24/2024] Open
Abstract
The neuronal signatures of sensory and cognitive load provide access to brain activities related to complex listening situations. Sensory and cognitive loads are typically reflected in measures like response time (RT) and event-related potentials (ERPs) components. It's, however, strenuous to distinguish the underlying brain processes solely from these measures. In this study, along with RT- and ERP-analysis, we performed time-frequency analysis and source localization of oscillatory activity in participants performing two different auditory tasks with varying degrees of complexity and related them to sensory and cognitive load. We studied neuronal oscillatory activity in both periods before the behavioral response (pre-response) and after it (post-response). Robust oscillatory activities were found in both periods and were differentially affected by sensory and cognitive load. Oscillatory activity under sensory load was characterized by decrease in pre-response (early) theta activity and increased alpha activity. Oscillatory activity under cognitive load was characterized by increased theta activity, mainly in post-response (late) time. Furthermore, source localization revealed specific brain regions responsible for processing these loads, such as temporal and frontal lobe, cingulate cortex and precuneus. The results provide evidence that in complex listening situations, the brain processes sensory and cognitive loads differently. These neural processes have specific oscillatory signatures and are long lasting, extending beyond the behavioral response.
Collapse
Affiliation(s)
- Brilliant
- Department of Experimental Otology, Hannover Medical School, 30625 Hannover, Germany.
| | - Y Yaar-Soffer
- Department of Communication Disorder, Tel Aviv University, 5262657 Tel Aviv, Israel; Hearing, Speech and Language Center, Sheba Medical Center, 5265601 Tel Hashomer, Israel
| | - C S Herrmann
- Experimental Psychology Division, University of Oldenburg, 26111 Oldenburg, Germany
| | - Y Henkin
- Department of Communication Disorder, Tel Aviv University, 5262657 Tel Aviv, Israel; Hearing, Speech and Language Center, Sheba Medical Center, 5265601 Tel Hashomer, Israel
| | - A Kral
- Department of Experimental Otology, Hannover Medical School, 30625 Hannover, Germany
| |
Collapse
|
10
|
Simpson EH, Akam T, Patriarchi T, Blanco-Pozo M, Burgeno LM, Mohebi A, Cragg SJ, Walton ME. Lights, fiber, action! A primer on in vivo fiber photometry. Neuron 2024; 112:718-739. [PMID: 38103545 PMCID: PMC10939905 DOI: 10.1016/j.neuron.2023.11.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 10/16/2023] [Accepted: 11/15/2023] [Indexed: 12/19/2023]
Abstract
Fiber photometry is a key technique for characterizing brain-behavior relationships in vivo. Initially, it was primarily used to report calcium dynamics as a proxy for neural activity via genetically encoded indicators. This generated new insights into brain functions including movement, memory, and motivation at the level of defined circuits and cell types. Recently, the opportunity for discovery with fiber photometry has exploded with the development of an extensive range of fluorescent sensors for biomolecules including neuromodulators and peptides that were previously inaccessible in vivo. This critical advance, combined with the new availability of affordable "plug-and-play" recording systems, has made monitoring molecules with high spatiotemporal precision during behavior highly accessible. However, while opening exciting new avenues for research, the rapid expansion in fiber photometry applications has occurred without coordination or consensus on best practices. Here, we provide a comprehensive guide to help end-users execute, analyze, and suitably interpret fiber photometry studies.
Collapse
Affiliation(s)
- Eleanor H Simpson
- Department of Psychiatry, Columbia University Medical Center, New York, NY, USA; New York State Psychiatric Institute, New York, NY, USA.
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - Tommaso Patriarchi
- Institute of Pharmacology and Toxicology, University of Zürich, Zürich, Switzerland; Neuroscience Center Zürich, University and ETH Zürich, Zürich, Switzerland.
| | - Marta Blanco-Pozo
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Lauren M Burgeno
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK
| | - Ali Mohebi
- Department of Neurology, University of California, San Francisco, San Francisco, CA, USA
| | - Stephanie J Cragg
- Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, UK; Aligning Science Across Parkinson's (ASAP) Collaborative Research Network, Chevy Chase, MD, USA
| | - Mark E Walton
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| |
Collapse
|
11
|
Venditto SJC, Miller KJ, Brody CD, Daw ND. Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.28.582617. [PMID: 38464244 PMCID: PMC10925334 DOI: 10.1101/2024.02.28.582617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step,reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.
Collapse
|
12
|
Colas JT, O’Doherty JP, Grafton ST. Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts. PLoS Comput Biol 2024; 20:e1011950. [PMID: 38552190 PMCID: PMC10980507 DOI: 10.1371/journal.pcbi.1011950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 02/26/2024] [Indexed: 04/01/2024] Open
Abstract
Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.
Collapse
Affiliation(s)
- Jaron T. Colas
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, California, United States of America
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, United States of America
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, California, United States of America
| | - John P. O’Doherty
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, United States of America
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, California, United States of America
| | - Scott T. Grafton
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, California, United States of America
| |
Collapse
|
13
|
Cruz AS, Cruz S, Remondes M. Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task. OXFORD OPEN NEUROSCIENCE 2024; 3:kvae002. [PMID: 38595941 PMCID: PMC10939314 DOI: 10.1093/oons/kvae002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 01/16/2024] [Accepted: 01/19/2024] [Indexed: 04/11/2024]
Abstract
Working memory is a fundamental cognitive ability, allowing us to keep information in memory for the time needed to perform a given task. A complex neural circuit fulfills these functions, among which is the anterior cingulate cortex (CG). Functionally and anatomically connected to the medial prefrontal, retrosplenial, midcingulate and hippocampus, as well as motor cortices, CG has been implicated in retrieving appropriate information when needed to select and control appropriate behavior. The role of cingulate cortex in working memory-guided behaviors remains unclear due to the lack of studies reversibly interfering with its activity during specific epochs of working memory. We used eNpHR3.0 to silence cingulate neurons while animals perform a standard delayed non-match to trajectory task, and found that, while not causing an absolute impairment in working memory, silencing cingulate neurons during retrieval decreases the mean performance if compared to silencing during encoding. Such retrieval-associated changes are accompanied by longer delays observed when light is delivered to control animals, when compared to eNpHR3.0+ ones, consistent with an adaptive recruitment of additional cognitive resources.
Collapse
Affiliation(s)
- Ana S Cruz
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
| | - Sara Cruz
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
| | - Miguel Remondes
- Instituto de Medicina Molecular João Lobo Antunes, Faculdade de Medicina Universidade de Lisboa, Lisbon 1649-028, Portugal
- Faculdade de Medicina Veterinária Universidade Lusófona, Lisbon 1749-024, Portugal
| |
Collapse
|
14
|
Fetcho RN, Parekh PK, Chou J, Kenwood M, Chalençon L, Estrin DJ, Johnson M, Liston C. A stress-sensitive frontostriatal circuit supporting effortful reward-seeking behavior. Neuron 2024; 112:473-487.e4. [PMID: 37963470 DOI: 10.1016/j.neuron.2023.10.020] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 07/06/2023] [Accepted: 10/18/2023] [Indexed: 11/16/2023]
Abstract
Effort valuation-a process for selecting actions based on the anticipated value of rewarding outcomes and expectations about the work required to obtain them-plays a fundamental role in decision-making. Effort valuation is disrupted in chronic stress states and is supported by the anterior cingulate cortex (ACC), but the circuit-level mechanisms by which the ACC regulates effort-based decision-making are unclear. Here, we show that ACC neurons projecting to the nucleus accumbens (ACC-NAc) play a critical role in effort valuation behavior in mice. Activity in ACC-NAc cells integrates both reward- and effort-related information, encoding a reward-related signal that scales with effort requirements and is necessary for supporting future effortful decisions. Chronic corticosterone exposure reduces motivation, suppresses effortful reward-seeking, and disrupts ACC-NAc signals. Together, our results delineate a stress-sensitive ACC-NAc circuit that supports effortful reward-seeking behavior by integrating reward and effort signals and reinforcing effort allocation in the service of maximizing reward.
Collapse
Affiliation(s)
- Robert N Fetcho
- Feil Family Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY 10021, USA; Weill Cornell/Rockefeller/Sloan Kettering Tri-Institutional MD-PhD Program, Weill Cornell Medicine, New York, NY 10021, USA; Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Puja K Parekh
- Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Jolin Chou
- Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Margaux Kenwood
- Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Laura Chalençon
- Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - David J Estrin
- Feil Family Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY 10021, USA; Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Megan Johnson
- Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA
| | - Conor Liston
- Feil Family Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY 10021, USA; Department of Psychiatry, Weill Cornell Medicine, New York, NY 10021, USA.
| |
Collapse
|
15
|
Blanco-Pozo M, Akam T, Walton ME. Dopamine-independent effect of rewards on choices through hidden-state inference. Nat Neurosci 2024; 27:286-297. [PMID: 38216649 PMCID: PMC10849965 DOI: 10.1038/s41593-023-01542-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 12/01/2023] [Indexed: 01/14/2024]
Abstract
Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement. Nonetheless, although rewards strongly influenced choices and dopamine activity, neither activating nor inhibiting dopamine neurons at trial outcome affected future choice. These data were recapitulated by a neural network model where cortex learned to track hidden task states by predicting observations, while basal ganglia learned values and actions via RPEs. This shows that the influence of rewards on choices can stem from dopamine-independent information they convey about the world's state, not the dopaminergic RPEs they produce.
Collapse
Affiliation(s)
- Marta Blanco-Pozo
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| | - Thomas Akam
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| | - Mark E Walton
- Department of Experimental Psychology, Oxford University, Oxford, UK.
- Wellcome Centre for Integrative Neuroimaging, Oxford University, Oxford, UK.
| |
Collapse
|
16
|
Sato Y, Sakai Y, Hirata S. State-transition-free reinforcement learning in chimpanzees (Pan troglodytes). Learn Behav 2023; 51:413-427. [PMID: 37369920 DOI: 10.3758/s13420-023-00591-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/07/2023] [Indexed: 06/29/2023]
Abstract
The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.
Collapse
Grants
- 16H06283 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 18H05524 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 19J22889 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- 26245069 Ministry of Education, Culture, Sports, Science, Japan Society for the Promotion of Science
- U04 Program for Leading Graduate Schools
Collapse
Affiliation(s)
- Yutaro Sato
- Wildlife Research Center, Kyoto University, Kyoto, Japan.
- University Administration Office, Headquarters for Management Strategy, Niigata University, Niigata, Japan.
| | - Yutaka Sakai
- Brain Science Institute, Tamagawa University, Tokyo, Japan
| | - Satoshi Hirata
- Wildlife Research Center, Kyoto University, Kyoto, Japan
| |
Collapse
|
17
|
Sznabel D, Land R, Kopp B, Kral A. The relation between implicit statistical learning and proactivity as revealed by EEG. Sci Rep 2023; 13:15787. [PMID: 37737452 PMCID: PMC10516964 DOI: 10.1038/s41598-023-42116-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 09/05/2023] [Indexed: 09/23/2023] Open
Abstract
Environmental events often occur on a probabilistic basis but can sometimes be predicted based on specific cues and thus approached proactively. Incidental statistical learning enables the acquisition of knowledge about probabilistic cue-target contingencies. However, the neural mechanisms of statistical learning about contingencies (SLC), the required conditions for successful learning, and the role of implicit processes in the resultant proactive behavior are still debated. We examined changes in behavior and cortical activity during an SLC task in which subjects responded to visual targets. Unbeknown to them, there were three types of target cues associated with high-, low-, and zero target probabilities. About half of the subjects spontaneously gained explicit knowledge about the contingencies (contingency-aware group), and only they showed evidence of proactivity: shortened response times to predictable targets and enhanced event-related brain responses (cue-evoked P300 and contingent negative variation, CNV) to high probability cues. The behavioral and brain responses were strictly associated on a single-trial basis. Source reconstruction of the brain responses revealed activation of fronto-parietal brain regions associated with cognitive control, particularly the anterior cingulate cortex and precuneus. We also found neural correlates of SLC in the contingency-unaware group, but these were restricted to post-target latencies and visual association areas. Our results document a qualitative difference between explicit and implicit learning processes and suggest that in certain conditions, proactivity may require explicit knowledge about contingencies.
Collapse
Affiliation(s)
- Dorota Sznabel
- Department of Experimental Otology, Hannover Medical School, Hannover, Germany.
- Cluster of Excellence "Hearing4all", Hannover, Germany.
| | - Rüdiger Land
- Department of Experimental Otology, Hannover Medical School, Hannover, Germany
| | - Bruno Kopp
- Department of Neurology, Hannover Medical School, Hannover, Germany
| | - Andrej Kral
- Department of Experimental Otology, Hannover Medical School, Hannover, Germany
- Cluster of Excellence "Hearing4all", Hannover, Germany
| |
Collapse
|
18
|
Ulloa Severino FP, Lawal OO, Sakers K, Wang S, Kim N, Friedman AD, Johnson SA, Sriworarat C, Hughes RH, Soderling SH, Kim IH, Yin HH, Eroglu C. Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control. Nat Commun 2023; 14:5522. [PMID: 37684234 PMCID: PMC10491649 DOI: 10.1038/s41467-023-41078-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2022] [Accepted: 08/17/2023] [Indexed: 09/10/2023] Open
Abstract
Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ-1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ-1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ-1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.
Collapse
Affiliation(s)
- Francesco Paolo Ulloa Severino
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA.
- Department of Psychology and Neuroscience, Duke University, Durham, NC, 27710, USA.
- Cajal Institute (CSIC), Madrid, 28001, Spain.
| | | | - Kristina Sakers
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA
| | - Shiyi Wang
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA
| | - Namsoo Kim
- Department of Psychology and Neuroscience, Duke University, Durham, NC, 27710, USA
| | | | - Sarah Anne Johnson
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA
| | | | - Ryan H Hughes
- Department of Psychology and Neuroscience, Duke University, Durham, NC, 27710, USA
| | - Scott H Soderling
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA
- Department of Neurobiology, Duke University Medical Center, Durham, NC, 27710, USA
- Duke Institute for Brain Sciences (DIBS), Durham, NC, 27710, USA
| | - Il Hwan Kim
- Department of Anatomy & Neurobiology, University of Tennessee Health and Science Center, Memphis, TN, 38103, USA
| | - Henry H Yin
- Department of Psychology and Neuroscience, Duke University, Durham, NC, 27710, USA.
- Department of Neurobiology, Duke University Medical Center, Durham, NC, 27710, USA.
- Duke Institute for Brain Sciences (DIBS), Durham, NC, 27710, USA.
| | - Cagla Eroglu
- Department of Cell Biology, Duke University Medical Center, Durham, NC, 27710, USA.
- Department of Neurobiology, Duke University Medical Center, Durham, NC, 27710, USA.
- Duke Institute for Brain Sciences (DIBS), Durham, NC, 27710, USA.
- Howard Hughes Medical Institute, Duke University, Durham, NC, 27710, USA.
| |
Collapse
|
19
|
Dumsky J, Maier ME, Steinhauser M. Effects of the number of competing responses on neural signatures of pre- and post-response conflict. Biol Psychol 2023; 182:108643. [PMID: 37467845 DOI: 10.1016/j.biopsycho.2023.108643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 06/08/2023] [Accepted: 07/14/2023] [Indexed: 07/21/2023]
Abstract
The optimization of human performance requires the continuous monitoring of behavioral conflicts. According to conflict monitoring theory, the dorsal anterior cingulate cortex registers response conflict which is reflected by two electrophysiological signatures, the N2 and the Ne/ERN. The theory assumes that, if a stimulus activates an incorrect response that competes with the correct response, pre-response conflict on correct trials (reflected by the N2) is enhanced but post-response conflict on error trials (reflected by the Ne/ERN) is reduced. Here, we asked whether response conflict depends on the number of competing incorrect responses activated by a stimulus, that is, whether the N2 is further enhanced and the Ne/ERN is further reduced if two incorrect responses are activated as compared to one. To this end, we used a modified flanker paradigm, in which the two flankers were associated either with the same incorrect response or with different incorrect responses. Our results indicate an increased N2 on correct trials and a reduced Ne/ERN on error trials in the latter as compared to the former condition. These results confirm central predictions of conflict monitoring theory and demonstrate that response conflict is directly related to the number of competing incorrect responses.
Collapse
Affiliation(s)
- Julia Dumsky
- Department of Psychology, Catholic University of Eichstätt-Ingolstadt, Ostenstraße 27, 85072 Eichstätt, Germany.
| | - Martin E Maier
- Department of Psychology, Catholic University of Eichstätt-Ingolstadt, Ostenstraße 27, 85072 Eichstätt, Germany
| | - Marco Steinhauser
- Department of Psychology, Catholic University of Eichstätt-Ingolstadt, Ostenstraße 27, 85072 Eichstätt, Germany
| |
Collapse
|
20
|
Jendryka MM, Lewin U, van der Veen B, Kapanaiah SKT, Prex V, Strahnen D, Akam T, Liss B, Pekcec A, Nissen W, Kätzel D. Control of sustained attention and impulsivity by G q-protein signalling in parvalbumin interneurons of the anterior cingulate cortex. Transl Psychiatry 2023; 13:243. [PMID: 37407615 DOI: 10.1038/s41398-023-02541-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 06/19/2023] [Accepted: 06/23/2023] [Indexed: 07/07/2023] Open
Abstract
The anterior cingulate cortex (ACC) has been implicated in attention deficit hyperactivity disorder (ADHD). More specifically, an appropriate balance of excitatory and inhibitory activity in the ACC may be critical for the control of impulsivity, hyperactivity, and sustained attention which are centrally affected in ADHD. Hence, pharmacological augmentation of parvalbumin- (PV) or somatostatin-positive (Sst) inhibitory ACC interneurons could be a potential treatment strategy. We, therefore, tested whether stimulation of Gq-protein-coupled receptors (GqPCRs) in these interneurons could improve attention or impulsivity assessed with the 5-choice-serial reaction-time task in male mice. When challenging impulse control behaviourally or pharmacologically, activation of the chemogenetic GqPCR hM3Dq in ACC PV-cells caused a selective decrease of active erroneous-i.e. incorrect and premature-responses, indicating improved attentional and impulse control. When challenging attention, in contrast, omissions were increased, albeit without extension of reward latencies or decreases of attentional accuracy. These effects largely resembled those of the ADHD medication atomoxetine. Additionally, they were mostly independent of each other within individual animals. GqPCR activation in ACC PV-cells also reduced hyperactivity. In contrast, if hM3Dq was activated in Sst-interneurons, no improvement of impulse control was observed, and a reduction of incorrect responses was only induced at high agonist levels and accompanied by reduced motivational drive. These results suggest that the activation of GqPCRs expressed specifically in PV-cells of the ACC may be a viable strategy to improve certain aspects of sustained attention, impulsivity and hyperactivity in ADHD.
Collapse
Affiliation(s)
- Martin M Jendryka
- Institute of Applied Physiology, Ulm University, Ulm, Germany
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Uwe Lewin
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | | | | | - Vivien Prex
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Daniel Strahnen
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Thomas Akam
- Department of Experimental Psychology and Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Birgit Liss
- Institute of Applied Physiology, Ulm University, Ulm, Germany
- Linacre College and New College, University of Oxford, Oxford, UK
| | - Anton Pekcec
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Wiebke Nissen
- Boehringer Ingelheim Pharma GmbH & Co. KG, Div. Research Germany, Biberach an der Riss, Germany
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm University, Ulm, Germany.
| |
Collapse
|
21
|
van Opheusden B, Kuperwajs I, Galbiati G, Bnaya Z, Li Y, Ma WJ. Expertise increases planning depth in human gameplay. Nature 2023; 618:1000-1005. [PMID: 37258667 DOI: 10.1038/s41586-023-06124-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 04/24/2023] [Indexed: 06/02/2023]
Abstract
A hallmark of human intelligence is the ability to plan multiple steps into the future1,2. Despite decades of research3-5, it is still debated whether skilled decision-makers plan more steps ahead than novices6-8. Traditionally, the study of expertise in planning has used board games such as chess, but the complexity of these games poses a barrier to quantitative estimates of planning depth. Conversely, common planning tasks in cognitive science often have a lower complexity9,10 and impose a ceiling for the depth to which any player can plan. Here we investigate expertise in a complex board game that offers ample opportunity for skilled players to plan deeply. We use model fitting methods to show that human behaviour can be captured using a computational cognitive model based on heuristic search. To validate this model, we predict human choices, response times and eye movements. We also perform a Turing test and a reconstruction experiment. Using the model, we find robust evidence for increased planning depth with expertise in both laboratory and large-scale mobile data. Experts memorize and reconstruct board features more accurately. Using complex tasks combined with precise behavioural modelling might expand our understanding of human planning and help to bridge the gap with progress in artificial intelligence.
Collapse
Affiliation(s)
- Bas van Opheusden
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA.
- Department of Computer Science, Princeton University, Princeton, NJ, USA.
| | - Ionatan Kuperwajs
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Gianni Galbiati
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
- Vidrovr, New York, NY, USA
| | - Zahy Bnaya
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Yunqi Li
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| | - Wei Ji Ma
- Center for Neural Science and Department of Psychology, New York University, New York, NY, USA
| |
Collapse
|
22
|
Kamarajan C, Pandey AK, Chorlian DB, Meyers JL, Kinreich S, Pandey G, Subbie-Saenz de Viteri S, Zhang J, Kuang W, Barr PB, Aliev F, Anokhin AP, Plawecki MH, Kuperman S, Almasy L, Merikangas A, Brislin SJ, Bauer L, Hesselbrock V, Chan G, Kramer J, Lai D, Hartz S, Bierut LJ, McCutcheon VV, Bucholz KK, Dick DM, Schuckit MA, Edenberg HJ, Porjesz B. Predicting Alcohol-Related Memory Problems in Older Adults: A Machine Learning Study with Multi-Domain Features. Behav Sci (Basel) 2023; 13:bs13050427. [PMID: 37232664 DOI: 10.3390/bs13050427] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2023] [Revised: 05/08/2023] [Accepted: 05/11/2023] [Indexed: 05/27/2023] Open
Abstract
Memory problems are common among older adults with a history of alcohol use disorder (AUD). Employing a machine learning framework, the current study investigates the use of multi-domain features to classify individuals with and without alcohol-induced memory problems. A group of 94 individuals (ages 50-81 years) with alcohol-induced memory problems (the memory group) were compared with a matched control group who did not have memory problems. The random forests model identified specific features from each domain that contributed to the classification of the memory group vs. the control group (AUC = 88.29%). Specifically, individuals from the memory group manifested a predominant pattern of hyperconnectivity across the default mode network regions except for some connections involving the anterior cingulate cortex, which were predominantly hypoconnected. Other significant contributing features were: (i) polygenic risk scores for AUD, (ii) alcohol consumption and related health consequences during the past five years, such as health problems, past negative experiences, withdrawal symptoms, and the largest number of drinks in a day during the past twelve months, and (iii) elevated neuroticism and increased harm avoidance, and fewer positive "uplift" life events. At the neural systems level, hyperconnectivity across the default mode network regions, including the connections across the hippocampal hub regions, in individuals with memory problems may indicate dysregulation in neural information processing. Overall, the study outlines the importance of utilizing multidomain features, consisting of resting-state brain connectivity data collected ~18 years ago, together with personality, life experiences, polygenic risk, and alcohol consumption and related consequences, to predict the alcohol-related memory problems that arise in later life.
Collapse
Affiliation(s)
- Chella Kamarajan
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Ashwini K Pandey
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - David B Chorlian
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Jacquelyn L Meyers
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Sivan Kinreich
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Gayathri Pandey
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Stacey Subbie-Saenz de Viteri
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Jian Zhang
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Weipeng Kuang
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Peter B Barr
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| | - Fazil Aliev
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers University, Piscataway, NJ 08854, USA
| | - Andrey P Anokhin
- Department of Psychiatry, School of Medicine, Washington University, St. Louis, MO 63110, USA
| | | | - Samuel Kuperman
- Department of Psychiatry, University of Iowa, Iowa City, IA 52242, USA
| | - Laura Almasy
- The Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Alison Merikangas
- The Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Sarah J Brislin
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers University, Piscataway, NJ 08854, USA
| | - Lance Bauer
- Department of Psychiatry, University of Connecticut, Farmington, CT 06030, USA
| | - Victor Hesselbrock
- Department of Psychiatry, University of Connecticut, Farmington, CT 06030, USA
| | - Grace Chan
- Department of Psychiatry, University of Iowa, Iowa City, IA 52242, USA
- Department of Psychiatry, University of Connecticut, Farmington, CT 06030, USA
| | - John Kramer
- Department of Psychiatry, University of Iowa, Iowa City, IA 52242, USA
| | - Dongbing Lai
- Indiana University School of Medicine, Indianapolis, IN 46202, USA
| | - Sarah Hartz
- Department of Psychiatry, School of Medicine, Washington University, St. Louis, MO 63110, USA
| | - Laura J Bierut
- Department of Psychiatry, School of Medicine, Washington University, St. Louis, MO 63110, USA
| | - Vivia V McCutcheon
- Department of Psychiatry, School of Medicine, Washington University, St. Louis, MO 63110, USA
| | - Kathleen K Bucholz
- Department of Psychiatry, School of Medicine, Washington University, St. Louis, MO 63110, USA
| | - Danielle M Dick
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers University, Piscataway, NJ 08854, USA
| | - Marc A Schuckit
- Department of Psychiatry, University of California, San Diego, CA 92103, USA
| | | | - Bernice Porjesz
- Henri Begleiter Neurodynamics Lab, Department of Psychiatry and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY 11203, USA
| |
Collapse
|
23
|
Kruithof ES, Klaus J, Schutter DJLG. The human cerebellum in reward anticipation and reward outcome processing: An activation likelihood estimation meta-analysis. Neurosci Biobehav Rev 2023; 149:105171. [PMID: 37060968 DOI: 10.1016/j.neubiorev.2023.105171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 03/10/2023] [Accepted: 04/11/2023] [Indexed: 04/17/2023]
Abstract
The cerebellum generates internal prediction models and actively compares anticipated and actual outcomes in order to reach a desired end state. In this process, reward can serve as a reinforcer that shapes internal prediction models, enabling context-appropriate behavior. While the involvement of the cerebellum in reward processing has been established in animals, there is no detailed account of which cerebellar regions are involved in reward anticipation and reward outcome processing in humans. To this end, an activation likelihood estimation meta-analysis of functional neuroimaging studies was performed to investigate cerebellar functional activity patterns associated with reward anticipation and reward outcome processing in healthy adults. Results showed that reward anticipation (k=31) was associated with regional activity in the bilateral anterior lobe, bilateral lobule VI, left Crus I and the posterior vermis, while reward outcome (k=16) was associated with regional activity in the declive and left lobule VI. The findings of this meta-analysis show distinct involvement of the cerebellum in reward anticipation and reward outcome processing as part of a predictive coding routine.
Collapse
Affiliation(s)
- Eline S Kruithof
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, the Netherlands.
| | - Jana Klaus
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, the Netherlands
| | - Dennis J L G Schutter
- Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, the Netherlands
| |
Collapse
|
24
|
Ye T, Romero-Sosa JL, Rickard A, Aguirre CG, Wikenheiser AM, Blair HT, Izquierdo A. Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning. OXFORD OPEN NEUROSCIENCE 2023; 2:kvad005. [PMID: 37456140 PMCID: PMC10348740 DOI: 10.1093/oons/kvad005] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 03/20/2023] [Accepted: 03/23/2023] [Indexed: 07/18/2023]
Abstract
Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition, there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.
Collapse
Affiliation(s)
- Tony Ye
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
| | | | - Anne Rickard
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
| | | | - Andrew M Wikenheiser
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
- The Brain Research Institute, UCLA, Los Angeles, CA 90095, USA
- Integrative Center for Learning and Memory, UCLA, Los Angeles, CA 90095, USA
- Integrative Center for Addictions, UCLA, Los Angeles, CA 90095, USA
| | - Hugh T Blair
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
- The Brain Research Institute, UCLA, Los Angeles, CA 90095, USA
- Integrative Center for Learning and Memory, UCLA, Los Angeles, CA 90095, USA
| | - Alicia Izquierdo
- Department of Psychology, UCLA, Los Angeles, CA 90095, USA
- The Brain Research Institute, UCLA, Los Angeles, CA 90095, USA
- Integrative Center for Learning and Memory, UCLA, Los Angeles, CA 90095, USA
- Integrative Center for Addictions, UCLA, Los Angeles, CA 90095, USA
| |
Collapse
|
25
|
Brandl F, Knolle F, Avram M, Leucht C, Yakushev I, Priller J, Leucht S, Ziegler S, Wunderlich K, Sorg C. Negative symptoms, striatal dopamine and model-free reward decision-making in schizophrenia. Brain 2023; 146:767-777. [PMID: 35875972 DOI: 10.1093/brain/awac268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 06/13/2022] [Accepted: 07/04/2022] [Indexed: 11/13/2022] Open
Abstract
Negative symptoms, such as lack of motivation or social withdrawal, are highly prevalent and debilitating in patients with schizophrenia. Underlying mechanisms of negative symptoms are incompletely understood, thereby preventing the development of targeted treatments. We hypothesized that in patients with schizophrenia during psychotic remission, impaired influences of both model-based and model-free reward predictions on decision-making ('reward prediction influence', RPI) underlie negative symptoms. We focused on psychotic remission, because psychotic symptoms might confound reward-based decision-making. Moreover, we hypothesized that impaired model-based/model-free RPIs depend on alterations of both associative striatum dopamine synthesis and storage (DSS) and executive functioning. Both factors influence RPI in healthy subjects and are typically impaired in schizophrenia. Twenty-five patients with schizophrenia with pronounced negative symptoms during psychotic remission and 24 healthy controls were included in the study. Negative symptom severity was measured by the Positive and Negative Syndrome Scale negative subscale, model-based/model-free RPI by the two-stage decision task, associative striatum DSS by 18F-DOPA positron emission tomography and executive functioning by the symbol coding task. Model-free RPI was selectively reduced in patients and associated with negative symptom severity as well as with reduced associative striatum DSS (in patients only) and executive functions (both in patients and controls). In contrast, model-based RPI was not altered in patients. Results provide evidence for impaired model-free reward prediction influence as a mechanism for negative symptoms in schizophrenia as well as for reduced associative striatum dopamine and executive dysfunction as relevant factors. Data suggest potential treatment targets for patients with schizophrenia and pronounced negative symptoms.
Collapse
Affiliation(s)
- Felix Brandl
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,Department of Neuroradiology, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,TUM-NIC Neuroimaging Center, School of Medicine, Technical University of Munich, Munich, 81675, Germany
| | - Franziska Knolle
- Department of Neuroradiology, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,TUM-NIC Neuroimaging Center, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,Department of Psychiatry, University of Cambridge, Cambridge CB20SZ, UK
| | - Mihai Avram
- Translational Psychiatry, Department of Psychiatry and Psychotherapy, University of Lübeck, Lübeck, 23538, Germany
| | - Claudia Leucht
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, 81675, Germany
| | - Igor Yakushev
- Department of Nuclear Medicine, School of Medicine, Technical University of Munich, Munich, 81675, Germany
| | - Josef Priller
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,Neuropsychiatry, Charité-Universitätsmedizin Berlin, and DZNE, Berlin, 10117, Germany.,UK DRI at University of Edinburgh, Edinburgh EH16 4SB, UK.,IoPPN, King's College London, London SE5 8AF, UK
| | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,Department of Psychosis studies, King's College London, London, UK
| | - Sibylle Ziegler
- Department of Nuclear Medicine, Ludwig-Maximilians University Munich, Munich, 81377, Germany
| | - Klaus Wunderlich
- Department of Psychology, Ludwig-Maximilians University Munich, Munich, 81377, Germany
| | - Christian Sorg
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,Department of Neuroradiology, School of Medicine, Technical University of Munich, Munich, 81675, Germany.,TUM-NIC Neuroimaging Center, School of Medicine, Technical University of Munich, Munich, 81675, Germany
| |
Collapse
|
26
|
Cox J, Minerva AR, Fleming WT, Zimmerman CA, Hayes C, Zorowitz S, Bandi A, Ornelas S, McMannon B, Parker NF, Witten IB. A neural substrate of sex-dependent modulation of motivation. Nat Neurosci 2023; 26:274-284. [PMID: 36646878 DOI: 10.1038/s41593-022-01229-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 11/01/2022] [Indexed: 01/18/2023]
Abstract
While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.
Collapse
Affiliation(s)
- Julia Cox
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.
| | - Adelaide R Minerva
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Weston T Fleming
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Cameron Hayes
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Samuel Zorowitz
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Akhil Bandi
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Sharon Ornelas
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Nathan F Parker
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
27
|
Moin Afshar N, Cinotti F, Martin D, Khamassi M, Calu DJ, Taylor JR, Groman SM. Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related. J Neurosci 2023; 43:458-471. [PMID: 36216504 PMCID: PMC9864557 DOI: 10.1523/jneurosci.1113-22.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 10/03/2022] [Accepted: 10/06/2022] [Indexed: 01/25/2023] Open
Abstract
Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.
Collapse
Affiliation(s)
- Neema Moin Afshar
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
| | - François Cinotti
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - David Martin
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland 21201
| | - Mehdi Khamassi
- Institute of Intelligent Systems and Robotics, Centre National de la Recherche Scientifique, Sorbonne University, 75005 Paris, France
| | - Donna J Calu
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland 21201
- Program in Neuroscience, University of Maryland School of Medicine, Baltimore, Maryland 21201
| | - Jane R Taylor
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
- Department of Psychology, Yale University, New Haven, Connecticut 06520
| | - Stephanie M Groman
- Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
- Department of Neuroscience, University of Minnesota Medical School, Minneapolis, Minnesota 55455
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| |
Collapse
|
28
|
Diehl GW, Redish AD. Differential processing of decision information in subregions of rodent medial prefrontal cortex. eLife 2023; 12:82833. [PMID: 36652289 PMCID: PMC9848391 DOI: 10.7554/elife.82833] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Accepted: 01/05/2023] [Indexed: 01/19/2023] Open
Abstract
Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.
Collapse
Affiliation(s)
- Geoffrey W Diehl
- Department of Neuroscience, University of MinnesotaMinneapolisUnited States
| | - A David Redish
- Department of Neuroscience, University of MinnesotaMinneapolisUnited States
| |
Collapse
|
29
|
Nees F, Usai K, Kandić M, Zidda F, Heukamp NJ, Moliadze V, Löffler M, Flor H. The association of spouse interactions and emotional learning in interference related to chronic back pain. NEUROBIOLOGY OF PAIN (CAMBRIDGE, MASS.) 2023; 13:100122. [PMID: 36910586 PMCID: PMC9996357 DOI: 10.1016/j.ynpai.2023.100122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 02/07/2023] [Accepted: 02/20/2023] [Indexed: 02/24/2023]
Abstract
Social interactions affect individual behaviours, preferences, and attitudes. This is also critical in the context of experiencing pain and expressing pain behaviours, and may relate to learned emotional responses. In this respect, individual variability in the medial prefrontal cortex (mPFC), which is involved in adjusting an organism's behaviour to its environment by evaluating and interpreting information within the context of past experiences, is important. It is critical for selecting suitable behavioural responses within a social environment and may reinforce maladaptation in chronic pain. In our study, we used brain imaging during appetitive and aversive pavlovian conditioning in persons with chronic back pain (CBP), subacute back pain (SABP), and healthy controls (HC), together with information on spouse responses to pain behaviours. We also examined the relationship of these responses with pain-related interference in the patients. Our findings yielded a significant negative association between mPFC responses to appetitive and aversive learning in CBP. We also observed a significant negative association for mPFC responses during aversive learning and distracting spouse responses, and a significant positive association between mPFC responses during appetitive learning and solicitous spouse responses in CBP. Both significantly predicted pain-related interference in the CBP group (explained variance up to 53%). Significant associations were not found for SABP or HC. Our findings support an association between appetitive and aversive pavlovian learning, related brain circuits and spouse responses to pain in CBP, where appetitive and aversive learning processes seem to be differentially involved. This can inform prevention and early intervention in a mechanistic approach.
Collapse
Affiliation(s)
- Frauke Nees
- Institute of Medical Psychology and Medical Sociology, University Medical Center Schleswig-Holstein, Kiel University, Kiel, Germany.,Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Katrin Usai
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Mina Kandić
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Francesca Zidda
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Nils Jannik Heukamp
- Institute of Medical Psychology and Medical Sociology, University Medical Center Schleswig-Holstein, Kiel University, Kiel, Germany
| | - Vera Moliadze
- Institute of Medical Psychology and Medical Sociology, University Medical Center Schleswig-Holstein, Kiel University, Kiel, Germany
| | - Martin Löffler
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Herta Flor
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| |
Collapse
|
30
|
Hart EE, Gardner MPH, Panayi MC, Kahnt T, Schoenbaum G. Calcium activity is a degraded estimate of spikes. Curr Biol 2022; 32:5364-5373.e4. [PMID: 36368324 PMCID: PMC9772124 DOI: 10.1016/j.cub.2022.10.037] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 09/20/2022] [Accepted: 10/18/2022] [Indexed: 11/12/2022]
Abstract
Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function-hippocampal neurons respond to locations in space,1 motor cortex neurons encode movement direction,2 and dopamine neurons signal reward prediction errors3-observations undergirding current theories of cognition,4 movement,5 and learning.6 Recently it has become possible to measure calcium flux, an internal cellular signal related to spiking. The ability to image calcium flux in anatomically7,8 or genetically9 identified neurons can extend our knowledge of neural circuit function by allowing activity to be monitored in specific cell types or projections, or in the same neurons across many days. However, while initial studies were grounded in prior unit recording work, it has become fashionable to assume that calcium is identical to spiking, even though the spike-to-fluorescence transformation is nonlinear, noisy, and unpredictable under real-world conditions.10 It remains an open question whether calcium provides a high-fidelity representation of single-unit activity in awake, behaving subjects. Here, we have addressed this question by recording both signals in the lateral orbitofrontal cortex (OFC) of rats during olfactory discrimination learning. Activity in the OFC during olfactory learning has been well-studied in humans,11,12,13,14 nonhuman primates,15,16 and rats,17,18,19,20,21 where it has been shown to signal information about both the sensory properties of odor cues and the rewards they predict. Our single-unit results replicated prior findings, whereas the calcium signal provided only a degraded estimate of the information available in the single-unit spiking, reflecting primarily reward value.
Collapse
Affiliation(s)
- Evan E Hart
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- National Institute of General Medical Sciences, 45 Center Drive, Bethesda, MD 20892, USA
| | - Matthew PH Gardner
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Psychology, Concordia University, 7141 Sherbrooke West, Montreal, QC H4B 1R6, CA
| | - Marios C Panayi
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, 110 S Paca Street, Baltimore, MD 21201, USA
- Department of Neuroscience, Johns Hopkins School of Medicine, 251 Bayview Boulevard, Baltimore, MD 21224, USA
- Department of Psychiatry, University of Maryland School of Medicine, 110 S Paca Street, Baltimore, MD 21201, USA
| |
Collapse
|
31
|
Prospective and retrospective values integrated in frontal cortex drive predictive choice. Proc Natl Acad Sci U S A 2022; 119:e2206067119. [PMID: 36417435 PMCID: PMC9889848 DOI: 10.1073/pnas.2206067119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
To make a deliberate action in a volatile environment, the brain must frequently reassess the value of each action (action-value). Choice can be initially made from the experience of trial-and-errors, but once the dynamics of the environment is learned, the choice can be made from the knowledge of the environment. The action-values constructed from the experience (retrospective value) and the ones from the knowledge (prospective value) were identified in various regions of the brain. However, how and which neural circuit integrates these values and executes the chosen action remains unknown. Combining reinforcement learning and two-photon calcium imaging, we found that the preparatory activity of neurons in a part of the frontal cortex, the anterior-lateral motor (ALM) area, initially encodes retrospective value, but after extensive training, they jointly encode the retrospective and prospective value. Optogenetic inhibition of ALM preparatory activity specifically abolished the expert mice's predictive choice behavior and returned them to the novice-like state. Thus, the integrated action-value encoded in the preparatory activity of ALM plays an important role to bias the action toward the knowledge-dependent, predictive choice behavior.
Collapse
|
32
|
Pearce AL, Fuchs BA, Keller KL. The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors. Front Nutr 2022; 9:1021868. [PMID: 36483928 PMCID: PMC9722736 DOI: 10.3389/fnut.2022.1021868] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Accepted: 11/04/2022] [Indexed: 11/23/2022] Open
Abstract
The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.
Collapse
Affiliation(s)
- Alaina L. Pearce
- Social Science Research Institute, Pennsylvania State University, University Park, PA, United States
- Department of Nutritional Sciences, Pennsylvania State University, University Park, PA, United States
| | - Bari A. Fuchs
- Department of Nutritional Sciences, Pennsylvania State University, University Park, PA, United States
| | - Kathleen L. Keller
- Social Science Research Institute, Pennsylvania State University, University Park, PA, United States
- Department of Nutritional Sciences, Pennsylvania State University, University Park, PA, United States
- Department of Food Science, Pennsylvania State University, University Park, PA, United States
| |
Collapse
|
33
|
Manelis A, Halchenko YO, Satz S, Ragozzino R, Iyengar S, Swartz HA, Levine MD. The interaction between depression diagnosis and BMI is related to altered activation pattern in the right inferior frontal gyrus and anterior cingulate cortex during food anticipation. Brain Behav 2022; 12:e2695. [PMID: 35962573 PMCID: PMC9480896 DOI: 10.1002/brb3.2695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Accepted: 06/22/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Depression and overweight/obesity often cooccur but the underlying neural mechanisms for this bidirectional link are not well understood. METHODS In this functional magnetic resonance imaging study, we scanned 54 individuals diagnosed with depressive disorders (DD) and 48 healthy controls (HC) to examine how diagnostic status moderates the relationship between body mass index (BMI) and brain activation during anticipation and pleasantness rating of food versus nonfood stimuli. RESULTS We found a significant BMI-by-diagnosis interaction effect on activation in the right inferior frontal gyrus (RIFG) and anterior cingulate cortex (ACC) during food versus nonfood anticipation (p < .0125). Brain activation in these regions was greater in HC with higher BMI than in HC with lower BMI. Individuals with DD showed an opposite pattern of activation. Structural equation modeling revealed that the relationship between BMI, activation in the RIFG and ACC, and participants' desire to eat food items shown in the experiment depended on the diagnostic status. CONCLUSIONS Considering that food anticipation is an important component of appetitive behavior and that the RIFG and ACC are involved in emotion regulation, response inhibition and conflict monitoring necessary to control this behavior, we propose that future clinical trials targeting weight loss in DD should investigate whether adequate mental preparation positively affects subsequent food consumption behaviors in these individuals.
Collapse
Affiliation(s)
- A Manelis
- Department of PsychiatryUniversity of PittsburghPittsburghPennsylvania
| | - YO Halchenko
- Department of Psychological and Brain SciencesDartmouth CollegeHanoverNew Hampshire
| | - S Satz
- Department of PsychiatryUniversity of PittsburghPittsburghPennsylvania
| | - R Ragozzino
- Department of PsychiatryUniversity of PittsburghPittsburghPennsylvania
| | - S Iyengar
- Department of StatisticsUniversity of PittsburghPittsburghPennsylvania
| | - HA Swartz
- Department of PsychiatryUniversity of PittsburghPittsburghPennsylvania
| | - MD Levine
- Department of PsychiatryUniversity of PittsburghPittsburghPennsylvania
| |
Collapse
|
34
|
Cingulate-motor circuits update rule representations for sequential choice decisions. Nat Commun 2022; 13:4545. [PMID: 35927275 PMCID: PMC9352796 DOI: 10.1038/s41467-022-32142-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Accepted: 07/19/2022] [Indexed: 12/04/2022] Open
Abstract
Anterior cingulate cortex mediates the flexible updating of an animal’s choice responses upon rule changes in the environment. However, how anterior cingulate cortex entrains motor cortex to reorganize rule representations and generate required motor outputs remains unclear. Here, we demonstrate that chemogenetic silencing of the terminal projections of cingulate cortical neurons in secondary motor cortex in the rat disrupts choice performance in trials immediately following rule switches, suggesting that these inputs are necessary to update rule representations for choice decisions stored in the motor cortex. Indeed, the silencing of cingulate cortex decreases rule selectivity of secondary motor cortical neurons. Furthermore, optogenetic silencing of cingulate cortical neurons that is temporally targeted to error trials immediately after rule switches exacerbates errors in the following trials. These results suggest that cingulate cortex monitors behavioral errors and updates rule representations in motor cortex, revealing a critical role for cingulate-motor circuits in adaptive choice behaviors. The anterior cingulate cortex allows an animal to update its behaviour when the environment changes. In this work, the authors identify a pathway from cingulate to secondary motor cortex, critical for updating motor rules following behavioural errors.
Collapse
|
35
|
Kane GA, James MH, Shenhav A, Daw ND, Cohen JD, Aston-Jones G. Rat Anterior Cingulate Cortex Continuously Signals Decision Variables in a Patch Foraging Task. J Neurosci 2022; 42:5730-5744. [PMID: 35688627 PMCID: PMC9302469 DOI: 10.1523/jneurosci.1940-21.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2021] [Revised: 04/24/2022] [Accepted: 04/28/2022] [Indexed: 01/22/2023] Open
Abstract
In patch foraging tasks, animals must decide whether to remain with a depleting resource or to leave it in search of a potentially better source of reward. In such tasks, animals consistently follow the general predictions of optimal foraging theory (the marginal value theorem; MVT): to leave a patch when the reward rate in the current patch depletes to the average reward rate across patches. Prior studies implicate an important role for the anterior cingulate cortex (ACC) in foraging decisions based on MVT: within single trials, ACC activity increases immediately preceding foraging decisions, and across trials, these dynamics are modulated as the value of staying in the patch depletes to the average reward rate. Here, we test whether these activity patterns reflect dynamic encoding of decision-variables and whether these signals are directly involved in decision-making. We developed a leaky accumulator model based on the MVT that generates estimates of decision variables within and across trials, and tested model predictions against ACC activity recorded from male rats performing a patch foraging task. Model predicted changes in MVT decision variables closely matched rat ACC activity. Next, we pharmacologically inactivated ACC in male rats to test the contribution of these signals to decision-making. ACC inactivation had a profound effect on rats' foraging decisions and response times (RTs) yet rats still followed the MVT decision rule. These findings indicate that the ACC encodes foraging-related variables for reasons unrelated to patch-leaving decisions.SIGNIFICANCE STATEMENT The ability to make adaptive patch-foraging decisions, to remain with a depleting resource or search for better alternatives, is critical to animal well-being. Previous studies have found that anterior cingulate cortex (ACC) activity is modulated at different points in the foraging decision process, raising questions about whether the ACC guides ongoing decisions or serves a more general purpose of regulating cognitive control. To investigate the function of the ACC in foraging, the present study developed a dynamic model of behavior and neural activity, and tested model predictions using recordings and inactivation of ACC. Findings revealed that ACC continuously signals decision variables but that these signals are more likely used to monitor and regulate ongoing processes than to guide foraging decisions.
Collapse
Affiliation(s)
- Gary A Kane
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts 02155
| | - Morgan H James
- Department of Psychiatry, Robert Wood Johnson Medical School, Rutgers University, Piscataway, New Jersey 08854
- Brain Health Institute, Rutgers University, Pisccataway, New Jersey 08854
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, & Psychological Sciences and Carney Institute for Brain Science, Brown University, Providence, Rhode Island 02912
| | - Nathaniel D Daw
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
| | - Jonathan D Cohen
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, New Jersey 08544
| | - Gary Aston-Jones
- Brain Health Institute, Rutgers University, Pisccataway, New Jersey 08854
| |
Collapse
|
36
|
Brain-inspired meta-reinforcement learning cognitive control in conflictual inhibition decision-making task for artificial agents. Neural Netw 2022; 154:283-302. [DOI: 10.1016/j.neunet.2022.06.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2021] [Revised: 06/09/2022] [Accepted: 06/16/2022] [Indexed: 11/21/2022]
|
37
|
Castro-Rodrigues P, Akam T, Snorasson I, Camacho M, Paixão V, Maia A, Barahona-Corrêa JB, Dayan P, Simpson HB, Costa RM, Oliveira-Maia AJ. Explicit knowledge of task structure is a primary determinant of human model-based action. Nat Hum Behav 2022; 6:1126-1141. [PMID: 35589826 DOI: 10.1038/s41562-022-01346-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Revised: 03/19/2022] [Accepted: 03/31/2022] [Indexed: 11/09/2022]
Abstract
Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.
Collapse
Affiliation(s)
- Pedro Castro-Rodrigues
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Centro Hospitalar Psiquiátrico de Lisboa, Lisbon, Portugal
| | - Thomas Akam
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Ivar Snorasson
- Center for Obsessive-Compulsive & Related Disorders, New York State Psychiatric Institute, New York, NY, USA
| | - Marta Camacho
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,John Van Geest Center for Brain Repair, University of Cambridge, Cambridge, UK
| | - Vitor Paixão
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal
| | - Ana Maia
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Department of Psychiatry and Mental Health, Centro Hospitalar de Lisboa Ocidental, Lisbon, Portugal
| | - J Bernardo Barahona-Corrêa
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal.,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany.,The University of Tübingen, Tübingen, Germany
| | - H Blair Simpson
- Center for Obsessive-Compulsive & Related Disorders, New York State Psychiatric Institute, New York, NY, USA.,Department of Psychiatry, Columbia University, New York, NY, USA
| | - Rui M Costa
- Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal.,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Albino J Oliveira-Maia
- Champalimaud Clinical Centre, Champalimaud Foundation, Lisbon, Portugal. .,Champalimaud Research, Champalimaud Foundation, Lisbon, Portugal. .,NOVA Medical School, NMS, Universidade Nova de Lisboa, Lisbon, Portugal.
| |
Collapse
|
38
|
Karvelis P, Diaconescu AO. A Computational Model of Hopelessness and Active-Escape Bias in Suicidality. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2022; 6:34-59. [PMID: 38774778 PMCID: PMC11104346 DOI: 10.5334/cpsy.80] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 02/15/2022] [Indexed: 12/27/2022]
Abstract
Currently, psychiatric practice lacks reliable predictive tools and a sufficiently detailed mechanistic understanding of suicidal thoughts and behaviors (STB) to provide timely and personalized interventions. Developing computational models of STB that integrate across behavioral, cognitive and neural levels of analysis could help better understand STB vulnerabilities and guide personalized interventions. To that end, we present a computational model based on the active inference framework. With this model, we show that several STB risk markers - hopelessness, Pavlovian bias and active-escape bias - are interrelated via the drive to maximize one's model evidence. We propose four ways in which these effects can arise: (1) increased learning from aversive outcomes, (2) reduced belief decay in response to unexpected outcomes, (3) increased stress sensitivity and (4) reduced sense of stressor controllability. These proposals stem from considering the neurocircuits implicated in STB: how the locus coeruleus - norepinephrine (LC-NE) system together with the amygdala (Amy), the dorsal prefrontal cortex (dPFC) and the anterior cingulate cortex (ACC) mediate learning in response to acute stress and volatility as well as how the dorsal raphe nucleus - serotonin (DRN-5-HT) system together with the ventromedial prefrontal cortex (vmPFC) mediate stress reactivity based on perceived stressor controllability. We validate the model by simulating performance in an Avoid/Escape Go/No-Go task replicating recent behavioral findings. This serves as a proof of concept and provides a computational hypothesis space that can be tested empirically and be used to distinguish planful versus impulsive STB subtypes. We discuss the relevance of the proposed model for treatment response prediction, including pharmacotherapy and psychotherapy, as well as sex differences as it relates to stress reactivity and suicide risk.
Collapse
Affiliation(s)
- Povilas Karvelis
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health (CAMH), Toronto, Ontario, Canada
| | - Andreea O. Diaconescu
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health (CAMH), Toronto, Ontario, Canada
- University of Toronto, Department of Psychiatry, Toronto, Ontario, Canada
- Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada
- Department of Psychology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
39
|
Posner MI, Weible AP, Voelker P, Rothbart MK, Niell CM. Decision Making as a Learned Skill in Mice and Humans. Front Neurosci 2022; 16:834701. [PMID: 35360159 PMCID: PMC8963179 DOI: 10.3389/fnins.2022.834701] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 02/08/2022] [Indexed: 11/18/2022] Open
Abstract
Attention is a necessary component in many forms of human and animal learning. Numerous studies have described how attention and memory interact when confronted with a choice point during skill learning. In both animal and human studies, pathways have been found that connect the executive and orienting networks of attention to the hippocampus. The anterior cingulate cortex, part of the executive attention network, is linked to the hippocampus via the nucleus reuniens of the thalamus. The parietal cortex, part of the orienting attention network, accesses the hippocampus via the entorhinal cortex. These studies have led to specific predictions concerning the functional role of each pathway in connecting the cortex to the hippocampus. Here, we review some of the predictions arising from these studies. We then discuss potential methods for manipulating the two pathways and assessing the directionality of their functional connection using viral expression techniques in mice. New studies may allow testing of a behavioral model specifying how the two pathways work together during skill learning.
Collapse
Affiliation(s)
- Michael I. Posner
- Institute of Neuroscience, University of Oregon, Eugene, OR, United States
- Department of Psychology, University of Oregon, Eugene, OR, United States
- *Correspondence: Michael I. Posner,
| | - Aldis P. Weible
- Institute of Neuroscience, University of Oregon, Eugene, OR, United States
| | - Pascale Voelker
- Department of Psychology, University of Oregon, Eugene, OR, United States
| | - Mary K. Rothbart
- Department of Psychology, University of Oregon, Eugene, OR, United States
| | - Cristopher M. Niell
- Institute of Neuroscience, University of Oregon, Eugene, OR, United States
- Department of Biology, University of Oregon, Eugene, OR, United States
| |
Collapse
|
40
|
Brinkman BAW, Yan H, Maffei A, Park IM, Fontanini A, Wang J, La Camera G. Metastable dynamics of neural circuits and networks. APPLIED PHYSICS REVIEWS 2022; 9:011313. [PMID: 35284030 PMCID: PMC8900181 DOI: 10.1063/5.0062603] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Accepted: 01/31/2022] [Indexed: 05/14/2023]
Abstract
Cortical neurons emit seemingly erratic trains of action potentials or "spikes," and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns, which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this Review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed "metastable" and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal, and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals; and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation, or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.
Collapse
Affiliation(s)
| | - H. Yan
- State Key Laboratory of Electroanalytical Chemistry, Changchun Institute of Applied Chemistry, Chinese Academy of Sciences, Changchun, Jilin 130022, People's Republic of China
| | | | | | | | - J. Wang
- Authors to whom correspondence should be addressed: and
| | - G. La Camera
- Authors to whom correspondence should be addressed: and
| |
Collapse
|
41
|
Akam T, Lustig A, Rowland JM, Kapanaiah SKT, Esteve-Agraz J, Panniello M, Márquez C, Kohl MM, Kätzel D, Costa RM, Walton ME. Open-source, Python-based, hardware and software for controlling behavioural neuroscience experiments. eLife 2022; 11:e67846. [PMID: 35043782 PMCID: PMC8769647 DOI: 10.7554/elife.67846] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 01/03/2022] [Indexed: 01/05/2023] Open
Abstract
Laboratory behavioural tasks are an essential research tool. As questions asked of behaviour and brain activity become more sophisticated, the ability to specify and run richly structured tasks becomes more important. An increasing focus on reproducibility also necessitates accurate communication of task logic to other researchers. To these ends, we developed pyControl, a system of open-source hardware and software for controlling behavioural experiments comprising a simple yet flexible Python-based syntax for specifying tasks as extended state machines, hardware modules for building behavioural setups, and a graphical user interface designed for efficiently running high-throughput experiments on many setups in parallel, all with extensive online documentation. These tools make it quicker, easier, and cheaper to implement rich behavioural tasks at scale. As important, pyControl facilitates communication and reproducibility of behavioural experiments through a highly readable task definition syntax and self-documenting features. Here, we outline the system's design and rationale, present validation experiments characterising system performance, and demonstrate example applications in freely moving and head-fixed mouse behaviour.
Collapse
Affiliation(s)
- Thomas Akam
- Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Champalimaud Neuroscience Program, Champalimaud Centre for the UnknownLisbonPortugal
| | - Andy Lustig
- Janelia Research Campus, Howard Hughes Medical InstituteAshburnUnited States
| | - James M Rowland
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
| | | | - Joan Esteve-Agraz
- Instituto de Neurociencias (Universidad Miguel Hernández-Consejo Superior de Investigaciones Científicas)Sant Joan d’AlacantSpain
| | - Mariangela Panniello
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
| | - Cristina Márquez
- Instituto de Neurociencias (Universidad Miguel Hernández-Consejo Superior de Investigaciones Científicas)Sant Joan d’AlacantSpain
| | - Michael M Kohl
- Department of Physiology Anatomy & Genetics, University of OxfordOxfordUnited Kingdom
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm UniversityUlmGermany
| | - Rui M Costa
- Champalimaud Neuroscience Program, Champalimaud Centre for the UnknownLisbonPortugal
- Department of Neuroscience and Neurology, Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Mark E Walton
- Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of OxfordOxfordUnited Kingdom
| |
Collapse
|
42
|
Abstract
Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.
Collapse
|
43
|
Keogh C, Deli A, Zand APD, Zorman MJ, Boccard-Binet SG, Parrott M, Sigalas C, Weiss AR, Stein JF, FitzGerald JJ, Aziz TZ, Green AL, Gillies MJ. Spatial and Temporal Distribution of Information Processing in the Human Dorsal Anterior Cingulate Cortex. Front Hum Neurosci 2022; 16:780047. [PMID: 35370577 PMCID: PMC8973009 DOI: 10.3389/fnhum.2022.780047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 02/18/2022] [Indexed: 11/13/2022] Open
Abstract
The dorsal anterior cingulate cortex (dACC) is a key node in the human salience network. It has been ascribed motor, pain-processing and affective functions. However, the dynamics of information flow in this complex region and how it responds to inputs remain unclear and are difficult to study using non-invasive electrophysiology. The area is targeted by neurosurgery to treat neuropathic pain. During deep brain stimulation surgery, we recorded local field potentials from this region in humans during a decision-making task requiring motor output. We investigated the spatial and temporal distribution of information flow within the dACC. We demonstrate the existence of a distributed network within the anterior cingulate cortex where discrete nodes demonstrate directed communication following inputs. We show that this network anticipates and responds to the valence of feedback to actions. We further show that these network dynamics adapt following learning. Our results provide evidence for the integration of learning and the response to feedback in a key cognitive region.
Collapse
Affiliation(s)
- Conor Keogh
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Alceste Deli
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | | | - Mark Jernej Zorman
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | | | - Matthew Parrott
- St Hilda’s College, University of Oxford, Oxford, United Kingdom
| | | | - Alexander R. Weiss
- Department of Neurology, Johns Hopkins University, Baltimore, MD, United States
| | - John Frederick Stein
- Department of Physiology, Anatomy and Genetics, University of Oxford, Oxford, United Kingdom
| | - James J. FitzGerald
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Tipu Z. Aziz
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Alexander L. Green
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Martin John Gillies
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
- *Correspondence: Martin John Gillies,
| |
Collapse
|
44
|
Soltani A, Koechlin E. Computational models of adaptive behavior and prefrontal cortex. Neuropsychopharmacology 2022; 47:58-71. [PMID: 34389808 PMCID: PMC8617006 DOI: 10.1038/s41386-021-01123-1] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 07/19/2021] [Accepted: 07/20/2021] [Indexed: 02/07/2023]
Abstract
The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus-action associations through rewards; (2) predictive models learning stimulus- and/or action-outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.
Collapse
Affiliation(s)
- Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Etienne Koechlin
- Institut National de la Sante et de la Recherche Medicale, Universite Pierre et Marie Curie, Ecole Normale Superieure, Paris, France.
| |
Collapse
|
45
|
Kapanaiah SKT, van der Veen B, Strahnen D, Akam T, Kätzel D. A low-cost open-source 5-choice operant box system optimized for electrophysiology and optophysiology in mice. Sci Rep 2021; 11:22279. [PMID: 34782697 PMCID: PMC8593009 DOI: 10.1038/s41598-021-01717-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 11/01/2021] [Indexed: 11/16/2022] Open
Abstract
Operant boxes enable the application of complex behavioural paradigms to support circuit neuroscience and drug discovery research. However, commercial operant box systems are expensive and often not optimised for combining behaviour with neurophysiology. Here we introduce a fully open-source Python-based operant-box system in a 5-choice design (pyOS-5) that enables assessment of multiple cognitive and affective functions. It is optimized for fast turn-over between animals, and for testing of tethered mice for simultaneous physiological recordings or optogenetic manipulation. For reward delivery, we developed peristaltic and syringe pumps based on a stepper motor and 3D-printed parts. Tasks are specified using a Python-based syntax implemented on custom-designed printed circuit boards that are commercially available at low cost. We developed an open-source graphical user interface (GUI) and task definition scripts to conduct assays assessing operant learning, attention, impulsivity, working memory, or cognitive flexibility, alleviating the need for programming skills of the end user. All behavioural events are recorded with millisecond resolution, and TTL-outputs and -inputs allow straightforward integration with physiological recordings and closed-loop manipulations. This combination of features realizes a cost-effective, nose-poke-based operant box system that allows reliable circuit-neuroscience experiments investigating correlates of cognition and emotion in large cohorts of subjects.
Collapse
Affiliation(s)
| | | | - Daniel Strahnen
- Institute of Applied Physiology, Ulm University, Ulm, Germany
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Dennis Kätzel
- Institute of Applied Physiology, Ulm University, Ulm, Germany.
| |
Collapse
|
46
|
Dissociable mechanisms of information sampling in prefrontal cortex and the dopaminergic system. Curr Opin Behav Sci 2021. [DOI: 10.1016/j.cobeha.2021.04.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
47
|
Ávila-Villanueva M, Gómez-Ramírez J, Ávila J, Fernández-Blázquez MA. Alzheimer's Disease and Empathic Abilities: The Proposed Role of the Cingulate Cortex. J Alzheimers Dis Rep 2021; 5:345-352. [PMID: 34189406 PMCID: PMC8203285 DOI: 10.3233/adr-200282] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
In recent years there has been increasing interest in examining the role of empathic abilities in Alzheimer’s disease (AD). Empathy, the ability to understand and share another person’s feelings, implies the existence of emotional and cognitive processes and is a pivotal aspect for success in social interactions. In turn, self-empathy is oriented to one’s thoughts and feelings. Decline of empathy and self-empathy can occur during the AD continuum and can be linked to different neuroanatomical pathways in which the cingulate cortex may play a crucial role. Here, we will summarize the involvement of empathic abilities through the AD continuum and further discuss the potential neurocognitive mechanisms that contribute to decline of empathy and self-empathy in AD.
Collapse
Affiliation(s)
- Marina Ávila-Villanueva
- Alzheimer Disease Research Unit, CIEN Foundation, Carlos III Institute of Health, Queen Sofía Foundation Alzheimer Center, Madrid, Spain
| | - Jaime Gómez-Ramírez
- Alzheimer Disease Research Unit, CIEN Foundation, Carlos III Institute of Health, Queen Sofía Foundation Alzheimer Center, Madrid, Spain
| | - Jesús Ávila
- Center of Molecular Biology Severo Ochoa (CSIC-UAM), Campus de Cantoblanco, Madrid, Spain.,Network Center for Biomedical Research in Neurodegenerative Diseases (CIBERNED), Madrid, Spain
| | - Miguel A Fernández-Blázquez
- Alzheimer Disease Research Unit, CIEN Foundation, Carlos III Institute of Health, Queen Sofía Foundation Alzheimer Center, Madrid, Spain.,Department of Experimental Psychology, Cognitive Processes and Speech Therapy, Complutense University of Madrid (UCM), Campus de Somosaguas, Pozuelo de Alarcón, Madrid, Spain
| |
Collapse
|
48
|
Tervo DGR, Kuleshova E, Manakov M, Proskurin M, Karlsson M, Lustig A, Behnam R, Karpova AY. The anterior cingulate cortex directs exploration of alternative strategies. Neuron 2021; 109:1876-1887.e6. [PMID: 33852896 DOI: 10.1016/j.neuron.2021.03.028] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 12/31/2020] [Accepted: 03/22/2021] [Indexed: 12/26/2022]
Abstract
The ability to adjust one's behavioral strategy in complex environments is at the core of cognition. Doing so efficiently requires monitoring the reliability of the ongoing strategy and, when appropriate, switching away from it to evaluate alternatives. Studies in humans and non-human primates have uncovered signals in the anterior cingulate cortex (ACC) that reflect the pressure to switch away from the ongoing strategy, whereas other ACC signals relate to the pursuit of alternatives. However, whether these signals underlie computations that actually underpin strategy switching or merely reflect tracking of related variables remains unclear. Here we provide causal evidence that the rodent ACC actively arbitrates between persisting with the ongoing behavioral strategy and temporarily switching away to re-evaluate alternatives. Furthermore, by individually perturbing distinct output pathways, we establish that the two associated computations-determining whether to switch strategy and committing to the pursuit of a specific alternative-are segregated in the ACC microcircuitry.
Collapse
Affiliation(s)
| | - Elena Kuleshova
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA; Institute of Higher Nervous Activity and Neurophysiology of the Russian Academy of Sciences, Moscow, Russia
| | - Maxim Manakov
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA; Department of Neuroscience, Johns Hopkins University Medical School, Baltimore, MD, USA
| | - Mikhail Proskurin
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA; Department of Neuroscience, Johns Hopkins University Medical School, Baltimore, MD, USA
| | - Mattias Karlsson
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA; SpikeGadgets, San Francisco, CA, USA
| | - Andy Lustig
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | - Reza Behnam
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | - Alla Y Karpova
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA.
| |
Collapse
|
49
|
Abstract
The Ouroboros Model has been proposed as a biologically-inspired comprehensive cognitive architecture for general intelligence, comprising natural and artificial manifestations. The approach addresses very diverse fundamental desiderata of research in natural cognition and also artificial intelligence, AI. Here, it is described how the postulated structures have met with supportive evidence over recent years. The associated hypothesized processes could remedy pressing problems plaguing many, and even the most powerful current implementations of AI, including in particular deep neural networks. Some selected recent findings from very different fields are summoned, which illustrate the status and substantiate the proposal.
Collapse
|
50
|
Distinct roles for dopamine clearance mechanisms in regulating behavioral flexibility. Mol Psychiatry 2021; 26:7188-7199. [PMID: 34193974 PMCID: PMC8872990 DOI: 10.1038/s41380-021-01194-y] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/08/2021] [Revised: 05/21/2021] [Accepted: 06/08/2021] [Indexed: 02/06/2023]
Abstract
Dopamine plays a crucial role in adaptive behavior, and dysfunctional dopamine is implicated in multiple psychiatric conditions characterized by inflexible or inconsistent choices. However, the precise relationship between dopamine and flexible decision making remains unclear. One reason is that, while many studies have focused on the activity of dopamine neurons, efficient dopamine signaling also relies on clearance mechanisms, notably the dopamine transporter (DAT), which predominates in striatum, and catechol-O-methyltransferase (COMT), which predominates in cortex. The exact locus, extent, and timescale of the effects of DAT and COMT are uncertain. Moreover, there is limited data on how acute disruption of either mechanism affects flexible decision making strategies mediated by cortico-striatal networks. To address these issues, we combined pharmacological modulation of DAT and COMT with electrochemistry and behavior in mice. DAT blockade, but not COMT inhibition, regulated sub-second dopamine release in the nucleus accumbens core, but surprisingly neither clearance mechanism affected evoked release in prelimbic cortex. This was not due to a lack of sensitivity, as both amphetamine and atomoxetine changed the kinetics of sub-second release. In a multi-step decision making task where mice had to respond to reversals in either reward probabilities or the choice sequence to reach the goal, DAT blockade selectively impaired, and COMT inhibition improved, performance after reward reversals, but neither manipulation affected the adaptation of choices after action-state transition reversals. Together, our data suggest that DAT and COMT shape specific aspects of behavioral flexibility by regulating different aspects of the kinetics of striatal and cortical dopamine, respectively.
Collapse
|