Spatial encoding in dorsomedial prefrontal cortex and hippocampus is related during deliberation

Bridging flexible goal-directed cognition and consciousness: The Goal-Aligning Representation Internal Manipulation theory

Goal-directed manipulation of internal representations is a key element of human flexible behaviour, while consciousness is commonly associated with higher-order cognition and human flexibility. Current perspectives have only partially linked these processes, thus preventing a clear understanding of how they jointly generate flexible cognition and behaviour. Moreover, these limitations prevent an effective exploitation of this knowledge for technological scopes. We propose a new theoretical perspective that extends our 'three-component theory of flexible cognition' toward higher-order cognition and consciousness, based on the systematic integration of key concepts from Cognitive Neuroscience and AI/Robotics. The theory proposes that the function of conscious processes is to support the alignment of representations with multi-level goals. This higher alignment leads to more flexible and effective behaviours. We analyse here our previous model of goal-directed flexible cognition (validated with more than 20 human populations) as a starting GARIM-inspired model. By bridging the main theories of consciousness and goal-directed behaviour, the theory has relevant implications for scientific and technological fields. In particular, it contributes to developing new experimental tasks and interpreting clinical evidence. Finally, it indicates directions for improving machine learning and robotics systems and for informing real-world applications (e.g., in digital-twin healthcare and roboethics).

Resting-state functional connectivity of goal-directed and habitual-learning systems: The efficacy of cognitive-behavioral therapy for obsessive-compulsive disorder

BACKGROUND

There is an imbalance between goal-directed and habitual-learning system in patients with obsessive-compulsive disorder (OCD). At present, the relationship between cognitive behavior therapy (CBT) as a first-line therapy and goal-directed and habitual-learning disorder is still unclear. We attempted to discuss the effect of CBT treatment in patients with OCD, using abnormalities in goal-directed and habitual-learning-related brain regions at baseline as predictive factors.

METHODS

A total of 71 subjects, including 35 OCD patients and 36 healthy controls, were recruited. The OCD patients underwent 8 weeks of CBT. These patients were divided into two groups based on treatment response (Nresponders = 18, Nnonresponders = 17). Further subgroup analysis was conducted based on disease duration (Nshort = 17, Nlong = 18) and age of onset (Nearly = 14, Nlate = 21). We collected resting-state ROI-ROI functional connectivity data and apply repeated-measures linear mixed-effects models to investigate the differences of different subgroups.

RESULTS

CBT led to symptom improvement in OCD patients, with varying degrees of effectiveness across subgroups. The orbitofrontal cortex (OFC) and insula, key regions for goal-directed behavior and habitual-learning, respectively, showed significant impacts on CBT efficacy in subgroups with different disease durations and ages of onset.

CONCLUSION

The findings suggest that the goal-directed system may influence the efficacy of CBT through goal selection, maintenance, and emotion regulation. Furthermore, we found that disease duration and age of onset may affect treatment outcomes by modulating functional connectivity between goal-directed and habitual-learning brain regions.

Memory consolidation affects the interplay of place and response navigation

Navigation through space is based on memory representations of landmarks ('place') or movement sequences ('response'). Over time, memory representations transform through consolidation. However, it is unclear how the transformation affects place and response navigation in humans. In the present study, healthy adults navigated to target locations in a virtual maze. The preference for using place and response strategies and the ability to recall place and response memories were tested after a delay of one hour (n = 31), one day (n = 30), or two weeks (n = 32). The different delays captured early-phase synaptic changes, changes after one night of sleep, and long-delay changes due to the reorganization of navigation networks. Our results show that the relative contributions of place and response navigation changed as a function of time. After a short delay of up to one day, participants preferentially used a place strategy and exhibited a high degree of visual landmark exploration. After a longer delay of two weeks, place strategy use decreased significantly. Participants now equally relied on place and response strategy use and increasingly repeated previously taken paths. Further analyses indicate that response strategy use predominantly occurred as a compensatory strategy in the absence of sufficient place memory. Over time, place memory faded before response memory. We suggest that the observed shift from place to response navigation is context-dependent since detailed landmark information, which strongly relied on hippocampal function, decayed faster than sequence information, which required less detail and depended on extra-hippocampal areas. We conclude that changes in place and response navigation likely reflect the reorganization of navigation networks during systems consolidation.

Dynamic prediction of goal location by coordinated representation of prefrontal-hippocampal theta sequences

Prefrontal (PFC) and hippocampal (HPC) sequences of neuronal firing modulated by theta rhythms could represent upcoming choices during spatial memory-guided decision-making. How the PFC-HPC network dynamically coordinates theta sequences to predict specific goal locations and how it is interrupted in memory impairments induced by amyloid beta (Aβ) remain unclear. Here, we detected theta sequences of firing activities of PFC neurons and HPC place cells during goal-directed spatial memory tasks. We found that PFC ensembles exhibited predictive representation of the specific goal location since the starting phase of memory retrieval, earlier than the hippocampus. High predictive accuracy of PFC theta sequences existed during successful memory retrieval and positively correlated with memory performance. Coordinated PFC-HPC sequences showed PFC-dominant prediction of goal locations during successful memory retrieval. Furthermore, we found that theta sequences of both regions still existed under Aβ accumulation, whereas their predictive representation of goal locations was weakened with disrupted spatial representation of HPC place cells and PFC neurons. These findings highlight the essential role of coordinated PFC-HPC sequences in successful memory retrieval of a precise goal location.

Neural signatures of opioid-induced risk-taking behavior in the prelimbic prefrontal cortex

Opioid use disorder occurs alongside impaired risk-related decision-making, but the underlying neural correlates are unclear. We developed a novel approach-avoidance conflict model using a modified conditioned place preference paradigm to study neural signals of risky opioid seeking in the prefrontal cortex, a region implicated in executive decision making. Upon establishment of morphine conditioned place preference, rats underwent a subsequent conflict test in which fear-inducing cat odor was introduced in the previously drug-paired side of the apparatus. While the saline control group avoided the cat odor side, the morphine group maintained preference for the paired side despite the presence of cat odor. K-means clustering identified two subsets of morphine-treated rats that exhibited either persistent drug seeking (Risk-Takers) or increased avoidance (Risk-Avoiders) during conflict. Single-unit recordings from the prelimbic cortex (PL) revealed decreased neuronal firing rates upon acute morphine exposure in both Risk-Takers and Risk-Avoiders, but this firing rate suppression was absent after repeated administration. Risk-Avoiders also displayed distinct post-morphine excitation in PL which persisted across conditioning. During the preference test, subpopulations of PL neurons in all groups were either excited or inhibited when rats entered the paired side. Interestingly, while this inhibitory signal was lost during the subsequent conflict test in both saline and Risk-Avoider groups, these inhibitory responses persisted in Risk-Takers. Our results suggest that loss of PL inhibition after opioid conditioning is associated with the formation of contextual reward memory. Furthermore, persistent PL inhibitory signaling in the drug-associated context during conflict may underlie increased risk taking following opioid exposure.

New information triggers prospective codes to adapt for flexible navigation

Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a novel task that precisely introduces new information using virtual reality, and we recorded neural activity as mice flexibly adapted their planned destinations. We found that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not flexibly adapt, prefrontal choice codes failed to switch, despite relatively intact hippocampal goal representations. Prospective code updating depended on the commitment to the initial choice and degree of adaptation needed. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.

Pausing and reorienting behaviors enhance the performance of a spatial working memory task

During spatial working memory tasks, animals need to retain information about a previous trial in order to successfully select their next trajectory. Specifically, the delayed non-match to position task requires rats to follow a cued sample trajectory, then select the opposite route after a delay period. When faced with this choice, rats will occasionally exhibit complex behaviors, such as pausing and sweeping their head back and forth. These behaviors, called vicarious trial and error (VTE), are thought to be a behavioral manifestation of deliberation. However, we identified similarly complex behaviors during sample-phase traversals, despite the fact that these laps do not require a decision. First, we identified that these behaviors occurred more often after incorrect trials than before them, indicating that rats are retaining information between trials. Next, we determined that these pause-and-reorient (PAR) behaviors increased the likelihood of the next choice being selected correctly, suggesting that these behaviors assist the rat in successful task performance. Finally, we identified similarities between PARs and choice-phase VTEs, suggesting that VTEs may not only be reflective of deliberation, but may also contribute to a strategy for successful performance of spatial working memory tasks.

Geometric transformation of cognitive maps for generalization across hippocampal-prefrontal circuits

The ability to abstract information to guide decisions during navigation across changing environments is essential for adaptation and requires the integrity of the hippocampal-prefrontal circuitry. The hippocampus encodes navigational information in a cognitive map, but it remains unclear how cognitive maps are transformed across hippocampal-prefrontal circuits to support abstraction and generalization. Here, we simultaneously record hippocampal-prefrontal ensembles as rats generalize navigational rules across distinct environments. We find that, whereas hippocampal representational maps maintain specificity of separate environments, prefrontal maps generalize across environments. Furthermore, while both maps are structured within a neural manifold of population activity, they have distinct representational geometries. Prefrontal geometry enables abstraction of rule-informative variables, a representational format that generalizes to novel conditions of existing variable classes. Hippocampal geometry lacks such abstraction. Together, these findings elucidate how cognitive maps are structured into distinct geometric representations to support abstraction and generalization while maintaining memory specificity.

Multiple-Timescale Representations of Space: Linking Memory to Navigation

When navigating through space, we must maintain a representation of our position in real time; when recalling a past episode, a memory can come back in a flash. Interestingly, the brain's spatial representation system, including the hippocampus, supports these two distinct timescale functions. How are neural representations of space used in the service of both real-world navigation and internal mnemonic processes? Recent progress has identified sequences of hippocampal place cells, evolving at multiple timescales in accordance with either navigational behaviors or internal oscillations, that underlie these functions. We review experimental findings on experience-dependent modulation of these sequential representations and consider how they link real-world navigation to time-compressed memories. We further discuss recent work suggesting the prevalence of these sequences beyond hippocampus and propose that these multiple-timescale mechanisms may represent a general algorithm for organizing cell assemblies, potentially unifying the dual roles of the spatial representation system in memory and navigation.

The ventral midline thalamus coordinates prefrontal-hippocampal neural synchrony during vicarious trial and error

When faced with difficult choices, the possible outcomes are considered through a process known as deliberation. In rats, deliberation is thought to be reflected by pause-and-reorienting behaviors, better known as vicarious trial and errors (VTEs). While VTEs are thought to require medial prefrontal cortex (mPFC) and dorsal hippocampal (dHPC) interactions, no empirical evidence has yet demonstrated such a dual requirement. The nucleus reuniens (Re) of the ventral midline thalamus is anatomically connected with both the mPFC and dHPC, is required for HPC-dependent spatial memory tasks, and is critical for mPFC-dHPC neural synchronization. Currently, it is unclear if, or how, the Re is involved in deliberation. Therefore, by examining the role of the Re on VTE behaviors, we can better understand the anatomical and physiological mechanisms supporting deliberation. Here, we examined the impact of Re suppression on VTE behaviors and mPFC-dHPC theta synchrony during asymptotic performance of a HPC-dependent delayed alternation (DA) task. Pharmacological suppression of the Re increased VTE behaviors that occurred with repetitive choice errors. These errors were best characterized as perseverative behaviors, in which some rats repeatedly selected a goal arm that previously yielded no reward. We then examined the impact of Re suppression on mPFC-dHPC theta synchrony during VTEs. We found that during VTEs, Re inactivation was associated with a reduction in mPFC-dHPC theta coherence and mPFC-to-dHPC theta directionality. Our findings suggest that the Re contributes to deliberation by coordinating mPFC-dHPC neural interactions.

Prefrontal pyramidal neurons are critical for all phases of working memory

The prefrontal cortex (PFC) is essential for working memory (WM) and has primarily been viewed as being responsible for maintaining information over a delay, but it is unclear whether it also plays a more general role during WM. Using task phase-specific optogenetic silencing of pyramidal neurons in the medial PFC (mPFC) of mice performing a spatial WM task, we find that the mPFC is required not only during the delay phase of the task but also during other phases requiring the encoding and retrieval of spatial information. Imaging of mPFC pyramidal neurons reveals that they are most strongly influenced by the animals' position and running direction, indicating a fundamental role in spatial navigation. Pyramidal neuron ensembles also represent to-be-remembered goal locations in a dynamic manner. Taken together, these results delineate the functional contribution of mPFC pyramidal neurons to WM, extending their role beyond the maintenance of information.

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

Imbalance in Functional and Structural Connectivity Underlying Goal-Directed and Habitual Learning Systems in Obsessive-Compulsive Disorder

An imbalance between the goal-directed and habitual learning systems has been proposed to underlie compulsivity in obsessive-compulsive disorder (OCD). In addition, the overall balance between these systems may be influenced by stress hormones. We examined the multimodal networks underlying these dual learning systems. Both functional and structural measures indicated reduced connectivity within the goal-directed subnetwork (FC: P = 0.042; SC-FN: P = 0.014) and reduced connectivity between the goal-directed and habitual subnetworks (FC: P = 0.014; SC-FA: P = 0.052), but no differences within the habitual subnetwork in patients with OCD compared with controls. Path modeling indicated that anatomical connectivity in the goal-directed subnetwork influenced compulsive symptoms (R2 = 0.41), whereas functional connectivity within the habit subnetwork and between goal-directed and habitual subnetworks influenced obsessive symptoms (R2 = 0.63). In addition, the relationship between anatomical connectivity in the goal-directed subnetwork and compulsion was moderated by the stress hormone ACTH (adrenocorticotropic hormone), such that at low levels of ACTH greater connectivity resulted in lower compulsion, but at high levels of ACTH this relationship was reversed. These results provide new insights into the neural correlates of the imbalance between dual learning systems, and their relationship with symptom dimensions in patients with OCD. It may further support the reconceptualization of OCD as "compulsive-obsessive disorder," with a greater focus on the transdiagnostic dimension of compulsivity.

Prefrontal cortical activity predicts the occurrence of nonlocal hippocampal representations during spatial navigation

The receptive field of a neuron describes the regions of a stimulus space where the neuron is consistently active. Sparse spiking outside of the receptive field is often considered to be noise, rather than a reflection of information processing. Whether this characterization is accurate remains unclear. We therefore contrasted the sparse, temporally isolated spiking of hippocampal CA1 place cells to the consistent, temporally adjacent spiking seen within their spatial receptive fields ("place fields"). We found that isolated spikes, which occur during locomotion, are strongly phase coupled to hippocampal theta oscillations and transiently express coherent nonlocal spatial representations. Further, prefrontal cortical activity is coordinated with and can predict the occurrence of future isolated spiking events. Rather than local noise within the hippocampus, sparse, isolated place cell spiking reflects a coordinated cortical-hippocampal process consistent with the generation of nonlocal scenario representations during active navigation.

Spatial Representations in Rat Orbitofrontal Cortex

The orbitofrontal cortex (OFC) and hippocampus share striking cognitive and functional similarities. As a result, both structures have been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. However, while this function has been exemplified by spatial coding in neurons of hippocampal regions-particularly place and grid cells-spatial representations in the OFC have been investigated far less. Here we sought to address this by recording OFC neurons from male rats engaged in an open-field foraging task like that originally developed to characterize place fields in rodent hippocampal neurons. Single-unit activity was recorded as rats searched for food pellets scattered randomly throughout a large enclosure. In some sessions, particular flavors of food occurred more frequently in particular parts of the enclosure; in others, only a single flavor was used. OFC neurons showed spatially localized firing fields in both conditions, and representations changed between flavored and unflavored foraging periods in a manner reminiscent of remapping in the hippocampus. Compared with hippocampal recordings taken under similar behavioral conditions, OFC spatial representations were less temporally reliable, and there was no significant evidence of grid tuning in OFC neurons. These data confirm that OFC neurons show spatial firing fields in a large, two-dimensional environment in a manner similar to hippocampus. Consistent with the focus of the OFC on biological meaning and goals, spatial coding was weaker than in hippocampus and influenced by outcome identity.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) and hippocampus have both been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. This function is exemplified by place and grid cells identified in hippocampus, the activity of which maps spatial environments. The current study directly demonstrates very similar, though not identical, spatial representatives in OFC neurons, confirming that OFC-like hippocampus-can represent a spatial map under the appropriate experimental conditions.

If I do A, B will happen: Dissecting circuits detecting causal relations between actions and outcomes in marmoset prefrontal cortex

In this issue of Neuron, Duan et al. (2021) use pharmacological manipulation to reveal opposing influences of anterior cingulate and orbitofrontal cortex of marmosets on decisions that are based on action-outcome associations.

Navigating for reward

An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.

A Machine Learning Approach for Detecting Vicarious Trial and Error Behaviors

Vicarious trial and error behaviors (VTEs) indicate periods of indecision during decision-making, and have been proposed as a behavioral marker of deliberation. In order to understand the neural underpinnings of these putative bridges between behavior and neural dynamics, researchers need the ability to readily distinguish VTEs from non-VTEs. Here we utilize a small set of trajectory-based features and standard machine learning classifiers to identify VTEs from non-VTEs for rats performing a spatial delayed alternation task (SDA) on an elevated plus maze. We also show that previously reported features of the hippocampal field potential oscillation can be used in the same types of classifiers to separate VTEs from non-VTEs with above chance performance. However, we caution that the modest classifier success using hippocampal population dynamics does not identify many trials where VTEs occur, and show that combining oscillation-based features with trajectory-based features does not improve classifier performance compared to trajectory-based features alone. Overall, we propose a standard set of features useful for trajectory-based VTE classification in binary decision tasks, and support previous suggestions that VTEs are supported by a network including, but likely extending beyond, the hippocampus.

The Radically Embodied Conscious Cybernetic Bayesian Brain: From Free Energy to Free Will and Back Again

Drawing from both enactivist and cognitivist perspectives on mind, I propose that explaining teleological phenomena may require reappraising both "Cartesian theaters" and mental homunculi in terms of embodied self-models (ESMs), understood as body maps with agentic properties, functioning as predictive-memory systems and cybernetic controllers. Quasi-homuncular ESMs are suggested to constitute a major organizing principle for neural architectures due to their initial and ongoing significance for solutions to inference problems in cognitive (and affective) development. Embodied experiences provide foundational lessons in learning curriculums in which agents explore increasingly challenging problem spaces, so answering an unresolved question in Bayesian cognitive science: what are biologically plausible mechanisms for equipping learners with sufficiently powerful inductive biases to adequately constrain inference spaces? Drawing on models from neurophysiology, psychology, and developmental robotics, I describe how embodiment provides fundamental sources of empirical priors (as reliably learnable posterior expectations). If ESMs play this kind of foundational role in cognitive development, then bidirectional linkages will be found between all sensory modalities and frontal-parietal control hierarchies, so infusing all senses with somatic-motoric properties, thereby structuring all perception by relevant affordances, so solving frame problems for embodied agents. Drawing upon the Free Energy Principle and Active Inference framework, I describe a particular mechanism for intentional action selection via consciously imagined (and explicitly represented) goal realization, where contrasts between desired and present states influence ongoing policy selection via predictive coding mechanisms and backward-chained imaginings (as self-realizing predictions). This embodied developmental legacy suggests a mechanism by which imaginings can be intentionally shaped by (internalized) partially-expressed motor acts, so providing means of agentic control for attention, working memory, imagination, and behavior. I further describe the nature(s) of mental causation and self-control, and also provide an account of readiness potentials in Libet paradigms wherein conscious intentions shape causal streams leading to enaction. Finally, I provide neurophenomenological handlings of prototypical qualia including pleasure, pain, and desire in terms of self-annihilating free energy gradients via quasi-synesthetic interoceptive active inference. In brief, this manuscript is intended to illustrate how radically embodied minds may create foundations for intelligence (as capacity for learning and inference), consciousness (as somatically-grounded self-world modeling), and will (as deployment of predictive models for enacting valued goals).

Multiple time-scales of decision-making in the hippocampus and prefrontal cortex

The prefrontal cortex and hippocampus are crucial for memory-guided decision-making. Neural activity in the hippocampus exhibits place-cell sequences at multiple timescales, including slow behavioral sequences (~seconds) and fast theta sequences (~100-200 ms) within theta oscillation cycles. How prefrontal ensembles interact with hippocampal sequences to support decision-making is unclear. Here, we examined simultaneous hippocampal and prefrontal ensemble activity in rats during learning of a spatial working-memory decision task. We found clear theta sequences in prefrontal cortex, nested within its behavioral sequences. In both regions, behavioral sequences maintained representations of current choices during navigation. In contrast, hippocampal theta sequences encoded alternatives for deliberation and were coordinated with prefrontal theta sequences that predicted upcoming choices. During error trials, these representations were preserved to guide ongoing behavior, whereas replay sequences during inter-trial periods were impaired prior to navigation. These results establish cooperative interaction between hippocampal and prefrontal sequences at multiple timescales for memory-guided decision-making.

Potential roles of the rodent medial prefrontal cortex in conflict resolution between multiple decision-making systems

Mammalian decision-making is mediated by the interaction of multiple, neurally and computationally separable decision systems. Having multiple systems requires a mechanism to manage conflict and converge onto the selection of singular actions. A long history of evidence has pointed to the prefrontal cortex as a central component in processing the interactions between distinct decision systems and resolving conflicts among them. In this chapter we review four theories of how that interaction might occur and identify how the medial prefrontal cortex in the rodent may be involved in each theory. We then present experimental predictions implied by the neurobiological data in the context of each theory as a starting point for future investigation of medial prefrontal cortex and decision-making.