Closed-Loop Theta Stimulation in the Orbitofrontal Cortex Prevents Reward-Based Learning

Reward recalibrates rule representations in human amygdala and hippocampus intracranial recordings

Adaptive behavior requires the ability to shift responding within (intra-dimensional) or between (extra-dimensional) stimulus dimensions when reward contingencies change. Studies of shifting in humans have focused mainly on the prefrontal cortex and/ or have been restricted to indirect measures of neural activity such as fMRI and lesions. Here, we demonstrate the importance of the amygdala and hippocampus by recording local field potentials directly from these regions intracranially in human epilepsy patients. Reward signals were coded in the high frequency gamma activity (HFG; 60-250 Hz) of both regions and synchronised via low frequency (3-5 Hz) phase-locking only after a shift when patients did not already know the rule and it signalled to stop shifting ("Win-Stay"). In contrast, HFG punishment signals were only seen in the amygdala when the rule then changed and it signalled to start shifting ("Lose-Shift"). During decision-making, hippocampal HFG was more inhibited on non-shift relative to shift trials, suggesting a role in preventing interference in rule representation and amygdala HFG was sensitive to stimulus novelty. The findings expand our understanding of human amygdala-hippocampal function and shifting processes, the disruption of which could contribute to shifting deficits in neuropsychiatric disorders.

Optimizing real-time phase detection in diverse rhythmic biological signals for phase-specific neuromodulation

Closed-loop, phase-specific neurostimulation is a powerful method to modulate ongoing brain activity for clinical and research applications. Phase-specific stimulation relies on estimating the phase of an ongoing oscillation in real time and issuing a control command at a target phase. Phase detection algorithms based on Fast Fourier transform (FFT) are widely used due to their computational efficiency and robustness. However, it is unclear how algorithm performance depends on the spectral properties of the input signal and how algorithm parameters can be optimized. We used offline simulation to evaluate the performance of three algorithms (endpoint-corrected Hilbert Transform, Hilbert Transform and phase mapping) on three rhythmic biological signals with distinct spectral properties (rodent hippocampal theta potential, human EEG alpha and human essential tremor). First, we found that algorithm performance was more strongly influenced by signal amplitude and frequency variation compared with signal to noise ratio. Second, our simulations showed that the size of the data window for phase estimation was critical for the performance of FFT-based algorithms, where the optimal data window corresponds to the period of the oscillation. We validated this prediction with real time phase detection of hippocampal theta oscillations in freely behaving rats performing spatial navigation. Our findings define the relationship between signal properties and algorithm performance and provide a convenient method for optimizing FFT-based phase detection algorithms.

Neural mechanisms of credit assignment for delayed outcomes during contingent learning

Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (FPl). Our results indicate that FPl holds previous causal choices in a "pending" state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.

Closed-loop microstimulations of the orbitofrontal cortex during real-life gaze interaction enhance dynamic social attention

Neurons from multiple prefrontal areas encode several key variables of social gaze interaction. To explore the causal roles of the primate prefrontal cortex in real-life gaze interaction, we applied weak closed-loop microstimulations that were precisely triggered by specific social gaze events. Microstimulations of the orbitofrontal cortex, but not the dorsomedial prefrontal cortex or the anterior cingulate cortex, enhanced momentary dynamic social attention in the spatial dimension by decreasing the distance of fixations relative to a partner's eyes and in the temporal dimension by reducing the inter-looking interval and the latency to reciprocate the other's directed gaze. By contrast, on a longer timescale, microstimulations of the dorsomedial prefrontal cortex modulated inter-individual gaze dynamics relative to one's own gaze positions. These findings demonstrate that multiple regions in the primate prefrontal cortex may serve as functionally accessible nodes in controlling different aspects of dynamic social attention and suggest their potential for a therapeutic brain interface.

Beta and theta oscillations track effort and previous reward in the human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we report an exploratory investigation into this with chronic intracranial recordings from the prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12 to 20 Hz) oscillations tracking effort on a single-trial basis and PFC theta (4 to 7 Hz) signaling previous trial reward, with no effects of net subjective value. Stimulation of PFC increased overall acceptance of offers and sensitivity to reward while decreasing the impact of effort on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, supports a causal role of PFC for such choices, and seeds hypotheses for future studies.

State-dependent dynamics of cuttlefish mantle activity

Cuttlefish skin is a powerful rendering device, capable of producing extraordinary changes in visual appearance over a broad range of temporal scales. This unique ability is typically associated with camouflage; however, cuttlefish often produce skin patterns that do not appear connected with the surrounding environment, such as fast large-scale fluctuations with wave-like characteristics. Little is known about the functional significance of these dynamic patterns. In this study, we developed novel tools for analyzing pattern dynamics, and demonstrate their utility for detecting changes in feeding state that occur without concomitant changes in sensory stimulation. Under these conditions, we found that the dynamic properties of specific pattern components differ for different feeding states, despite no measurable change in the overall expression of those components. Therefore, these dynamic changes are not detectable by conventional analyses focusing on pattern expression, requiring analytical tools specifically targeted to pattern dynamics.

Orbitofrontal high-gamma reflects spike-dissociable value and decision mechanisms

The orbitofrontal cortex (OFC) plays a crucial role in value-based decision-making. While previous research has focused on spiking activity in OFC neurons, the role of OFC local field potentials (LFPs) in decision-making remains unclear. LFPs are important because they can reflect synaptic and subthreshold activity not directly coupled to spiking, and because they are potential targets for less invasive forms of brain-machine interface (BMI). We recorded LFPs and spiking activity using multi-channel vertical probes while monkeys performed a two-option value-based decision-making task. We compared the value- and decision-coding properties of high-gamma range LFPs (HG, 50-150 Hz) to the coding properties of spiking multi-unit activity (MUA) recorded concurrently on the same electrodes. Results show that HG and MUA both represent the values of decision targets, and that their representations have similar temporal profiles in a trial. However, we also identified value-coding properties of HG that were dissociable from the concurrently-measured MUA. On average across channels, HG amplitude increased monotonically with value, whereas the average value encoding in MUA was net neutral. HG also encoded a signal consistent with a comparison between the values of the two targets, a signal which was much weaker in MUA. In individual channels, HG was better able to predict choice outcomes than MUA; however, when simultaneously recorded channels were combined in population-based decoder, MUA provided more accurate predictions than HG. Interestingly, HG value representations were accentuated in channels in or near shallow cortical layers, suggesting a dissociation between neuronal sources of HG and MUA. In summary, we find that HG signals are dissociable from MUA with respect to cognitive variables encoded in prefrontal cortex, evident in the monotonic encoding of value, stronger encoding of value comparisons, and more accurate predictions about behavior. High-frequency LFPs may therefore be a viable - or even preferable - target for BMIs to assist cognitive function, opening the possibility for less invasive access to mental contents that would otherwise be observable only with spike-based measures.

Dissociable contributions of the hippocampus and orbitofrontal cortex to representing task space in a social context

The hippocampus (HC) and the orbitofrontal cortex (OFC) jointly encode a map-like representation of a task space to guide behavior. It remains unclear how the OFC and HC interact in encoding this map-like representation, though previous studies indicated that both regions have different functions. We acquired the functional magnetic resonance imaging data under a social navigation task in which participants interacted with characters in a two-dimensional "social space." We calculate the social relationships between the participants and characters and used a drift-diffusion model to capture the inner process of social interaction. Then we used multivoxel pattern analysis to explore the brain-behavior relationship. We found that (i) both the HC and the OFC showed higher activations during the selective trial than the narrative trial; (ii) the neural pattern of the right HC was associated with evidence accumulation during social interaction, and the pattern of the right lateral OFC was associated with the social relationship; (iii) the neural pattern of the HC can decode the participants choices, while the neural pattern of the OFC can decode the task information about trials. The study provided evidence for distinct roles of the HC and the OFC in encoding different information when representing social space.

Prefrontal network engagement by deep brain stimulation in limbic hubs

Prefrontal circuits in the human brain play an important role in cognitive and affective processing. Neuromodulation therapies delivered to certain key hubs within these circuits are being used with increasing frequency to treat a host of neuropsychiatric disorders. However, the detailed neurophysiological effects of stimulation to these hubs are largely unknown. Here, we performed intracranial recordings across prefrontal networks while delivering electrical stimulation to two well-established white matter hubs involved in cognitive regulation and depression: the subcallosal cingulate (SCC) and ventral capsule/ventral striatum (VC/VS). We demonstrate a shared frontotemporal circuit consisting of the ventromedial prefrontal cortex, amygdala, and lateral orbitofrontal cortex where gamma oscillations are differentially modulated by stimulation target. Additionally, we found participant-specific responses to stimulation in the dorsal anterior cingulate cortex and demonstrate the capacity for further tuning of neural activity using current-steered stimulation. Our findings indicate a potential neurophysiological mechanism for the dissociable therapeutic effects seen across the SCC and VC/VS targets for psychiatric neuromodulation and our results lay the groundwork for personalized, network-guided neurostimulation therapy.

Closed-loop microstimulations of the orbitofrontal cortex during real-life gaze interaction enhance dynamic social attention

The prefrontal cortex is extensively involved in social exchange. During dyadic gaze interaction, multiple prefrontal areas exhibit neuronal encoding of social gaze events and context-specific mutual eye contact, supported by a widespread neural mechanism of social gaze monitoring. To explore causal manipulation of real-life gaze interaction, we applied weak closed-loop microstimulations that were precisely triggered by specific social gaze events to three prefrontal areas in monkeys. Microstimulations of orbitofrontal cortex (OFC), but not dorsomedial prefrontal or anterior cingulate cortex, enhanced momentary dynamic social attention in the spatial dimension by decreasing distance of one's gaze fixations relative to partner monkey's eyes. In the temporal dimension, microstimulations of OFC reduced the inter-looking interval for attending to another agent and the latency to reciprocate other's directed gaze. These findings demonstrate that primate OFC serves as a functionally accessible node in controlling dynamic social attention and suggest its potential for a therapeutic brain interface.

A cognitive map for value-guided choice in ventromedial prefrontal cortex

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Beta and theta oscillations track effort and previous reward in human basal ganglia and prefrontal cortex during decision making

Choosing whether to exert effort to obtain rewards is fundamental to human motivated behavior. However, the neural dynamics underlying the evaluation of reward and effort in humans is poorly understood. Here, we investigate this with chronic intracranial recordings from prefrontal cortex (PFC) and basal ganglia (BG; subthalamic nuclei and globus pallidus) in people with Parkinson's disease performing a decision-making task with offers that varied in levels of reward and physical effort required. This revealed dissociable neural signatures of reward and effort, with BG beta (12-20 Hz) oscillations tracking subjective effort on a single trial basis and PFC theta (4-7 Hz) signaling previous trial reward. Stimulation of PFC increased overall acceptance of offers in addition to increasing the impact of reward on choices. This work uncovers oscillatory mechanisms that guide fundamental decisions to exert effort for reward across BG and PFC, as well as supporting a causal role of PFC for such choices.

Behavioral read-out from population value signals in primate orbitofrontal cortex

The primate orbitofrontal cortex (OFC) has long been recognized for its role in value-based decisions; however, the exact mechanism linking value representations in the OFC to decision outcomes has remained elusive. Here, to address this question, we show, in non-human primates, that trial-wise variability in choices can be explained by variability in value signals decoded from many simultaneously recorded OFC neurons. Mechanistically, this relationship is consistent with the projection of activity within a low-dimensional value-encoding subspace onto a potentially higher-dimensional, behaviorally potent output subspace. Identifying this neural-behavioral link answers longstanding questions about the role of the OFC in economic decision-making and suggests population-level read-out mechanisms for the OFC similar to those recently identified in sensory and motor cortex.

Common Mechanisms of Learning in Motor and Cognitive Systems

Rapid progress in our understanding of the brain's learning mechanisms has been accomplished over the past decade, particularly with conceptual advances, including representing behavior as a dynamical system, large-scale neural population recordings, and new methods of analysis of neuronal populations. However, motor and cognitive systems have been traditionally studied with different methods and paradigms. Recently, some common principles, evident in both behavior and neural activity, that underlie these different types of learning have become to emerge. Here we review results from motor and cognitive learning, relying on different techniques and studying different systems to understand the mechanisms of learning. Movement is intertwined with cognitive operations, and its dynamics reflect cognitive variables. Training, in either motor or cognitive tasks, involves recruitment of previously unresponsive neurons and reorganization of neural activity in a low dimensional manifold. Mapping of new variables in neural activity can be very rapid, instantiating flexible learning of new tasks. Communication between areas is just as critical a part of learning as are patterns of activity within an area emerging with learning. Common principles across systems provide a map for future research.

Human thalamic low-frequency oscillations correlate with expected value and outcomes during reinforcement learning

Reinforcement-based adaptive decision-making is believed to recruit fronto-striatal circuits. A critical node of the fronto-striatal circuit is the thalamus. However, direct evidence of its involvement in human reinforcement learning is lacking. We address this gap by analyzing intra-thalamic electrophysiological recordings from eight participants while they performed a reinforcement learning task. We found that in both the anterior thalamus (ATN) and dorsomedial thalamus (DMTN), low frequency oscillations (LFO, 4-12 Hz) correlated positively with expected value estimated from computational modeling during reward-based learning (after outcome delivery) or punishment-based learning (during the choice process). Furthermore, LFO recorded from ATN/DMTN were also negatively correlated with outcomes so that both components of reward prediction errors were signaled in the human thalamus. The observed differences in the prediction signals between rewarding and punishing conditions shed light on the neural mechanisms underlying action inhibition in punishment avoidance learning. Our results provide insight into the role of thalamus in reinforcement-based decision-making in humans.

Aberrant reward dynamics in depression with anticipatory anhedonia

OBJECTIVE

Previous studies have shown that anticipatory anhedonia is linked to abnormal reward processing. The present study aimed to explore the underlying neural mechanism of the influence of anticipatory anhedonia symptoms on reward processing.

METHODS

Electrophysiological activities in the anticipatory and consummatory phase were recorded during the Monetary Incentive Delay (MID) task in 24 depressed high anticipatory anhedonia (HAA) patients, 25 depressed low anticipatory anhedonia (LAA) patients, and 29 healthy controls (HC).

RESULTS

We suggested a significant condition × group interaction effect on feedback-related negativity (FRN) amplitudes during the consummatory phase, a smaller FRN in reward cue trails compared with neutral cue trail was revealed in the HC and LAA group, but such reward-related effect was not found in the HAA group. In addition, we found significant correlations between FRN, fb-P3 and cue-N1, cue-N2 in the HC group, besides, significant correlations between FRN, fb-P3 and cue-P2 was also revealed in the HC and LAA group. However, no significant correlation was found in HAA patients.

CONCLUSIONS

Our results suggest that the link between the anticipatory and consummatory phase was interrupted in depressed HAA patients, which may be driven by the aberrant consummatory reward processing.

SIGNIFICANCE

The current study is the first one to demonstrate the influence of anticipatory anhedonia symptom on the association between anticipatory and consummatory phase of reward process.

Value dynamics affect choice preparation during decision-making

During decision-making, neurons in the orbitofrontal cortex (OFC) sequentially represent the value of each option in turn, but it is unclear how these dynamics are translated into a choice response. One brain region that may be implicated in this process is the anterior cingulate cortex (ACC), which strongly connects with OFC and contains many neurons that encode the choice response. We investigated how OFC value signals interacted with ACC neurons encoding the choice response by performing simultaneous high-channel count recordings from the two areas in nonhuman primates. ACC neurons encoding the choice response steadily increased their firing rate throughout the decision-making process, peaking shortly before the time of the choice response. Furthermore, the value dynamics in OFC affected ACC ramping-when OFC represented the more valuable option, ACC ramping accelerated. Because OFC tended to represent the more valuable option more frequently and for a longer duration, this interaction could explain how ACC selects the more valuable response.

Neurons in human pre-supplementary motor area encode key computations for value-based choice

Adaptive behaviour in real-world environments requires that choices integrate several variables, including the novelty of the options under consideration, their expected value and uncertainty in value estimation. Here, to probe how integration over decision variables occurs during decision-making, we recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex and dorsal anterior cingulate. Unlike the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option but also encoded an integrated utility signal for each choice option and, subsequently, the decision itself. Post-decision encoding of variables for the chosen option was more widely distributed and especially prominent in the ventromedial prefrontal cortex. Our findings position the human preSMA as central to the implementation of value-based decisions.

Orbitofrontal cortex-hippocampus potentiation mediates relief for depression: A randomized double-blind trial and TMS-EEG study

It has been 15 years since repetitive transcranial magnetic stimulation (rTMS) targeting the dorsolateral prefrontal cortex (DLPFC) was approved by the FDA for clinical depression treatment. Yet, the underlying mechanisms for rTMS-induced depression relief are not fully elucidated. This study analyzes TMS-electroencephalogram (EEG) data from 64 healthy control (HC) subjects and 53 patients with major depressive disorder (MDD) before and after rTMS treatment. Prior to treatment, patients with MDD have lower activity in the DLPFC, the hippocampus (HPC), the orbitofrontal cortex (OFC), and DLPFC-OFC connectivity compared with HCs. Following active rTMS treatment, patients with MDD show a significant increase in the DLPFC, HPC, and OFC. Notably, the increase in HPC activity is specifically associated with amelioration of depressive symptoms but not anxiety or sleep quality. The orbitofrontal-hippocampal pathway plays a crucial role in mediating depression relief following rTMS treatment. These findings suggest potential alternative targets for brain stimulation therapy against depression (chictr.org.cn: ChiCTR2100052007).

Cytoelectric Coupling: Electric fields sculpt neural activity and "tune" the brain's infrastructure

We propose and present converging evidence for the Cytoelectric Coupling Hypothesis: Electric fields generated by neurons are causal down to the level of the cytoskeleton. This could be achieved via electrodiffusion and mechanotransduction and exchanges between electrical, potential and chemical energy. Ephaptic coupling organizes neural activity, forming neural ensembles at the macroscale level. This information propagates to the neuron level, affecting spiking, and down to molecular level to stabilize the cytoskeleton, "tuning" it to process information more efficiently.

Theta oscillations in anterior cingulate cortex and orbitofrontal cortex differentially modulate accuracy and speed in flexible reward learning

Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition, there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.

Large-scale brain correlates of sweet versus cocaine reward in rats

Cocaine induces many supranormal changes in neuronal activity in the brain, notably in learning- and reward-related regions, in comparison with nondrug rewards-a difference that is thought to contribute to its relatively high addictive potential. However, when facing a choice between cocaine and a nondrug reward (e.g., water sweetened with saccharin), most rats do not choose cocaine, as one would expect from the extent and magnitude of its global activation of the brain, but instead choose the nondrug option. We recently showed that cocaine, though larger in magnitude, is also an inherently more delayed reward than sweet water, thereby explaining why it has less value during choice and why rats opt for the more immediate nondrug option. Here, we used a large-scale Fos brain mapping approach to measure brain responses to each option in saccharin-preferring rats, with the hope to identify brain regions whose activity may explain the preference for the nondrug option. In total, Fos expression was measured in 142 brain levels corresponding to 52 brain subregions and composing 5 brain macrosystems. Overall, our findings confirm in rats with a preference for saccharin that cocaine induces more global brain activation than the preferred nondrug option does. Only very few brain regions were uniquely activated by saccharin. They included regions involved in taste processing (i.e., anterior gustatory cortex) and also regions involved in processing reward delay and intertemporal choice (i.e., some components of the septohippocampal system and its connections with the lateral habenula).

When do bursts matter in the primary motor cortex? Investigating changes in the intermittencies of beta rhythms associated with movement states

Brain activity exhibits significant temporal structure that is not well captured in the power spectrum. Recently, attention has shifted to characterising the properties of intermittencies in rhythmic neural activity (i.e. bursts), yet the mechanisms that regulate them are unknown. Here, we present evidence from electrocorticography recordings made over the motor cortex to show that the statistics of bursts, such as duration or amplitude, in the beta frequency (14-30 Hz) band, significantly aid the classification of motor states such as rest, movement preparation, execution, and imagery. These features reflect nonlinearities not detectable in the power spectrum, with states increasing in nonlinearity from movement execution to preparation to rest. Further, we show using a computational model of the cortical microcircuit, constrained to account for burst features, that modulations of laminar specific inhibitory interneurons are responsible for the temporal organisation of activity. Finally, we show that the temporal characteristics of spontaneous activity can be used to infer the balance of cortical integration between incoming sensory information and endogenous activity. Critically, we contribute to the understanding of how transient brain rhythms may underwrite cortical processing, which in turn, could inform novel approaches for brain state classification, and modulation with novel brain-computer interfaces.

Top-down control of exogenous attentional selection is mediated by beta coherence in prefrontal cortex

Salience-driven exogenous and goal-driven endogenous attentional selection are two distinct forms of attention that guide selection of task-irrelevant and task-relevant targets in primates. During conflict i.e, when salience and goal each favor the selection of different targets, endogenous selection of the task-relevant target relies on top-down control. Top-down attentional control mechanisms enable selection of the task-relevant target by limiting the influence of sensory information. Although the lateral prefrontal cortex (LPFC) is known to mediate top-down control, the neuronal mechanisms of top-down control of attentional selection are poorly understood. Here, using a two-target free-choice luminance-reward selection task, we demonstrate that visual-movement neurons and not visual neurons or movement neurons encode exogenous and endogenous selection. We then show that coherent-beta activity selectively modulates mechanisms of exogenous selection specifically during conflict and consequently may support top-down control. These results reveal the VM-neuron-specific network mechanisms of attentional selection and suggest a functional role for beta-frequency coherent neural dynamics in the modulation of sensory communication channels for the top-down control of attentional selection.

Sleep strengthens resting-state functional communication between brain areas involved in the consolidation of problem-solving skills

Sleep consolidates procedural memory for motor skills, and this process is associated with strengthened functional connectivity in hippocampal-striatal-cortical areas. It is unknown whether similar processes occur for procedural memory that requires cognitive strategies needed for problem-solving. It is also unclear whether a full night of sleep is indeed necessary for consolidation to occur, compared with a daytime nap. We examined how resting-state functional connectivity within the hippocampal-striatal-cortical network differs after offline consolidation intervals of sleep, nap, or wake. Resting-state fMRI data were acquired immediately before and after training on a procedural problem-solving task that requires the acquisition of a novel cognitive strategy and immediately prior to the retest period (i.e., following the consolidation interval). ROI to ROI and seed to whole-brain functional connectivity analyses both specifically and consistently demonstrated strengthened hippocampal-prefrontal functional connectivity following a period of sleep versus wake. These results were associated with task-related gains in behavioral performance. Changes in functional communication were also observed between groups using the striatum as a seed. Here, we demonstrate that at the behavioral level, procedural strategies benefit from both a nap and a night of sleep. However, a full night of sleep is associated with enhanced functional communication between regions that support problem-solving skills.

The ventrolateral prefrontal cortex is part of the modular working memory system: A functional neuroanatomical perspective

For many years, the functional role of the ventrolateral Pre-Frontal Cortex (PFC) was associated with executive functions, specifically in the context of non-affective cognitive processes. However, recent research has suggested that the ventrolateral PFC is also involved in the attention system. The Ben Shalom model of the functional organization of the prefrontal cortex (2019) posits that the ventrolateral PFC selects perceptual stimuli after integration by the adjacent ventromedial PFC. This article reviews the state-of-the-art findings to better understand the role of the ventrolateral PFC in the selection of perceptual information as grounded in the Ben Shalom model. Numerous studies have reported converging evidence for the selective role of this area. However, most argue that this perceptual selection takes place through the active updating of information values linked to goal-oriented actions. These studies thus view the ventrolateral PFC as part of a system that actively manipulates and changes processed information such as the working memory function, rather than being part of the attention system. In agreement with this view, this review suggests that this area is part of a complex and modular working memory system and illustrates with reference to Diamond's work on ADD. This working memory system is functionally and anatomically dispersed and includes the dorsolateral PFC, the ACC, the parietal cortex, the basal ganglia, and the cerebellum. Hence, future research should continue to explore the specific neurofunctional roles of these areas in working memory systems, and the connections between the different subareas in this complex array.

Large-scale, closed-loop interrogation of neural circuits underlying cognition

Cognitive functions are increasingly understood to involve coordinated activity patterns between multiple brain regions, and their disruption by neuropsychiatric disorders is similarly complex. Closed-loop neurostimulation can directly modulate neural signals with temporal and spatial precision. How to leverage such an approach to effectively identify and target distributed neural networks implicated in mediating cognition remains unclear. We review current conceptual and technical advances in this area, proposing that devices that enable large-scale acquisition, integrated processing, and multiregion, arbitrary waveform stimulation will be critical for mechanistically driven manipulation of cognitive processes in physiological and pathological brain networks.

Complementary task representations in hippocampus and prefrontal cortex for generalizing the structure of problems

Humans and other animals effortlessly generalize prior knowledge to solve novel problems, by abstracting common structure and mapping it onto new sensorimotor specifics. To investigate how the brain achieves this, in this study, we trained mice on a series of reversal learning problems that shared the same structure but had different physical implementations. Performance improved across problems, indicating transfer of knowledge. Neurons in medial prefrontal cortex (mPFC) maintained similar representations across problems despite their different sensorimotor correlates, whereas hippocampal (dCA1) representations were more strongly influenced by the specifics of each problem. This was true for both representations of the events that comprised each trial and those that integrated choices and outcomes over multiple trials to guide an animal's decisions. These data suggest that prefrontal cortex and hippocampus play complementary roles in generalization of knowledge: PFC abstracts the common structure among related problems, and hippocampus maps this structure onto the specifics of the current situation.

Hippocampal signals modify orbitofrontal representations to learn new paths

We often remember the consequences of past choices to adapt to changing circumstances. Recalling past events requires the hippocampus (HPC), and using stimuli to anticipate outcome values requires the orbitofrontal cortex (OFC).1-3 Spatial reversal tasks require both structures to navigate newly rewarded paths.4,5 Both HPC place6 and OFC value cells7,8 fire in phase with theta (4-12 Hz) oscillations. Both structures are described as cognitive maps: HPC maps space9 and OFC maps task states.10 These similarities imply that OFC-HPC interactions are crucial for using memory to predict outcomes when circumstances change, but the mechanisms remain largely unknown. To investigate possible interactions, we simultaneously recorded ensembles in OFC and CA1 as rats learned spatial reversals in a plus maze. Striking interactions occurred only while rats learned their first reversal: CA1 population vectors predicted changes in OFC activity but not vice versa, OFC spikes phase locked to hippocampal theta oscillations, mixed pairs of CA1 and OFC neurons fired together within single theta cycles, and CA1 led OFC spikes by ∼30 ms. After the new contingency became familiar, CA1 ensembles stably represented distinct spatial paths, whereas OFC ensembles developed more generalized goal arm representations in different paths to identical rewards. These frontotemporal interactions, engaged selectively when new task features inform decision-making, suggest a mechanism for linking novel episodes with expected outcomes, when HPC signals trigger "cognitive remapping" by OFC.11.

Fast and slow contributions to decision-making in corticostriatal circuits

We make complex decisions using both fast judgments and slower, more deliberative reasoning. For example, during value-based decision-making, animals make rapid value-guided orienting eye movements after stimulus presentation that bias the upcoming decision. The neural mechanisms underlying these processes remain unclear. To address this, we recorded from the caudate nucleus and orbitofrontal cortex while animals made value-guided decisions. Using population-level decoding, we found a rapid, phasic signal in caudate that predicted the choice response and closely aligned with animals' initial orienting eye movements. In contrast, the dynamics in orbitofrontal cortex were more consistent with a deliberative system serially representing the value of each available option. The phasic caudate value signal and the deliberative orbitofrontal value signal were largely independent from each other, consistent with value-guided orienting and value-guided decision-making being independent processes.

Taking stock of value in the orbitofrontal cortex

People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.

Neural mechanisms of credit assignment for inferred relationships in a structured world

Animals abstract compact representations of a task's structure, which supports accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown. We develop a hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge. We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes. Furthermore, the mPFC and lateral orbitofrontal cortex track the current "position" within a latent association space that generalizes over stimuli. Collectively, these findings demonstrate the importance of both tracking the current position in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.

Theta low-gamma phase amplitude coupling in the human orbitofrontal cortex increases during a conflict-processing task

Objective. The human orbitofrontal cortex (OFC) is involved in automatic response inhibition and conflict processing, but the mechanism of frequency-specific power changes that control these functions is unknown. Theta and gamma activity have been independently observed in the OFC during conflict processing, while theta-gamma interactions in other brain areas have been noted primarily in studies of memory. Within the OFC, it is possible that theta-gamma phase amplitude coupling (PAC) drives conflict processing. This study aims to characterize the coupled relationship between theta and gamma frequency bands in the OFC during conflict processing using a modified Stroop task.Approach. Eight epilepsy patients implanted with OFC stereotactic electroencephalography electrodes participated in a color-word modified Stroop task. PAC between theta phase and gamma amplitude was assessed to determine the timing and magnitude of neural oscillatory changes. Group analysis was conducted using a non-parametric cluster-permutationt-test on coherence values.Main results.Theta-low gamma (LG) PAC significantly increased in five out of eight patients during successful trials of the incongruent condition compared with the congruent condition. Significant increases in theta-LG PAC were most prominent during cue processing 200-800 ms after cue presentation. On group analysis, trial-averaged mean theta-LG PAC was statistically significantly greater in the incongruent condition compared to the congruent condition (p< 0.001, Cohen'sd= 0.51).Significance.For the first time, we report that OFC theta phase and LG amplitude coupling increases during conflict resolution. Given the delayed onset after cue presentation, OFC theta-LG PAC may contribute to conflict processing after conflict detection and before motor response. This explanation follows the hypothesis that global theta waves modulate local gamma signals. Understanding this relationship within the OFC will help further elucidate the neural mechanisms of human conflict resolution.

Value representation in the monkey hippocampus

The hippocampus is thought to form cognitive maps across different domains of experience, including space and time. Recent work by Knudsen and Wallis identifies a map of abstract value space in the monkey hippocampus. We consider how these abstract variables might contribute to a comprehensive hippocampal representation of ongoing experience.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Temporally resolved neural dynamics underlying handwriting

How do the temporal dynamics of neural activity encode highly coordinated visual-motor behaviour? To capture the millisecond-resolved neural activations associated with fine visual-motor skills, we devised a co-registration system to simultaneously record electroencephalogram and handwriting kinematics while participants were performing four handwriting tasks (writing in Chinese/English scripts with their dominant/non-dominant hand). The neural activation associated with each stroke was clearly identified with a well-structured and reliable pattern. The functional significance of this pattern was validated by its significant associations with language, hand and the cognitive stages and kinematics of handwriting. Furthermore, the handwriting rhythmicity was found to be synchronised to the brain's ongoing theta oscillation, and the synchronisation was associated with the factor of language and hand. These major findings imply an implication between motor skill formation and the interplay between the rhythms in the brain and the peripheral systems.

The hippocampus and orbitofrontal cortex jointly represent task structure during memory-guided decision making

The hippocampus, well known for its role in episodic memory, might also be an important brain region for extracting structure from our experiences in order to guide future decisions. Recent evidence in rodents suggests that the hippocampus supports decision making by representing task structure in cooperation with the orbitofrontal cortex (OFC). Here, we examine how the human hippocampus and OFC represent task structure during an associative learning task that required learning of both context-determined and context-invariant probabilistic associations. We find that after learning, hippocampal and lateral OFC representations differentiated between context-determined and context-invariant task structures. The degree of this differentiation within the hippocampus and lateral OFC is highly correlated. These results advance our understanding of the hippocampus and suggest that the hippocampus and OFC support goal-directed behavior by representing information that guides the selection of appropriate decision strategies.

Mizrak et al. use fMRI to demonstrate that hippocampus and orbitofrontal cortex generalize across decisions that share the same task sub-structure compared with different task sub-structures. Results show that the hippocampus, in coordination with OFC, supports decision making by extracting structure from past experiences.

Striatal BOLD and Midfrontal Theta Power Express Motivation for Action

Action selection is biased by the valence of anticipated outcomes. To assess mechanisms by which these motivational biases are expressed and controlled, we measured simultaneous EEG-fMRI during a motivational Go/NoGo learning task (N = 36), leveraging the temporal resolution of EEG and subcortical access of fMRI. VmPFC BOLD encoded cue valence, importantly predicting trial-by-trial valence-driven response speed differences and EEG theta power around cue onset. In contrast, striatal BOLD encoded selection of active Go responses and correlated with theta power around response time. Within trials, theta power ramped in the fashion of an evidence accumulation signal for the value of making a "Go" response, capturing the faster responding to reward cues. Our findings reveal a dual nature of midfrontal theta power, with early components reflecting the vmPFC contribution to motivational biases, and late components reflecting their striatal translation into behavior, in line with influential recent "value of work" theories of striatal processing.

Is the core function of orbitofrontal cortex to signal values or make predictions?

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

The visual prefrontal cortex of anthropoids: interaction with temporal cortex in decision making and its role in the making of "visual animals"

The ventral prefrontal cortex (PFC) of primates-a region strongly implicated in decision making-receives highly processed visual sensory inputs from the inferior temporal cortex (ITC) and perirhinal cortex (PRC) and can therefore be considered visual PFC. Usually, the functions of temporal cortex and visual PFC have been discussed in separate literatures. By considering them together, we aim to clarify the ways in which fronto-temporal networks guide decision making. After discussing the ways in which visual PFC interacts with temporal cortex to promote decision making, we offer specific predictions about the selective roles of the ITC- and PRC-based fronto-temporal networks. Finally, we suggest that an increased reliance on visual PFC in anthropoid primates led to our emergence as 'visual' animals.

Inferences on a multidimensional social hierarchy use a grid-like code

Generalizing experiences to guide decision-making in novel situations is a hallmark of flexible behavior. Cognitive maps of an environment or task can theoretically afford such flexibility, but direct evidence has proven elusive. In this study, we found that discretely sampled abstract relationships between entities in an unseen two-dimensional social hierarchy are reconstructed into a unitary two-dimensional cognitive map in the hippocampus and entorhinal cortex. We further show that humans use a grid-like code in entorhinal cortex and medial prefrontal cortex for inferred direct trajectories between entities in the reconstructed abstract space during discrete decisions. These grid-like representations in the entorhinal cortex are associated with decision value computations in the medial prefrontal cortex and temporoparietal junction. Collectively, these findings show that grid-like representations are used by the human brain to infer novel solutions, even in abstract and discrete problems, and suggest a general mechanism underpinning flexible decision-making and generalization.

Reinstatement of Cortical Outcome Representations during Higher-Order Learning

Naturalistic learning scenarios are characterized by infrequent experience of external feedback to guide behavior. Higher-order learning mechanisms like second-order conditioning (SOC) may allow stimuli that were never experienced together with reinforcement to acquire motivational value. Despite its explanatory potential for real-world learning, surprisingly little is known about the neural mechanism underlying such associative transfer of value in SOC. Here, we used multivariate cross-session, cross-modality searchlight classification on functional magnetic resonance imaging data obtained from humans during SOC. We show that visual first-order conditioned stimuli (CS) reinstate cortical patterns representing previously paired gustatory outcomes in the lateral orbitofrontal cortex (OFC). During SOC, this OFC region showed increased functional covariation with amygdala, where neural pattern similarity between second-order CS and outcomes increased from early to late stages of SOC. Our data suggest a mechanism by which motivational value is conferred to stimuli that were never paired with reinforcement.

Spatial Representations in Rat Orbitofrontal Cortex

The orbitofrontal cortex (OFC) and hippocampus share striking cognitive and functional similarities. As a result, both structures have been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. However, while this function has been exemplified by spatial coding in neurons of hippocampal regions-particularly place and grid cells-spatial representations in the OFC have been investigated far less. Here we sought to address this by recording OFC neurons from male rats engaged in an open-field foraging task like that originally developed to characterize place fields in rodent hippocampal neurons. Single-unit activity was recorded as rats searched for food pellets scattered randomly throughout a large enclosure. In some sessions, particular flavors of food occurred more frequently in particular parts of the enclosure; in others, only a single flavor was used. OFC neurons showed spatially localized firing fields in both conditions, and representations changed between flavored and unflavored foraging periods in a manner reminiscent of remapping in the hippocampus. Compared with hippocampal recordings taken under similar behavioral conditions, OFC spatial representations were less temporally reliable, and there was no significant evidence of grid tuning in OFC neurons. These data confirm that OFC neurons show spatial firing fields in a large, two-dimensional environment in a manner similar to hippocampus. Consistent with the focus of the OFC on biological meaning and goals, spatial coding was weaker than in hippocampus and influenced by outcome identity.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) and hippocampus have both been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. This function is exemplified by place and grid cells identified in hippocampus, the activity of which maps spatial environments. The current study directly demonstrates very similar, though not identical, spatial representatives in OFC neurons, confirming that OFC-like hippocampus-can represent a spatial map under the appropriate experimental conditions.

Hippocampal neurons construct a map of an abstract value space

The hippocampus is thought to encode a "cognitive map," a structural organization of knowledge about relationships in the world. Place cells, spatially selective hippocampal neurons that have been extensively studied in rodents, are one component of this map, describing the relative position of environmental features. However, whether this map extends to abstract, cognitive information remains unknown. Using the relative reward value of cues to define continuous "paths" through an abstract value space, we show that single neurons in primate hippocampus encode this space through value place fields, much like a rodent's place neurons encode paths through physical space. Value place fields remapped when cues changed but also became increasingly correlated across contexts, allowing maps to become generalized. Our findings help explain the critical contribution of the hippocampus to value-based decision-making, providing a mechanism by which knowledge of relationships in the world can be incorporated into reward predictions for guiding decisions.

Causal manipulation of self-other mergence in the dorsomedial prefrontal cortex

To navigate social environments, people must simultaneously hold representations about their own and others’ abilities. During self-other mergence, people estimate others’ abilities not only on the basis of the others’ past performance, but the estimates are also influenced by their own performance. For example, if we perform well, we overestimate the abilities of those with whom we are co-operating and underestimate competitors. Self-other mergence is associated with specific activity patterns in the dorsomedial prefrontal cortex (dmPFC). Using a combination of non-invasive brain stimulation, functional magnetic resonance imaging, and computational modeling, we show that dmPFC neurostimulation silences these neural signatures of self-other mergence in relation to estimation of others’ abilities. In consequence, self-other mergence behavior increases, and our assessments of our own performance are projected increasingly onto other people. This suggests an inherent tendency to form interdependent social representations and a causal role of the dmPFC in separating self and other representations.

•

During self-other mergence (SOM), people confuse one’s own with another’s performance

•

Brain stimulation over dorsomedial prefrontal cortex (dmPFC) alters neural SOM

•

Brain stimulation over dmPFC simultaneously alters behavioral SOM

•

This suggests a causal role of dmPFC in separating self and other representations

What is dopamine doing in model-based reinforcement learning?

Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine's involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine's involvement in model-based RL.

Post-error recruitment of frontal sensory cortical projections promotes attention in mice

The frontal cortex, especially the anterior cingulate cortex area (ACA), is essential for exerting cognitive control after errors, but the mechanisms that enable modulation of attention to improve performance after errors are poorly understood. Here we demonstrate that during a mouse visual attention task, ACA neurons projecting to the visual cortex (VIS; ACA_VIS neurons) are recruited selectively by recent errors. Optogenetic manipulations of this pathway collectively support the model that rhythmic modulation of ACA_VIS neurons in anticipation of visual stimuli is crucial for adjusting performance following errors. 30-Hz optogenetic stimulation of ACA_VIS neurons in anesthetized mice recapitulates the increased gamma and reduced theta VIS oscillatory changes that are associated with endogenous post-error performance during behavior and subsequently increased visually evoked spiking, a hallmark feature of visual attention. This frontal sensory neural circuit links error monitoring with implementing adjustments of attention to guide behavioral adaptation, pointing to a circuit-based mechanism for promoting cognitive control.