An Integrated Model of Action Selection: Distinct Modes of Cortical Control of Striatal Decision Making

Model of a striatal circuit exploring biological mechanisms underlying decision-making during normal and disordered states

Decision-making requires continuous adaptation to internal and external contexts. Changes in decision-making are reliable transdiagnostic symptoms of neuropsychiatric disorders. We created a computational model demonstrating how the striosome compartment of the striatum constructs a mathematical space for decision-making computations depending on context, and how the matrix compartment defines action value depending on the space. The model explains multiple experimental results and unifies other theories like reward prediction error, roles of the direct versus indirect pathways, and roles of the striosome versus matrix, under one framework. We also found, through new analyses, that striosome and matrix neurons increase their synchrony during difficult tasks, caused by a necessary increase in dimensionality of the space. The model makes testable predictions about individual differences in disorder susceptibility, decision-making symptoms shared among neuropsychiatric disorders, and differences in neuropsychiatric disorder symptom presentation. The model reframes the role of the striosomal circuit in neuroeconomic and disorder-affected decision-making.

Highlights

Striosomes prioritize decision-related data used by matrix to set action values. Striosomes and matrix have different roles in the direct and indirect pathways. Abnormal information organization/valuation alters disorder presentation. Variance in data prioritization may explain individual differences in disorders.

eTOC

Beck et al. developed a computational model of how a striatal circuit functions during decision-making. The model unifies and extends theories about the direct versus indirect pathways. It further suggests how aberrant circuit function underlies decision-making phenomena observed in neuropsychiatric disorders.

Cortico-striatal action control inherent of opponent cognitive-motivational styles

Turning on cue or stopping at a red light requires the detection of such cues to select action sequences, or suppress action, in accordance with cue-associated action rules. Cortico-striatal projections are an essential part of the brain's attention-motor interface. Glutamate-sensing microelectrode arrays were used to measure glutamate transients in the dorsomedial striatum (DMS) of male and female rats walking a treadmill and executing cued turns and stops. Prelimbic-DMS projections were chemogenetically inhibited to determine their behavioral necessity and the cortico-striatal origin of cue-evoked glutamate transients. Furthermore, we investigated rats exhibiting preferably goal-directed (goal trackers, GTs) versus cue-driven attention (sign trackers, STs), to determine the impact of such cognitive-motivational biases on cortico-striatal control. GTs executed more cued turns and initiated such turns more slowly than STs. During turns, but not missed turns or cued stops, cue-evoked glutamate concentrations were higher in GTs than in STs. In conjunction with turn cue-evoked glutamate spike levels, the presence of a single spike rendered GTs to be almost twice as likely to turn than STs. In contrast, multiple glutamate spikes predicted GTs to be less likely to turn than STs. In GTs, but not STs, inhibition of prelimbic-DMS projections attenuated turn rates, turn cue-evoked glutamate peaks, and increased the number of spikes. These findings suggest that turn cue-evoked glutamate release in GTs is tightly controlled by cortico-striatal neuronal activity. In contrast, in STs, glutamate release from DMS glutamatergic terminals may be regulated by other striatal circuitry, preferably mediating cued suppression of action and reward tracking.

Impaired striatal glutamate/GABA regulation in violent offenders with antisocial personality disorder and psychopathy

Men with antisocial personality disorder (ASPD) with or without psychopathy (+/-P) are responsible for most violent crime in society. Development of effective treatments is hindered by poor understanding of the neurochemical underpinnings of the condition. Men with ASPD with and without psychopathy demonstrate impulsive decision-making, associated with striatal abnormalities in functional neuroimaging studies. However, to date, no study has directly examined the potential neurochemical underpinnings of such abnormalities. We therefore investigated striatal glutamate: GABA ratio using Magnetic Resonance Spectroscopy in 30 violent offenders (16 ASPD-P, 14 ASPD + P) and 21 healthy non-offenders. Men with ASPD +/- P had a significant reduction in striatal glutamate : GABA ratio compared to non-offenders. We report, for the first time, striatal Glutamate/GABA dysregulation in ASPD +/- P, and discuss how this may be related to core behavioral abnormalities in the disorders.

Functional diversity of dopamine axons in prefrontal cortex during classical conditioning

Midbrain dopamine neurons impact neural processing in the prefrontal cortex (PFC) through mesocortical projections. However, the signals conveyed by dopamine projections to the PFC remain unclear, particularly at the single-axon level. Here, we investigated dopaminergic axonal activity in the medial PFC (mPFC) during reward and aversive processing. By optimizing microprism-mediated two-photon calcium imaging of dopamine axon terminals, we found diverse activity in dopamine axons responsive to both reward and aversive stimuli. Some axons exhibited a preference for reward, while others favored aversive stimuli, and there was a strong bias for the latter at the population level. Long-term longitudinal imaging revealed that the preference was maintained in reward- and aversive-preferring axons throughout classical conditioning in which rewarding and aversive stimuli were paired with preceding auditory cues. However, as mice learned to discriminate reward or aversive cues, a cue activity preference gradually developed only in aversive-preferring axons. We inferred the trial-by-trial cue discrimination based on machine learning using anticipatory licking or facial expressions, and found that successful discrimination was accompanied by sharper selectivity for the aversive cue in aversive-preferring axons. Our findings indicate that a group of mesocortical dopamine axons encodes aversive-related signals, which are modulated by both classical conditioning across days and trial-by-trial discrimination within a day.

Adolescents rats engage the orbitofrontal-striatal pathway differently than adults during impulsive actions

Adolescence is characterized by increased impulsive and risk-taking behaviors. To better understand the neural networks that subserves impulsivity in adolescents, we used a reward-guided behavioral model that quantifies age differences in impulsive actions in adult and adolescent rats of both sexes. Using chemogenetics, we identified orbitofrontal cortex (OFC) projections to the dorsomedial striatum (DMS) as a critical pathway for age-related execution of impulsive actions. Simultaneous recording of single units and local field potentials in the OFC and DMS during task performance revealed an overall muted response in adolescents during impulsive actions as well as age-specific differences in theta power and OFC-DMS functional connectivity. Collectively, these data reveal that the OFC-DMS pathway is critical for age-differences in reward-guided impulsive actions and provide a network mechanism to enhance our understanding of how adolescent and adult brains coordinate behavioral inhibition.

Paternal cocaine-seeking motivation defines offspring's vulnerability to addiction by down-regulating GABAergic GABRG3 in the ventral tegmental area

Epidemiological investigations indicate that parental drug abuse experiences significantly influenced the addiction vulnerability of offspring. Studies using animal models have shown that paternal cocaine use and highly motivated drug-seeking behavior are important determinants of offspring addiction susceptibility. However, the key molecules contributing to offspring addiction susceptibility are currently unclear. The motivation for cocaine-seeking behavior in offspring of male rats was compared between those whose fathers self-administered cocaine (SA) and those who were yoked with them and received non-contingent cocaine administrations (Yoke). We found that paternal experience with cocaine-seeking behavior, but not direct cocaine exposure, could lead to increased lever-pressing behavior in male F1 offspring. This effect was observed without significant changes to the dose-response relationship. The transcriptomes of ventral tegmental area (VTA) in offspring were analyzed under both naive state and after self-administration training. Specific transcriptomic changes in response to paternal cocaine-seeking experiences were found, which mainly affected biological processes such as synaptic connections and receptor signaling pathways. Through joint analysis of these candidate genes and parental drug-seeking motivation scores, we found that gamma-aminobutyric acid receptor subunit gamma-3 (Gabrg3) was in the hub position of the drug-seeking motivation-related module network and highly correlated with parental drug-seeking motivation scores. The downregulation of Gabrg3 expression, caused by paternal motivational cocaine-seeking, mainly occurred in GABAergic neurons in the VTA. Furthermore, down-regulating GABAergic Gabrg3 in VTA resulted in an increase in cocaine-seeking behavior in the Yoke F1 group. This down-regulation also reduced transcriptome differences between the Yoke and SA groups, affecting processes related to synaptic formation and neurotransmitter transmission. Taken together, we propose that paternal cocaine-seeking behavior, rather than direct drug exposure, significantly influences offspring addiction susceptibility through the downregulation of Gabrg3 in GABAergic neurons of the VTA, highlighting the importance of understanding specific molecular pathways in the intergenerational inheritance of addiction vulnerability.

Cholinergic Activation of Corticofugal Circuits in the Adult Mouse Prefrontal Cortex

Acetylcholine (ACh) promotes neocortical output to the thalamus and brainstem by preferentially enhancing the postsynaptic excitability of layer 5 pyramidal tract (PT) neurons relative to neighboring intratelencephalic (IT) neurons. Less is known about how ACh regulates the excitatory synaptic drive of IT and PT neurons. To address this question, spontaneous excitatory postsynaptic potentials (sEPSPs) were recorded in dual recordings of IT and PT neurons in slices of prelimbic cortex from adult female and male mice. ACh (20 µM) enhanced sEPSP amplitudes, frequencies, rise-times, and half-widths preferentially in PT neurons. These effects were blocked by the muscarinic receptor antagonist atropine (1 µM). When challenged with pirenzepine (1 µM), an antagonist selective for M1-type muscarinic receptors, ACh instead reduced sEPSP frequencies, suggesting that ACh may generally suppress synaptic transmission in the cortex via non-M1 receptors. Cholinergic enhancement of sEPSPs in PT neurons was not sensitive to antagonism of GABA receptors with gabazine (10 µM) and CGP52432 (2.5 µM) but was blocked by tetrodotoxin (1 µM), suggesting that ACh enhances action-potential-dependent excitatory synaptic transmission in PT neurons. ACh also preferentially promoted the occurrence of synchronous sEPSPs in dual recordings of PT neurons relative to IT-PT and IT-IT parings. Finally, selective chemogenetic silencing of hM4Di-expressing PT, but not commissural IT, neurons blocked cholinergic enhancement of sEPSP amplitudes and frequencies in PT neurons. These data suggest that, in addition to selectively enhancing the postsynaptic excitability of PT neurons, M1 receptor activation promotes corticofugal output by amplifying recurrent excitation within networks of PT neurons.

Context learning: Dividing up the camp

A new study reveals that ventral hippocampus integrates information about where we can get rewards into a decision to seek out those rewards. This helps maximise the number of rewards received, while reducing the effort we expend to procure them.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

The cognitive (lateral) hypothalamus

Despite the physiological complexity of the hypothalamus, its role is typically restricted to initiation or cessation of innate behaviors. For example, theories of lateral hypothalamus argue that it is a switch to turn feeding 'on' and 'off' as dictated by higher-order structures that render when feeding is appropriate. However, recent data demonstrate that the lateral hypothalamus is critical for learning about food-related cues. Furthermore, the lateral hypothalamus opposes learning about information that is neutral or distal to food. This reveals the lateral hypothalamus as a unique arbitrator of learning capable of shifting behavior toward or away from important events. This has relevance for disorders characterized by changes in this balance, including addiction and schizophrenia. Generally, this suggests that hypothalamic function is more complex than increasing or decreasing innate behaviors.

Prefrontal neuronal ensembles link prior knowledge with novel actions during flexible action selection

We make decisions based on currently perceivable information or an internal model of the environment. The medial prefrontal cortex (mPFC) and its interaction with the hippocampus have been implicated in the latter, model-based decision-making; however, the underlying computational properties remain incompletely understood. We have examined mPFC spiking and hippocampal oscillatory activity while rats flexibly select new actions using a known associative structure of environmental cues and outcomes. During action selection, the mPFC reinstates representations of the associative structure. These awake reactivation events are accompanied by synchronous firings among neurons coding the associative structure and those coding actions. Moreover, their functional coupling is strengthened upon the reactivation events leading to adaptive actions. In contrast, only cue-coding neurons improve functional coupling during hippocampal sharp wave ripples. Thus, the lack of direct experience disconnects the mPFC from the hippocampus to independently form self-organized neuronal ensemble dynamics linking prior knowledge with novel actions.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

We aim to differentiate the brain regions involved in the learning and encoding of Pavlovian associations sensitive to changes in outcome value from those that are not sensitive to such changes by combining a learning task with outcome devaluation, eye-tracking, and functional magnetic resonance imaging in humans. Contrary to theoretical expectation, voxels correlating with reward prediction errors in the ventral striatum and subgenual cingulate appear to be sensitive to devaluation. Moreover, regions encoding state prediction errors appear to be devaluation insensitive. We can also distinguish regions encoding predictions about outcome taste identity from predictions about expected spatial location. Regions encoding predictions about taste identity seem devaluation sensitive while those encoding predictions about an outcome's spatial location seem devaluation insensitive. These findings suggest the existence of multiple and distinct associative mechanisms in the brain and help identify putative neural correlates for the parallel expression of both devaluation sensitive and insensitive conditioned behaviors.

The orbitofrontal cortex: A goal-directed cognitive map framework for social and non-social behaviors

The orbitofrontal cortex (OFC) is regarded as one of the core brain areas in a variety of value-based behaviors. Over the past two decades, tremendous knowledge about the OFC function was gained from studying the behaviors of single subjects. As a result, our previous understanding of the OFC's function of encoding decision variables, such as the value and identity of choices, has evolved to the idea that the OFC encodes a more complex representation of the task space as a cognitive map. Accumulating evidence also indicates that the OFC importantly contributes to behaviors in social contexts, especially those involved in cooperative interactions. However, it remains elusive how exactly OFC neurons contribute to social functions and how non-social and social behaviors are related to one another in the computations performed by OFC neurons. In this review, we aim to provide an integrated view of the OFC function across both social and non-social behavioral contexts. We propose that seemingly complex functions of the OFC may be explained by its role in providing a goal-directed cognitive map to guide a wide array of adaptive reward-based behaviors in both social and non-social domains.

Cholinergic activation of corticofugal circuits in the adult mouse prefrontal cortex

In layer 5 of the neocortex, ACh promotes cortical output to the thalamus and brainstem by preferentially enhancing the postsynaptic excitability of pyramidal tract (PT) neurons relative to neighboring intratelencephalic (IT) neurons. Less is known about how ACh regulates the excitatory synaptic drive of IT and PT neurons. To address this question, spontaneous excitatory postsynaptic potentials (sEPSPs) were recorded in pairs of IT and PT neurons in slices of prelimbic cortex from adult female and male mice. ACh (20 µM) enhanced sEPSP amplitudes, frequencies, rise-times, and half-widths preferentially in PT neurons. These effects were blocked by the muscarinic acetylcholine receptor antagonist atropine (1 µM). When challenged with pirenzepine (1 µM), an antagonist selective for M1-type muscarinic receptors, ACh instead reduced sEPSP frequencies. The cholinergic increase in sEPSP amplitudes and frequencies in PT neurons was not sensitive to blockade of GABAergic receptors with gabazine (10 µM) and CGP52432 (2.5 µM), but was blocked by tetrodotoxin (1 µM), suggesting that ACh enhances action-potential-dependent excitatory synaptic transmission in PT neurons. ACh also preferentially promoted the occurrence of synchronous sEPSPs in pairs of PT neurons relative to IT-PT and IT-IT pairs. Finally, selective chemogenetic silencing of hM4Di-expressing PT, but not IT, neurons with clozapine-N-oxide (5 µM) blocked cholinergic enhancement of sEPSP amplitudes and frequencies in PT neurons. These data suggest that, in addition to enhancing the postsynaptic excitability of PT neurons, M1 receptor activation promotes corticofugal output by preferentially amplifying recurrent excitation within networks of PT neurons.

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

Closed-loop enhancement and neural decoding of cognitive control in humans

Deficits in cognitive control-that is, in the ability to withhold a default pre-potent response in favour of a more adaptive choice-are common in depression, anxiety, addiction and other mental disorders. Here we report proof-of-concept evidence that, in participants undergoing intracranial epilepsy monitoring, closed-loop direct stimulation of the internal capsule or striatum, especially the dorsal sites, enhances the participants' cognitive control during a conflict task. We also show that closed-loop stimulation upon the detection of lapses in cognitive control produced larger behavioural changes than open-loop stimulation, and that task performance for single trials can be directly decoded from the activity of a small number of electrodes via neural features that are compatible with existing closed-loop brain implants. Closed-loop enhancement of cognitive control might remediate underlying cognitive deficits and aid the treatment of severe mental disorders.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans

Pavlovian learning depends on multiple and parallel associations leading to distinct classes of conditioned responses that vary in their flexibility following changes in the value of an associated outcome. Here, we aimed to differentiate brain areas involved in learning and encoding associations that are sensitive to changes in the value of an outcome from those that are not sensitive to such changes. To address this question, we combined a Pavlovian learning task with outcome devaluation, eye-tracking and functional magnetic resonance imaging. We used computational modeling to identify brain regions involved in learning stimulus-reward associations and stimulus-stimulus associations, by testing for brain areas correlating with reward-prediction errors and state-prediction errors, respectively. We found that, contrary to theoretical predictions about reward prediction errors being exclusively model-free, voxels correlating with reward prediction errors in the ventral striatum and subgenual anterior cingulate cortex were sensitive to devaluation. On the other hand, brain areas correlating with state prediction errors were found to be devaluation insensitive. In a supplementary analysis, we distinguished brain regions encoding predictions about outcome taste identity from those involved in encoding predictions about its expected spatial location. A subset of regions involved in taste identity predictions were devaluation sensitive while those involved in encoding predictions about spatial location were devaluation insensitive. These findings provide insights into the role of multiple associative mechanisms in the brain in mediating Pavlovian conditioned behavior - illustrating how distinct neural pathways can in parallel produce both devaluation sensitive and devaluation insensitive behaviors.

Impairments in goal-directed action and reversal learning in a proportion of individuals with psychosis

Cognitive impairment in psychosis is one of the strongest predictors of functional decline. Problems with decision-making processes, such as goal-directed action and reversal learning, can reflect cortico-striatal dysfunction. The heterogenous symptoms and neurobiology observed in those with psychosis suggests that specific cognitive phenotypes may reflect differing causative mechanisms. As such, decision-making performance could identify subgroups of individuals with more severe cortico-striatal dysfunction and help to predict their functional decline. The present work evaluated the relationship between goal-directed action, reversal learning, and symptom profiles in those with psychosis. We assessed decision-making processes in healthy controls (N = 34) and those with persistent psychosis (N = 45), subclassifying subjects based on intact/impaired goal-directed action. Compared with healthy controls (<20%), a large proportion (58%) of those with persistent psychosis displayed impaired goal-directed action, predicting poor serial reversal learning performance. Computational approaches indicated that those with impaired goal-directed action had a decreased capacity to rapidly update their prior beliefs in the face of changing contingencies. Impaired decision-making also was associated with reduced levels of grandiosity and increased problems with abstract thinking. These findings suggest that prominent decision-making deficits, indicative of cortico-striatal dysfunction, are present in a large proportion of people with persistent psychosis. Moreover, these impairments would have significant functional implications in terms of planning and abstract thinking.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Medial prefrontal cortex lesions disrupt prepotent action selection signals in dorsomedial striatum

The ability to inhibit or adapt unwanted actions or movements is a critical feature of almost all forms of behavior. Many have attributed this ability to frontal brain areas such as the anterior cingulate cortex (ACC) and the medial prefrontal cortex (mPFC), but the exact contribution of each brain region is often debated because their functions are not examined in animals performing the same task. Recently, we have shown that ACC signals a need for cognitive control and is crucial for the adaptation of action selection signals in dorsomedial striatum (DMS) in rats performing a stop-change task. Here, we show that unlike ACC, the prelimbic region of mPFC does not disrupt the inhibition or adaption of an action plan at either the level of behavior or downstream firing in DMS. Instead, lesions to mPFC correlate with changes in DMS signals involved in action initiation and disrupt performance on GO trials while improving performance on STOP trials.

Differential Dopamine Receptor-Dependent Sensitivity Improves the Switch Between Hard and Soft Selection in a Model of the Basal Ganglia

The problem of selecting one action from a set of different possible actions, simply referred to as the problem of action selection, is a ubiquitous challenge in the animal world. For vertebrates, the basal ganglia (BG) are widely thought to implement the core computation to solve this problem, as its anatomy and physiology are well suited to this end. However, the BG still display physiological features whose role in achieving efficient action selection remains unclear. In particular, it is known that the two types of dopaminergic receptors (D1 and D2) present in the BG give rise to mechanistically different responses. The overall effect will be a difference in sensitivity to dopamine, which may have ramifications for action selection. However, which receptor type leads to a stronger response is unclear due to the complexity of the intracellular mechanisms involved. In this study, we use an existing, high-level computational model of the BG, which assumes that dopamine contributes to action selection by enabling a switch between different selection regimes, to predict which of D1 or D2 has the greater sensitivity. Thus, we ask, Assuming dopamine enables a switch between action selection regimes in the BG, what functional sensitivity values would result in improved action selection computation? To do this, we quantitatively assessed the model's capacity to perform action selection as we parametrically manipulated the sensitivity weights of D1 and D2. We show that differential (rather than equal) D1 and D2 sensitivity to dopaminergic input improves the switch between selection regimes during the action selection computation in our model. Specifically, greater D2 sensitivity compared to D1 led to these improvements.

Alcohol dependence decreases functional activation of the caudate nucleus during model-based decision processes

BACKGROUND

Impaired decision making, a key characteristic of alcohol dependence (AD), manifests in continuous alcohol consumption despite severe negative consequences. The neural basis of this impairment in individuals with AD and differences with known neural decision mechanisms among healthy subjects are not fully understood. In particular, it is unclear whether the choice behavior among individuals with AD is based on a general impairment of decision mechanisms or is mainly explained by altered value attribution, with an overly high subjective value attributed to alcohol-related stimuli.

METHODS

Here, we use a functional magnetic resonance imaging (fMRI) monetary reward task to compare the neural processes of model-based decision making and value computation between AD individuals (n = 32) and healthy controls (n = 32). During fMRI, participants evaluated monetary offers with respect to dynamically changing constraints and different levels of uncertainty.

RESULTS

Individuals with AD showed lower activation associated with model-based decision processes in the caudate nucleus than controls, but there were no group differences in value-related neural activity or task performance.

CONCLUSIONS

Our findings highlight the role of the caudate nucleus in impaired model-based decisions of alcohol-dependent individuals.

Extending neural systems for navigation to hunting behavior

For decades, a central question in neuroscience has been: How does the brain support navigation? Recent research on navigation has explored how brain regions support the capacity to adapt to changes in the environment and track the distance and direction to goal locations. Here, we provide a brief review of this literature and speculate how these neural systems may be involved in another, parallel behavior-hunting. Hunting shares many of the same challenges as navigation. Like navigation, hunting requires the hunter to orient towards a goal while minimizing their distance from it while traveling. Likewise, hunting may require the accommodation of detours to locate prey or the exploitation of shortcuts for a quicker capture. Recent research suggests that neurons in the periaqueductal gray, hypothalamus, and dorsal anterior cingulate play key roles in such hunting behavior. In this review, we speculate on how these regions may operate functionally with other key brain regions involved in navigation, such as the hippocampus, to support hunting. Additionally, we posit that hunting in a group presents an additional set of challenges, where success relies on multicentric tracking and prediction of prey position as well as the position of co-hunters.

Model-based learning retrospectively updates model-free values

Reinforcement learning (RL) is widely regarded as divisible into two distinct computational strategies. Model-free learning is a simple RL process in which a value is associated with actions, whereas model-based learning relies on the formation of internal models of the environment to maximise reward. Recently, theoretical and animal work has suggested that such models might be used to train model-free behaviour, reducing the burden of costly forward planning. Here we devised a way to probe this possibility in human behaviour. We adapted a two-stage decision task and found evidence that model-based processes at the time of learning can alter model-free valuation in healthy individuals. We asked people to rate subjective value of an irrelevant feature that was seen at the time a model-based decision would have been made. These irrelevant feature value ratings were updated by rewards, but in a way that accounted for whether the selected action retrospectively ought to have been taken. This model-based influence on model-free value ratings was best accounted for by a reward prediction error that was calculated relative to the decision path that would most likely have led to the reward. This effect occurred independently of attention and was not present when participants were not explicitly told about the structure of the environment. These findings suggest that current conceptions of model-based and model-free learning require updating in favour of a more integrated approach. Our task provides an empirical handle for further study of the dialogue between these two learning systems in the future.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information. While dysfunction in dopaminergic signalling has long been linked to the pathophysiology of schizophrenia, a cohesive framework that accounts for this learning paradox remains elusive. Recently, there has been an explosion of new research investigating how dopamine contributes to reinforcement learning, which illustrates that midbrain dopamine contributes in complex ways to reinforcement learning, not previously envisioned. This new data brings new possibilities for how dopamine signalling contributes to the symptomatology of schizophrenia. Building on recent work, we present a new neural framework for how we might envision specific dopamine circuits contributing to this learning paradox in schizophrenia in the context of models of reinforcement learning. Further, we discuss avenues of preclinical research with the use of cutting-edge neuroscience techniques where aspects of this model may be tested. Ultimately, it is hoped that this review will spur to action more research utilising specific reinforcement learning paradigms in preclinical models of schizophrenia, to reconcile seemingly disparate symptomatology and develop more efficient therapeutics.

Temporal dynamics of affect in the brain: Evidence from human imaging and animal models

Emotions are time-varying internal states that promote survival in the face of dynamic environments and shifting homeostatic needs. Research in non-human organisms has recently afforded specific insights into the neural mechanisms that support the emergence, persistence, and decay of affective states. Concurrently, a separate affective neuroscience literature has begun to dissect the neural bases of affective dynamics in humans. However, the circuit-level mechanisms identified in animals lack a clear mapping to the human neuroscience literature. As a result, critical questions pertaining to the neural bases of affective dynamics in humans remain unanswered. To address these shortcomings, the present review integrates findings from humans and non-human organisms to highlight the neural mechanisms that govern the temporal features of emotional states. Using the theory of affective chronometry as an organizing framework, we describe the specific neural mechanisms and modulatory factors that arbitrate the rise-time, intensity, and duration of emotional states.

Uncovering the global task-modulated brain network in chunk decomposition with Chinese characters

Chunk decomposition, which requires the mental representation transformation in accordance with behavioral goals, is of vital importance to problem solving and creative thinking. Previous studies have identified that the frontal, parietal, and occipital cortex in the cognitive control network selectively activated in response to chunk tightness, however, functional localization strategy may overlook the interaction brain regions. Based on the notion of a global brain network, we proposed that multiple specialized regions have to be interconnected to maintain goal representation during the course of chunk decomposition. Therefore, the present study applied a beta-series correlation method to investigate interregional functional connectivity in the event-related design of chunk decomposition tasks using Chinese characters, which would highlight critical nodes irrespective to chunk tightness. The results reveal a network of functional hubs with highly within or between module connections, including the orbitofrontal cortex, superior/inferior parietal lobule, hippocampus, and thalamus. We speculate that the thalamus integrates information across modular as an integrative hub while the orbitofrontal cortex tracks the mental states of chunk decomposition on a moment-to-moment basis. The superior and inferior parietal lobule collaborate to manipulate the mental representation of chunk decomposition and the hippocampus associates the relationship between elements in the question and solution phase. Furthermore, the tightness of chunks is not only associated with different processors in visual systems but also leads to increased intermodular connections in right superior frontal gyrus and left precentral gyrus. To summary up, the present study first reveals the task-modulated brain network of chunk decomposition in addition to the tightness-related nodes in the frontal and occipital cortex.

Executive Function Assigns Value to Novel Goal-Congruent Outcomes

People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from abstract novel outcomes harnesses the same circuitry that supports learning from familiar secondary reinforcers. Behavior and neuroimaging revealed that novel images can act as a substitute for rewards during instrumental learning, producing reliable reward-like signals in dopaminergic circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. These results suggest that learning from novel outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow for human reinforcement learning over arbitrarily abstract reward functions.

Higher-Order Conditioning and Dopamine: Charting a Path Forward

Higher-order conditioning involves learning causal links between multiple events, which then allows one to make novel inferences. For example, observing a correlation between two events (e.g., a neighbor wearing a particular sports jersey), later helps one make new predictions based on this knowledge (e.g., the neighbor's wife's favorite sports team). This type of learning is important because it allows one to benefit maximally from previous experiences and perform adaptively in complex environments where many things are ambiguous or uncertain. Two procedures in the lab are often used to probe this kind of learning, second-order conditioning (SOC) and sensory preconditioning (SPC). In second-order conditioning (SOC), we first teach subjects that there is a relationship between a stimulus and an outcome (e.g., a tone that predicts food). Then, an additional stimulus is taught to precede the predictive stimulus (e.g., a light leads to the food-predictive tone). In sensory preconditioning (SPC), this order of training is reversed. Specifically, the two neutral stimuli (i.e., light and tone) are first paired together and then the tone is paired separately with food. Interestingly, in both SPC and SOC, humans, rodents, and even insects, and other invertebrates will later predict that both the light and tone are likely to lead to food, even though they only experienced the tone directly paired with food. While these processes are procedurally similar, a wealth of research suggests they are associatively and neurobiologically distinct. However, midbrain dopamine, a neurotransmitter long thought to facilitate basic Pavlovian conditioning in a relatively simplistic manner, appears critical for both SOC and SPC. These findings suggest dopamine may contribute to learning in ways that transcend differences in associative and neurological structure. We discuss how research demonstrating that dopamine is critical to both SOC and SPC places it at the center of more complex forms of cognition (e.g., spatial navigation and causal reasoning). Further, we suggest that these more sophisticated learning procedures, coupled with recent advances in recording and manipulating dopamine neurons, represent a new path forward in understanding dopamine's contribution to learning and cognition.

Beyond the hippocampus: The role of parahippocampal-prefrontal communication in context-modulated behavior

Multiple paradigms indicate that the physical environment can influence spontaneous and learned behavior. In rodents, context-dependent behavior is putatively supported by the prefrontal cortex and the medial temporal lobe. A preponderance of the literature has targeted the role of the hippocampus. In addition to the hippocampus proper, the medial temporal lobe also comprises parahippocampal areas, including the perirhinal and postrhinal cortices. These parahippocampal areas directly connect with multiple regions in the prefrontal cortex. The function of these connections, however, is not well understood. This article first reviews the involvement of the perirhinal, postrhinal, and prefrontal cortices in context-dependent behavior in rodents. Then, based on functional and anatomical evidence, we suggest that perirhinal and postrhinal contributions to context-dependent behavior go beyond supporting context representation in the hippocampus. Specifically, we propose that the perirhinal and postrhinal cortices act as a contextual-support network that directly provides contextual and spatial information to the prefrontal cortex. In turn, the perirhinal and postrhinal cortices modulate prefrontal input to the hippocampus in the service of context-guided behavior.

Dissociable and Paradoxical Roles of Rat Medial and Lateral Orbitofrontal Cortex in Visual Serial Reversal Learning

Much evidence suggests that reversal learning is mediated by cortico-striatal circuitries with the orbitofrontal cortex (OFC) playing a prominent role. The OFC is a functionally heterogeneous region, but potential differential roles of lateral (lOFC) and medial (mOFC) portions in visual reversal learning have yet to be determined. We investigated the effects of pharmacological inactivation of mOFC and lOFC on a deterministic serial visual reversal learning task for rats. For reference, we also targeted other areas previously implicated in reversal learning: prelimbic (PrL) and infralimbic (IL) prefrontal cortex, and basolateral amygdala (BLA). Inactivating mOFC and lOFC produced opposite effects; lOFC impairing, and mOFC improving, performance in the early, perseverative phase specifically. Additionally, mOFC inactivation enhanced negative feedback sensitivity, while lOFC inactivation diminished feedback sensitivity in general. mOFC and lOFC inactivation also affected novel visual discrimination learning differently; lOFC inactivation paradoxically improved learning, and mOFC inactivation had no effect. We also observed dissociable roles of the OFC and the IL/PrL. Whereas the OFC inactivation affected only perseveration, IL/PrL inactivation improved learning overall. BLA inactivation did not affect perseveration, but improved the late phase of reversal learning. These results support opponent roles of the rodent mOFC and lOFC in deterministic visual reversal learning.

Lost in translation: no effect of repeated optogenetic cortico-striatal stimulation on compulsivity in rats

The orbitofrontal cortex-ventromedial striatum (OFC-VMS) circuitry is widely believed to drive compulsive behavior. Hyperactivating this pathway in inbred mice produces excessive and persistent self-grooming, which has been considered a model for human compulsivity. We aimed to replicate these findings in outbred rats, where there are few reliable compulsivity models. Male Long-Evans rats implanted with optical fibers into VMS and with opsins delivered into OFC received optical stimulation at parameters that produce OFC-VMS plasticity and compulsive grooming in mice. We then evaluated rats for compulsive self-grooming at six timepoints: before, during, immediately after, and 1 h after each stimulation, 1 and 2 weeks after the ending of a 6-day stimulation protocol. To further test for effects of OFC-VMS hyperstimulation, we ran animals in three standard compulsivity assays: marble burying, nestlet shredding, and operant attentional set-shifting. OFC-VMS stimulation did not increase self-grooming or induce significant changes in nestlet shredding, marble burying, or set-shifting in rats. Follow-on evoked potential studies verified that the stimulation protocol altered OFC-VMS synaptic weighting. In sum, although we induced physiological changes in the OFC-VMS circuitry, we could not reproduce in a strongly powered study in rats a model of compulsive behavior previously reported in mice. This suggests possible limitations to translation of mouse findings to species higher on the phylogenetic chain.

The orbitofrontal cartographer

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency contingent upon them has become contentious, with several plausible theories advocating different answers to these questions. In this review we will focus on three of these ideas-the economic value, credit assignment, and cognitive map hypotheses-describing both their successes and failures. We will propose that these failures hint at a more nuanced and perhaps unique role for the OFC, particularly the lateral subdivision, in supporting the proposed functions when an underlying model or map of the causal structures in the environment must be constructed or updated. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Parafascicular Thalamic and Orbitofrontal Cortical Inputs to Striatum Represent States for Goal-Directed Action Selection

Several lines of evidence accrued over the last 5–10 years have converged to suggest that the parafascicular nucleus of the thalamus and the lateral orbitofrontal cortex each represent or contribute to internal state/context representations that guide action selection in partially observable task situations. In rodents, inactivations of each structure have been found to selectively impair performance in paradigms testing goal-directed action selection, but only when that action selection relies on state representations. Electrophysiological evidence has suggested that each structure achieves this function via inputs onto cholinergic interneurons (CINs) in the dorsomedial striatum. Here, we briefly review these studies, then point to anatomical evidence regarding the afferents of each structure and what they suggest about the specific features that each contribute to internal state representations. Finally, we speculate as to whether this role might be achieved interdependently through direct PF→OFC projections, or through the convergence of independent direct orbitofrontal cortex (OFC) and parafascicular nucleus of the thalamus (PF) inputs onto striatal targets.

The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation

The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.

Coordinated Prefrontal State Transition Leads Extinction of Reward-Seeking Behaviors

Extinction learning suppresses conditioned reward responses and is thus fundamental to adapt to changing environmental demands and to control excessive reward seeking. The medial prefrontal cortex (mPFC) monitors and controls conditioned reward responses. Abrupt transitions in mPFC activity anticipate changes in conditioned responses to altered contingencies. It remains, however, unknown whether such transitions are driven by the extinction of old behavioral strategies or by the acquisition of new competing ones. Using in vivo multiple single-unit recordings of mPFC in male rats, we studied the relationship between single-unit and population dynamics during extinction learning, using alcohol as a positive reinforcer in an operant conditioning paradigm. To examine the fine temporal relation between neural activity and behavior, we developed a novel behavioral model that allowed us to identify the number, onset, and duration of extinction-learning episodes in the behavior of each animal. We found that single-unit responses to conditioned stimuli changed even under stable experimental conditions and behavior. However, when behavioral responses to task contingencies had to be updated, unit-specific modulations became coordinated across the whole population, pushing the network into a new stable attractor state. Thus, extinction learning is not associated with suppressed mPFC responses to conditioned stimuli, but is anticipated by single-unit coordination into population-wide transitions of the internal state of the animal.SIGNIFICANCE STATEMENT The ability to suppress conditioned behaviors when no longer beneficial is fundamental for the survival of any organism. While pharmacological and optogenetic interventions have shown a critical involvement of the mPFC in the suppression of conditioned responses, the neural dynamics underlying such a process are still largely unknown. Combining novel analysis tools to describe behavior, single-neuron response, and population activity, we found that widespread changes in neuronal firing temporally coordinate across the whole mPFC population in anticipation of behavioral extinction. This coordination leads to a global transition in the internal state of the network, driving extinction of conditioned behavior.

Orbitofrontal State Representations Are Related to Choice Adaptations and Reward Predictions

Animals can categorize the environment into "states," defined by unique sets of available action-outcome contingencies in different contexts. Doing so helps them choose appropriate actions and make accurate outcome predictions when in each given state. State maps have been hypothesized to be held in the orbitofrontal cortex (OFC), an area implicated in decision-making and encoding information about outcome predictions. Here we recorded neural activity in OFC in 6 male rats to test state representations. Rats were trained on an odor-guided choice task consisting of five trial blocks containing distinct sets of action-outcome contingencies, constituting states, with unsignaled transitions between them. OFC neural ensembles were analyzed using decoding algorithms. Results indicate that the vast majority of OFC neurons contributed to representations of the current state at any point in time, independent of odor cues and reward delivery, even at the level of individual neurons. Across state transitions, these representations gradually integrated evidence for the new state; the rate at which this integration happened in the prechoice part of the trial was related to how quickly the rats' choices adapted to the new state. Finally, OFC representations of outcome predictions, often thought to be the primary function of OFC, were dependent on the accuracy of OFC state representations.SIGNIFICANCE STATEMENT A prominent hypothesis proposes that orbitofrontal cortex (OFC) tracks current location in a "cognitive map" of state space. Here we tested this idea in detail by analyzing neural activity recorded in OFC of rats performing a task consisting of a series of states, each defined by a set of available action-outcome contingencies. Results show that most OFC neurons contribute to state representations and that these representations are related to the rats' decision-making and OFC reward predictions. These findings suggest new interpretations of emotional dysregulation in pathologies, such as addiction, which have long been known to be related to OFC dysfunction.

Make a Left Turn: Cortico-Striatal Circuitry Mediating the Attentional Control of Complex Movements

BACKGROUND

In movement disorders such as Parkinson's disease (PD), cholinergic signaling is disrupted by the loss of basal forebrain cholinergic neurons, as well as aberrant activity in striatal cholinergic interneurons (ChIs). Several lines of evidence suggest that gait imbalance, a key disabling symptom of PD, may be driven by alterations in high-level frontal cortical and cortico-striatal processing more typically associated with cognitive dysfunction.

METHODS

Here we describe the corticostriatal circuitry that mediates the cognitive-motor interactions underlying such complex movement control. The ability to navigate dynamic, obstacle-rich environments requires the continuous integration of information about the environment with movement selection and sequencing. The cortical-attentional processing of extero- and interoceptive cues requires modulation by cholinergic activity to guide striatal movement control. Cue-derived information is "transferred" to striatal circuitry primarily via fronto-striatal glutamatergic projections.

RESULT

Evidence from parkinsonian fallers and from a rodent model reproducing the dual cholinergic-dopaminergic losses observed in these patients supports the main hypotheses derived from this neuronal circuitry-guided conceptualization of parkinsonian falls. Furthermore, in the striatum, ChIs constitute a particularly critical node for the integration of cortical with midbrain dopaminergic afferents and thus for cues to control movements.

CONCLUSION

Procholinergic treatments that enhance or rescue cortical and striatal mechanisms may improve complex movement control in parkinsonian fallers and perhaps also in older persons suffering from gait disorders and a propensity for falls. © 2021 International Parkinson and Movement Disorder Society.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

Neural substrates of the interplay between cognitive load and emotional involvement in bilingual decision making

Prior work has reported that foreign language influences decision making by either reducing access to emotion or imposing additional cognitive demands. In this fMRI study, we employed a cross-task design to assess at the neural level whether and how the interaction between cognitive load and emotional involvement is affected by language (native L1 vs. foreign L2). Participants completed a Lexico-semantic task where in each trial they were presented with a neutrally or a negatively valenced word either in L1 or L2, either under cognitive load or not. We manipulated cognitive load by varying the difficulty of the task: to increase cognitive demands, we used traditional characters instead of simplified ones in L1 (Chinese), and words with capital letters instead of lowercase letters in L2 (English). After each trial, participants decided whether to take a risky decision in a gambling game. During the Gamling task, left amygdala and right insula were more activated after having processed a negative word under cognitive load in the Lexico-semantic task. However, this was true for L1 but not for L2. In particular, in L1, cognitive load facilitated rather than hindered access to emotion. Further suggesting that cognitive load can enhance emotional sensitivity in L1 but not in L2, we found that functional connectivity between reward-related striatum and right insula increased under cognitive load only in L1. Overall, results suggest that cognitive load in L1 can favor access to emotion and lead to impulsive decision making, whereas cognitive load in L2 can attenuate access to emotion and lead to more rational decisions.

The Hippocampus and Dorsolateral Striatum Integrate Distinct Types of Memories through Time and Space, Respectively

Several decades of research have established that different kinds of memories result from the activity of discrete neural networks. Studying how these networks process information in experiments that target specific types of mnemonic representations has provided deep insights into memory architecture and its neural underpinnings. However, in natural settings reality confronts organisms with problems that are not neatly compartmentalized. Thus, a critical problem in memory research that still needs to be addressed is how distinct types of memories are ultimately integrated. Here we demonstrate how two memory networks, the hippocampus and dorsolateral striatum, may accomplish such a goal. The hippocampus supports memory for facts and events, collectively known as declarative memory and often studied as spatial memory in rodents. The dorsolateral striatum provides the basis for habits that are assessed in stimulus-response types of tasks. Expanding previous findings, the current work revealed that in male Long-Evans rats, the hippocampus and dorsolateral striatum use time and space in distinct and largely complementary ways to integrate spatial and habitual representations. Specifically, the hippocampus supported both types of memories when they were formed in temporal juxtaposition, even if the learning took place in different environments. In contrast, the lateral striatum supported both types of memories if they were formed in the same environment, even at temporally distinct points. These results reveal for the first time that by using fundamental aspects of experience in specific ways, the hippocampus and dorsolateral striatum can transcend their attributed roles in information storage.SIGNIFICANCE STATEMENT The current paradigm in memory research postulates that different types of memories reflected in separate types of behavioral strategies result from activity in distinct neural circuits. However, recent data have shown that when rats concurrently acquired in the same environment of hippocampal-dependent spatial navigation and striatal-dependent approach of a visual cue, each of the two types of memories became dependent on both the hippocampus and dorsolateral striatum. The current work reveals that the hippocampus and dorsolateral striatum use distinct and complementary principles to integrate different types of memories in time and space: the hippocampus integrates memories formed in temporal proximity, while the lateral striatum integrates memories formed in the same space.

The orbitofrontal cortex: reward, emotion and depression

The orbitofrontal cortex in primates including humans is the key brain area in emotion, and in the representation of reward value and in non-reward, that is not obtaining an expected reward. Cortical processing before the orbitofrontal cortex is about the identity of stimuli, i.e. 'what' is present, and not about reward value. There is evidence that this holds for taste, visual, somatosensory and olfactory stimuli. The human medial orbitofrontal cortex represents many different types of reward, and the lateral orbitofrontal cortex represents non-reward and punishment. Not obtaining an expected reward can lead to sadness, and feeling depressed. The concept is advanced that an important brain region in depression is the orbitofrontal cortex, with depression related to over-responsiveness and over-connectedness of the non-reward-related lateral orbitofrontal cortex, and to under-responsiveness and under-connectivity of the reward-related medial orbitofrontal cortex. Evidence from large-scale voxel-level studies and supported by an activation study is described that provides support for this hypothesis. Increased functional connectivity of the lateral orbitofrontal cortex with brain areas that include the precuneus, posterior cingulate cortex and angular gyrus is found in patients with depression and is reduced towards the levels in controls when treated with medication. Decreased functional connectivity of the medial orbitofrontal cortex with medial temporal lobe areas involved in memory is found in patients with depression. Some treatments for depression may act by reducing activity or connectivity of the lateral orbitofrontal cortex. New treatments that increase the activity or connectivity of the medial orbitofrontal cortex may be useful for depression. These concepts, and that of increased activity in non-reward attractor networks, have potential for advancing our understanding and treatment of depression. The focus is on the orbitofrontal cortex in primates including humans, because of differences of operation of the orbitofrontal cortex, and indeed of reward systems, in rodents. Finally, the hypothesis is developed that the orbitofrontal cortex has a special role in emotion and decision-making in part because as a cortical area it can implement attractor networks useful in maintaining reward and emotional states online, and in decision-making.

Differential encoding of action selection by orbitofrontal and striatal population dynamics

Survival relies on the ability to flexibly choose between different actions according to varying environmental circumstances. Many lines of evidence indicate that action selection involves signaling in corticostriatal circuits, including the orbitofrontal cortex (OFC) and dorsomedial striatum (DMS). While choice-specific responses have been found in individual neurons from both areas, it is unclear whether populations of OFC or DMS neurons are better at encoding an animal's choice. To address this, we trained head-fixed mice to perform an auditory guided two-alternative choice task, which required moving a joystick forward or backward. We then used silicon microprobes to simultaneously measure the spiking activity of OFC and DMS ensembles, allowing us to directly compare population dynamics between these areas within the same animals. Consistent with previous literature, both areas contained neurons that were selective for specific stimulus-action associations. However, analysis of concurrently recorded ensemble activity revealed that the animal's trial-by-trial behavior could be decoded more accurately from DMS dynamics. These results reveal substantial regional differences in encoding action selection, suggesting that DMS neural dynamics are more specialized than OFC at representing an animal's choice of action.NEW & NOTEWORTHY While previous literature shows that both orbitofrontal cortex (OFC) and dorsomedial striatum (DMS) represent information relevant to selecting specific actions, few studies have directly compared neural signals between these areas. Here we compared OFC and DMS dynamics in mice performing a two-alternative choice task. We found that the animal's choice could be decoded more accurately from DMS population activity. This work provides among the first evidence that OFC and DMS differentially represent information about an animal's selected action.

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models

Schizophrenia is characterized by positive, negative and cognitive symptoms. All current antipsychotic treatments feature dopamine-receptor antagonism that is relatively effective at addressing the psychotic (positive) symptoms of schizophrenia. However, there is no clear evidence that these medications improve the negative or cognitive symptoms, which are the greatest predictors of functional outcomes. One of the most robust pathophysiological observations in patients with schizophrenia is increased subcortical dopamine neurotransmission, primarily in the associative striatum. This brain area has an important role in a range of cognitive processes. Dopamine is also known to play a major part in regulating a number of cognitive functions impaired in schizophrenia but much of this research has been focused on cortical dopamine. Emerging research highlights the strong influence subcortical dopamine has on a range of cognitive domains, including attention, reward learning, goal-directed action and decision-making. Nonetheless, the precise role of the associative striatum in the cognitive impairments observed in schizophrenia remains poorly understood, presenting an opportunity to revisit its contribution to schizophrenia. Without a better understanding of the mechanisms underlying cognitive dysfunction, treatment development remains at a standstill. For this reason, improved preclinical animal models are needed if we are to understand the complex relationship between subcortical dopamine and cognition. A range of new techniques are facillitating the discrete manipulation of dopaminergic neurotransmission and measurements of cognitive performance, which can be investigated using a variety of sensitive translatable tasks. This has the potential to aid the successful incorporation of recent clinical research to address the lack of treatment strategies for cognitive symptoms in schizophrenia. This review will give an overview on the current state of research focused on subcortical dopamine and cognition in the context of schizophrenia research. We also discuss future strategies and approaches aimed at improving the translational outcomes for the treatment of cognitive deficits in schizophrenia.

Psychomotor Slowing in Schizophrenia: Implications for Endophenotype and Biomarker Development

Motor abnormalities (e.g., dyskinesia, psychomotor slowing, neurological soft signs) are core features of schizophrenia that occur independent of drug treatment and are associated with the genetic vulnerability and pathophysiology for the illness. Among this list, psychomotor slowing in particular is one of the most consistently observed and robust findings in the field. Critically, psychomotor slowing may serve as a uniquely promising endophenotype and/or biomarker for schizophrenia considering it is frequently observed in those with genetic vulnerability for the illness, predicts transition in subjects at high-risk for the disorder, and is associated with symptoms and recovery in patients. The purpose of the present review is to provide an overview of the history of psychomotor slowing in psychosis, discuss its possible neural underpinnings, and review the current literature supporting slowing as a putative endophenotype and/or biomarker for the illness. This review summarizes substantial evidence from a diverse array of methodologies and research designs that supports the notion that psychomotor slowing not only reflects genetic vulnerability, but is also sensitive to disease processes and the pathophysiology of the illness. Furthermore, there are unique deficits across the cognitive (prefix "psycho") and motor execution (root word "motor") aspects of slowing, with cognitive processes such as planning and response selection being particularly affected. These findings suggest that psychomotor slowing may serve as a promising endophenotype and biomarker for schizophrenia that may prove useful for identifying individuals at greatest risk and tracking the course of the illness and recovery.

Striatal and hippocampal contributions to flexible navigation in rats and humans

The hippocampus has been firmly established as playing a crucial role in flexible navigation. Recent evidence suggests that dorsal striatum may also play an important role in such goal-directed behaviour in both rodents and humans. Across recent studies, activity in the caudate nucleus has been linked to forward planning and adaptation to changes in the environment. In particular, several human neuroimaging studies have found the caudate nucleus tracks information traditionally associated with that by the hippocampus. In this brief review, we examine this evidence and argue the dorsal striatum encodes the transition structure of the environment during flexible, goal-directed behaviour. We highlight that future research should explore the following: (1) Investigate neural responses during spatial navigation via a biophysically plausible framework explained by reinforcement learning models and (2) Observe the interaction between cortical areas and both the dorsal striatum and hippocampus during flexible navigation.

Meta-analysis of reward processing in major depressive disorder reveals distinct abnormalities within the reward circuit

Many neuroimaging studies have investigated reward processing dysfunction in major depressive disorder. These studies have led to the common idea that major depressive disorder is associated with blunted responses within the reward circuit, particularly in the ventral striatum. Yet, the link between major depressive disorder and reward-related responses in other regions remains inconclusive, thus limiting our understanding of the pathophysiology of major depressive disorder. To address this issue, we performed a coordinate-based meta-analysis of 41 whole-brain neuroimaging studies encompassing reward-related responses from a total of 794 patients with major depressive disorder and 803 healthy controls. Our findings argue against the common idea that major depressive disorder is primarily linked to deficits within the reward system. Instead, our results demonstrate that major depressive disorder is associated with opposing abnormalities in the reward circuit: hypo-responses in the ventral striatum and hyper-responses in the orbitofrontal cortex. The current findings suggest that dysregulated corticostriatal connectivity may underlie reward-processing abnormalities in major depressive disorder, providing an empirical foundation for a more refined understanding of abnormalities in the reward circuitry in major depressive disorder.

Prefrontal cortex and cognitive control: new insights from human electrophysiology

Cognitive control, the ability to regulate one's cognition and actions on the basis of over-riding goals, is impaired in many psychiatric conditions. Although control requires the coordinated function of several prefrontal cortical regions, it has been challenging to determine how they work together, in part because doing so requires simultaneous recordings from multiple regions. Here, we provide a précis of cognitive control and describe the beneficial consequences of recent advances in neurosurgical practice that make large-scale prefrontal cortical network recordings possible in humans. Such recordings implicate inter-regional theta (5-8 Hz) local field potential (LFP) synchrony as a key element in cognitive control. Major open questions include how theta might influence other oscillations within these networks, the precise timing of information flow between these regions, and how perturbations such as brain stimulation might demonstrate the causal role of LFP phenomena. We propose that an increased focus on human electrophysiology is essential for an understanding of the neural basis of cognitive control.