Prefrontal Regulation of Neuronal Activity in the Ventral Tegmental Area

Orbitofrontal cortex modulates auditory cortical sensitivity and sound perception in Mongolian gerbils

Sensory perception is dynamic, quickly adapting to sudden shifts in environmental or behavioral context. Although decades of work have established that these dynamics are mediated by rapid fluctuations in sensory cortical activity, we have a limited understanding of the brain regions and pathways that orchestrate these changes. Neurons in the orbitofrontal cortex (OFC) encode contextual information, and recent data suggest that some of these signals are transmitted to sensory cortices. Whether and how these signals shape sensory encoding and perceptual sensitivity remain uncertain. Here, we asked whether the OFC mediates context-dependent changes in auditory cortical sensitivity and sound perception by monitoring and manipulating OFC activity in freely moving Mongolian gerbils of both sexes under two behavioral contexts: passive sound exposure and engagement in an amplitude modulation (AM) detection task. We found that the majority of OFC neurons, including the specific subset that innervates the auditory cortex, were strongly modulated by task engagement. Pharmacological inactivation of the OFC prevented rapid context-dependent changes in auditory cortical firing and significantly impaired behavioral AM detection. Our findings suggest that contextual information from the OFC mediates rapid plasticity in the auditory cortex and facilitates the perception of behaviorally relevant sounds.

Decoupling Dopamine Synthesis from Impulsive Action, Risk-Related Decision-Making, and Propensity to Cocaine Intake: A Longitudinal [18F]-FDOPA PET Study in Roman High- and Low-Avoidance Rats

Impulsive action and risk-related decision-making (RDM) are two facets of impulsivity linked to a hyperdopaminergic release in the striatum and an increased propensity to cocaine intake. We previously showed that with repeated cocaine exposure, this initial hyperdopaminergic release is blunted in impulsive animals, potentially signaling drug-induced tolerance. Whether such dopaminergic dynamics involve changes in dopamine (DA) synthesis as a function of impulsivity is currently unknown. Here, we investigated the predictive value of DA synthesis for impulsive action, RDM, and the propensity to take cocaine in a rat model of vulnerability to cocaine abuse. Additionally, we assessed the effects of cocaine intake on these variables. Rats were tested sequentially in the rat Gambling Task (rGT) and were scanned with positron emission tomography and [18F]-FDOPA to respectively assess both impulsivity facets and striatal DA synthesis before and after cocaine self-administration (SA). Our results revealed that baseline striatal levels of DA synthesis did not significantly predict impulsive action, RDM, or a greater propensity to cocaine SA in impulsive animals. Besides, we showed that impulsive action, but not RDM, predicted higher rates of cocaine taking. However, chronic cocaine exposure had no impact on DA synthesis, nor affected impulsive action and RDM. These findings indicate that the hyper-responsive DA system associated with impulsivity and a propensity for cocaine consumption, along with the reduction in this hyper-responsive DA state in impulsive animals with a history of cocaine use, might not be mediated by dynamic changes in DA synthesis.

Orbitofrontal Cortex Modulates Auditory Cortical Sensitivity and Sound Perception

Sensory perception is dynamic, quickly adapting to sudden shifts in environmental or behavioral context. Though decades of work have established that these dynamics are mediated by rapid fluctuations in sensory cortical activity, we have a limited understanding of the brain regions and pathways that orchestrate these changes. Neurons in the orbitofrontal cortex (OFC) encode contextual information, and recent data suggest that some of these signals are transmitted to sensory cortices. Whether and how these signals shape sensory encoding and perceptual sensitivity remains uncertain. Here, we asked whether the OFC mediates context-dependent changes in auditory cortical sensitivity and sound perception by monitoring and manipulating OFC activity in freely moving animals under two behavioral contexts: passive sound exposure and engagement in an amplitude modulation (AM) detection task. We found that the majority of OFC neurons, including the specific subset that innervate the auditory cortex, were strongly modulated by task engagement. Pharmacological inactivation of the OFC prevented rapid context-dependent changes in auditory cortical firing, and significantly impaired behavioral AM detection. Our findings suggest that contextual information from the OFC mediates rapid plasticity in the auditory cortex and facilitates the perception of behaviorally relevant sounds.

Significance Statement

Sensory perception depends on the context in which stimuli are presented. For example, perception is enhanced when stimuli are informative, such as when they are important to solve a task. Perceptual enhancements result from an increase in the sensitivity of sensory cortical neurons; however, we do not fully understand how such changes are initiated in the brain. Here, we tested the role of the orbitofrontal cortex (OFC) in controlling auditory cortical sensitivity and sound perception. We found that OFC neurons change their activity when animals perform a sound detection task. Inactivating OFC impairs sound detection and prevents task-dependent increases in auditory cortical sensitivity. Our findings suggest that the OFC controls contextual modulations of the auditory cortex and sound perception.

Chronic Voluntary Morphine Intake Is Associated with Changes in Brain Structures Involved in Drug Dependence in a Rat Model of Polydrug Use

Chronic opioid intake leads to several brain changes involved in the development of dependence, whereby an early hedonistic effect (liking) extends to the need to self-administer the drug (wanting), the latter being mostly a prefrontal-striatal function. The development of animal models for voluntary oral opioid intake represents an important tool for identifying the cellular and molecular alterations induced by chronic opioid use. Studies mainly in humans have shown that polydrug use and drug dependence are shared across various substances. We hypothesize that an animal bred for its alcohol preference would develop opioid dependence and further that this would be associated with the overt cortical abnormalities clinically described for opioid addicts. We show that Wistar-derived outbred UChB rats selected for their high alcohol preference additionally develop: (i) a preference for oral ingestion of morphine over water, resulting in morphine intake of 15 mg/kg/day; (ii) marked opioid dependence, as evidenced by the generation of strong withdrawal signs upon naloxone administration; (iii) prefrontal cortex alterations known to be associated with the loss of control over drug intake, namely, demyelination, axonal degeneration, and a reduction in glutamate transporter GLT-1 levels; and (iv) glial striatal neuroinflammation and brain oxidative stress, as previously reported for chronic alcohol and chronic nicotine use. These findings underline the relevance of polydrug animal models and their potential in the study of the wide spectrum of brain alterations induced by chronic morphine intake. This study should be valuable for future evaluations of therapeutic approaches for this devastating condition.

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

Dopaminergic prediction errors in the ventral tegmental area reflect a multithreaded predictive model

Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can explain error signals observed in many studies, it cannot cope with settings where subjects might infer multiple independent events and outcomes. In the present study, we recorded dopamine neurons in the ventral tegmental area in such a setting to test the validity of the single-stream assumption. Rats were trained in an odor-based choice task, in which the timing and identity of one of several rewards delivered in each trial changed across trial blocks. This design revealed an error signaling pattern that requires the dopamine neurons to access and update multiple independent predictive streams reflecting the subject's belief about timing and potentially unique identities of expected rewards.

Analysis of individual differences in neurofeedback training illuminates successful self-regulation of the dopaminergic midbrain

The dopaminergic midbrain is associated with reinforcement learning, motivation and decision-making – functions often disturbed in neuropsychiatric disorders. Previous research has shown that dopaminergic midbrain activity can be endogenously modulated via neurofeedback. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examine whether the activation of particular brain regions associates with successful regulation transfer when feedback is no longer available. Moreover, to elucidate mechanisms underlying effective self-regulation, we study the relation of successful transfer with learning (temporal difference coding) outside the midbrain during neurofeedback training and with individual reward sensitivity in a monetary incentive delay (MID) task. Fifty-nine participants underwent neurofeedback training either in standard (Study 1 N = 15, Study 2 N = 28) or control feedback group (Study 1, N = 16). We find that successful self-regulation is associated with prefrontal reward sensitivity in the MID task (N = 25), with a decreasing relation between prefrontal activity and midbrain learning signals during neurofeedback training and with increased activity within cognitive control areas during transfer. The association between midbrain self-regulation and prefrontal temporal difference and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings provide insights in the control of midbrain activity and may facilitate individually tailoring neurofeedback training.

Analysis of real-time fMRI data from 59 participants undergoing neurofeedback training suggests that reinforcement learning contributes to successful self-regulation in the dopaminergic midbrain.

Dopamine signals as temporal difference errors: recent advances

In the brain, dopamine is thought to drive reward-based learning by signaling temporal difference reward prediction errors (TD errors), a 'teaching signal' used to train computers. Recent studies using optogenetic manipulations have provided multiple pieces of evidence supporting that phasic dopamine signals function as TD errors. Furthermore, novel experimental results have indicated that when the current state of the environment is uncertain, dopamine neurons compute TD errors using 'belief states' or a probability distribution over potential states. It remains unclear how belief states are computed but emerging evidence suggests involvement of the prefrontal cortex and the hippocampus. These results refine our understanding of the role of dopamine in learning and the algorithms by which dopamine functions in the brain.

Orbitofrontal control of visual cortex gain promotes visual associative learning

The orbitofrontal cortex (OFC) encodes expected outcomes and plays a critical role in flexible, outcome-guided behavior. The OFC projects to primary visual cortex (V1), yet the function of this top-down projection is unclear. We find that optogenetic activation of OFC projection to V1 reduces the amplitude of V1 visual responses via the recruitment of local somatostatin-expressing (SST) interneurons. Using mice performing a Go/No-Go visual task, we show that the OFC projection to V1 mediates the outcome-expectancy modulation of V1 responses to the reward-irrelevant No-Go stimulus. Furthermore, V1-projecting OFC neurons reduce firing during expectation of reward. In addition, chronic optogenetic inactivation of OFC projection to V1 impairs, whereas chronic activation of SST interneurons in V1 improves the learning of Go/No-Go visual task, without affecting the immediate performance. Thus, OFC top-down projection to V1 is crucial to drive visual associative learning by modulating the response gain of V1 neurons to non-relevant stimulus.

The orbitofrontal cortex (OFC) encodes expected outcomes and plays a key role in outcome-guided behavior. The authors show here that the top-down projection from the OFC to the visual cortex drives visual associative learning by modulating the response gain of V1 neurons to non-relevant stimuli.

Functional differentiation in the human ventromedial frontal lobe: A data-driven parcellation

Ventromedial regions of the frontal lobe (vmFL) are thought to play a key role in decision-making and emotional regulation. However, aspects of this area's functional organization, including the presence of a multiple subregions, their functional and anatomical connectivity, and the cross-species homologies of these subregions with those of other species, remain poorly understood. To address this uncertainty, we employed a two-stage parcellation of the region to identify six distinct structures within the region on the basis of data-driven classification of functional connectivity patterns obtained using the meta-analytic connectivity modeling (MACM) approach. From anterior to posterior, the derived subregions included two lateralized posterior regions, an intermediate posterior region, a dorsal and ventral central region, and a single anterior region. The regions were characterized further by functional connectivity derived using resting-state fMRI and functional decoding using the Brain Map database. In general, the regions could be differentiated on the basis of different patterns of functional connectivity with canonical "default mode network" regions and/or subcortical regions such as the striatum. Together, the findings suggest the presence of functionally distinct neural structures within vmFL, consistent with data from experimental animals as well prior demonstrations of anatomical differences within the region. Detailed correspondence with the anterior cingulate, medial orbitofrontal cortex, and rostroventral prefrontal cortex, as well as specific animal homologs are discussed. The findings may suggest future directions for resolving potential functional and structural correspondence of subregions within the frontal lobe across behavioral contexts, and across mammalian species.

Synergy of Distinct Dopamine Projection Populations in Behavioral Reinforcement

Dopamine neurons of the ventral tegmental area (VTA) regulate reward association and motivation. It remains unclear whether there are distinct dopamine populations to mediate these functions. Using mouse genetics, we isolated two populations of dopamine-producing VTA neurons with divergent projections to the nucleus accumbens (NAc) core and shell. Inhibition of VTA-core-projecting neurons disrupted Pavlovian reward learning, and activation of these cells promoted the acquisition of an instrumental response. VTA-shell-projecting neurons did not regulate Pavlovian reward learning and could not facilitate acquisition of an instrumental response, but their activation could drive robust responding in a previously learned instrumental task. Both populations are activated simultaneously by cues, actions, and rewards, and this co-activation is required for robust reinforcement of behavior. Thus, there are functionally distinct dopamine populations in the VTA for promoting motivation and reward association, which operate on the same timescale to optimize behavioral reinforcement.

Expectancy-Related Changes in Dopaminergic Error Signals Are Impaired by Cocaine Self-Administration

Addiction is a disorder of behavioral control and learning. While this may reflect pre-existing propensities, drug use also clearly contributes by causing changes in outcome processing in prefrontal and striatal regions. This altered processing is associated with behavioral deficits, including changes in learning. These areas provide critical input to midbrain dopamine neurons regarding expected outcomes, suggesting that effects on learning may result from changes in dopaminergic error signaling. Here, we show that dopamine neurons recorded in rats that had self-administered cocaine failed to suppress firing on omission of an expected reward and exhibited lower amplitude and imprecisely timed increases in firing to an unexpected reward. Learning also appeared to have less of an effect on reward-evoked and cue-evoked firing in the cocaine-experienced rats. Overall, the changes are consistent with reduced fidelity of input regarding the expected outcomes, such as their size, timing, and overall value, because of cocaine use.

Single-cell activity tracking reveals that orbitofrontal neurons acquire and maintain a long-term memory to guide behavioral adaptation

Learning to predict rewards based on environmental cues is essential for survival. The orbitofrontal cortex (OFC) contributes to such learning by conveying reward-related information to brain areas such as the ventral tegmental area (VTA). Despite this, how cue-reward memory representations form in individual OFC neurons and are modified based on new information is unknown. To address this, using in vivo two-photon calcium imaging in mice, we tracked the response evolution of thousands of OFC output neurons, including those projecting to VTA, through multiple days and stages of cue-reward learning. Collectively, we show that OFC contains several functional clusters of neurons distinctly encoding cue-reward memory representations, with only select responses routed downstream to VTA. Unexpectedly, these representations were stably maintained by the same neurons even after extinction of the cue-reward pairing, and supported behavioral learning and memory. Thus, OFC neuronal activity represents a long-term cue-reward associative memory to support behavioral adaptation.

Sensory prediction errors in the human midbrain signal identity violations independent of perceptual distance

The firing of dopaminergic midbrain neurons is thought to reflect prediction errors (PE) that depend on the difference between the value of expected and received rewards. However, recent work has demonstrated that unexpected changes in value-neutral outcome features, such as identity, can evoke similar responses. It remains unclear whether the magnitude of these identity PEs scales with the perceptual dissimilarity of expected and received rewards, or whether they are independent of perceptual similarity. We used a Pavlovian transreinforcer reversal task to elicit identity PEs for value-matched food odor rewards, drawn from two perceptual categories (sweet, savory). Replicating previous findings, identity PEs were correlated with fMRI activity in midbrain, OFC, piriform cortex, and amygdala. However, the magnitude of identity PE responses was independent of the perceptual distance between expected and received outcomes, suggesting that identity comparisons underlying sensory PEs may occur in an abstract state space independent of straightforward sensory percepts.

Dopamine Neurons Reflect the Uncertainty in Fear Generalization

Generalized fear is a maladaptive behavior in which non-threatening stimuli elicit a fearful response. Here, we demonstrate that discrimination between predictive and non-predictive threat stimuli is highly sensitive to probabilistic discounting and increasing threat intensity in mice. We find that dopamine neurons of the ventral tegmental area (VTA) encode both the negative valence of threat-predictive cues and the certainty of threat prediction. As fear generalization emerges, the dopamine neurons that are activated by a threat predictive cue (CS+) decrease the amplitude of activation and an equivalent signal emerges to a non-predictive cue (CS-). Temporally precise enhancement of dopamine neurons during threat conditioning to high threat levels or uncertain threats can prevent generalization. Moreover, phasic enhancement of genetically captured dopamine neurons activated by threat cues can reverse fear generalization. These findings demonstrate the dopamine neurons reflect the certainty of threat prediction that can be used to inform and update the fear engram.

Impaired recruitment of dopamine neurons during working memory in mice with striatal D2 receptor overexpression

The dopamine (DA) system plays a major role in cognitive functions through its interactions with several brain regions including the prefrontal cortex (PFC). Conversely, disturbances in the DA system contribute to cognitive deficits in psychiatric diseases, yet exactly how they do so remains poorly understood. Here we show, using mice with disease-relevant alterations in DA signaling (D2R-OE mice), that deficits in working memory (WM) are associated with impairments in the WM-dependent firing patterns of DA neurons in the ventral tegmental area (VTA). The WM-dependent phase-locking of DA neurons to 4 Hz VTA-PFC oscillations is absent in D2R-OE mice and VTA-PFC synchrony deficits scale with their WM impairments. We also find reduced 4 Hz synchrony between VTA DA neurons and selective impairments in their representation of WM demand. These results identify how altered DA neuron activity—at the level of long-range network activity and task-related firing patterns—may underlie cognitive impairments.

Disrupted dopamine neuron firing is thought to contribute to cognitive dysfunction in psychiatric disorders. Here the authors show that mice overexpressing D2R in the striatum, commonly seen in schizophrenia, are also impaired in recruitment of dopamine neurons during working memory performance.

The Medial Prefrontal Cortex Shapes Dopamine Reward Prediction Errors under State Uncertainty

Animals make predictions based on currently available information. In natural settings, sensory cues may not reveal complete information, requiring the animal to infer the "hidden state" of the environment. The brain structures important in hidden state inference remain unknown. A previous study showed that midbrain dopamine neurons exhibit distinct response patterns depending on whether reward is delivered in 100% (task 1) or 90% of trials (task 2) in a classical conditioning task. Here we found that inactivation of the medial prefrontal cortex (mPFC) affected dopaminergic signaling in task 2, in which the hidden state must be inferred ("will reward come or not?"), but not in task 1, where the state was known with certainty. Computational modeling suggests that the effects of inactivation are best explained by a circuit in which the mPFC conveys inference over hidden states to the dopamine system. VIDEO ABSTRACT.

Depression: the search for separable behaviors and circuits

Major depressive disorder can manifest as different combinations of symptoms, ranging from a profound and incapacitating sadness, to a loss of interest in daily life, to an inability to engage in effortful, goal-directed behavior. Recent research has focused on defining the neural circuits that mediate separable features of depression in patients and preclinical animal models, and connections between frontal cortex and brainstem neuromodulators have emerged as candidate targets. The development of methods permitting recording and manipulation of neural circuits defined by connectivity has enabled the investigation of prefrontal-neuromodulatory circuit dynamics in animal models of depression with exquisite precision, a systems-level approach that has brought new insights by integrating these fields of depression research.

Effects of inference on dopaminergic prediction errors depend on orbitofrontal processing

Dopaminergic reward prediction errors in monkeys reflect inferential reward predictions that well-trained animals can make when associative rules change. Here, in a new analysis of previously described data, we test whether dopaminergic error signals in rats are influenced by inferential predictions and whether such effects depend on the orbitofrontal cortex (OFC). Dopamine neurons were recorded from controls or rats with ipsilateral OFC lesions during performance of a choice task in which odor cues signaled the availability of sucrose reward in 2 wells. To induce prediction errors, we manipulated either the timing or number of rewards delivered in each well across blocks of trials. Of importance, a change in reward at 1 well predicted a change in reward at the other on later trials. We compared behavior and neural activity on trials when such inference was possible versus trials involving the same reward change when inference was not possible. Rats responded faster when they could infer an increase in reward compared to when the same reward was coming but they could not infer a change. This inferential prediction was reflected in the firing of dopamine neurons in controls, which changed less to unexpected delivery (or omission) of reward and more to the new high-value cue on inference versus noninference trials. These effects were absent in dopamine neurons recorded in rats with ipsilateral OFC lesions. Thus, dopaminergic error signals recorded in rats are influenced by both experiential and inferential reward predictions, and the effects of inferential predictions depend on OFC. (PsycINFO Database Record

Roundabout receptor 2 maintains inhibitory control of the adult midbrain

The maintenance of excitatory and inhibitory balance in the brain is essential for its function. Here we find that the developmental axon guidance receptor Roundabout 2 (Robo2) is critical for the maintenance of inhibitory synapses in the adult ventral tegmental area (VTA), a brain region important for the production of the neurotransmitter dopamine. Following selective genetic inactivation of Robo2 in the adult VTA of mice, reduced inhibitory control results in altered neural activity patterns, enhanced phasic dopamine release, behavioral hyperactivity, associative learning deficits, and a paradoxical inversion of psychostimulant responses. These behavioral phenotypes could be phenocopied by selective inactivation of synaptic transmission from local GABAergic neurons of the VTA, demonstrating an important function for Robo2 in regulating the excitatory and inhibitory balance of the adult brain.

DOI:http://dx.doi.org/10.7554/eLife.23858.001

Although no two people are alike, we all share the same basic brain structure. This similarity arises because the same developmental program takes place in every human embryo. Specific genes are activated in a designated sequence to generate the structure of a typical human brain. But what happens to these genes when development is complete – do they remain active in the adult brain?

A gene known as Robo2 encodes a protein that helps neurons find their way through the developing brain. Many of these neurons will ultimately form part of the brain’s reward system. This is a network of brain regions that communicate with one another using a chemical called dopamine. The reward system contributes to motivation, learning and memory, and also underlies drug addiction. In certain mental illnesses such as Parkinson’s disease and schizophrenia, the dopamine-producing neurons in the reward system work incorrectly or die.

To find out whether Robo2 is active in the mature nervous system, Gore et al. used genetic techniques to selectively remove the gene from the reward system of adult mice. Doing so reduced the ability of the dopamine neurons within the reward system to inhibit one another, which in turn increased their activity. This changed the behavior of the mice, making them hyperactive and less able to learn and remember. Cocaine makes normal mice more active; however, mice that lacked the Robo2 gene became less active when given cocaine.

Overall, the work of Gore et al. suggests that developmental axon guidance genes remain important in the adult brain. Studying developmental genes such as Robo2 may therefore open up new treatment possibilities for a number of mental illnesses and brain disorders.

DOI:http://dx.doi.org/10.7554/eLife.23858.002

Dopamine reward prediction errors reflect hidden-state inference across time

Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a ‘belief state’). In this work, we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling exhibited a striking difference between two tasks that differed only with respect to whether reward was delivered deterministically. Our results favor an associative learning rule that combines cached values with hidden state inference.

Analysis of morphine-induced changes in the activity of periaqueductal gray neurons in the intact rat

Microinjection of morphine into the periaqueductal gray (PAG) produces antinociception. In vitro slice recordings indicate that all PAG neurons are sensitive to morphine either by direct inhibition or indirect disinhibition. We tested the hypothesis that all PAG neurons respond to opioids in vivo by examining the extracellular activity of PAG neurons recorded in lightly anesthetized and awake rats. Spontaneous activity was less than 1Hz in most neurons. Noxious stimuli (heat, pinch) caused an increase in activity in 57% and 75% of the neurons recorded in anesthetized and awake rats, respectively. The same noxious stimuli caused a decrease in activity in only 17% and 6% of neurons recorded in anesthetized and awake rats. Systemic administration of morphine caused approximately equal numbers of neurons to increase, decrease, or show no change in activity in lightly anesthetized rats. In contrast, administration of morphine caused an increase in the activity of 22 of the 27 neurons recorded in awake rats. No change in activity was evident in the remaining five neurons. Changes in activity caused by morphine were surprisingly modest (a median increase from 0.7 to 1.3Hz). The small inconsistent effects of morphine are in stark contrast to the large changes produced by morphine on the activity of rostral ventromedial medulla (RVM) neurons or the widespread inhibition and excitation of PAG neurons treated with opioids in in vitro slice experiments. The relatively modest effects of morphine in the present study suggest that morphine produces antinociception by causing small changes in the activity of many PAG neurons.

Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex

The hippocampus and the orbitofrontal cortex (OFC) both have important roles in cognitive processes such as learning, memory and decision making. Nevertheless, research on the OFC and hippocampus has proceeded largely independently, and little consideration has been given to the importance of interactions between these structures. Here, evidence is reviewed that the hippocampus and OFC encode parallel, but interactive, cognitive 'maps' that capture complex relationships between cues, actions, outcomes and other features of the environment. A better understanding of the interactions between the OFC and hippocampus is important for understanding the neural bases of flexible, goal-directed decision making.

Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning

Different subregions of the prefrontal cortex (PFC) contribute to the ability to respond flexibly to changes in reward contingencies, with the medial versus orbitofrontal cortex (OFC) subregions contributing differentially to processes such as set-shifting and reversal learning. To date, the manner in which these regions may facilitate reversal learning in situations involving reward uncertainty remains relatively unexplored. We investigated the involvement of five distinct regions of the rat OFC (lateral and medial) and medial PFC (prelimbic, infralimbic, and anterior cingulate) on probabilistic reversal learning wherein "correct" versus "incorrect" responses were rewarded on 80% and 20% of trials, respectively. Contingencies were reversed repeatedly within a session. In well trained rats, inactivation of the medial or lateral OFC induced dissociable impairments in performance (indexed by fewer reversals completed) when outcomes were probabilistic, but not when they were assured. Medial OFC inactivation impaired probabilistic learning during the first discrimination, increased perseverative responding and reduced sensitivity to positive and negative feedback, suggestive of a deficit in incorporating information about previous action outcomes to guide subsequent behavior. Lateral OFC inactivation preferentially impaired performance during reversal phases. In contrast, prelimbic inactivation caused an apparent improvement in performance by increasing the number of reversals completed. This was associated with enhanced sensitivity to recently rewarded actions and reduced sensitivity to negative feedback. Infralimbic inactivation had no effect, whereas the anterior cingulate appeared to play a permissive role in this form of reversal learning. These results clarify the dissociable contributions of different regions of the frontal lobes to probabilistic learning.

SIGNIFICANCE STATEMENT

The ability to adjust behavior in response to changes involving uncertain or probabilistic reward contingencies is an essential survival skill that is impaired in a variety of psychiatric disorders. It is well established that different forms of cognitive flexibility are mediated by anatomically distinct regions of the frontal lobes when reinforcement contingencies are assured, however, less is known about the contribution of these regions to probabilistic reinforcement learning. Here we show that different regions of the orbitofrontal and medial prefrontal cortex make distinct contributions to probabilistic reversal learning. These findings provide novel information about the complex interplay between frontal lobe regions in mediating these processes and accordingly provide insight into possible pathophysiology that underlies impairments in cognitive flexibility observed in mental illnesses.

Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum

Dopamine neurons signal reward prediction errors. This requires accurate reward predictions. It has been suggested that the ventral striatum provides these predictions. Here we tested this hypothesis by recording from putative dopamine neurons in the VTA of rats performing a task in which prediction errors were induced by shifting reward timing or number. In controls, the neurons exhibited error signals in response to both manipulations. However, dopamine neurons in rats with ipsilateral ventral striatal lesions exhibited errors only to changes in number and failed to respond to changes in timing of reward. These results, supported by computational modeling, indicate that predictions about the temporal specificity and the number of expected reward are dissociable and that dopaminergic prediction-error signals rely on the ventral striatum for the former but not the latter.

Human ventromedial prefrontal lesions alter incentivisation by reward

Although medial frontal brain regions are implicated in valuation of rewards, evidence from focal lesions to these areas is scant, with many conflicting results regarding motivation and affect, and no human studies specifically examining incentivisation by reward. Here, 19 patients with isolated, focal damage in ventral and medial prefrontal cortex were selected from a database of 453 individuals with subarachnoid haemorrhage. Using a speeded saccadic task based on the oculomotor capture paradigm, we manipulated the maximum reward available on each trial using an auditory incentive cue. Modulation of behaviour by motivation permitted quantification of reward sensitivity. At the group level, medial frontal damage was overall associated with significantly reduced effects of reward on invigorating saccadic velocity and autonomic (pupil) responses compared to age-matched, healthy controls. Crucially, however, some individuals instead showed abnormally strong incentivisation effects for vigour. Increased sensitivity to rewards within the lesion group correlated with damage in subgenual ventromedial prefrontal cortex (vmPFC) areas, which have recently become the target for deep brain stimulation (DBS) in depression. Lesion correlations with clinical apathy suggested that the apathy associated with prefrontal damage is in fact reduced by damage at those coordinates. Reduced reward sensitivity showed a trend to correlate with damage near nucleus accumbens. Lesions did not, on the other hand, influence reward sensitivity of cognitive control, as measured by distractibility. Thus, although medial frontal lesions may generally reduce reward sensitivity, damage to key subregions paradoxically protect from this effect.