Tonic or Phasic Stimulation of Dopaminergic Projections to Prefrontal Cortex Causes Mice to Maintain or Deviate from Previously Learned Behavioral Strategies

Context-Dependent Multiplexing by Individual VTA Dopamine Neurons

Dopamine (DA) neurons of the VTA track cues and rewards to generate a reward prediction error signal during Pavlovian conditioning. Here we explored how these neurons respond to a self-paced, operant task in freely moving mice. The animal could trigger a reward-predicting cue by remaining in a specific location of an operant box for a brief time before moving to a spout for reward collection. VTA DA neurons were identified using DAT-Cre male mice that carried an optrode with minimal impact on the behavioral task. In vivo single-unit recordings revealed transient fast spiking responses to the cue and reward in correct trials, while for incorrect ones the activity paused, reflecting positive and negative error signals of a reward prediction. In parallel, a majority of VTA DA neurons simultaneously encoded multiple actions (e.g., movement velocity, acceleration, distance to goal, and licking) in sustained slow firing modulation. Applying a GLM, we show that such multiplexed encoding of rewarding and motor variables by individual DA neurons was only apparent while the mouse was engaged in the task. Downstream targets may exploit such goal-directed multiplexing of VTA DA neurons to adjust actions to optimize the task's outcome.SIGNIFICANCE STATEMENT VTA DA neurons code for multiple functions, including the reward prediction error but also motivation and locomotion. Here we show that about half of the recorded VTA DA neurons perform multiplexing: they exploit the phasic and tonic activity modes to encode, respectively, the cue/reward responses and motor parameters, most prominently when the mouse engages in a self-paced operand task. VTA non-DA neurons, by contrast, encode motor parameters regardless of task engagement.

Cross-hemispheric gamma synchrony between prefrontal parvalbumin interneurons supports behavioral adaptation during rule shift learning

Organisms must learn new strategies to adapt to changing environments. Activity in different neurons often exhibits synchronization that can dynamically enhance their communication and might create flexible brain states that facilitate changes in behavior. We studied the role of gamma-frequency (~40 Hz) synchrony between prefrontal parvalbumin (PV) interneurons in mice learning multiple new cue-reward associations. Voltage indicators revealed cell-type-specific increases of cross-hemispheric gamma synchrony between PV interneurons when mice received feedback that previously learned associations were no longer valid. Disrupting this synchronization by delivering out-of-phase optogenetic stimulation caused mice to perseverate on outdated associations, an effect not reproduced by in-phase stimulation or out-of-phase stimulation at other frequencies. Gamma synchrony was specifically required when new associations used familiar cues that were previously irrelevant to behavioral outcomes, not when associations involved new cues or for reversing previously learned associations. Thus, gamma synchrony is indispensable for reappraising the behavioral salience of external cues.

Dopamine Differentially Regulates Response Dynamics of Prefrontal Cortical Principal Neurons and Interneurons to Optogenetic Stimulation of Inputs from Ventral Tegmental Area

Prefrontal cortex (PFC) is highly influenced by the inputs from ventral tegmental area (VTA); however, how the projection from VTA impacts PFC neurons and how the synaptically released dopamine affects PFC activity are largely unclear. Using optogenetics and electrophysiological approaches, we examined the impact of VTA stimulation on PFC principal neurons and parvalbumin-positive (PV+) interneurons and the modulatory role of dopamine. We found that the brief activation of the VTA-PFC circuit immediately induced action potential firing, which was mediated by glutamatergic transmission. However, strong stimulation of VTA gradually induced a marked and prolonged enhancement of the excitability of PFC PV+ interneurons and a modest and short-lived enhancement of the excitability of PFC principal neurons. Blocking dopamine receptors (DARs) shortened the VTA excitation of PFC PV+ interneurons and prolonged the VTA excitation of PFC principal neurons. Blocking GABAA receptors induced a similar effect as DAR antagonists in PFC principal neurons, suggesting that the dopaminergic effect is through influencing the inhibitory transmission system. These results have revealed a role of dopamine in regulating the temporal dynamics of excitation/inhibition balance in VTA-PFC circuit, which provides insights into the functional consequence of activating dopamine system in the mesocortical system.

Unanticipated Stressful and Rewarding Experiences Engage the Same Prefrontal Cortex and Ventral Tegmental Area Neuronal Populations

Brain networks that mediate motivated behavior in the context of aversive and rewarding experiences involve the prefrontal cortex (PFC) and ventral tegmental area (VTA). Neurons in both regions are activated by stress and reward, and by learned cues that predict aversive or appetitive outcomes. Recent studies have proposed that separate neuronal populations and circuits in these regions encode learned aversive versus appetitive contexts. But how about the actual experience? Do the same or different PFC and VTA neurons encode unanticipated aversive and appetitive experiences? To address this, we recorded unit activity and local field potentials (LFPs) in the dorsomedial PFC (dmPFC) and VTA of male rats as they were exposed, in the same recording session, to reward (sucrose) or stress (tail pinch) spaced 1 h apart. As expected, experience-specific neuronal responses were observed. Approximately 15–25% of single units in each region responded by excitation or inhibition to either stress or reward, and only stress increased LFP theta oscillation power in both regions and coherence between regions. But the largest number of responses (29% dmPFC and 30% VTA units) involved dual-valence neurons that responded to both stress and reward exposure. Moreover, the temporal profile of neuronal population activity in dmPFC and VTA as assessed by principal component analysis (PCA) were similar during both types of experiences. These results reveal that aversive and rewarding experiences engage overlapping neuronal populations in the dmPFC and the VTA. These populations may provide a locus of vulnerability for stress-related disorders, which are often associated with anhedonia.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times

Dopamine (DA) neurons are thought to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction. We applied deep learning to classify mouse freezing behavior, eliminating the need for human scoring. Our fiber photometry recordings showed DA neurons in medial and lateral VTA have distinct activity profiles during fear extinction: medial VTA activity more closely reflected RPE, while lateral VTA activity more closely reflected a salience-like signal. Optogenetic inhibition of DA neurons in either region slowed fear extinction, with the relevant time period for inhibition differing across regions. Our results indicate salience-like signals can have similar downstream consequences to RPE-like signals, although with different temporal dependencies.

Dopamine Modulation of Prefrontal Cortex Activity Is Manifold and Operates at Multiple Temporal and Spatial Scales

Although the function of dopamine in subcortical structures is largely limited to reward and movement, dopamine neurotransmission in the prefrontal cortex (PFC) is critical to a multitude of temporally and functionally diverse processes, such as attention, working memory, behavioral flexibility, action planning, and sustained motivational and affective states. How does dopamine influence computation of these temporally complex functions? We find causative links between sustained and burst patterns of phasic dopamine neuron activation and modulation of medial PFC neuronal activity at multiple spatiotemporal scales. These include a multidirectional and weak impact on individual neuron rate activity but a robust influence on coordinated ensemble activity, gamma oscillations, and gamma-theta coupling that persisted for minutes. In addition, PFC network responses to burst pattern of dopamine firing were selectively strengthened in behaviorally active states. This multiplex mode of modulation by dopamine input may enable PFC to compute and generate spatiotemporally diverse and specialized outputs.

Communication between the Anterior Cingulate Cortex and Ventral Tegmental Area during a Cost-Benefit Reversal Task

The anterior cingulate cortex (ACC) is implicated in value-based decision making, anticipation, and adaptation; however, how ACC activity modulates these behaviors is unclear. One possibility is via the ACC's connections with the ventral tegmental area (VTA), a dopaminergic region implicated in motivation and feedback processing. We tested this by monitoring ACC and VTA local field potentials in rats performing a cost-benefit reversal task that elicited both value-based and anticipatory choices. Partial directed coherence analyses revealed that elevated 4-Hz ACC-to-VTA signaling accompanied decisions that appeared to be anticipatory. ACC-to-VTA signaling also occurred post-reversal, consistent with it being involved in the initiation of non-default behavior. An analysis of 4-Hz signals in the other direction (VTA-to-ACC) revealed that it was elevated when the rats committed errors and that this signal was followed by behavioral adaptation. Together, these findings suggest that bidirectional communication between the ACC and VTA supports behavioral flexibility.

Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats' ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Aversion hot spots in the dopamine system

Through the development of optogenetics and other viral vector-based technologies, our view of the dopamine system has substantially advanced over the last decade. In particular, progress has been made in the reclassification of dopamine neurons based on subtypes displaying specific projections, which are associated with different features at the anatomical, molecular and behavioral level. Together, these discoveries have raised the possibility that individual groups of dopamine cells make a unique contribution to the processing of reward and aversion. Here, we review recent studies that have identified non-canonical dopamine pathways that are excited in response to aversive stimuli, including dopamine projections to the ventromedial shell of the nucleus accumbens, prefrontal cortex, tail of the striatum, and amygdala.

Effects of Dopaminergic Drugs on Cognitive Control Processes Vary by Genotype

Dopamine (DA) has been implicated in modulating multiple cognitive control processes, including the robust maintenance of task sets and memoranda in the face of distractors (cognitive stability) and, conversely, the ability to switch task sets or update the contents of working memory when it is advantageous to do so (cognitive flexibility). In humans, the limited specificity of available pharmacological probes has posed a challenge for understanding the mechanisms by which DA, acting on multiple receptor families across the PFC and striatum, differentially influences these cognitive processes. Using a within-subject, placebo-controlled design, we contrasted the impact of two mechanistically distinct DA drugs, tolcapone (an inhibitor of catechol-O-methyltransferase [COMT], a catecholamine inactivator) and bromocriptine (a DA agonist with preferential affinity for the D2 receptor), on the maintenance and switching of task rules. Given previous work demonstrating that drug effects on behavior are dependent on baseline DA tone, participants were stratified according to genetic polymorphisms associated with cortical (COMT Val158Met) and striatal (Taq1A) DA system function. Our results were partially consistent with an inverted-U-shaped relationship between tolcapone and robust rule maintenance (interaction with COMT genotype) and between bromocriptine and cued rule switching (interaction with Taq1A genotype). However, when task instructions were ambiguous, a third relationship emerged to explain drug effects on spontaneous task switching (interaction of COMT genotype and bromocriptine). Together, this pattern of results suggests that the effects of DA drugs vary not only as a function of the DA system component upon which they act but also on subtle differences in task demands and context.

Neuropsychopharmacological regulation of performance on creativity-related tasks

A number of factors affect performance on tasks associated with creativity. Two pharmacological systems in particularly been identified as important for their impact on creativity, the noradrenergic system and the dopaminergic systems. Furthermore, stress is also established as an important factor impacting performance, most likely mediated by its effects on these neurotransmitter systems. Herein, we review the current literature on the relationships between stress, the noradrenergic system, the dopaminergic system, and other pharmacological factors and their effects on performance on tasks associated with creativity.

Dopamine and Cognitive Control in Prefrontal Cortex

Cognitive control, the ability to orchestrate behavior in accord with our goals, depends on the prefrontal cortex. These cognitive functions are heavily influenced by the neuromodulator dopamine. We review here recent insights exploring the influence of dopamine on neuronal response properties in prefrontal cortex (PFC) during ongoing behaviors in primates. This review suggests three major computational roles of dopamine in cognitive control: (i) gating sensory input, (ii) maintaining and manipulating working memory contents, and (iii) relaying motor commands. For each of these roles, we propose a neuronal microcircuit based on known mechanisms of action of dopamine in PFC, which are corroborated by computational network models. This conceptual approach accounts for the various roles of dopamine in prefrontal executive functioning.

Dopamine tunes prefrontal outputs to orchestrate aversive processing

Decades of research suggest that the mesocortical dopamine system exerts powerful control over mPFC physiology and function. Indeed, dopamine signaling in the medial prefrontal cortex (mPFC) is implicated in a vast array of processes, including working memory, stimulus discrimination, stress responses, and emotional and behavioral control. Consequently, even slight perturbations within this delicate system result in profound disruptions of mPFC-mediated processes. Many neuropsychiatric disorders are associated with dysregulation of mesocortical dopamine, including schizophrenia, depression, attention deficit hyperactivity disorder, post-traumatic stress disorder, among others. Here, we review the anatomy and functions of the mesocortical dopamine system. In contrast to the canonical role of striatal dopamine in reward-related functions, recent work has revealed that mesocortical dopamine fine-tunes distinct efferent projection populations in a manner that biases subsequent behavior towards responding to stimuli associated with potentially aversive outcomes. We propose a framework wherein dopamine can serve as a signal for switching mPFC states by orchestrating how information is routed to the rest of the brain.

Neural Circuit Motifs in Valence Processing

How do our brains determine whether something is good or bad? How is this computational goal implemented in biological systems? Given the critical importance of valence processing for survival, the brain has evolved multiple strategies to solve this problem at different levels. The psychological concept of "emotional valence" is now beginning to find grounding in neuroscience. This review aims to bridge the gap between psychology and neuroscience on the topic of emotional valence processing. Here, I highlight a subset of studies that exemplify circuit motifs that repeatedly appear as implementational systems in valence processing. The motifs I identify as being important in valence processing include (1) Labeled Lines, (2) Divergent Paths, (3) Opposing Components, and (4) Neuromodulatory Gain. Importantly, the functionality of neural substrates in valence processing is dynamic, context-dependent, and changing across short and long timescales due to synaptic plasticity, competing mechanisms, and homeostatic need.

Direct Glutamatergic Signaling From Midbrain Dopaminergic Neurons Onto Pyramidal Prefrontal Cortex Neurons

The dopaminergic neurons of the ventral tegmental area (VTA) have been identified with the ability to co-release dopamine and glutamate. This ability was first documented in the nucleus accumbens but showed to be absent in the dorsal striatum. Recently the ability to release glutamate from a subpopulation of the VTA dopaminergic neurons has been shown to control the prefrontal cortex (PFC) excitation through the exclusive innervation of GABAergic fast spiking interneurons. Here, using an optogenetic approach, we expand this view by presenting that the VTA dopaminergic neurons do not only innervate interneurons but also pyramidal PFC neurons. This finding opens the range of possibilities for the VTA dopaminergic neurons to modulate the activity of PFC.

The hot 'n' cold of cue-induced drug relapse

Environmental cues associated with rewards can acquire motivational properties. However, there is considerable variation in the extent to which a reward cue gains motivational control over behavior, depending on the individual and the form of the cue. When a discrete cue is paired with food reward, it acquires greater control over motivated behavior in some rats (sign-trackers, STs) than others (goal-trackers, GTs) as indicated by the propensity to approach the cue, the willingness to work to obtain it, and its ability to reinstate reward-seeking behavior. Here, we review studies that employ this ST/GT animal model to investigate characteristics of individuals that are especially susceptible to reward cue-elicited behavior and the involvement of dopamine and acetylcholine neuromodulator systems in the susceptibility to cue-induced drug relapse. First, we discuss individual differences in the attribution of incentive salience to different forms of reward cues and the involvement of the mesolimbic dopamine system. We then discuss individual differences in cognitive/attentional control and the contributions of the cholinergic system in processing reward cues. It is suggested that in STs a propensity to attribute motivational properties to a drug cue is combined with poor attentional control in the face of these cues, making them particularly vulnerable to transition from casual/experimental patterns of drug use to addiction and to cue-induced relapse.

Conflict and adaptation signals in the anterior cingulate cortex and ventral tegmental area

The integration and utilization of feedback in order to determine which decision strategy to use in different contexts is the core of executive function. The anterior cingulate cortex (ACC) is central to these processes but how feedback is made available to the ACC is unclear. To address this question, we trained rats with implants in the ACC and the ventral tegmental area (VTA), a dopaminergic brain region implicated in feedback processing, in a spatial decision reversal task with rule switching occurring approximately every 12 trials. Following a rule switch, the rats had to shift and sustain responses to the alternative side in order to obtain reward. Partial directed coherence (PDC) models of signal directionality between the ACC and VTA indicated that VTA → ACC communication (near 4 Hz) increased immediately prior to incorrect choices and during post-error decisions. This increase did not occur during correct choices. These data indicate that the VTA provides a feedback-driven, bottom-up modulating signal to the ACC which may be involved in assessing, and correcting for, decision conflict.

Depression: the search for separable behaviors and circuits

Major depressive disorder can manifest as different combinations of symptoms, ranging from a profound and incapacitating sadness, to a loss of interest in daily life, to an inability to engage in effortful, goal-directed behavior. Recent research has focused on defining the neural circuits that mediate separable features of depression in patients and preclinical animal models, and connections between frontal cortex and brainstem neuromodulators have emerged as candidate targets. The development of methods permitting recording and manipulation of neural circuits defined by connectivity has enabled the investigation of prefrontal-neuromodulatory circuit dynamics in animal models of depression with exquisite precision, a systems-level approach that has brought new insights by integrating these fields of depression research.

Roles of Prefrontal Cortex and Mediodorsal Thalamus in Task Engagement and Behavioral Flexibility

Behavioral tasks involving auditory cues activate inhibitory neurons within auditory cortex, leading to a reduction in the amplitude of auditory evoked response potentials (ERPs). One hypothesis is that this process, termed "task engagement," may enable context-dependent behaviors. Here we set out to determine (1) whether the medial prefrontal cortex (mPFC) plays a role in task engagement and (2) how task engagement relates to the context-dependent processing of auditory cues in male and female mice performing a decision-making task that can be guided by either auditory or visual cues. We found that, in addition to auditory ERP suppression, task engagement is associated with increased mPFC activity and an increase in theta band (4-7 Hz) synchronization between the mPFC and auditory cortex. Optogenetically inhibiting the mPFC eliminates the task engagement-induced auditory ERP suppression, while also preventing mice from switching between auditory and visual cue-based rules. However, mPFC inhibition, which eliminates task engagement-induced auditory ERP suppression, did not prevent mice from making decisions based on auditory cues. Furthermore, a more specific manipulation, selective disruption of mPFC outputs to the mediodorsal (MD) thalamus, is sufficient to prevent switching between auditory and visual rules but does not affect auditory ERPs. Based on these findings, we conclude that (1) the mPFC contributes to both task engagement and behavioral flexibility; (2) mPFC-MD projections are important for behavioral flexibility but not task engagement; and (3) task engagement, evidenced by the suppression of cortical responses to sensory input, is not required for sensory cue-guided decision making.SIGNIFICANCE STATEMENT When rodents perform choice-selection tasks based on sensory cues, neural responses to these cues are modulated compared with task-free conditions. Here we demonstrate that this phenomenon depends on the prefrontal cortex and thus represents a form of "top-down" regulation. However, we also show that this phenomenon is not critical for task performance, as rodents can make decisions based on specific sensory cues even when the task-dependent modulation of responses to those cues is abolished. Furthermore, disrupting one specific set of prefrontal outputs impairs rule switching but not the task-dependent modulation of sensory responses. These results show that the prefrontal cortex comprises multiple circuits that mediate dissociable functions related to behavioral flexibility and sensory processing.

The neuroscience of cognitive-motivational styles: Sign- and goal-trackers as animal models

Cognitive-motivational styles describe predominant patterns of processing or biases that broadly influence human cognition and performance. Here we focus on the impact of cognitive-motivational styles on the response to cues predicting the availability of food or addictive drugs. An individual may preferably conduct an analysis of the motivational significance of reward cues, with the result that such cues per se are perceived as rewarding and worth approaching and working for. Alternatively, a propensity for a "cold" analysis of the behavioral utility of a reward cue may yield search behavior for food or drugs but not involve cue approach. Animal models for studying the neuronal mechanisms mediating such styles have originated from research concerning behavioral indices that predict differential vulnerability to addiction-like behaviors. Rats classified as sign- or goal-trackers (STs, GTs) were found to have opposed attentional biases (bottom-up or cue-driven attention vs. top-down or goal-driven attentional control) that are mediated primarily via relatively unresponsive versus elevated levels of cholinergic neuromodulation in the cortex. The capacity for cholinergic neuromodulation in STs is limited by a neuronal choline transporter (CHT) that fails to support increases in cholinergic activity. Moreover, in contrast to STs, the frontal dopamine system in GTs does not respond to the presence of drug cues and, thus, biases against cue-oriented behavior. The opponent cognitive-motivational styles that are indexed by sign- and goal-tracking bestow different cognitive-behavioral vulnerabilities that may contribute to the manifestation of a wide range of neuropsychiatric disorders. (PsycINFO Database Record

'Hot' vs. 'cold' behavioural-cognitive styles: motivational-dopaminergic vs. cognitive-cholinergic processing of a Pavlovian cocaine cue in sign- and goal-tracking rats

Discrete Pavlovian reward cues acquire more potent incentive motivational properties (incentive salience) in some animals (sign-trackers; STs) compared to others (goal-trackers; GTs). Conversely, GTs appear to be better than STs in processing more complex contextual cues, perhaps reflecting their relatively greater bias for goal-directed cue processing. Here, we investigated the activity of two major prefrontal neuromodulatory input systems, dopamine (DA) and acetylcholine (ACh), in response to a discrete Pavlovian cue that was previously paired with cocaine administration in STs and GTs. Rats underwent Pavlovian training in which light cue presentations were either paired or unpaired with an intravenous cocaine infusion. Following a 10-day abstinence period, prefrontal dialysates were collected in STs and GTs during cue presentations in the absence of cocaine. In STs, the cue previously paired with cocaine significantly increased prefrontal DA levels. DA levels remained elevated over baseline across multiple cue presentation blocks, and DA levels and approaches to the cue were significantly correlated. In STs, ACh levels were unaffected by cue presentations. In contrast, in GTs, presentations of the cocaine cue increased prefrontal ACh, but not DA, levels. GTs oriented towards the cue at rates similar to STs, but they did not approach it and elevated ACh levels did not correlate with conditioned orientation. The results indicate a double dissociation between the role of prefrontal DA and ACh in STs and GTs, and suggest that these phenotypes will be useful for studying the role of neuromodulator systems in mediating opponent behavioural-cognitive styles.