Neural Activity in Ventral Medial Prefrontal Cortex Is Modulated More Before Approach Than Avoidance During Reinforced and Extinction Trial Blocks

Where Actions Meet Outcomes: Medial Prefrontal Cortex, Central Thalamus, and the Basal Ganglia

Medial prefrontal cortex (mPFC) interacts with distributed networks that give rise to goal-directed behavior through afferent and efferent connections with multiple thalamic nuclei and recurrent basal ganglia-thalamocortical circuits. Recent studies have revealed individual roles for different thalamic nuclei: mediodorsal (MD) regulation of signaling properties in mPFC neurons, intralaminar control of cortico-basal ganglia networks, ventral medial facilitation of integrative motor function, and hippocampal functions supported by ventral midline and anterior nuclei. Large scale mapping studies have identified functionally distinct cortico-basal ganglia-thalamocortical subnetworks that provide a structural basis for understanding information processing and functional heterogeneity within the basal ganglia. Behavioral analyses comparing functional deficits produced by lesions or inactivation of specific thalamic nuclei or subregions of mPFC or the basal ganglia have elucidated the interdependent roles of these areas in adaptive goal-directed behavior. Electrophysiological recordings of mPFC neurons in rats performing delayed non-matching-to position (DNMTP) and other complex decision making tasks have revealed populations of neurons with activity related to actions and outcomes that underlie these behaviors. These include responses related to motor preparation, instrumental actions, movement, anticipation and delivery of action outcomes, memory delay, and spatial context. Comparison of results for mPFC, MD, and ventral pallidum (VP) suggest critical roles for mPFC in prospective processes that precede actions, MD for reinforcing task-relevant responses in mPFC, and VP for providing feedback about action outcomes. Synthesis of electrophysiological and behavioral results indicates that different networks connecting mPFC with thalamus and the basal ganglia are organized to support distinct functions that allow organisms to act efficiently to obtain intended outcomes.

Cortico-Striatal Control over Adaptive Goal-Directed Responding Elicited by Cues Signaling Sucrose Reward or Punishment

The medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) have been associated with the expression of adaptive and maladaptive behavior elicited by fear-related and drug-associated cues. However, reported effects of mPFC manipulations on cue-elicited natural reward-seeking and inhibition thereof have been varied, with few studies examining cortico-striatal contributions in tasks that require adaptive responding to cues signaling reward and punishment within the same session. The current study aimed to better elucidate the role of mPFC and NAc subdivisions, and their functional connectivity in cue-elicited adaptive responding using a novel discriminative cue responding task. Male Long-Evans rats learned to lever-press on a VR5 schedule for a discriminative cue signaling reward, and to avoid pressing the same lever in the presence of another cue signaling punishment. Postacquisition, prelimbic (PL) and infralimbic (IL) areas of the mPFC, NAc core, shell, PL-core, or IL-shell circuits were pharmacologically or chemogenetically inhibited while animals performed under (1) nonreinforced (extinction) conditions, where the appetitive and aversive cues were presented in alternating trials alone or as a compound stimulus; and (2) reinforced conditions, whereby cued responding was accompanied by associated outcomes. PL and IL inactivation attenuated nonreinforced and reinforced goal-directed cue responding, whereas NAc core and shell inactivation impaired nonreinforced responding for the appetitive, but not aversive cue. Furthermore, PL-core and IL-shell inhibition disinhibited nonreinforced but not reinforced cue responding. Our findings implicate the mPFC as a site of confluence of motivationally significant cues and outcomes, and in the regulation of nonreinforced cue responding via downstream NAc targets.SIGNIFICANCE STATEMENT The ability to discriminate and respond appropriately to environmental cues that signal availability of reward or punishment is essential for survival. The medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) have been implicated in adaptive and maladaptive behavior elicited by fear-related and drug-associated cues. However, less is known about the role they play in orchestrating adaptive responses to natural reward and punishment cues within the same behavioral task. Here, using a novel discriminative cue responding task combined with pharmacological or chemogenetic inhibition of mPFC, NAc and mPFC-NAc circuits, we report that mPFC is critically involved in responding to changing cued response-outcomes, both when the responses are reinforced, and nonreinforced. Furthermore, the mPFC coordinates nonreinforced discriminative cue responding by suppressing inappropriate responding via downstream NAc targets.

The rodent medial prefrontal cortex and associated circuits in orchestrating adaptive behavior under variable demands

Emerging evidence implicates rodent medial prefrontal cortex (mPFC) in tasks requiring adaptation of behavior to changing information from external and internal sources. However, the computations within mPFC and subsequent outputs that determine behavior are incompletely understood. We review the involvement of mPFC subregions, and their projections to the striatum and amygdala in two broad types of tasks in rodents: 1) appetitive and aversive Pavlovian and operant conditioning tasks that engage mPFC-striatum and mPFC-amygdala circuits, and 2) foraging-based tasks that require decision making to optimize reward. We find support for region-specific function of the mPFC, with dorsal mPFC and its projections to the dorsomedial striatum supporting action control with higher cognitive demands, and ventral mPFC engagement in translating affective signals into behavior via discrete projections to the ventral striatum and amygdala. However, we also propose that defined mPFC subdivisions operate as a functional continuum rather than segregated functional units, with crosstalk that allows distinct subregion-specific inputs (e.g., internal, affective) to influence adaptive behavior supported by other subregions.

Information capacity and robustness of encoding in the medial prefrontal cortex are modulated by the bioavailability of serotonin and the time elapsed from the cue during a reward-driven task

Serotonin (5-HT) is a key neuromodulator of medial prefrontal cortex (mPFC) functions. Pharmacological manipulation of systemic 5-HT bioavailability alters the electrical activity of mPFC neurons. However, 5-HT modulation at the population level is not well characterized. In the present study, we made single neuron extracellular recordings in the mPFC of rats performing an operant conditioning task, and analyzed the effect of systemic administration of fluoxetine (a selective serotonin reuptake inhibitor) on the information encoded in the firing activity of the neural population. Chronic (longer than 15 days), but not acute (less than 15 days), fluoxetine administration reduced the firing rate of mPFC neurons. Moreover, fluoxetine treatment enhanced pairwise entropy but diminished noise correlation and redundancy in the information encoded, thus showing how mPFC differentially encodes information as a function of 5-HT bioavailability. Information about the occurrence of the reward-predictive stimulus was maximized during reward consumption, around 3 to 4 s after the presentation of the cue, and it was higher under chronic fluoxetine treatment. However, the encoded information was less robust to noise corruption when compared to control conditions.

Food-Seeking Behavior Is Mediated by Fos-Expressing Neuronal Ensembles Formed at First Learning in Rats

Neuronal ensembles in the infralimbic cortex (IL) develop after prolonged food self-administration training. However, rats demonstrate evidence of learning the food self-administration response as early as day 1, with responding quickly increasing to asymptotic levels. Since the contribution of individual brain regions to task performance shifts over the course of training, it remains unclear whether IL ensembles are gradually formed and refined over the course of extensive operant training, or whether functionally-relevant ensembles might be recruited and formed as early as the initial acquisition of food self-administration behavior. Here, we aimed to determine the role of IL ensembles at the earliest possible point after demonstrable learning of a response-outcome association. We first allowed rats to lever press for palatable food pellets and stopped training rats once their behavior evidenced the response-outcome association (learners). We compared their food-seeking behavior and neuronal activation (Fos protein expression) to similarly trained rats that did not form this association (non-learners). Learners had greater food-seeking behavior and neuronal activation within the medial prefrontal cortex (mPFC), suggesting that mPFC subregions might encode initial food self-administration memories. To test the functional relevance of mPFC Fos-expressing ensembles to subsequent food seeking, we tested region-wide inactivation of the IL using muscimol+baclofen and neuronal ensemble-specific ablation using the Daun02 inactivation procedure. Both region-wide inactivation and ensemble-specific inactivation of the IL significantly decreased food seeking. These data suggest that IL neuronal ensembles form during initial learning of food self-administration behavior, and furthermore, that these ensembles play a functional role in food seeking.

Integration of value and action in medial prefrontal neural systems

The rodent medial prefrontal cortex (mPFC) plays a key role in regulating cognition, emotion, and behavior. mPFC neurons are activated in diverse experimental paradigms, raising the questions of whether there are specific task elements or dimensions encoded by mPFC neurons, and whether these encoded parameters are selective to neurons in particular mPFC subregions or networks. Here, we consider the role of mPFC neurons in processing appetitive and aversive cues, outcomes, and related behaviors. mPFC neurons are strongly activated in tasks probing value and outcome-associated actions, but these responses vary across experimental paradigms. Can we identify specific categories of responses (e.g., positive or negative value), or do mPFC neurons exhibit response properties that are too heterogeneous/complex to cluster into distinct conceptual groups? Based on a review of relevant studies, we consider what has been done and what needs to be further explored in order to address these questions.

Ethanol Alters Variability, But Not Rate, of Firing in Medial Prefrontal Cortex Neurons of Awake-Behaving Rats

BACKGROUND

The medial prefrontal cortex (mPFC) is a brain region involved in the evaluation and selection of motivationally relevant outcomes. Neural activity in mPFC is altered following acute ethanol (EtOH) use and, in rodent models, doses as low as 0.75 g/kg yield cognitive deficits. Deficits in decision making following acute EtOH are thought to be mediated, at least in part, by decreases in mPFC firing rates (FRs). However, the data leading to this conclusion have been generated exclusively in anesthetized rodents. The present study characterizes the effects of acute EtOH injections on mPFC neural activity in awake-behaving rodents.

METHODS

Awake-behaving and anesthetized in vivo electrophysiological recordings were performed. We utilized 3 groups: the first received 2 saline injections, the second received a saline injection followed by 1.0 g/kg EtOH, and the last received saline followed by 2 g/kg EtOH. One week later, an anesthetized recording occurred where a saline injection was followed by an injection of 1.0 g/kg EtOH.

RESULTS

The anesthetized condition showed robust decreases in neural activity and differences in up-down states (UDS) dynamics. In the awake-behaving condition, FRs were grouped according to behavioral state: moving, not-moving, and sleep. The differences in median FRs were found for each treatment and behavioral state combination. A FR decrease was only found in the 2.0 g/kg EtOH treatment during not-moving states. However, robust decreases in FR variability were found across behavioral state in both the 1.0 and 2.0 g/kg EtOH treatment. Sleep was separately analyzed. EtOH modulated the UDS during sleep producing decreases in FRs.

CONCLUSIONS

In conclusion, the changes in neural activity following EtOH administration in anesthetized animals are not conserved in awake-behaving animals. The most prominent difference following EtOH was a decrease in FR variability suggesting that acute EtOH may be affecting decision making via this mechanism.

The neural and computational systems of social learning

Learning the value of stimuli and actions from others - social learning - adaptively contributes to individual survival and plays a key role in cultural evolution. We review research across species targeting the neural and computational systems of social learning in both the aversive and appetitive domains. Social learning generally follows the same principles as self-experienced value-based learning, including computations of prediction errors and is implemented in brain circuits activated across task domains together with regions processing social information. We integrate neural and computational perspectives of social learning with an understanding of behaviour of varying complexity, from basic threat avoidance to complex social learning strategies and cultural phenomena.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Avoidance Problems Reconsidered

Active avoidance is the prototypical paradigm for studying aversively-motivated instrumental behavior. However, avoidance research stalled amid heated theoretical debates and the hypothesis that active avoidance is essentially Pavlovian flight. Here I reconsider key "avoidance problems" and review neurobehavioral data collected with modern tools. Although the picture remains incomplete, these studies strongly suggest that avoidance has an instrumental component and is mediated by brain circuits that resemble appetitive instrumental actions more than Pavlovian fear reactions. Rapid progress may be possible if investigators consider important factors like safety signals, response-competition, goal-directed vs. habitual control and threat imminence in avoidance study design. Since avoidance responses likely contribute to active coping, this research has important implications for understanding human resilience and disorders of control.