Contributions of basolateral amygdala and nucleus accumbens subregions to mediating motivational conflict during punished reward-seeking

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

The FAAH inhibitor URB597 reduces cocaine intake during conditioned punishment and mitigates cocaine seeking during withdrawal

The endocannabinoid system is prominently implicated in the control of cocaine reinforcement due to its relevant role in synaptic plasticity and neurotransmitter modulation in the mesocorticolimbic system. The inhibition of fatty acid amide hydrolase (FAAH), and the resulting increase in anandamide and other N-acylethanolamines, represents a promising strategy for reducing drug seeking. In the present study, we aimed to assess the effects of the FAAH inhibitor URB597 (1 mg/kg) on crucial features of cocaine addictive-like behaviour in mice. Therefore, we tested the effects of URB597 on acquisition of cocaine (0.6 mg/kg/inf) self-administration, compulsive-like cocaine intake and cue-induced drug-seeking behaviour during withdrawal. URB597 reduced cocaine intake under conditioned punishment while having no impact on acquisition. This result was associated to increased cannabinoid receptor 1 gene expression in the ventral striatum and medium spiny neurons activation in the nucleus accumbens shell. Moreover, URB597 mitigated cue-induced drug-seeking behaviour during prolonged abstinence and prevented the withdrawal-induced increase in FAAH gene expression in the ventral striatum. In this case, URB597 decreased activation of medium spiny neurons in the nucleus accumbens core. Our findings evidence the prominent role of endocannabinoids in the development of cocaine addictive-like behaviours and support the potential of FAAH inhibition as a therapeutical target for the treatment of cocaine addiction.

The central amygdala modulates distinctive conflict-like behaviors in a naturalistic foraging task

Conflict situations elicit a diverse range of behaviors that extend beyond the simplistic approach or avoidance dichotomy. However, many conflict-related studies have primarily focused on approach suppression, neglecting the complexity of these behaviors. In our study, we exposed rats to a semi-naturalistic foraging task, presenting them with a trade-off between a food reward and a predatory threat posed by a robotic agent. We observed that rats displayed two conflict-like behaviors (CLBs)-diagonal approach and stretched posture-when facing a robotic predator guarding a food pellet. After electrolytic lesions to the central amygdala (CeA), both conflict behaviors were significantly reduced, accompanied by a decrease in avoidance behavior (hiding) and an increase in approach behavior (frequency of interactions with the robot). A significant negative correlation between avoidance and approach behaviors emerged after the CeA lesion; however, our data suggest that CLBs are not tightly coupled with either approach or avoidance behaviors, showing no significant correlation to those behaviors. Our findings indicate that the CeA plays a crucial role in modulating conflict behaviors, competing with approach suppression in risky situations.

Circuit and Cell-Specific Contributions to Decision Making Involving Risk of Explicit Punishment in Male and Female Rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine (DA) D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit-specific and cell type-specific optogenetic approaches in rats during a decision making task involving risk of punishment. In experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased preference for the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.SIGNIFICANCE STATEMENT Until recently, the ability to dissect the neural substrates of decision making involving risk of punishment (risk taking) in a circuit-specific and cell-specific manner has been limited by the tools available for use in rats. Here, we leveraged the temporal precision of optogenetics, together with transgenic rats, to probe contributions of a specific circuit and cell population to different phases of risk-based decision making. Our findings reveal basolateral amygdala (BLA)→nucleus accumbens shell (NAcSh) is involved in evaluation of punished rewards in a sex-dependent manner. Further, NAcSh D2 receptor (D2R)-expressing neurons make unique contributions to risk taking that vary across the decision making process. These findings advance our understanding of the neural principles of decision making and provide insight into how risk taking may become compromised in neuropsychiatric diseases.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Amygdala activity and amygdala-hippocampus connectivity: Metabolic diseases, dementia, and neuropsychiatric issues

With rapid aging of the population worldwide, the number of people with dementia is dramatically increasing. Some studies have emphasized that metabolic syndrome, which includes obesity and diabetes, leads to increased risks of dementia and cognitive decline. Factors such as insulin resistance, hyperglycemia, high blood pressure, dyslipidemia, and central obesity in metabolic syndrome are associated with synaptic failure, neuroinflammation, and imbalanced neurotransmitter levels, leading to the progression of dementia. Due to the positive correlation between diabetes and dementia, some studies have called it "type 3 diabetes". Recently, the number of patients with cognitive decline due to metabolic imbalances has considerably increased. In addition, recent studies have reported that neuropsychiatric issues such as anxiety, depressive behavior, and impaired attention are common factors in patients with metabolic disease and those with dementia. In the central nervous system (CNS), the amygdala is a central region that regulates emotional memory, mood disorders, anxiety, attention, and cognitive function. The connectivity of the amygdala with other brain regions, such as the hippocampus, and the activity of the amygdala contribute to diverse neuropathological and neuropsychiatric issues. Thus, this review summarizes the significant consequences of the critical roles of amygdala connectivity in both metabolic syndromes and dementia. Further studies on amygdala function in metabolic imbalance-related dementia are needed to treat neuropsychiatric problems in patients with this type of dementia.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

The effects of early life stress on impulsivity

Elevated impulsivity is a symptom shared by various psychiatric disorders such as substance use disorder, bipolar disorder, and attention-deficit/hyperactivity disorder. However, impulsivity is not a unitary construct and impulsive behaviors fall into two subcategories: impulsive action and impulsive choice. Impulsive choice refers to the tendency to prefer immediate, small rewards over delayed, large rewards, whereas impulsive action involves difficulty inhibiting rash, premature, or mistimed behaviors. These behaviors are mediated by the mesocorticolimbic dopamine (DA) system, which consists of projections from the ventral tegmental area to the nucleus accumbens and prefrontal cortex. Early life stress (ELS) alters both impulsive choice and impulsive action in rodents. ELS also changes DA receptor expression, transmission, and activity within the mesocorticolimbic system. This review integrates the dopamine, impulsivity, and ELS literature to provide evidence that ELS alters impulsivity via inducing changes in the mesocorticolimbic DA system. Understanding how ELS affects brain circuits associated with impulsivity can help advance treatments aimed towards reducing impulsivity symptoms in a variety of psychiatric disorders.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Dialectical Thinking Is Linked With Smaller Left Nucleus Accumbens and Right Amygdala

Our current work examined the interface between thinking style and emotional experience at both the behavioral and neuropsychological levels. Thirty-nine Chinese participants completed the triad task, and we calculated the rate of individually selected relationship pairings to overall selections to represent their holistic thinking tendencies. In addition, participants in the top one-third of the ratio score were classified into the high holistic thinking group, while those in the bottom one-third of the ratio score were classified into the low holistic thinking group. We used the sensitivity to punishment and sensitivity to reward questionnaire (SPSRQ) to examine how people elicit positive and negative affective behaviors. Additionally, we examined the volume of the amygdala and nucleus accumbens and their functional connectivity in the resting-state. We found that high holistic thinkers were much less sensitive to rewards than low holistic thinkers. In other words, individuals with high holistic thinking are less likely to pursue behaviors that have positive emotional outcomes. Furthermore, their bilateral nucleus accumbens and right amygdala volumes were smaller than those of low holistic thinkers. Hierarchical regression analysis showed that holistic thinking tendency can negatively predict the volume of the left nucleus accumbens and right amygdala. Finally, resting-state functional connectivity results showed increased functional connectivity FC between left nucleus accumbens and bilateral amygdala in high holistic thinkers. These findings provide emotion-related manifestations of thinking styles at the behavioral and neural levels.

Alcohol Seeking Under Risk of Punishment Is Associated With Activation of Cortical and Subcortical Brain Regions

In humans, stimuli associated with alcohol availability can provoke relapse during abstinence. In this study, we investigated the role of discriminative stimuli (DS) in the control of alcohol seeking in two types of behavioral tests. The first test examined the ability of an alcohol-associated DS to promote alcohol seeking (relapse) after punishment-imposed abstinence in the presence of a different DS. Following this, we tested whether the differentially associated DS can promote and suppress alcohol self-administration in a within-session discrimination task. During the within-session discrimination task, we also tested the rate of alcohol self-administration when two DS are presented in a compound. We first trained Long-Evans male rats (n = 24) to self-administer alcohol in the presence of one DS (reward-associated discriminative stimulus, rewDS) and then punished that behavior in the presence of a different DS (punishment-associated discriminative stimulus, punDS). On the test, we found that rats tested with the rewDS showed higher alcohol seeking than rats tested with the punDS. This result shows that a single Cue DS can promote alcohol seeking in a manner comparable to contexts. Subsequently, we trained 16 of these rats in a within-session trial-based discrimination task, comprised of intervening 2-min trials of rewDS, punDS, or conflict with rewDS and punDS in compound and a reduced probability of punishment. We found that alcohol self-administration is bi-directionally regulated by the rewDS and punDS. In conflict trials, alcohol self-administration was at a rate that was intermediate between the rewDS and punDS trials. In a final test, rats were presented with one of the three trial conditions and perfused for Fos immunohistochemistry. We found Fos expression was higher in the rats tested in the conflict condition in three interconnected sub-cortical brain regions. This study demonstrated the important role that alcohol-associated DS plays an important role in promoting relapse to alcohol seeking after punishment-imposed abstinence. We also implemented a within-session discrimination task that allows for the study of alcohol seeking under motivational conflict, which may be relevant for alcohol use despite negative consequences. The results from the Fos data suggest that higher alcohol seeking in approach-avoidance motivational conflict is associated with activation of sub-cortical regions but not cortical regions.

Melatonin Blocks Morphine-Induced Place Preference: Involvement of GLT-1, NF-κB, BDNF, and CREB in the Nucleus Accumbens

Opioid addiction remains a widespread issue despite continuous attempts by the FDA to help maintain abstinence. Melatonin is a neurohormone considered to be involved only in the neuroendocrine and reproductive systems; however, recent reports have demonstrated its potential to attenuate drug addiction and dependence. Cumulative studies have suggested that melatonin can attenuate the rewarding effects of several drugs of abuse, including opioids. This study aimed to investigate the effect of melatonin (50 mg/kg) on morphine (5 mg/kg) to produce place preference. We also investigated the effect of melatonin and morphine on the expression of GLT-1, BDNF, NF-κB, and CREB within the nucleus accumbens. Male Wistar rats were divided into control, morphine, melatonin, and the morphine + melatonin groups. The study involved a two-phase habituation phase from day 1 to day 3 and an acquisition phase from day 5 to day 14. The conditioned place preference (CPP) score, distance traveled, resting time, ambulatory count, and total activity count were measured for all animals. Rats that received morphine showed a significant increase in CPP score compared to those in the control group. Morphine treatment reduced the mRNA expression of GLT-1, BDNF, and CREB and increased that of NF-κB. However, melatonin treatment administered 30 min before morphine treatment attenuated morphine place preference and reversed GLT-1, BDNF, NF-κB, and CREB expression levels. In conclusion, the study results indicate, for the first time, the new potential targets of melatonin in modulating morphine-induced CPP.

Chronic Distress in Male Mice Impairs Motivation Compromising Both Effort and Reward Processing With Altered Anterior Insular Cortex and Basolateral Amygdala Neural Activation

In humans and mammals, effort-based decision-making for monetary or food rewards paradigms contributes to the study of adaptive goal-directed behaviours acquired through reinforcement learning. Chronic distress modelled by repeated exposure to glucocorticoids in rodents induces suboptimal decision-making under uncertainty by impinging on instrumental acquisition and prompting negative valence behaviours. In order to further disentangle the motivational tenets of adaptive decision-making, this study addressed the consequences of enduring distress on relevant effort and reward-processing dimensions. Experimentally, appetitive and consummatory components of motivation were evaluated in adult C57BL/6JRj male mice experiencing chronic distress induced by oral corticosterone (CORT), using multiple complementary discrete behavioural tests. Behavioural data (from novelty suppressed feeding, operant effort-based choice, free feeding, and sucrose preference tasks) collectively show that behavioural initiation, effort allocation, and hedonic appreciation and valuation are altered in mice exposed to several weeks of oral CORT treatment. Additionally, data analysis from FosB immunohistochemical processing of postmortem brain samples highlights CORT-dependent dampening of neural activation in the anterior insular cortex (aIC) and basolateral amygdala (BLA), key telencephalic brain regions involved in appetitive and consummatory motivational processing. Combined, these results suggest that chronic distress-induced irregular aIC and BLA neural activations with reduced effort production and attenuated reward value processing during reinforcement-based instrumental learning could result in maladaptive decision-making under uncertainty. The current study further illustrates how effort and reward processing contribute to adjust the motivational threshold triggering goal-directed behaviours in versatile environments.

Conflict Test Battery for Studying the Act of Facing Threats in Pursuit of Rewards

Survival depends on the ability of animals to avoid threats and approach rewards. Traditionally, these two opposing motivational systems have been studied separately. In nature, however, they regularly compete for the control of behavior. When threat- and reward-eliciting stimuli (learned or unlearned) occur simultaneously, a motivational conflict emerges that challenges individuals to weigh available options and execute a single behavioral response (avoid or approach). Most previous animal models using approach/avoidance conflicts have often focused on the ability to avoid threats by forgoing or delaying the opportunity to obtain rewards. In contrast, behavioral tasks designed to capitalize on the ability to actively choose to execute approach behaviors despite threats are scarce. Thus, we developed a behavioral test battery composed of three conflict tasks to directly study rats confronting threats to obtain rewards guided by innate and conditioned cues. One conflict task involves crossing a potentially electrified grid to obtain food on the opposite end of a straight alley, the second task is based on the step-down threat avoidance paradigm, and the third one is a modified version of the open field test. We used diazepam to pharmacologically validate conflict behaviors in our tasks. We found that, regardless of whether competing stimuli were conditioned or innate, a low diazepam dose decreased risk assessment and facilitated taking action to obtain rewards in the face of threats during conflict, without affecting choice behavior when there was no conflict involved. Using this pharmacologically validated test battery of ethologically designed innate/learned conflict tasks could help understand the fundamental brain mechanisms underlying the ability to confront threats to achieve goals.

A Decision Architecture for Safety Computations

Accurately estimating safety is critical to pursuing nondefensive survival behaviors. However, little attention has been paid to how the human brain computes safety. We conceptualize a model that consists of two components: (i) threat-oriented evaluations that focus on threat value, imminence, and predictability; and (ii) self-oriented evaluations that focus on the agent's experience, strategies, and ability to control the situation. Our model points to the dynamic interaction between these two components as a mechanism of safety estimation. Based on a growing body of human literature, we hypothesize that distinct regions of the ventromedial prefrontal cortex (vmPFC) respond to threat and safety to facilitate survival decisions. We suggest safety is not an inverse of danger, but reflects independent computations that mediate defensive circuits and behaviors.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Medial prefrontal cortical control of reward- and aversion-based behavioral output: Bottom-up modulation

How does the brain guide our actions? This is a complex issue, where the medial prefrontal cortex (mPFC) plays a crucial role. The mPFC is essential for cognitive flexibility and decision making. These functions are related to reward- and aversion-based learning, which ultimately drive behavior. Though, cortical projections and modulatory systems that may regulate those processes in the mPFC are less understood. How does the mPFC regulate approach-avoidance behavior in the case of conflicting aversive and appetitive stimuli? This is likely dependent on the bottom-up neuromodulation of the mPFC projection neurons. In this review, we integrate behavioral-, pharmacological-, and viral-based circuit manipulation data showing the involvement of mPFC dopaminergic, noradrenergic, cholinergic, and serotoninergic inputs in reward and aversion processing. Given that an incorrect balance of reward and aversion value could be a key problem in mental diseases such as substance use disorders, we discuss outstanding questions for future research on the role of mPFC modulation in reward and aversion.

Characterizing Different Strategies for Resolving Approach-Avoidance Conflict

The ability of animals to maximize benefits and minimize costs during approach-avoidance conflicts is an important evolutionary tool, but little is known about the emergence of specific strategies for conflict resolution. Accordingly, we developed a simple approach-avoidance conflict task in rats that pits the motivation to press a lever for sucrose against the motivation to step onto a distant platform to avoid a footshock delivered at the end of a 30 s tone (sucrose is available only during the tone). Rats received conflict training for 16 days to give them a chance to optimize their strategy by learning to properly time the expression of both behaviors across the tone. Rats unexpectedly separated into three distinct subgroups: those pressing early in the tone and avoiding later (Timers, 49%); those avoiding throughout the tone (Avoidance-preferring, 32%); and those pressing throughout the tone (Approach-preferring, 19%). The immediate early gene cFos revealed that Timers showed increased activity in the ventral striatum and midline thalamus relative to the other two subgroups, Avoidance-preferring rats showed increased activity in the amygdala, and Approach-preferring rats showed decreased activity in the prefrontal cortex. This pattern is consistent with low fear and high behavioral flexibility in Timers, suggesting the potential of this task to reveal the neural mechanisms of conflict resolution.

Activation of Basolateral Amygdala to Nucleus Accumbens Projection Neurons Attenuates Chronic Corticosterone-Induced Behavioral Deficits in Male Mice

The basolateral amygdala (BLA) is critical for reward behaviors via a projection to the nucleus accumbens (NAc). Specifically, BLA-NAc projections are involved in reinforcement learning, reward-seeking, sustained instrumental responding, and risk behaviors. However, it remains unclear whether chronic stress interacts with BLA-NAc projection neurons to result in maladaptive behaviors. Here we take a chemogenetic, projection-specific approach to clarify how NAc-projecting BLA neurons affect avoidance, reward, and feeding behaviors in male mice. Then, we examine whether chemogenetic activation of NAc-projecting BLA neurons attenuates the maladaptive effects of chronic corticosterone (CORT) administration on these behaviors. CORT mimics the behavioral and neural effects of chronic stress exposure. We found a nuanced role of BLA-NAc neurons in mediating reward behaviors. Surprisingly, activation of BLA-NAc projections rescues CORT-induced deficits in the novelty suppressed feeding, a behavior typically associated with avoidance. Activation of BLA-NAc neurons also increases instrumental reward-seeking without affecting free-feeding in chronic CORT mice. Taken together, these data suggest that NAc-projecting BLA neurons are involved in chronic CORT-induced maladaptive reward and motivation behaviors.

Motivational competition and the paraventricular thalamus

Although significant progress has been made in understanding the behavioral and brain mechanisms for motivational systems, much less is known about competition between motivational states or motivational conflict (e.g., approach - avoidance conflict). Despite being produced under diverse conditions, behavior during motivational competition has two signatures: bistability and metastability. These signatures reveal the operation of positive feedback mechanisms in behavioral selection. Different neuronal architectures can instantiate this selection to achieve bistability and metastability in behavior, but each relies on circuit-level inhibition to achieve rapid and stable selection between competing tendencies. Paraventricular thalamus (PVT) is identified as critical to this circuit level inhibition, resolving motivational competition via its extensive projections to local inhibitory networks in the ventral striatum and extended amygdala, enabling adaptive responding under motivational conflict.

Hippocampal Interaction With Area 25, but not Area 32, Regulates Marmoset Approach-Avoidance Behavior

Affective disorders are associated with increased sensitivity to negative feedback that influences approach-avoidance decision making. Although neuroimaging studies of these disorders reveal dysregulation in primate cingulate areas 25 and 32 and the anterior hippocampus (aHipp), the causal involvement of these structures and their interaction in the primate brain is unknown. We therefore investigated the effects of localized pharmacological manipulations of areas 25 and 32 and/or the aHipp of the marmoset monkey on performance of an anxiolytic-sensitive instrumental decision-making task in which an approach-avoidance conflict is created by pairing a response with reward and punishment. During control infusions animals avoided punishment, but this bias was reduced by increasing glutamate release within the aHipp or area 32, and inactivation or 5-HT1a antagonism within area 25. Conversely, increasing glutamate release in area 25 enhanced punishment avoidance but, in contrast to previous reports, area 32 and aHipp inactivations had no effect. Simultaneous inactivation or 5-HT1a antagonism within area 25, but not area 32, abolished the reduced punishment avoidance seen after increasing aHipp glutamate. Besides providing causal evidence that these primate areas differentially regulate negative feedback sensitivity, this study links the decision-making deficits in affective disorders to aberrant aHipp-area 25 circuit activity.

Prefrontal cortical and nucleus accumbens contributions to discriminative conditioned suppression of reward-seeking

Fear can potently inhibit ongoing behavior, including reward-seeking, yet the neural circuits that underlie such suppression remain to be clarified. Prior studies have demonstrated that distinct subregions of the rodent medial prefrontal cortex (mPFC) differentially affect fear behavior, whereby fear expression is promoted by the more dorsal prelimbic cortex (PL) and inhibited by the more ventral infralimbic cortex (IL). These mPFC regions project to subregions of the nucleus accumbens, the core (NAcC) and shell (NAcS), that differentially contribute to reward-seeking as well as affective processes that may be relevant to fear expression. Here, we investigated how these mPFC and NAc subregions contribute to discriminative fear conditioning, assessed by conditioned suppression of reward-seeking. Bilateral inactivation of the NAcS or PL reduced the expression of conditioned suppression to a shock-associated CS+, whereas NAcC inactivation reduced reward-seeking without affecting suppression. IL inactivation caused a general reduction in conditioned suppression following discriminative conditioning, but not when using a single-stimulus design. Pharmacological disconnection of the PL → NAcS pathway revealed that this projection mediates conditioned suppression. These data add to a growing literature implicating discrete cortico-striatal pathways in the suppression of reward-seeking in response to aversive stimuli. Dysfunction within related structures may contribute to aberrant patterns of behavior in psychiatric illnesses including substance use disorders.

Prefrontal Regulation of Punished Ethanol Self-administration

BACKGROUND

A clinical hallmark of alcohol use disorder is persistent drinking despite potential adverse consequences. The ventromedial prefrontal cortex (vmPFC) and dorsomedial prefrontal cortex (dmPFC) are positioned to exert top-down control over subcortical regions, such as the nucleus accumbens shell (NAcS) and basolateral amygdala, which encode positive and negative valence of ethanol (EtOH)-related stimuli. Prior rodent studies have implicated these regions in regulation of punished EtOH self-administration (EtOH-SA).

METHODS

We conducted in vivo electrophysiological recordings in mouse vmPFC and dmPFC to obtain neuronal correlates of footshock-punished EtOH-SA. Ex vivo recordings were performed in NAcS D₁ receptor-expressing medium spiny neurons receiving vmPFC input to examine punishment-related plasticity in this pathway. Optogenetic photosilencing was employed to assess the functional contribution of the vmPFC, dmPFC, vmPFC projections to NAcS, or vmPFC projections to basolateral amygdala, to punished EtOH-SA.

RESULTS

Punishment reduced EtOH lever pressing and elicited aborted presses (lever approach followed by rapid retraction). Neurons in the vmPFC and dmPFC exhibited phasic firing to EtOH lever presses and aborts, but only in the vmPFC was there a population-level shift in coding from lever presses to aborts with punishment. Closed-loop vmPFC, but not dmPFC, photosilencing on a postpunishment probe test negated the reduction in EtOH lever presses but not in aborts. Punishment was associated with altered plasticity at vmPFC inputs to D₁ receptor-expressing medium spiny neurons in the NAcS. Photosilencing vmPFC projections to the NAcS, but not to the basolateral amygdala, partially reversed suppression of EtOH lever presses on probe testing.

CONCLUSIONS

These findings demonstrate a key role for the vmPFC in regulating EtOH-SA after punishment, with implications for understanding the neural basis of compulsive drinking in alcohol use disorder.

Basolateral amygdala - nucleus accumbens circuitry regulates optimal cue-guided risk/reward decision making

Maladaptive decision making is a characteristic feature of substance use disorder and pathological gambling. Studies in humans and animals have implicated neural circuits that include the basolateral amygdala (BLA) and nucleus accumbens (NAc) in facilitating risk/reward decision making. However, the preclinical literature has focussed primarily on situations where animals use internally-generated information to adapt to changes in reward likelihood, whereas many real-life situations require the use of external stimuli to facilitate context-appropriate behavior. We recently developed the "Blackjack" task, to measure cued risk/reward decision making requiring rats to chose between Small/Certain and Large/Risky rewards, with auditory cues at the start of each trial explicitly informing that the probability of obtaining a large reward was either good (50%) or poor (12.5%). Here we investigated the contribution of the BLA and its interaction with the NAc in guiding these types of decisions. In well-trained male rats, bilateral inactivation of the BLA induced suboptimal decision making, primarily by reducing risky choice on good-odds trials. In comparison, pharmacological disconnection of the BLA and NAc-shell also induced suboptimal decision making, diverting choice from more preferred option by reducing or increasing risky choice on good vs. poor odds trials respectively. Together, these results suggest that the BLA-NAc circuitry plays a crucial role in integrating information provided by discriminative stimuli. Furthermore, this circuitry may aid in guiding action selection of advantageous options in situations to maximize rewards. Finally, they suggest that perturbations in optimal decision making observed in substance abuse and gambling disorders may be driven in part by dysfunction within this circuitry.

Contribution of the prefrontal cortex and basolateral amygdala to behavioral decision-making under reward/punishment conflict

RATIONALE

Control of reward-seeking behavior under conditions of punishment is an important function for survival.

OBJECTIVES AND METHODS

We designed a task in which rats could choose to either press a lever and obtain a food pellet accompanied by a footshock or refrain from pressing the lever to avoid footshock, in response to tone presentation. In the task, footshock intensity steadily increased, and the task was terminated when the lever press probability reached < 25% (last intensity). Rats were trained until the last intensity was stable. Subsequently, we investigated the effects of the pharmacological inactivation of the ventromedial prefrontal cortex (vmPFC), lateral orbitofrontal cortex (lOFC), and basolateral amygdala (BLA) on task performance.

RESULTS

Bilateral inactivation of the vmPFC, lOFC, and BLA did not alter lever press responses at the early stage of the task. The number of lever presses increased following vmPFC and BLA inactivation but decreased following lOFC inactivation during the later stage of the task. The last intensity was elevated by vmPFC or BLA inactivation but lowered by lOFC inactivation. Disconnection of the vmPFC-BLA pathway induced behavioral alterations that were similar to vmPFC or BLA inactivation. Inactivation of any regions did not alter footshock sensitivity and anxiety levels.

CONCLUSIONS

Our results demonstrate a strong role of the vmPFC and BLA and their interactions in reward restraint to avoid punishment and a prominent role of the lOFC in reward-seeking under reward/punishment conflict situations.

Dissociable roles for the ventral and dorsal medial prefrontal cortex in cue-guided risk/reward decision making

Converging evidence from studies with animals and humans have implicated separate regions of the medial prefrontal cortex (mPFC) corresponding to the anterior cingulate cortex (ACC), in mediating different aspects of reward-related decisions involving uncertainty or risk. However, the dissociable contributions of subregions of the ACC remain unclear, as discrepancies exist between human neuroimaging findings and preclinical rodent studies. To clarify how ventral vs. dorsal regions of the mPFC contribute to risk/reward decision making, the present study assessed the effects of inactivation of different subregions on performance of a "Blackjack task" that measured cue-guided decision making and shares similarities with paradigms used with humans. Male, Long-Evans rats were well-trained to choose between a Small/Certain reward vs a Large/Risky reward delivered with variable probabilities (i.e., good vs. poor-odds, 50% vs. 12.5%). The odds of obtaining the larger reward was signaled by auditory cues at the start of each trial. Inactivation of the ventral, infralimbic region of the mPFC increased risky choice selectively when the odds of winning were poor. By contrast, inactivation of the prelimbic and anterior cingulate regions of the dorsal mPFC led to suboptimal reductions in risky choice on good-odds trials. The effects of prelimbic vs anterior cingulate inactivations were associated with context-dependent alterations in reward vs negative feedback, respectively. These results further clarify the distinct yet complementary manners in which separate ACC regions promote optimal risk/reward decision making and complement neuroimaging findings that activity in human ventral vs dorsal ACC promotes risk aversion or risky choices.

Risky decision-making predicts dopamine release dynamics in nucleus accumbens shell

The risky decision-making task (RDT) measures risk-taking in a rat model by assessing preference between a small, safe reward and a large reward with increasing risk of punishment (mild foot shock). It is well-established that dopaminergic drugs modulate risk-taking; however, little is known about how differences in baseline phasic dopamine signaling drive individual differences in risk preference. Here, we used in vivo fixed potential amperometry in male Long-Evans rats to test if phasic nucleus accumbens shell (NACs) dopamine dynamics are associated with risk-taking. We observed a positive correlation between medial forebrain bundle-evoked dopamine release in the NACs and risky decision-making, suggesting that risk-taking is associated with elevated dopamine sensitivity. Moreover, "risk-taking" subjects were found to demonstrate greater phasic dopamine release than "risk-averse" subjects. Risky decision-making also predicted enhanced sensitivity to the dopamine reuptake inhibitor nomifensine, and elevated autoreceptor function. Importantly, this hyperdopaminergic phenotype was selective for risky decision-making, as delay discounting performance was not predictive of phasic dopamine release or dopamine supply. These data identify phasic NACs dopamine release as a possible therapeutic target for alleviating the excessive risk-taking observed across multiple forms of psychopathology.

Random access to palatable food stimulates similar addiction-like responses as a fixed schedule, but only a fixed schedule elicits anticipatory activation

Restricted intermittent food access to palatable food (PF) induces addiction-like behaviors and plastic changes in corticolimbic brain areas. Intermittent access protocols normally schedule PF to a fixed time, enabling animals to predict the arrival of PF. Because outside the laboratory the presence of PF may occur in a random unpredictable manner, the present study explored whether random access to PF would stimulate similar addiction-like responses as observed under a fixed scheduled. Rats were randomly assigned to a control group without chocolate access, to ad libitum access to chocolate, to fixed intermittent access (CH-F), or to random unpredictable access (CH-R) to chocolate. Only the CH-F group developed behavioral and core temperature anticipation to PF access. Both groups exposed to intermittent access to PF showed binge eating, increased effort behaviors to obtain chocolate, as well as high FosB/ΔFosB in corticolimbic areas. Moreover, FosB/ΔFosB in all areas correlated with the intensity of binge eating and effort behaviors. We conclude that both conditions of intermittent access to PF stimulate addiction-like behaviors and FosB/ΔFosB accumulation in brain reward areas; while only a fixed schedule, which provides a time clue, elicited anticipatory activation, which is strongly associated with craving behaviors and may favor relapse during withdrawal.

Three Rostromedial Tegmental Afferents Drive Triply Dissociable Aspects of Punishment Learning and Aversive Valence Encoding

Persistence of reward seeking despite punishment or other negative consequences is a defining feature of mania and addiction, and numerous brain regions have been implicated in such punishment learning, but in disparate ways that are difficult to reconcile. We now show that the ability of an aversive punisher to inhibit reward seeking depends on coordinated activity of three distinct afferents to the rostromedial tegmental nucleus (RMTg) arising from cortex, brainstem, and habenula that drive triply dissociable RMTg responses to aversive cues, outcomes, and prediction errors, respectively. These three pathways drive correspondingly dissociable aspects of punishment learning. The RMTg in turn drives negative, but not positive, valence encoding patterns in the ventral tegmental area (VTA). Hence, punishment learning involves pathways and functions that are highly distinct, yet tightly coordinated.

A Focus on the Functions of Area 25

Subcallosal area 25 is one of the least understood regions of the anterior cingulate cortex, but activity in this area is emerging as a crucial correlate of mood and affective disorder symptomatology. The cortical and subcortical connectivity of area 25 suggests it may act as an interface between the bioregulatory and emotional states that are aberrant in disorders such as depression. However, evidence for such a role is limited because of uncertainty over the functional homologue of area 25 in rodents, which hinders cross-species translation. This emphasizes the need for causal manipulations in monkeys in which area 25, and the prefrontal and cingulate regions in which it is embedded, resemble those of humans more than rodents. In this review, we consider physiological and behavioral evidence from non-pathological and pathological studies in humans and from manipulations of area 25 in monkeys and its putative homologue, the infralimbic cortex (IL), in rodents. We highlight the similarities between area 25 function in monkeys and IL function in rodents with respect to the regulation of reward-driven responses, but also the apparent inconsistencies in the regulation of threat responses, not only between the rodent and monkey literatures, but also within the rodent literature. Overall, we provide evidence for a causal role of area 25 in both the enhanced negative affect and decreased positive affect that is characteristic of affective disorders, and the cardiovascular and endocrine perturbations that accompany these mood changes. We end with a brief consideration of how future studies should be tailored to best translate these findings into the clinic.

Context-induced relapse after extinction versus punishment: similarities and differences

Results from clinical studies suggest that drug relapse and craving are often provoked by exposure to drug-associated contexts. Since 2002, this phenomenon has been modeled in laboratory animals using the ABA renewal model. In the classical version of this model, rats with a history of drug self-administration in one context (A) undergo extinction in a different context (B) and reinstate (or relapse to) drug seeking after exposure to the original drug-associated context (A). In a more recent version of the model introduced in 2013, the experimental conditions in context A are identical to those used in the classical model, but drug-reinforced responding in context B is suppressed by probabilistic punishment. The punishment-based ABA renewal model is proposed to resemble abstinence in humans, which is often initiated by the desire to avoid the negative consequences of drug use. The goal of our review is to discuss similarities and differences in mechanisms that play a role in suppression of drug seeking in context B and context-induced relapse to drug seeking in context A in the two models. We first describe psychological mechanisms that mediate extinction and punishment of drug-reinforced responding in context B. We then summarize recent findings on brain mechanisms of context-induced relapse of drug seeking after extinction, or punishment-imposed abstinence. These findings demonstrate both similarities and differences in brain mechanisms underlying relapse in the two variations of the ABA renewal model. We conclude by briefly discussing clinical implications of the preclinical studies.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Behavioral and neurobiological mechanisms of punishment: implications for psychiatric disorders

Punishment involves learning about the relationship between behavior and its adverse consequences. Punishment is fundamental to reinforcement learning, decision-making and choice, and is disrupted in psychiatric disorders such as addiction, depression, and psychopathy. However, little is known about the brain mechanisms of punishment and much of what is known is derived from study of superficially similar, but fundamentally distinct, forms of aversive learning such as fear conditioning and avoidance learning. Here we outline the unique conditions that support punishment, the contents of its learning, and its behavioral consequences. We consider evidence implicating GABA and monoamine neurotransmitter systems, as well as corticostriatal, amygdala, and dopamine circuits in punishment. We show how maladaptive punishment processes are implicated in addictions, impulse control disorders, psychopathy, anxiety, and depression and argue that a better understanding of the cellular, circuit, and cognitive mechanisms of punishment will make important contributions to next generation therapeutic approaches.

Differential Contributions of Nucleus Accumbens Subregions to Cue-Guided Risk/Reward Decision Making and Implementation of Conditional Rules

The nucleus accumbens (NAc) is a key node within corticolimbic circuitry for guiding action selection and cost/benefit decision making in situations involving reward uncertainty. Preclinical studies have typically assessed risk/reward decision making using assays where decisions are guided by internally generated representations of choice-outcome contingencies. Yet, real-life decisions are often influenced by external stimuli that inform about likelihoods of obtaining rewards. How different subregions of the NAc mediate decision making in such situations is unclear. Here, we used a novel assay colloquially termed the "Blackjack" task that models these types of situations. Male Long-Evans rats were trained to choose between one lever that always delivered a one-pellet reward and another that delivered four pellets with different probabilities [either 50% (good-odds) or 12.5% (poor-odds)], which were signaled by one of two auditory cues. Under control conditions, rats selected the large/risky option more often on good-odds versus poor-odds trials. Inactivation of the NAc core caused indiscriminate choice patterns. In contrast, NAc shell inactivation increased risky choice, more prominently on poor-odds trials. Additional experiments revealed that both subregions contribute to auditory conditional discrimination. NAc core or shell inactivation reduced Pavlovian approach elicited by an auditory CS+, yet shell inactivation also increased responding during presentation of a CS-. These data highlight distinct contributions for NAc subregions in decision making and reward seeking guided by discriminative stimuli. The core is crucial for implementation of conditional rules, whereas the shell refines reward seeking by mitigating the allure of larger, unlikely rewards and reducing expression of inappropriate or non-rewarded actions.SIGNIFICANCE STATEMENT Using external cues to guide decision making is crucial for adaptive behavior. Deficits in cue-guided behavior have been associated with neuropsychiatric disorders, such as attention deficit hyperactivity disorder and schizophrenia, which in turn has been linked to aberrant processing in the nucleus accumbens. However, many preclinical studies have often assessed risk/reward decision making in the absence of explicit cues. The current study fills that gap by using a novel task that allows for the assessment of cue-guided risk/reward decision making in rodents. Our findings identified distinct yet complementary roles for the medial versus lateral portions of this nucleus that provide a broader understanding of the differential contributions it makes to decision making and reward seeking guided by discriminative stimuli.

Opposite Effects of Basolateral Amygdala Inactivation on Context-Induced Relapse to Cocaine Seeking after Extinction versus Punishment

Studies using the renewal procedure showed that basolateral amygdala (BLA) inactivation inhibits context-induced relapse to cocaine-seeking after extinction. Here, we determined whether BLA inactivation would also inhibit context-induced relapse after drug-reinforced responding is suppressed by punishment, an animal model of human relapse after self-imposed abstinence due to adverse consequences of drug use. We also determined the effect of central amygdala (CeA) inactivation on context-induced relapse.We trained rats to self-administer cocaine for 12 d (6 h/d) in Context A and then exposed them to either extinction or punishment training for 8 d in Context B. During punishment, 50% of cocaine-reinforced lever-presses produced an aversive footshock of increasing intensity (0.1-0.5 or 0.7 mA). We then tested the rats for relapse to cocaine seeking in the absence of cocaine or shock in Contexts A and B after BLA or CeA injections of vehicle or GABA agonists (muscimol-baclofen). We then retrained the rats for cocaine self-administration in Context A, repunished or re-extinguished lever pressing in Context B, and retested for relapse after BLA or CeA inactivation.BLA or CeA inactivation decreased context-induced relapse in Context A after extinction in Context B. BLA, but not CeA, inactivation increased context-induced relapse in Context A after punishment in Context B. BLA or CeA inactivation provoked relapse in Context B after punishment but not extinction. Results demonstrate that amygdala's role in relapse depends on the method used to achieve abstinence and highlights the importance of studying relapse under abstinence conditions that more closely mimic the human condition.SIGNIFICANCE STATEMENT Relapse to drug use during abstinence is often provoked by re-exposure to the drug self-administration environment or context. Studies using the established extinction-reinstatement rodent model of drug relapse have shown that inactivation of the basolateral amygdala inhibits context-induced drug relapse after extinction of the drug-reinforced responding. Here, we determined whether basolateral amygdala inactivation would also inhibit relapse after drug-reinforced responding is suppressed by punishment, a model of human relapse after self-imposed abstinence. Unexpectedly, we found that basolateral amygdala inactivation had opposite effects on relapse provoked by re-exposure to the drug self-administration environment after extinction versus punishment. Our results demonstrate that depending on the historical conditions that lead to abstinence, amygdala activity can either promote or inhibit relapse.

Optogenetic Inhibition Reveals Distinct Roles for Basolateral Amygdala Activity at Discrete Time Points during Risky Decision Making

Decision making is a multifaceted process, consisting of several distinct phases that likely require different cognitive operations. Previous work showed that the basolateral amygdala (BLA) is a critical substrate for decision making involving risk of punishment; however, it is unclear how the BLA is recruited at different stages of the decision process. To this end, the current study used optogenetics to inhibit the BLA during specific task phases in a model of risky decision making (risky decision-making task) in which rats choose between a small, "safe" reward and a large reward accompanied by varying probabilities of footshock punishment. Male Long-Evans rats received intra-BLA microinjections of viral vectors carrying either halorhodopsin (eNpHR3.0-mCherry) or mCherry alone (control) followed by optic fiber implants and were trained in the risky decision-making task. Laser delivery during the task occurred during intertrial interval, deliberation, or reward outcome phases, the latter of which was further divided into the three possible outcomes (small, safe; large, unpunished; large, punished). Inhibition of the BLA selectively during the deliberation phase decreased choice of the large, risky outcome (decreased risky choice). In contrast, BLA inhibition selectively during delivery of the large, punished outcome increased risky choice. Inhibition had no effect during the other phases, nor did laser delivery affect performance in control rats. Collectively, these data indicate that the BLA can either inhibit or promote choice of risky options, depending on the phase of the decision process in which it is active.SIGNIFICANCE STATEMENT To date, most behavioral neuroscience research on neural mechanisms of decision making has used techniques that preclude assessment of distinct phases of the decision process. Here we show that optogenetic inhibition of the BLA has opposite effects on choice behavior in a rat model of risky decision making, depending on the phase in which inhibition occurs. BLA inhibition during a period of deliberation between small, safe and large, risky outcomes decreased risky choice. In contrast, BLA inhibition during receipt of the large, punished outcome increased risky choice. These findings highlight the importance of temporally targeted approaches to understand neural substrates underlying complex cognitive processes. More importantly, they reveal novel information about dynamic BLA modulation of risky choice.