Prefrontal cortical contribution to risk-based decision making

Concurrent behavioral modeling and multimodal neuroimaging reveals how feedback affects the performance of decision making in internet gaming disorder

Internet gaming disorder (IGD) prompts inquiry into how feedback from prior gaming rounds influences subsequent risk-taking behavior and potential neural mechanisms. Forty-two participants, including 15 with IGD and 27 health controls (HCs), underwent a sequential risk-taking task. Hierarchy Bayesian modeling was adopted to measure risky propensity, behavioral consistence, and affection by emotion ratings from last trial. Concurrent electroencephalogram and functional near-infrared spectroscopy (EEG-fNIRS) recordings were performed to demonstrate when, where and how the previous-round feedback affects the decision making to the next round. We discovered that the IGD illustrated heightened risk-taking propensity as compared to the HCs, indicating by the computational modeling (p = 0.028). EEG results also showed significant time window differences in univariate and multivariate pattern analysis between the IGD and HCs after the loss of the game. Further, reduced brain activation in the prefrontal cortex during the task was detected in IGD as compared to that of the control group. The findings underscore the importance of understanding the aberrant decision-making processes in IGD and suggest potential implications for future interventions and treatments aimed at addressing this behavioral addiction.

Noradrenergic modulation of stress induced catecholamine release: Opposing influence of FG7142 and yohimbine

BACKGROUND

Life stress modulates decision making, particularly in the face of risk, in some cases prompting vulnerable populations to make suboptimal, life-altering choices. In the brain, stress is known to alter the extracellular release of catecholamines in structures such as basolateral amygdala (BLA) and nucleus accumbens (NAc), but the relationship between catecholamines and decision-making behavior under stress has not been systemically explored.

METHODS

We developed an operant touchscreen decision-making task for rats comprising elements of loss aversion and risk seeking behavior. Rats were first injected systemically with an adrenergic alpha 2A-receptor agonist (guanfacine) and antagonist (yohimbine), as well as a partial inverse GABAA agonist, FG 7142, known to induce anxiety and stress related physiological responses in a variety of species, including humans. We then used fiber photometry to monitor NE in the basolateral amygdala (BLA), and DA activity in the nucleus accumbens (NAc) while animals engaged in decision-making and following systemic injections of FG 7142 and yohimbine.

RESULTS

Neither yohimbine nor guanfacine had any impact on decision making strategy but altered motivational state with yohimbine making the animal almost insensitive to the reward outcome. The pharmacological induction of stress with FG 7142 biased the rats decisions towards safety, but this bias shifted toward risk when co-treated with yohimbine. In the BLA and NAc, the FG 7142 altered catecholamine release, with systemic yohimbine producing opposing effects on NE and DA release.

CONCLUSIONS

Stress induced changes in catecholamine release in the BLA and NAc can directly influence loss sensitivity, decisions and motivation, which can be modulated by the alpha 2A adrenoceptor antagonist, yohimbine.

Perturbations in risk/reward decision making and frontal cortical catecholamine regulation induced by mild traumatic brain injury

Mild traumatic brain injury (mTBI) disrupts cognitive processes that influence risk taking behavior. Little is known regarding the effects of repetitive mild injury (rmTBI) or whether these outcomes are sex specific. Risk/reward decision making is mediated by the prefrontal cortex (PFC), which is densely innervated by catecholaminergic fibers. Aberrant PFC catecholamine activity has been documented following TBI and may underlie TBI-induced risky behavior. The present study characterized the effects of rmTBI on risk/reward decision making behavior and catecholamine transmitter regulatory proteins within the PFC. Rats were exposed to sham, single (smTBI), or three closed-head controlled cortical impact (CH-CCI) injuries and assessed for injury-induced effects on risk/reward decision making using a probabilistic discounting task (PDT). In the first week post-final surgery, mTBI increased risky choice preference. By the fourth week, males exhibited increased latencies to make risky choices following rmTBI, demonstrating a delayed effect on processing speed. When levels of tyrosine hydroxylase (TH) and the norepinephrine reuptake transporter (NET) were measured within subregions of the PFC, females exhibited dramatic increases of TH levels within the orbitofrontal cortex (OFC) following smTBI. However, both males and females demonstrated reduced levels of OFC NET following rmTBI. These results indicate the OFC is susceptible to catecholamine instability after rmTBI and suggests that not all areas of the PFC contribute equally to TBI-induced imbalances. Overall, the CH-CCI model of rmTBI has revealed time-dependent and sex-specific changes in risk/reward decision making and catecholamine regulation following repetitive mild head injuries.

Medial prefrontal cortex suppresses reward-seeking behavior with risk of punishment by reducing sensitivity to reward

Reward-seeking behavior is frequently associated with risk of punishment. There are two types of punishment: positive punishment, which is defined as addition of an aversive stimulus, and negative punishment, involves the omission of a rewarding outcome. Although the medial prefrontal cortex (mPFC) is important in avoiding punishment, whether it is important for avoiding both positive and negative punishment and how it contributes to such avoidance are not clear. In this study, we trained male mice to perform decision-making tasks under the risks of positive (air-puff stimulus) and negative (reward omission) punishment, and modeled their behavior with reinforcement learning. Following the training, we pharmacologically inhibited the mPFC. We found that pharmacological inactivation of mPFC enhanced the reward-seeking choice under the risk of positive, but not negative, punishment. In reinforcement learning models, this behavioral change was well-explained as an increase in sensitivity to reward, rather than a decrease in the strength of aversion to punishment. Our results suggest that mPFC suppresses reward-seeking behavior by reducing sensitivity to reward under the risk of positive punishment.

Adolescent THC impacts on mPFC dopamine-mediated cognitive processes in male and female rats

Rationale

Adolescent cannabis use is linked to later-life changes in cognition, learning, and memory. Rodent experimental studies suggest Δ9-tetrahydrocannabinol (THC) influences development of circuits underlying these processes, especially in the prefrontal cortex, which matures during adolescence.

Objective

We determined how 14 daily THC injections (5mg/kg) during adolescence persistently impacts medial prefrontal cortex (mPFC) dopamine-dependent cognition.

Methods

In adult Long Evans rats treated as adolescents with THC (AdoTHC), we quantify performance on two mPFC dopamine-dependent reward-based tasks-strategy set shifting and probabilistic discounting. We also determined how acute dopamine augmentation with amphetamine (0, 0.25, 0.5 mg/kg), or specific chemogenetic stimulation of ventral tegmental area (VTA) dopamine neurons and their projections to mPFC impacts probabilistic discounting.

Results

AdoTHC sex-dependently impacts acquisition of cue-guided instrumental reward seeking, but has minimal effects on set-shifting or probabilistic discounting in either sex. When we challenged dopamine circuits acutely with amphetamine during probabilistic discounting, we found reduced discounting of improbable reward options, with AdoTHC rats being more sensitive to these effects than controls. In contrast, neither acute chemogenetic stimulation of VTA dopamine neurons nor pathway-specific chemogenetic stimulation of their projection to mPFC impacted probabilistic discounting in control rats, although stimulation of this cortical dopamine projection slightly disrupted choices in AdoTHC rats.

Conclusions

These studies confirm a marked specificity in the cognitive processes impacted by AdoTHC exposure. They also suggest that some persistent AdoTHC effects may alter amphetamine-induced cognitive changes in a manner independent of VTA dopamine projections to mPFC, or via alterations of non-VTA dopamine neurons.

Clinical efficacy of deep transcranial magnetic stimulation (dTMS) in psychiatric and cognitive disorders: A systematic review

Deep transcranial magnetic stimulation (dTMS) has gained attention as an enhanced form of traditional TMS, targeting broader and deeper regions of the brain. However, a fulsome synthesis of dTMS efficacy across psychiatric and cognitive disorders using sham-controlled trials is lacking. We systematically reviewed 28 clinical trials comparing active dTMS to a sham/controlled condition to characterize dTMS efficacy across diverse psychiatric and cognitive disorders. A comprehensive search of APA PsycINFO, Cochrane, Embase, Medline, and PubMed databases was conducted. Predominant evidence supports dTMS efficacy in patients with obsessive-compulsive disorder (OCD; n = 2), substance use disorders (SUDs; n = 8), and in those experiencing depressive episodes with major depressive disorder (MDD) or bipolar disorder (BD; n = 6). However, the clinical efficacy of dTMS in psychiatric disorders characterized by hyperactivity or hyperarousal (i.e., attention-deficit/hyperactivity disorder, posttraumatic stress disorder, and schizophrenia) was heterogeneous. Common side effects included headaches and pain/discomfort, with rare but serious adverse events such as seizures and suicidal ideation/attempts. Risk of bias ratings indicated a collectively low risk according to the Grading of Recommendations, Assessment, Development, and Evaluations checklist (Meader et al., 2014). Literature suggests promise for dTMS as a beneficial alternative or add-on treatment for patients who do not respond well to traditional treatment, particularly for depressive episodes, OCD, and SUDs. Mixed evidence and limited clinical trials for other psychiatric and cognitive disorders suggest more extensive research is warranted. Future research should examine the durability of dTMS interventions and identify moderators of clinical efficacy.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

Early resource scarcity causes cortical astrocyte enlargement and sex-specific changes in the orbitofrontal cortex transcriptome in adult rats

Astrocyte morphology affects function, including the regulation of glutamatergic signaling. This morphology changes dynamically in response to the environment. However, how early life manipulations alter adult cortical astrocyte morphology is underexplored. Our lab uses brief postnatal resource scarcity, the limited bedding and nesting (LBN) manipulation, in rats. We previously found that LBN augments maternal behaviors and promotes later resilience to adult addiction-related behaviors, reducing impulsivity, risky decision-making, and morphine self-administration. These behaviors rely on glutamatergic transmission in the medial orbitofrontal (mOFC) and medial prefrontal (mPFC) cortex. Here we tested whether LBN changed astrocyte morphology in the mOFC and mPFC of adult rats using a novel viral approach that, unlike traditional markers, fully labels astrocytes. Prior exposure to LBN causes an increase in the surface area and volume of astrocytes in the mOFC and mPFC of adult males and females relative to control-raised rats. We next used bulk RNA sequencing of OFC tissue to assess transcriptional changes that could increase astrocyte size in LBN rats. LBN caused mainly sex-specific changes in differentially expressed genes. Pathway analysis revealed that OFC glutamatergic signaling is altered by LBN in males and females, but the gene changes in that pathway differed across sex. This may represent a convergent sex difference where glutamatergic signaling, which affects astrocyte morphology, is altered by LBN via sex-specific mechanisms. Collectively, these studies highlight that astrocytes may be an important cell type that mediates the effect of early resource scarcity on adult brain function.

Still a "hidden island"? The rodent insular cortex in drug seeking, reward, and risk

The insular cortex (IC) is implicated in risky decision making and drug-seeking behaviors, in a manner dissociable from natural reward seeking. However, evidence from rodent studies of motivated behaviors suggests that the role of the IC is not always consistent across procedures. Moreover, there is evidence of dissociation of function between posterior (pIC) and anterior (aIC) subregions in these behaviors. Under which circumstances, and by which mechanisms, these IC subregions are recruited to regulate motivated behaviors remains unclear. Here, we discuss evidence of rodent pIC and aIC function across drug-related behaviors, natural reward seeking, and decision making under risk and highlight procedural differences that may account for seemingly conflicting findings. Although gaps in the literature persist, we hypothesize that IC activity is broadly important for selection of appropriate behaviors based on learned action-outcome contingencies and that associated risk is sufficient, but not necessary, to recruit the aIC in reward seeking without involving the pIC.

A calculus of probability or belief? Neural underpinnings of social decision-making in a card game

For decades, the prefrontal cortex (PFC) has been the focus of social neuroscience research, specifically regarding its role in competitive social decision-making. However, the distinct contributions of PFC subregions when making strategic decisions involving multiple types of information (social, non-social, and mixed information) remain unclear. This study investigates decision-making strategies (pure probability calculation vs. mentalizing) and their neural representations using functional near-infrared spectroscopy (fNIRS) data collected during a two-person card game. We observed individual differences in information processing strategy, indicating that some participants relied more on probability than others. Overall, the use of pure probability decreased over time in favor of other types of information (e.g., mixed information), with this effect being more pronounced within-round trials than across rounds. In the brain, (1) the lateral PFC activates when decisions are driven by probability calculations; (2) the right lateral PFC responds to trial difficulty; and (3) the anterior medial PFC is engaged when decision-making involves mentalizing. Furthermore, neural synchrony, which reflects the real-time interplay between individuals' cognitive processes, did not consistently contribute to correct decisions and fluctuated throughout the experiment, suggesting a hierarchical mentalizing mechanism at work.

Early resource scarcity causes cortical astrocyte enlargement and sex-specific changes in the orbitofrontal cortex transcriptome in adult rats

Astrocyte morphology affects function, including the regulation of glutamatergic signaling. This morphology changes dynamically in response to the environment. However, how early life manipulations alter adult cortical astrocyte morphology is underexplored. Our lab uses brief postnatal resource scarcity, the limited bedding and nesting (LBN) manipulation, in rats. We previously found that LBN promotes later resilience to adult addiction-related behaviors, reducing impulsivity, risky decision-making, and morphine self-administration. These behaviors rely on glutamatergic transmission in the medial orbitofrontal (mOFC) and medial prefrontal (mPFC) cortex. Here we tested whether LBN changed astrocyte morphology in the mOFC and mPFC of adult rats using a novel viral approach that, unlike traditional markers, fully labels astrocytes. Prior exposure to LBN causes an increase in the surface area and volume of astrocytes in the mOFC and mPFC of adult males and females relative to control-raised rats. We next used bulk RNA sequencing of OFC tissue to assess transcriptional changes that could increase astrocyte size in LBN rats. LBN caused mainly sex-specific changes in differentially expressed genes. However, Park7, which encodes for the protein DJ-1 that alters astrocyte morphology, was increased by LBN across sex. Pathway analysis revealed that OFC glutamatergic signaling is altered by LBN in males and females, but the gene changes in that pathway differed across sex. This may represent a convergent sex difference where glutamatergic signaling, which affects astrocyte morphology, is altered by LBN via sex-specific mechanisms. Collectively, these studies highlight that astrocytes may be an important cell type that mediates the effect of early resource scarcity on adult brain function.

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

The Effects of the Inhalant Toluene on Cognitive Function and Behavioral Flexibility: A Review of Recent Findings

Substance use disorder (SUD) is characterized, in part, by lack of control over drug seeking and taking. The prefrontal cortex (PFC) is highly involved in control of behavior and deficits in PFC structure and function have been demonstrated in clinical and preclinical studies of SUD. Of the various classes of drugs associated with the development of SUD, inhalants are among the least studied despite their widespread use among adolescents and children. In this work, we review what is currently known regarding the sites and mechanisms of action of inhalants with a focus on the volatile solvent toluene that is contained in a wide variety of legal and easily obtained products. We then describe how inhalants including toluene affect various behaviors with an emphasis on those associated with PFC function and how chronic use of inhalants alters brain structure and neuronal signaling. Findings from these studies highlight advances made in recent years that have expanded our understanding of the effects of inhalants on brain structure and reinforce the need for continued work in this field.

Effect of psilocybin on decision-making and motivation in the healthy rat

Psilocybin and its active metabolite psilocin are hallucinogenic serotonergic agonists with high affinity for several serotonin receptors. In addition to underlying the hallucinogenic effects of these compounds, serotonin receptor activation also has important effects on decision-making and goal-directed behaviors. The impact of psilocybin and psilocin on these cognitive systems, however, remains unclear. This study investigated the effects of psilocybin treatment on decision-making and motivation in healthy male and female rats. We compared probability and delay discounting performance of psilocybin treated (1 mg/kg) to vehicle rats (n = 10/sex/group), and further assessed motivation in each group using a progressive ratio task. We also confirmed drug action by assessing head twitch responses after psilocybin treatment (1 mg/kg). Results from this study demonstrated that exposure to 1 mg/kg psilocybin did not affect decision-making in the probability and delay discounting tasks and did not reduce response rates in the progressive ratio task. However, psilocybin treatment did cause the expected increase in head twitch responses in both male and female rats, demonstrating that the drug was delivered at a pharmacologically relevant dosage. Combined, these results suggest that psilocybin may not impair or improve decision-making and motivation. Considering recent interest in psilocybin as a potential fast-acting therapeutic for a variety of mental health disorders, our findings also suggest the therapeutic effects of this drug may not be mediated by changes to the brain systems underlying reward and decision-making. Finally, these results may have important implications regarding the relative safety of this compound, suggesting that widespread cognitive impairments may not be seen in subjects, even after chronic treatment.

Temporal Dynamics Underlying Prelimbic Prefrontal Cortical Regulation of Action Selection and Outcome Evaluation during Risk/Reward Decision-Making

Risk/reward decision-making is a dynamic process that includes periods of deliberation before action selection and evaluation of the action outcomes that bias subsequent choices. Inactivation of the prelimbic (PL) cortex has revealed its integral role in updating decision biases in the face of changes in probabilistic reward contingencies, yet how phasic PL signals during different phases of the decision process influence choice remains unclear. We used temporally specific optogenetic inhibition to selectively disrupt PL activity coinciding with action selection and outcome phases to examine how these signals influence choice. Male rats expressing the inhibitory opsin eArchT within PL excitatory neurons were well trained on a probabilistic discounting task, entailing choice between small/certain versus large/risky rewards, the probability of which varied over a session (50-12.5%). During testing, brief light pulses suppressed PL activity before choice or after different outcomes. Prechoice suppression reduced bias toward more preferred/higher utility options and disrupted how recent outcomes influenced subsequent choice. Inhibition during risky losses induced a similar profile, but here, the impact of reward omissions were either amplified or diminished, relative to the context of the estimated profitability of the risky option. Inhibition during large or small reward receipt reduced risky choice when this option was more profitable, suggesting these signals can both reinforce rewarded risky choices and also act as a relative value comparator signal that augments incentive for larger rewards. These findings reveal multifaceted contributions by the PL in implementing decisions and integrating action-outcome feedback to assign context to the decision space.SIGNIFICANCE STATEMENT The PL prefrontal cortex plays an integral role in guiding risk/reward decisions, but how activity in this region during different phases of the decision process influences choice is unclear. By using temporally specific optogenetic manipulations of this activity, the present study unveiled previously uncharacterized and differential contributions by PL in implementing decision policies and how evaluation of decision outcomes shape subsequent choice. These findings provide novel insight into the dynamic processes engaged by the PL that underlie action selection in situations involving reward uncertainty that may aid in understanding the mechanism underlying normal and aberrant decision-making processes.

Modulation of GABAB receptors in the insula bidirectionally affects associative memory of epilectic rats in both spatial and non-spatial operant tasks

Background

Stimulation of gamma-aminobutyric acid (GABA) activity through GABA receptor agonists is the basic mechanism of many anticonvulsant drugs. Nevertheless, many of these GABergic drugs have adverse cognitive effects. We previously found that GABAB receptors (GABA_BRs) in the insula regulate operant associative memory in healthy rats. The present study aimed at investigating the effects of GABA_BR modulation in the insula on operant associative memory in epileptic rats, along with the underlying mechanisms.

Methods

The lithium-pilocarpine model of temporal lobe epilepsy (TLE) was established in male Sprague-Dawley rats. A 22-gauge stainless-steel guide cannula was surgically implanted into the granular insula cortex of the epileptic rats. Baclofen (125 ng/μl, 1 μl), CGP35348 (12.5 μg/μl, 1 μl), or saline (1 μl) was slowly infused through the guide cannula. The Intellicage automated behavioral testing system was used to evaluate operant associative memory of the epileptic rats, including non-spatial operant tasks (basic nosepoke learning and skilled nosepoke learning) and spatial operant tasks (chamber position learning). The expression of the GABA_BR subunits GB1 and GB2 in the insula was examined by immunofluorescence and Western blotting.

Results

The Intellicage tests demonstrated that baclofen significantly impaired basic nosepoke learning, skilled nosepoke learning and chamber position learning of the epileptic rats, while CGP35348 boosted these functions. Immunofluorescence staining revealed that GB1 and GB2 were expressed in the insula of the epileptic rats, and Western blotting analysis showed that baclofen enhanced while CGP35348 inhibited the expression of these subunits.

Conclusion

GABA_BRs in the insula bidirectionally regulate both spatial and non-spatial operant associative memory of epileptic rats. Effects of GABA_BRs on cognition should be taken into account when evaluating new possible treatments for people with epilepsy.

Prelimbic cortex neural encoding dynamically tracks expected outcome value

Animals must modify their behavior based on updated expected outcomes in a changing environment. Prelimbic cortex (PrL) neural encoding during learning predicts, and is necessary for, appropriately altering behavior based on a new expected outcome value following devaluation. We aimed to determine how PrL neural activity encodes reward predictive cues after the expected outcome value of those cues is decreased following conditioned taste aversion. In one post-devaluation session, rats were tested under extinction to determine their ability to alter their behavior to the expected outcome values (i.e., extinction test). In a second post-devaluation session, rats were tested with the newly devalued outcome delivered so that the rats experienced the updated outcome value within the session (i.e., re-exposure test). We found that PrL neural encoding of the cue associated with the devalued reward predicted the ability of rats to suppress behavior in the extinction test session, but not in the re-exposure test session. While all rats were able to successfully devalue the outcome during conditioned taste aversion, a subset of rats continued to consume the devalued outcome in the re-exposure test session. We found differential patterns of PrL neural encoding in the population of rats that did not avoid the devalued outcome during the re-exposure test compared to the rats that successfully avoided the devalued outcome. Our findings suggest that PrL neural encoding dynamically tracks expected outcome values, and differential neural encoding in the PrL to reward predictive cues following expected outcome value changes may contribute to distinct behavioral phenotypes.

From concepts to treatment: a dialog between a preclinical researcher and a clinician in addiction medicine

The debate surrounding the brain disease model and the associated questioning of the relevance of animal models is polarizing the field of addiction, and tends to widen the gap between preclinical research and addiction medicine. Here, we aimed at bridging this gap by establishing a dialog between a preclinical researcher and a clinician in addiction medicine. Our objective was to evaluate animal models and the neuroscientific conceptualization of addiction in light of alcohol or drug dependence and treatment in patients struggling with an addiction. We sought to determine how preclinical research influenced addiction medicine over past decades, and reciprocally, what can preclinical researchers learn from addiction medicine that could lead to more effective approaches. In this dialog, we talk about the co-evolution of addiction concepts and treatments from neuroscientific and medical perspectives. This dialog illustrates the reciprocal influences and mutual enrichment between the two disciplines and reveals that, although preclinical research might not produce new pharmacotherapies, it does shape the theoretical conceptualization of addiction and could thereby contribute to the implementation of therapeutic approaches.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Prefrontal Dopaminergic Regulation of Cue-Guided Risky Decision-Making Performance in Rats

Risky decision-making is the decision made by individuals when they know the probability of each outcome. In order to survive in unpredictable environments, it is necessary for individuals to assess the probability of events occurring to an make appropriate decisions. There are few studies on the neural basis of risky decision-making behavior guided by external cues, which is related to the relative paucity of animal behavioral paradigms. Previous studies have shown that the prefrontal cortex (PFC) plays a key role in risk-based decision-making. The PFC receives projections from the dopamine (DA) system from the ventral tegmental area of the midbrain. The mesocorticolimbic DA system regulates the judgments of reward and value in decision-making. However, the specific receptor mechanism for prefrontal DA regulation of cue-guided risky decision-making behavior remains unclear. Here we established a cue-guided risky decision-making behavioral paradigm (RDM task) to detect the behavior of rats making decisions between a small certain reward and a large uncertain reward in a self-paced manner. The D1 receptor antagonist SCH-23390 (5 mM) or agonist SKF-82958 (5 mM), and the D2 receptor antagonist thioridazine hydrochloride (5 mM) or agonist MLS-1547 (5 mM) was injected into the mPFC, respectively, to investigate how the behavior in the RDM task was changed. The results showed that: (1) rats were able to master the operation of the cue-guided RDM task in a self-paced way; (2) a majority of rats were inclined to choose risk rather than a safe option when the reward expectations were equal; and (3) risk selection was reduced upon inhibition of D1 receptors or stimulation of D2 receptors, but increased upon stimulation of D1 receptors or inhibition of D2 receptors, suggesting that the RDM performance is regulated by D1 and D2 receptors in the mPFC. The present results suggest that DA receptors in the mPFC of rats are involved in regulating cue-guided RDM behavior, with differential involvement of D1 and D2 receptors in the regulation.

Cannabinoid receptor type 1 antagonists alter aspects of risk/reward decision making independent of toluene-mediated effects

Drugs of abuse including cannabis and inhalants impair risk/reward decision making. Cannabis use is often concurrent with inhalant intoxication; yet, preclinical studies investigating the role of endocannabinoids in inhalant misuse are limited. To address this gap in the literature, we used the well-validated probabilistic discounting task to assess risk/reward decision making in rodents following combinations of toluene vapor (a common inhalant) and manipulations of cannabinoid receptor type 1 (CB1R) signaling. As reported previously, acute exposure to toluene vapor disrupted behavioral flexibility during probabilistic discounting. Systemic administration of the CB1R inverse agonist AM281 did not prevent toluene-induced alterations in risky choices, but did independently reduce win-stay behavior, increase choice latency, and increase omissions. Toluene-induced deficits in probabilistic discounting are thought to involve impaired medial prefrontal cortex (mPFC) activity. As we previously reported that some of toluene's inhibitory effects on glutamatergic signaling in the mPFC are endocannabinoid-dependent, we tested the hypothesis that mPFC CB1R activity mediates toluene-induced deficits in discounting. However, bilateral injection of the CB1R inverse agonist AM251 prior to toluene vapor exposure had no effect on toluene-induced changes in risk behavior. In a final set of experiments, we injected the CB1R inverse agonist AM251 (5 and 50 ng), the CB1R agonist WIN55,212-2 (50 ng and 500 ng), or vehicle into the mPFC prior to testing. While mPFC CB1R stimulation did not affect any of the measures tested, the CB1R inverse agonist caused a dose-dependent reduction in win-stay behavior without altering any other measures. Together, these studies indicate that toluene-induced deficits in probabilistic discounting are largely distinct from CB1R-dependent effects that include decreased effectiveness of positive reinforcement (mPFC CB1Rs), decision making speed, and task engagement (non-mPFC CB1Rs).

The orbitofrontal cortex represents advantageous choice in the Iowa gambling task

A good‐based model, the central neurobiological model of economic decision‐making, proposes that the orbitofrontal cortex (OFC) represents binary choice outcome, that is, the chosen good. A good is defined by a group of determinants characterizing the conditions in which the commodity is offered, including commodity type, cost, risk, time delay, and ambiguity. Previous studies have found that the OFC represents the binary choice outcome in decision‐making tasks involving commodity type, cost, risk, and delay. Real‐life decisions are often complex and involve uncertainty, rewards, and penalties; however, whether the OFC represents binary choice outcomes in a complex decision‐making situation, for example, Iowa gambling task (IGT), remains unclear. Here, we propose that the OFC represents binary choice outcome, that is, advantageous choice versus disadvantageous choice, in the IGT. We propose two hypotheses: first, the activity pattern in the human OFC represents an advantageous choice; and second, choice induces an OFC‐related functional network. Using functional magnetic resonance imaging and advanced machine‐learning tools, we found that the OFC represented an advantageous choice in the IGT. The OFC representation of advantageous choice was related to decision‐making performance. Choice modulated the functional connectivity between the OFC and the superior medial gyrus. In conclusion, the OFC represents an advantageous choice during the IGT. In the framework of a good‐based model, the results extend the role of the OFC to complex decision‐making situation when making a binary choice.

Neurobiological and epigenetic perspectives on hedonism, altruism and conscience

This article examines notions of hedonism, altruism and conscience in relation to the activity of four neurotransmitter pathways: the dopamine reward, noradrenaline fight or flight, serotonin calming and glutamine learning pathways. Associated brain areas that modulate behaviour are highlighted: the prefrontal cortex (activity planning, risk mitigation), the hippocampus (memory retrieval) and the insular cortex (integration of information to decide on action). Putative epigenetic changes influencing adult behaviours after childhood privation are discussed. Pharmacological and psychological means of mitigating harmful behaviours are summarised, alongside the ethics of epigenetic screening to predict future addictive and violent tendencies.

GABAB receptors in prelimbic cortex and basolateral amygdala differentially influence intertemporal decision making and decline with age

The ability to decide adaptively between immediate vs. delayed gratification (intertemporal choice) is critical for well-being and is associated with a range of factors that influence quality of life. In contrast to young adults, many older adults show enhanced preference for delayed gratification; however, the neural mechanisms underlying this age difference in intertemporal choice are largely un-studied. Changes in signaling through GABA_B receptors (GABA_BRs) mediate several age-associated differences in cognitive processes linked to intertemporal choice. The current study used a rat model to determine how GABA_BRs in two brain regions known to regulate intertemporal choice (prelimbic cortex; PrL and basolateral amygdala; BLA) contribute to age differences in this form of decision making in male rats. As in humans, aged rats showed enhanced preference for large, delayed over small, immediate rewards during performance in an intertemporal choice task in operant test chambers. Activation of PrL GABA_BRs via microinfusion of the agonist baclofen increased choice of large, delayed rewards in young adult rats but did not influence choice in aged rats. Conversely, infusion of baclofen into the BLA strongly reduced choice of large, delayed rewards in both young adult and aged rats. Aged rats further showed a significant reduction in expression of GABA_BR1 subunit isoforms in the prefrontal cortex, a discovery that is consonant with the null effect of intra-PrL baclofen on intertemporal choice in aged rats. In contrast, expression of GABA_BR subunits was generally conserved with age in the BLA. Jointly, these findings elucidate a role for GABA_BRs in intertemporal choice and identify fundamental features of brain maturation and aging that mediate an improved ability to delay gratification.

Probabilistic Reinforcement Learning and Anhedonia

Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject's responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Better living through understanding the insula: Why subregions can make all the difference

Insula function is considered critical for many motivated behaviors, with proposed functions ranging from attention, behavioral control, emotional regulation, goal-directed and aversion-resistant responding. Further, the insula is implicated in many neuropsychiatric conditions including substance abuse. More recently, multiple insula subregions have been distinguished based on anatomy, connectivity, and functional contributions. Generally, posterior insula is thought to encode more somatosensory inputs, which integrate with limbic/emotional information in middle insula, that in turn integrate with cognitive processes in anterior insula. Together, these regions provide rapid interoceptive information about the current or predicted situation, facilitating autonomic recruitment and quick, flexible action. Here, we seek to create a robust foundation from which to understand potential subregion differences, and provide direction for future studies. We address subregion differences across humans and rodents, so that the latter's mechanistic interventions can best mesh with clinical relevance of human conditions. We first consider the insula's suggested roles in humans, then compare subregional studies, and finally describe rodent work. One primary goal is to encourage precision in describing insula subregions, since imprecision (e.g. including both posterior and anterior studies when describing insula work) does a disservice to a larger understanding of insula contributions. Additionally, we note that specific task details can greatly impact recruitment of various subregions, requiring care and nuance in design and interpretation of studies. Nonetheless, the central ethological importance of the insula makes continued research to uncover mechanistic, mood, and behavioral contributions of paramount importance and interest. This article is part of the special Issue on 'Neurocircuitry Modulating Drug and Alcohol Abuse'.

Does Ventromedial Prefrontal Cortex Damage Really Increase Impulsiveness? Delay and Probability Discounting in Patients with Focal Lesions

If the tendency to discount rewards reflects individuals' general level of impulsiveness, then the discounting of delayed and probabilistic rewards should be negatively correlated: The less a person is able to wait for delayed rewards, the more they should take chances on receiving probabilistic rewards. It has been suggested that damage to the ventromedial prefrontal cortex (vMPFC) increases individuals' impulsiveness, but both intertemporal choice and risky choice have only recently been assayed in the same patients with vMPFC damage. Here, we assess both delay and probability discounting in individuals with vMPFC damage (n = 8) or with medial temporal lobe (MTL) damage (n = 10), and in age- and education-matched controls (n = 30). On average, MTL-lesioned individuals discounted delayed rewards at normal rates but discounted probabilistic rewards more shallowly than controls. In contrast, vMPFC-lesioned individuals discounted delayed rewards more steeply but probabilistic rewards more shallowly than controls. These results suggest that vMPFC lesions affect the weighting of reward amount relative to delay and certainty in opposite ways. Moreover, whereas MTL-lesioned individuals and controls showed typical, nonsignificant correlations between the discounting of delayed and probabilistic rewards, vMPFC-lesioned individuals showed a significant negative correlation, as would be expected if vMPFC damage increases impulsiveness more in some patients than in others. Although these results are consistent with the hypothesis that vMPFC plays a role in impulsiveness, it is unclear how they could be explained by a single mechanism governing valuation of both delayed and probabilistic rewards.

Amygdala-hippocampal innervation modulates stress-induced depressive-like behaviors through AMPA receptors

Chronic stress is one of the most critical factors in the onset of depressive disorders; hence, environmental factors such as psychosocial stress are commonly used to induce depressive-​like traits in animal models of depression. Ventral CA1 (vCA1) in hippocampus and basal lateral amygdala (BLA) are critical sites during chronic stress-induced alterations in depressive subjects; however, the underlying neural mechanisms remain unclear. Here we employed chronic unpredictable mild stress (CUMS) to model depression in mice and found that the activity of the posterior BLA to vCA1 (pBLA-vCA1) innervation was markedly reduced. Mice subjected to CUMS showed reduction in dendritic complexity, spine density, and synaptosomal AMPA receptors (AMPARs). Stimulation of pBLA-vCA1 innervation via chemogenetics or administration of cannabidiol (CBD) could reverse CUMS-induced synaptosomal AMPAR decrease and efficiently alleviate depressive-like behaviors in mice. These findings demonstrate a critical role for AMPARs and CBD modulation of pBLA-vCA1 innervation in CUMS-induced depressive-like behaviors.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Reward motivation and cognitive flexibility in tau null-mutation mice

The reduction of tau or hyperphosphorylated tau (p-tau) has been proposed as a therapeutic strategy for Alzheimer's disease (AD) and frontotemporal dementia (FTD). Cognitive decline and sleep-wake dysregulation seen in AD and FTD patients are mimicked in transgenic and null-mutation mouse models of tauopathy. Alterations in the reward system are additional symptoms of AD and FTD. However, the role of tau in reward processes is not well understood. The present study aimed to examine reward and reward-motivated cognitive processes in male and female tau knockout (tau-/-) and wild-type mice using progressive ratio and reversal learning tasks. Tau-/- mice were heavier, ate more in the home cage, and reached criterion in operant lever training faster than wild-type mice. Tau-/- mice had a higher breakpoint in progressive ratio but were unimpaired in reversal learning or reward sensitivity. These data indicate that tau loss of function alters reward processing. This may help to explain aberrant reward-related behaviors in tauopathy patients and highlights a potentially important area for consideration in the development of anti-tau therapies.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Pair housing, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effect of probability presentation order on amphetamine-induced changes in risky choice

Probability discounting is often measured with independent schedules. Independent schedules have several limitations, such as confounding preference for one alternative with frequency of reward presentation and generating ceiling/floor effects at certain probabilities. To address this potential caveat, a controlled reinforcer frequency schedule can be used, in which the manipulandum that leads to reinforcement is pseudo-randomly determined before each trial. This schedule ensures subjects receive equal presentations of the small and large magnitude reinforcers across each block of trials. A total of 24 pair-housed and 11 individually housed female Sprague Dawley rats were tested in a controlled reinforcer frequency procedure. For half of the rats, the odds against (OA) receiving the large magnitude reinforcer increased across the session (ascending schedule); the OA decreased across the session for half of the rats (descending schedule). Following training, rats received treatments of amphetamine (AMPH; 0, 0.25, 0.5, 1.0 mg/kg; s.c.). For pair-housed rats, AMPH (0.5 mg/kg) increased risky choice, regardless of probability presentation order, whereas a higher dose of AMPH (1.0 mg/kg) decreased discriminability of reinforcer magnitude for rats trained on the descending schedule only. For individually housed rats, probability presentation order modulated the effects of AMPH on probability discounting, as AMPH (0.25 and 0.5 mg/kg) increased risky choice in rats trained on the ascending schedule but not on the descending schedule. These results show that pair-housing animals, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effects of probability presentation order on drug effects on risky choice.

Prefrontal Cortex Representation of Learning of Punishment Probability During Reward-Motivated Actions

Actions executed toward obtaining a reward are frequently associated with the probability of harm occurring during action execution. Learning this probability allows for appropriate computation of future harm to guide action selection. Impaired learning of this probability may be critical for the pathogenesis of anxiety or reckless and impulsive behavior. Here we designed a task for punishment probability learning during reward-guided actions to begin to understand the neuronal basis of this form of learning, and the biological or environmental variables that influence action selection after learning. Male and female Long-Evans rats were trained in a seek-take behavioral paradigm where the seek action was associated with varying probability of punishment. The take action remained safe and was followed by reward delivery. Learning was evident as subjects selectively adapted seek action behavior as a function of punishment probability. Recording of neural activity in the mPFC during learning revealed changes in phasic mPFC neuronal activity during risky-seek actions but not during the safe take actions or reward delivery, revealing that this region is involved in learning of probabilistic punishment. After learning, the variables that influenced behavior included reinforcer and punisher value, pretreatment with the anxiolytic diazepam, and biological sex. In particular, females were more sensitive to probabilistic punishment than males. These data demonstrate that flexible encoding of risky actions by mPFC is involved in probabilistic punishment learning and provide a novel behavioral approach for studying the pathogenesis of anxiety and impulsivity with inclusion of sex as a biological variable.SIGNIFICANCE STATEMENT Actions we choose to execute toward obtaining a reward are often associated with the probability of harm occurring. Impaired learning of this probability may be critical for the pathogenesis of anxiety or reckless behavior and impulsivity. We developed a behavioral model to assess this mode of learning. This procedure allowed us to determine biological and environmental factors that influence the resistance of reward seeking to probabilistic punishment and to identify the mPFC as a region that flexibly adapts its response to risky actions as contingencies are learned.

Touchscreen-based assessment of risky-choice in mice

Addictions are characterized by choices made to satisfy the addiction despite the risk it could produce an adverse consequence. Here, we developed a murine version of a 'risky decision-making' task (RDT), in which mice could respond on a touchscreen panel to obtain either a large milkshake reward associated with varying probability of footshock, or a smaller amount of the same reward that was never punished. Results showed that (the following font is incorrectly smaller/subscripted) mice shifted choice from the large to small reward stimulus as shock probability increased. Immunohistochemical analysis revealed more Fos-positive cells in prelimbic cortex (PL) and basal amygdala (BA) after RDT testing, and a strong anti-correlation between infralimbic cortex (IL) activity and choice of the large reward stimulus under likely (75-100 % probability) punishment. These findings establish an assay for risky choice in mice and provide preliminary insight into the underlying neural substrates.

Prefrontal Regulation of Punished Ethanol Self-administration

BACKGROUND

A clinical hallmark of alcohol use disorder is persistent drinking despite potential adverse consequences. The ventromedial prefrontal cortex (vmPFC) and dorsomedial prefrontal cortex (dmPFC) are positioned to exert top-down control over subcortical regions, such as the nucleus accumbens shell (NAcS) and basolateral amygdala, which encode positive and negative valence of ethanol (EtOH)-related stimuli. Prior rodent studies have implicated these regions in regulation of punished EtOH self-administration (EtOH-SA).

METHODS

We conducted in vivo electrophysiological recordings in mouse vmPFC and dmPFC to obtain neuronal correlates of footshock-punished EtOH-SA. Ex vivo recordings were performed in NAcS D₁ receptor-expressing medium spiny neurons receiving vmPFC input to examine punishment-related plasticity in this pathway. Optogenetic photosilencing was employed to assess the functional contribution of the vmPFC, dmPFC, vmPFC projections to NAcS, or vmPFC projections to basolateral amygdala, to punished EtOH-SA.

RESULTS

Punishment reduced EtOH lever pressing and elicited aborted presses (lever approach followed by rapid retraction). Neurons in the vmPFC and dmPFC exhibited phasic firing to EtOH lever presses and aborts, but only in the vmPFC was there a population-level shift in coding from lever presses to aborts with punishment. Closed-loop vmPFC, but not dmPFC, photosilencing on a postpunishment probe test negated the reduction in EtOH lever presses but not in aborts. Punishment was associated with altered plasticity at vmPFC inputs to D₁ receptor-expressing medium spiny neurons in the NAcS. Photosilencing vmPFC projections to the NAcS, but not to the basolateral amygdala, partially reversed suppression of EtOH lever presses on probe testing.

CONCLUSIONS

These findings demonstrate a key role for the vmPFC in regulating EtOH-SA after punishment, with implications for understanding the neural basis of compulsive drinking in alcohol use disorder.

Prelimbic and Infralimbic Prefrontal Regulation of Active and Inhibitory Avoidance and Reward-Seeking

Flexible initiation or suppression of actions to avoid aversive events is crucial for survival. The prelimbic (PL) and infralimbic (IL) regions of the medial prefrontal cortex (mPFC) have been implicated in different aspects of avoidance and reward-seeking, but their respective contribution in instigating versus suppressing actions in aversive contexts remains to be clarified. We examined mPFC involvement in different forms of avoidance in rats well trained on different cued lever-press avoidance tasks. Active/inhibitory avoidance required flexible discrimination between auditory cues signaling foot-shock could be avoided by making or withholding instrumental responses. On a simpler active avoidance task, a single cue signaled when a lever press would avoid shock. PL inactivation disrupted active but not inhibitory avoidance on the discriminative task while having no effect on single-cued avoidance. In comparison, IL inactivation broadly impaired active and inhibitory avoidance. Conversely, on a cued appetitive go/no-go task, both IL and PL inactivation impaired inhibitory but not active reward-seeking, the latter effect being diametrically opposite to that observed on the avoidance task. These findings highlight the complex manner in which different mPFC regions aid in initiating or inhibiting actions in the service of avoiding aversive outcomes or obtaining rewarding ones. IL facilitates active avoidance but suppress inappropriate actions in appetitive and aversive contexts. In contrast, contextual valence plays a critical role in how the PL is recruited in initiating or suppressing actions, which may relate to the degree of cognitive control required to flexibly negotiate response or motivational conflicts and override prepotent behaviors.SIGNIFICANCE STATEMENT Choosing to make or withhold actions in a context-appropriate manner to avoid aversive events or obtain other goals is a critical survival skill. Different medial prefrontal cortex (mPFC) regions have been implicated in certain aspects of avoidance, but their contributions to instigating or suppressing actions remains to be clarified. Here, we show that the dorsal, prelimbic (PL) region of the medial PFC aids active avoidance in situations requiring flexible mitigation of response conflicts, but also aids in withholding responses to obtain rewards. In comparison the ventral infralimbic (IL) cortex plays a broader role in active and inhibitory avoidance as well as suppressing actions to obtain rewards. These findings provide insight into mechanisms underlying normal and maladaptive avoidance behaviors and response inhibition.

Translating concepts of risk and loss in rodent models of gambling and the limitations for clinical applications

Gambling involves placing something of value at risk in exchange for the opportunity to potentially gain something of greater value in return. A variety of gambling paradigms have been designed to study the maladaptive decision-making that underlies problematic gambling. Central to these gambling models are the definitions of "risk" and "loss", especially when translating the results from rodent studies to clinical applications. Risk and loss are not mutually exclusive but rather share some overlap. With careful interpretation and consideration of the limitations of these behavioral paradigms, results from rodent models may provide insights into the neurobiology of risky decision-making that leads to problematic gambling in humans.

Dorsomedial prefrontal cortex and hippocampus represent strategic context even while simultaneously changing representation throughout a task session

Dorsomedial prefrontal cortex (dmPFC) and hippocampus (HPC) are thought to play complementary roles in a spatial working memory and decision-making network, where spatial information from HPC informs representations in dmPFC, and contextual information from dmPFC biases how HPC recalls that information. We recorded simultaneously from neural ensembles in rodent dmPFC and HPC as rats performed a rule-switching task, and found that ensembles in dmPFC and HPC simultaneously encoded task contingencies and other time-varying information. While ensembles in HPC transitioned to represent new contingencies at the same time as rats updated their strategies to be consistent with the new contingency, dmPFC ensembles transitioned earlier. Neural representations of other time-varying information also changed faster in dmPFC than in HPC. Our results suggest that HPC and dmPFC represent contingencies while simultaneously representing other information which changes over time, and that this contextual information is integrated into hippocampal representations more slowly than in dmPFC.

Contribution of the prefrontal cortex and basolateral amygdala to behavioral decision-making under reward/punishment conflict

RATIONALE

Control of reward-seeking behavior under conditions of punishment is an important function for survival.

OBJECTIVES AND METHODS

We designed a task in which rats could choose to either press a lever and obtain a food pellet accompanied by a footshock or refrain from pressing the lever to avoid footshock, in response to tone presentation. In the task, footshock intensity steadily increased, and the task was terminated when the lever press probability reached < 25% (last intensity). Rats were trained until the last intensity was stable. Subsequently, we investigated the effects of the pharmacological inactivation of the ventromedial prefrontal cortex (vmPFC), lateral orbitofrontal cortex (lOFC), and basolateral amygdala (BLA) on task performance.

RESULTS

Bilateral inactivation of the vmPFC, lOFC, and BLA did not alter lever press responses at the early stage of the task. The number of lever presses increased following vmPFC and BLA inactivation but decreased following lOFC inactivation during the later stage of the task. The last intensity was elevated by vmPFC or BLA inactivation but lowered by lOFC inactivation. Disconnection of the vmPFC-BLA pathway induced behavioral alterations that were similar to vmPFC or BLA inactivation. Inactivation of any regions did not alter footshock sensitivity and anxiety levels.

CONCLUSIONS

Our results demonstrate a strong role of the vmPFC and BLA and their interactions in reward restraint to avoid punishment and a prominent role of the lOFC in reward-seeking under reward/punishment conflict situations.

Dissociable roles for the ventral and dorsal medial prefrontal cortex in cue-guided risk/reward decision making

Converging evidence from studies with animals and humans have implicated separate regions of the medial prefrontal cortex (mPFC) corresponding to the anterior cingulate cortex (ACC), in mediating different aspects of reward-related decisions involving uncertainty or risk. However, the dissociable contributions of subregions of the ACC remain unclear, as discrepancies exist between human neuroimaging findings and preclinical rodent studies. To clarify how ventral vs. dorsal regions of the mPFC contribute to risk/reward decision making, the present study assessed the effects of inactivation of different subregions on performance of a "Blackjack task" that measured cue-guided decision making and shares similarities with paradigms used with humans. Male, Long-Evans rats were well-trained to choose between a Small/Certain reward vs a Large/Risky reward delivered with variable probabilities (i.e., good vs. poor-odds, 50% vs. 12.5%). The odds of obtaining the larger reward was signaled by auditory cues at the start of each trial. Inactivation of the ventral, infralimbic region of the mPFC increased risky choice selectively when the odds of winning were poor. By contrast, inactivation of the prelimbic and anterior cingulate regions of the dorsal mPFC led to suboptimal reductions in risky choice on good-odds trials. The effects of prelimbic vs anterior cingulate inactivations were associated with context-dependent alterations in reward vs negative feedback, respectively. These results further clarify the distinct yet complementary manners in which separate ACC regions promote optimal risk/reward decision making and complement neuroimaging findings that activity in human ventral vs dorsal ACC promotes risk aversion or risky choices.

Integrated Stress Response Inhibitor Reverses Sex-Dependent Behavioral and Cell-Specific Deficits after Mild Repetitive Head Trauma

Mild repetitive traumatic brain injury (rTBI) induces chronic behavioral and cognitive alterations and increases the risk for dementia. Currently, there are no therapeutic strategies to prevent or mitigate chronic deficits associated with rTBI. Previously we developed an animal model of rTBI that recapitulates the cognitive and behavioral deficits observed in humans. We now report that rTBI results in an increase in risk-taking behavior in male but not female mice. This behavioral phenotype is associated with chronic activation of the integrated stress response and cell-specific synaptic alterations in the type A subtype of layer V pyramidal neurons in the medial prefrontal cortex. Strikingly, by briefly treating animals weeks after injury with ISRIB, a selective inhibitor of the integrated stress response (ISR), we (1) relieve ISR activation, (2) reverse the increased risk-taking behavioral phenotype and maintain this reversal, and (3) restore cell-specific synaptic function in the affected mice. Our results indicate that targeting the ISR even at late time points after injury can permanently reverse behavioral changes. As such, pharmacological inhibition of the ISR emerges as a promising avenue to combat rTBI-induced behavioral dysfunction.

Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex

RATIONALE

During value-based decision-making, organisms make choices on the basis of reward expectations, which have been formed during prior action-outcome learning. Although it is known that neuronal manipulations of different subregions of the rat prefrontal cortex (PFC) have qualitatively different effects on behavioral tasks involving value-based decision-making, it is unclear how these regions contribute to the underlying component processes.

OBJECTIVES

Assessing how different regions of the rodent PFC contribute to component processes of value-based decision-making behavior, including reward (or positive feedback) learning, punishment (or negative feedback) learning, response persistence, and exploration versus exploitation.

METHODS

We performed behavioral modeling of data of rats in a probabilistic reversal learning task after pharmacological inactivation of five PFC subregions, to assess how inactivation of these different regions affected the structure of responding of animals in the task.

RESULTS

Our results show reductions in reward and punishment learning after PFC subregion inactivation. The prelimbic, infralimbic, lateral orbital, and medial orbital PFC particularly contributed to punishment learning, and the prelimbic and lateral orbital PFC to reward learning. In addition, response persistence depended on the infralimbic and medial orbital PFC. As a result, pharmacological inactivation of the infralimbic and lateral orbitofrontal cortex reduced the number of reversals achieved, whereas inactivation of the prelimbic and medial orbitofrontal cortex decreased the number of rewards obtained. Finally, using simulated data, we explain discrepancies with a previous study and demonstrate complex, interacting relationships between conventional measures of probabilistic reversal learning performance, such as win-stay/lose-switch behavior, and component processes of value-based decision-making.

CONCLUSIONS

Together, our data suggest that distinct components of value-based learning and decision-making are generated in medial and orbital PFC regions, displaying functional specialization and overlap, with a prominent role of large parts of the PFC in negative feedback processing.

The Abused Inhalant Toluene Impairs Medial Prefrontal Cortex Activity and Risk/Reward Decision-Making during a Probabilistic Discounting Task

Inhalant (e.g., toluene) misuse is linked to behavioral and cognitive deficits in humans, yet preclinical studies of the effect of inhalants on higher-order cognition are limited. We addressed this gap in the literature by examining the effect of toluene vapor exposure on risk/reward decision-making in male and female Sprague-Dawley rats using a probabilistic discounting task. In this task, rodents chose a risky/large reward or a safe/small reward, with the odds of risky reinforcement descending or ascending throughout the test session. We observed a dose-dependent, sex-independent deficit in behavioral flexibility during probabilistic discounting caused by acute toluene exposure. Rats exposed to toluene vapor during adolescence and tested as adults performed comparably to air-treated controls and were susceptible to the effects of an acute toluene challenge. These behavioral flexibility deficits observed suggests dysfunctional medial prefrontal cortex (mPFC) activity. To address this hypothesis, we virally expressed the genetically encoded calcium sensor GCaMP6f in glutamatergic mPFC neurons and monitored calcium transients in real-time using in vivo fiber photometry. mPFC activity peaked before either lever press during free-choice trials in toluene- and air-treated animals. During forced-choice trials, GCaMP6f transients shifted from pre-risky to pre-safe choice, an effect mitigated by acute toluene exposure. mPFC activity decreased during rewarded trials, with larger decreases following risky/large wins compared with safe/small wins. Toluene-treated animals also had decreased mPFC activity during rewarded trials, but there was no distinction between risky/large wins and safe/small wins. These results provide physiological evidence for mPFC-dependent behavioral deficits caused by toluene.SIGNIFICANCE STATEMENT Inhalants (e.g., toluene) are an understudied class of drugs of abuse that cause devastating behavioral and cognitive deficits in humans. Understanding the neurobiological interactions of toluene vapor using animal models is important for developing effective treatment strategies for inhalant addicts. Here we find that toluene vapor reduces behavioral flexibility in rodents making risk/reward-based decisions. The medial prefrontal cortex (mPFC) drives behavioral flexibility during this type of decision-making and we show that toluene reduces the ability of mPFC neurons to track optimal choices as reward probabilities change. Toluene also reduces these neurons' ability to distinguish between small and large rewards. A combination of these factors likely leads to the impaired performance in probabilistic discounting following acute toluene exposure.

Toward an animal model of borderline personality disorder

BACKGROUND

Borderline personality disorder (BPD) is a pervasive psychiatric disorder characterized by emotion dysregulation, impulsivity, impaired self-perceptions, and interpersonal relationships and currently affects 1-3% of the US population as reported by Torgersen et al. (Arch Gen Psychiatry 58:590-596, Torgersen et al. 2001), Lenzenweger et al. (Biol Psychiatry 62:553-564, Lenzenweger et al. 2007), and Tomko et al. (J Personal Disord 28:734-750, Tomko et al. 2014). One major obstacle to our understanding of the neural underpinnings of BPD is a lack of valid animal models that translate the key known features of the disorder to a system that is amenable to study.

OBJECTIVE

To summarize the etiology, major symptoms, and symptom triggers of BPD and then propose a blueprint for building an animal model of BPD by choosing key components of the disorder that can be implemented in rodents.

RESULTS

We identify the role of early life stress and subsequent mild stress in adulthood as contributing etiological factors and the potential use of altered communication between frontal cortices and the amygdala in extinction and habituation, increased impulsivity, dysregulation of the hypothalamic pituitary axis (HPA), and increased neuroinflammation as biological markers of BPD. Building upon these features of BPD, we propose a two-hit animal model that uses maternal abandonment to alter maturation of the HPA axis and mild secondary adult stress to evoke behavioral symptoms such as increased impulsivity and impaired extinction, habituation, and social interactions.

CONCLUSION

Through exploration of the etiology, symptom presentation, and altered neurological function, we propose an animal model of BPD. We believe that a number of existing animal paradigms that model other mental health disorders should be combined in a unique way to reflect the etiology, symptom presentation, and altered neurological function that is evident in BPD. These model, when compared with available human data, will inform research and treatment in humans for better understanding of systems from the micro-molecular level to more global physiology underlying BPD.

Avoid-approach conflict behaviors differentially affected by anxiolytics: implications for a computational model of risky decision-making

Whether fear or anxiety is expressed is thought to depend on an animal's proximity to threat. In general, fear is elicited when threat is proximal, while anxiety is a response to threat that is distal and uncertain. This threat gradient model suggests that fear and anxiety involve non-overlapping neural circuitry, yet few behavioral paradigms exist that elicit both states. We studied avoid-approach conflict in rats that were behaving in a predator-inhabited foraging arena task that involved tangible threat and reward incentives. In the task, rats exhibited a variety of both fearful and anxious behaviors corresponding to proximal and distal threat, respectively. We then administered ethanol or diazepam to the rats in order to study how anxiolytics affected these fear and anxiety behaviors. We discovered that both ethanol and diazepam attenuated proximal-threat fear-like behaviors. Furthermore, we found that diazepam, but not ethanol, increased distal-threat anxiety-like behavior but also made rats less risk-averse. Finally, we describe how decisional conflict can be modeled as a partially observable Markov decision process and characterize a potential relationship between anxious behavior, diazepam's ability to suppress hippocampal theta oscillations, and hippocampal representations of the future.