Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

Mu-opioid receptor knockout on Foxp2-expressing neurons reduces aversion-resistant alcohol drinking

Mu-opioid receptors (MORs) in the amygdala and striatum are important in addictive and rewarding behaviors. The transcription factor Foxp2 is a genetic marker of intercalated (ITC) cells in the amygdala and a subset of striatal medium spiny neurons (MSNs), both of which express MORs in wild-type mice and are neuronal subpopulations of potential relevance to alcohol-drinking behaviors. For the current series of studies, we characterized the behavior of mice with genetic deletion of the MOR gene Oprm1 in Foxp2-expressing neurons (Foxp2-Cre/Oprm1fl/fl). Male and female Foxp2-Cre/Oprm1fl/fl mice were generated and heterozygous Cre+ (knockout) and homozygous Cre- (control) animals were tested for aversion-resistant alcohol consumption using an intermittent access (IA) task, operant responding for a sucrose reward, conditioned place aversion (CPA) to morphine withdrawal, and locomotor sensitization to morphine. The results demonstrate that deletion of MOR on Foxp2-expressing neurons renders mice more sensitive to quinine-adulterated ethanol (EtOH). Mice with the deletion (vs. Cre- controls) also consumed less alcohol during the final sessions of the IA task, responded less for sucrose under an FR3 schedule, and were less active at baseline and following morphine injection. Foxp2-MOR deletion did not impair the ability to learn to respond for reward or develop a conditioned aversion to morphine withdrawal. Together, these investigations demonstrate that Foxp2-expressing neurons may be involved in escalation of alcohol consumption and the development of compulsive-like alcohol drinking.

Medial prefrontal cortex suppresses reward-seeking behavior with risk of punishment by reducing sensitivity to reward

Reward-seeking behavior is frequently associated with risk of punishment. There are two types of punishment: positive punishment, which is defined as addition of an aversive stimulus, and negative punishment, involves the omission of a rewarding outcome. Although the medial prefrontal cortex (mPFC) is important in avoiding punishment, whether it is important for avoiding both positive and negative punishment and how it contributes to such avoidance are not clear. In this study, we trained male mice to perform decision-making tasks under the risks of positive (air-puff stimulus) and negative (reward omission) punishment, and modeled their behavior with reinforcement learning. Following the training, we pharmacologically inhibited the mPFC. We found that pharmacological inactivation of mPFC enhanced the reward-seeking choice under the risk of positive, but not negative, punishment. In reinforcement learning models, this behavioral change was well-explained as an increase in sensitivity to reward, rather than a decrease in the strength of aversion to punishment. Our results suggest that mPFC suppresses reward-seeking behavior by reducing sensitivity to reward under the risk of positive punishment.

Examining a punishment-related brain circuit with miniature fluorescence microscopes and deep learning

In humans experiencing substance use disorder (SUD), abstinence from drug use is often motivated by a desire to avoid some undesirable consequence of further use: health effects, legal ramifications, etc. This process can be experimentally modeled in rodents by training and subsequently punishing an operant response in a context-induced reinstatement procedure. Understanding the biobehavioral mechanisms underlying punishment learning is critical to understanding both abstinence and relapse in individuals with SUD. To date, most investigations into the neural mechanisms of context-induced reinstatement following punishment have utilized discrete loss-of-function manipulations that do not capture ongoing changes in neural circuitry related to punishment-induced behavior change. Here, we describe a two-pronged approach to analyzing the biobehavioral mechanisms of punishment learning using miniature fluorescence microscopes and deep learning algorithms. We review recent advancements in both techniques and consider a target neural circuit.

The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Infralimbic cortex plays a similar role in the punishment and extinction of instrumental behavior

Learning to stop responding is a fundamental process in instrumental learning. Animals may learn to stop responding under a variety of conditions that include punishment-where the response earns an aversive stimulus in addition to a reinforcer-and extinction-where a reinforced response now earns nothing at all. Recent research suggests that punishment and extinction may be related manifestations of a common retroactive interference process. In both paradigms, animals learn to stop performing a specific response in a specific context, suggesting direct inhibition of the response by the context. This process may depend on the infralimbic cortex (IL), which has been implicated in a variety of interference-based learning paradigms including extinction and habit learning. Despite the behavioral parallels between extinction and punishment, a corresponding role for IL in punishment has not been identified. Here we report that, in a simple arrangement where either punishment or extinction was conducted in a context that differed from the context in which the behavior was first acquired, IL inactivation reduced response suppression in the inhibitory context, but not responding when it "renewed" in the original context. In a more complex arrangement in which two responses were first trained in different contexts and then extinguished or punished in the opposite one, IL inactivation had no effect. The results advance our understanding of the effects of IL in retroactive interference and the behavioral mechanisms that can produce suppression of a response.

A nociceptive amygdala-striatal pathway for chronic pain aversion

The basolateral amygdala (BLA) is essential for assigning positive or negative valence to sensory stimuli. Noxious stimuli that cause pain are encoded by an ensemble of nociceptive BLA projection neurons (BLAnoci ensemble). However, the role of the BLAnoci ensemble in mediating behavior changes and the molecular signatures and downstream targets distinguishing this ensemble remain poorly understood. Here, we show that the same BLAnoci ensemble neurons are required for both acute and chronic neuropathic pain behavior. Using single nucleus RNA-sequencing, we characterized the effect of acute and chronic pain on the BLA and identified enrichment for genes with known functions in axonal and synaptic organization and pain perception. We thus examined the brain-wide targets of the BLAnoci ensemble and uncovered a previously undescribed nociceptive hotspot of the nucleus accumbens shell (NAcSh) that mirrors the stability and specificity of the BLAnoci ensemble and is recruited in chronic pain. Notably, BLAnoci ensemble axons transmit acute and neuropathic nociceptive information to the NAcSh, highlighting this nociceptive amygdala-striatal circuit as a unique pathway for affective-motivational responses across pain states.

Enhanced Risky Choice in Male Rats Elicited by the Acute Pharmacological Stressor Yohimbine Involves Prefrontal Dopamine D1 Receptor Activation

BACKGROUND

Acute stress alters risk-based decision-making; however, the underlying neural and neurochemical substrates are underexplored. Given their well-documented stress-inducing effects in humans and laboratory animals, glucocorticoids such as cortisol and corticosterone and the α2-adrenoceptor antagonist yohimbine represent potent pharmacological tools to mimic some characteristics of acute stress.

METHODS

Here, we analyzed the effects of the pharmacological stressors corticosterone and yohimbine given systemically on risk-based decision-making in male rats. Moreover, we investigated whether pharmacological stressor effects on risk-based decision-making involve dopamine D1 receptor stimulation in the dorsal prelimbic cortex (PL). We used a risk discounting task that requires choosing between a certain/small reward lever that always delivered 1 pellet and a risky/large reward lever that delivered 4 pellets with a decreasing probability across subsequent trials.

RESULTS

Systemic administration of yohimbine increased the preference for the risky/large reward lever. By contrast, systemic single administration of corticosterone did not significantly promote risky choice. Moreover, co-administration of corticosterone did not enhance the effects of yohimbine on risky choice. The data further show that the increased preference for the risky/large reward lever under systemic yohimbine was lowered by a concurrent pharmacological blockade of dopamine D1 receptors in the PL.

CONCLUSIONS

Our rodent data provide causal evidence that stimulation of PL D1 receptors may represent a neurochemical mechanism by which the acute pharmacological stressor yohimbine, and possibly nonpharmacological stressors as well, promote risky choice.

Effects of reproductive experience on cost-benefit decision making in female rats

Many individuals undergo mating and/or other aspects of reproductive experience at some point in their lives, and pregnancy and childbirth in particular are associated with alterations in the prevalence of several psychiatric disorders. Research in rodents shows that maternal experience affects spatial learning and other aspects of hippocampal function. In contrast, there has been little work in animal models concerning how reproductive experience affects cost-benefit decision making, despite the relevance of this aspect of cognition for psychiatric disorders. To begin to address this issue, reproductively experienced (RE) and reproductively naïve (RN) female Long-Evans rats were tested across multiple tasks that assess different forms of cost-benefit decision making. In a risky decision-making task, in which rats chose between a small, safe food reward and a large food reward accompanied by variable probabilities of punishment, RE females chose the large risky reward significantly more frequently than RN females (greater risk taking). In an intertemporal choice task, in which rats chose between a small, immediate food reward and a large food reward delivered after a variable delay period, RE females chose the large reward less frequently than RN females. Together, these results show distinct effects of reproductive experience on different forms of cost-benefit decision making in female rats, and highlight reproductive status as a variable that could influence aspects of cognition relevant for psychiatric disorders.

Opioid withdrawal: role in addiction and neural mechanisms

Withdrawal from opioids involves a negative affective state that promotes maintenance of drug-seeking behavior and relapse. As such, understanding the neurobiological mechanisms underlying withdrawal from opioid drugs is critical as scientists and clinicians seek to develop new treatments and therapies. In this review, we focus on the neural systems known to mediate the affective and somatic signs and symptoms of opioid withdrawal, including the mesolimbic dopaminergic system, basolateral amygdala, extended amygdala, and brain and hormonal stress systems. Evidence from preclinical studies suggests that these systems are altered following opioid exposure and that these changes mediate behavioral signs of negative affect such as aversion and anxiety during withdrawal. Adaptations in these systems also parallel the behavioral and psychological features of opioid use disorder (OUD), highlighting the important role of withdrawal in the development of addictive behavior. Implications for relapse and treatment are discussed as well as promising avenues for future research, with the hope of promoting continued progress toward characterizing neural contributors to opioid withdrawal and compulsive opioid use.

Risk and aversion coding in human habenula high gamma activity

Neurons in the primate lateral habenula fire in response to punishments and are inhibited by rewards. Through its modulation of midbrain monoaminergic activity, the habenula is believed to play an important role in adaptive behavioural responses to punishment and underlie depressive symptoms and their alleviation with ketamine. However, its role in value-based decision-making in humans is poorly understood due to limitations with non-invasive imaging methods which measure metabolic, not neural, activity with poor temporal resolution. Here, we overcome these limitations to more closely bridge the gap between species by recording local field potentials directly from the habenula in 12 human patients receiving deep brain stimulation treatment for bipolar disorder (n = 4), chronic pain (n = 3), depression (n = 3) and schizophrenia (n = 2). This allowed us to record neural activity during value-based decision-making tasks involving monetary rewards and losses. High-frequency gamma (60-240 Hz) activity, a proxy for population-level spiking involved in cognitive computations, increased during the receipt of loss and decreased during receipt of reward. Furthermore, habenula high gamma also encoded risk during decision-making, being larger in amplitude for high compared to low risk. For both risk and aversion, differences between conditions peaked approximately between 400 and 750 ms after stimulus onset. The findings not only demonstrate homologies with the primate habenula but also extend its role to human decision-making, showing its temporal dynamics and suggesting revisions to current models. The findings suggest that habenula high gamma could be used to optimize real-time closed-loop deep brain stimulation treatment for mood disturbances and impulsivity in psychiatric disorders.

Age-Related Changes in Risky Decision Making and Associated Neural Circuitry in a Rat Model

Altered decision making at advanced ages can have a significant impact on an individual's quality of life and the ability to maintain personal independence. Relative to young adults, older adults make less impulsive and less risky choices; although these changes in decision making could be considered beneficial, they can also lead to choices with potentially negative consequences (e.g., avoidance of medical procedures). Rodent models of decision making have been invaluable for dissecting cognitive and neurobiological mechanisms that contribute to age-related changes in decision making, but they have predominantly used costs related to timing or probability of reward delivery and have not considered other equally important costs, such as the risk of adverse consequences. The current study therefore used a rat model of decision making involving risk of explicit punishment to examine age-related changes in this form of choice behavior in male rats, and to identify potential cognitive and neurobiological mechanisms that contribute to these changes. Relative to young rats, aged rats displayed greater risk aversion, which was not attributable to reduced motivation for food, changes in shock sensitivity, or impaired cognitive flexibility. Functional MRI analyses revealed that, overall, functional connectivity was greater in aged rats compared with young rats, particularly among brain regions implicated in risky decision making such as basolateral amygdala, orbitofrontal cortex, and ventral tegmental area. Collectively, these findings are consistent with greater risk aversion found in older humans, and reveal age-related changes in brain connectivity.

Sex-dependent effects of chronic stress on punished cocaine self-administration and cue-induced relapse to cocaine seeking

We tested the effects of chronic stress on cocaine relapse after drug-reinforced responding was suppressed by punishment, an animal model of human relapse after self-imposed abstinence due to the negative consequences of drug use. Male rats displayed greater resistance to punishment than females, but daily stress decreased this resistance. Only female rats with a history of chronic stress displayed increased responding for cocaine cues from abstinence Day 1 to Day 8. Thus, the effects of chronic stress on punished cocaine self-administration and cue-induced relapse are dependent on biological sex, and may have implications for more targeted treatments for cocaine addiction.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Researching Mitigation of Alcohol Binge Drinking in Polydrug Abuse: KCNK13 and RASGRF2 Gene(s) Risk Polymorphisms Coupled with Genetic Addiction Risk Severity (GARS) Guiding Precision Pro-Dopamine Regulation

Excessive alcohol intake, e.g., binge drinking, is a serious and mounting public health problem in the United States and throughout the world. Hence the need for novel insights into the underlying neurobiology that may help improve prevention and therapeutic strategies. Therefore, our group employed a darkness-induced alcohol intake protocol to define the reward deficiency domains of alcohol and other substance use disorders in terms of reward pathways' reduced dopamine signaling and its restoration via specifically-designed therapeutic compounds. It has been determined that KCNK13 and RASGRF2 genes, respectively, code for potassium two pore domain channel subfamily K member 13 and Ras-specific guanine nucleotide-releasing factor 2, and both genes have important dopamine-related functions pertaining to alcohol binge drinking. We present a hypothesis that identification of KCNK13 and RASGRF2 genes' risk polymorphism, coupled with genetic addiction risk score (GARS)-guided precision pro-dopamine regulation, will mitigate binge alcohol drinking. Accordingly, we review published reports on the benefits of this unique approach and provide data on favorable outcomes for both binge-drinking animals and drunk drivers, including reductions in alcohol intake and prevention of relapse to drinking behavior. Since driving under the influence of alcohol often leads to incarceration rather than rehabilitation, there is converging evidence to support the utilization of GARS with or without KCNK13 and RASGRF2 risk polymorphism in the legal arena, whereby the argument that "determinism" overrides the "free will" account may be a plausible defense strategy. Obviously, this type of research is tantamount to helping resolve a major problem related to polydrug abuse.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Aberrant integrity of the cortico-limbic-striatal circuit in major depressive disorder with suicidal ideation

BACKGROUND

Suicidal ideation is a common symptom of major depressive disorder (MDD) that reflects a cognitive alteration in the background of intense emotional dysregulation. Amygdala is a critical emotion processing center that facilitates moving from emotional appraisal to action. However, whether MDD patients with suicidal ideation show dysconnectivity of the amygdala within a large-scale neurocognitive circuitry remains unknown.

METHODS

Participants were 22 MDD patients without suicidal ideation (MDD-NSI), 59 MDD patients with suicidal ideation (MDD-SI), and 60 healthy controls (HCs). We compared the amygdala-based resting-state functional connectivity of four amygdala subregions across the three groups. We selected brain regions with significant between-group differences in amygdalar conectivity as the regions of interest (ROI) and performed ROI-to-ROI and graph-theoretical analyses to explore dysconnectivity patterns at various granularity levels.

RESULTS

Brain regions showing omnibus differences across the three groups were distributed across a cortico-limbic-striatal circuitry. MDD-SI had unique dysconnectivity of the lateral amygdala with caudate, middle temporal gyrus, and postcentral gyrus compared with the other two groups. MDD-SI and MDD-NSI had shared dysconnectivity of the medial amygdala with medial superior frontal gyrus and middle temporal gyrus. Within the derived cortico-limbic-striatal circuitry, MDD-SI exhibited lower global connectivity, reduced sigma (small-worldness), but increased lambda (path-length) than HCs. Reduced sigma correlated with increased severity of suicidal ideation. We achieved high classification accuracy (84.09%, with AUC 0.82) in distinguishing MDD-SI from MDD-NSI.

CONCLUSIONS

Aberrant integrity of the cortico-limbic-striatal circuit centered on the amygdala provides a promising neural substrate for suicidal ideation that requires further investigation in MDD.

Seeking motivation and reward: roles of dopamine, hippocampus and supramammillo-septal pathway

Reinforcement learning and goal-seeking behavior are thought to be mediated by midbrain dopamine neurons. However, little is known about neural substrates of curiosity and exploratory behavior, which occur in the absence of clear goal or reward. This is despite behavioral scientists having long suggested that curiosity and exploratory behaviors are regulated by an innate drive. We refer to such behavior as information-seeking behavior and propose 1) key neural substrates and 2) the concept of environment prediction error as a framework to understand information-seeking processes. The cognitive aspect of information-seeking behavior, including the perception of salience and uncertainty, involves, in part, the pathways from the posterior hypothalamic supramammillary region to the hippocampal formation. The vigor of such behavior is modulated by the following: supramammillary glutamatergic neurons; their projections to medial septal glutamatergic neurons; and the projections of medial septal glutamatergic neurons to ventral tegmental dopaminergic neurons. Phasic responses of dopaminergic neurons are characterized as signaling potentially important stimuli rather than rewards. This paper describes how novel stimuli and uncertainty trigger seeking motivation and how these neural substrates modulate information-seeking behavior.

Anhedonia and Hyperhedonia in Autism and Related Neurodevelopmental Disorders

Although autism spectrum disorder (ASD) is defined by impaired social communication and restricted and repetitive behaviors and interests, ASD is also characterized by impaired motivational processes. The "social motivation theory of autism" describes how social motivation disruptions in ASD in early childhood may impede the drive to engage in reciprocal social behaviors and ultimately interfere with the development of neural networks critical for social communication (Chevallier et al., Trends Cogn Sci 16:231-239, 2012b). Importantly, clinical studies and preclinical research using model organisms for ASD indicate that motivational impairments in ASD are not constrained to social rewards but are evident in response to a range of nonsocial rewards as well. Additionally, translational studies on certain genetically defined neurodevelopmental disorders associated with ASD indicate that these syndromic forms of ASD are also characterized by motivational deficits and mesolimbic dopamine impairments. In this chapter we summarize clinical and preclinical research relevant to reward processing impairments in ASD and related neurodevelopmental disorders. We also propose a nosology to describe reward processing impairments in these disorders that uses a three-axes model. In this triaxial nosology, the first axis defines the direction of the reward response (i.e., anhedonic, hyperhedonic); the second axis defines the construct of the reward process (e.g., reward liking, reward wanting); and the third axis defines the context of the reward response (e.g., social, nonsocial). A more precise nosology for describing reward processing impairments in ASD and related neurodevelopmental disorders will aid in the translation of preclinical research to clinical investigations which will ultimately help to speed up the development of interventions that target motivational systems for ASD and related neurodevelopmental disorders.

Impulsive trait mediates the relationship between white matter integrity of prefrontal-striatal circuits and the severity of dependence in alcoholism

BACKGROUND

Alcohol dependence (AD) remains one of the major public health concerns. Impulsivity plays a central role in the transfer from recreational alcohol use to dependence and relapse. White matter dysfunction has been implicated in alcohol addiction behaviors and impulsivity. However, little is known about the role of systematic striatal structural connections underlying the mechanism of impulsive traits in AD.

METHODS

In our study, we used seed-based classification by probabilistic tractography with five target masks of striatal circuits to explore the differences in white matter integrity (fractional anisotropy, FA) in AD male patients (N = 51) and healthy controls (N = 27). We mainly explored the correlation between FA of the striatal circuits and impulsive traits (Barratt Impulsiveness Scale, BIS-11), and the mediation role of impulsivity in white matter integrity and the severity of alcohol dependence.

RESULTS

Compared with healthy controls, AD showed much lower FA in the left and right striatum-supplementary motor area (SMA) and left striatum-amygdala. We also found the decreased FA of right striatum-vlPFC was correlated with higher impulsivity. Besides, the relationship between reduced FA of right striatum-vlPFC and severity of dependence could be mediated by impulsivity.

CONCLUSION

In our study, we found disrupted white matter integrity in systematic striatal circuits in AD and the decreased FA of right striatum-vlPFC was correlated with higher impulsivity in AD. Our main findings provide evidence for reduced white matter integrity of systematic striatal circuits and the underlying mechanisms of impulsivity in male AD individuals.

Should we or shouldn't we?: Preface to the Special Issue "Neurochemistry of Reward-Seeking"

Reward-seeking is critical for survival. Learning about the relationship between actions and outcomes helps us to make decisions and select behaviours that result in a specific outcome. This special issue entitled "The Neurochemistry of Reward-Seeking" addresses this crucial facet of behaviour and brings together a number of key thought-leaders to provide a timely update on the circuitry, chemistry and mechanisms underlying different aspects of reward-seeking. The reviews in this issue canvass unanswered questions in the field and provide a degree of forethought about how we may advance our understanding of reward-seeking by embracing novel technology alongside existing scholarship. This issue also highlights the neurochemical complexity of reward-seeking, and the reader will uncover both distinct and shared circuits and transmitters driving various forms of reward-seeking. Accordingly, we hope that this special issue will provide a valuable resource for the field and trigger future research on this topic.