Acute stressor effects on goal-directed action in rats

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

Enhanced Risky Choice in Male Rats Elicited by the Acute Pharmacological Stressor Yohimbine Involves Prefrontal Dopamine D1 Receptor Activation

BACKGROUND

Acute stress alters risk-based decision-making; however, the underlying neural and neurochemical substrates are underexplored. Given their well-documented stress-inducing effects in humans and laboratory animals, glucocorticoids such as cortisol and corticosterone and the α2-adrenoceptor antagonist yohimbine represent potent pharmacological tools to mimic some characteristics of acute stress.

METHODS

Here, we analyzed the effects of the pharmacological stressors corticosterone and yohimbine given systemically on risk-based decision-making in male rats. Moreover, we investigated whether pharmacological stressor effects on risk-based decision-making involve dopamine D1 receptor stimulation in the dorsal prelimbic cortex (PL). We used a risk discounting task that requires choosing between a certain/small reward lever that always delivered 1 pellet and a risky/large reward lever that delivered 4 pellets with a decreasing probability across subsequent trials.

RESULTS

Systemic administration of yohimbine increased the preference for the risky/large reward lever. By contrast, systemic single administration of corticosterone did not significantly promote risky choice. Moreover, co-administration of corticosterone did not enhance the effects of yohimbine on risky choice. The data further show that the increased preference for the risky/large reward lever under systemic yohimbine was lowered by a concurrent pharmacological blockade of dopamine D1 receptors in the PL.

CONCLUSIONS

Our rodent data provide causal evidence that stimulation of PL D1 receptors may represent a neurochemical mechanism by which the acute pharmacological stressor yohimbine, and possibly nonpharmacological stressors as well, promote risky choice.

Stress to inflammation and anhedonia: Mechanistic insights from preclinical and clinical models

Anhedonia, as evidenced by impaired pleasurable response to reward, reduced reward motivation, and/or deficits in reward-related learning, is a common feature of depression. Such deficits in reward processing are also an important clinical target as a risk factor for depression onset. Unfortunately, reward-related deficits remain difficult to treat. To address this gap and inform the development of effective prevention and treatment strategies, it is critical to understand the mechanisms that drive impairments in reward function. Stress-induced inflammation is a plausible mechanism of reward deficits. The purpose of this paper is to review evidence for two components of this psychobiological pathway: 1) the effects of stress on reward function; and 2) the effects of inflammation on reward function. Within these two areas, we draw upon preclinical and clinical models, distinguish between acute and chronic effects of stress and inflammation, and address specific domains of reward dysregulation. By addressing these contextual factors, the review reveals a nuanced literature which might be targeted for additional scientific inquiry to inform the development of precise interventions.

A matter of habit? Stressful life events and cognitive flexibility in 15-month-olds

Exposure to chronic stress is associated with habitual learning in adults. We studied the origins of this association by examining the link between stressful life events and infant cognitive flexibility. The final sample consisted of N = 72 fifteen-month-old infants and their mothers. Mothers completed a survey on pre- and postnatal negative life events. To assess chronic stress physiologically, infant and maternal hair cortisol concentrations were determined for cortisol accumulation during the past 3 months. Each infant participated in two cognitive tasks in the laboratory. An instrumental learning task tested infants' ability to disengage from a habituated action when this action became ineffective (Seehagen et al., 2015). An age-adequate version of the A-not-B task tested infants' ability to find a toy at location B after repeatedly finding it at location A. Correlations between cortisol concentrations and postnatal negative life events (number, perceived impact) did not yield significance. Infant and maternal hair cortisol concentrations were not correlated. Infants' ability to shift to a new action in either task, controlled for acute stress, correlated neither with pre- and postnatal negative life events nor with cortisol concentrations. Taken together, these results indicate that the potential link between long-term stress exposure and cognitive flexibility might not be present in samples with low levels of psychosocial stress.

Rats exposed to intermittent ethanol during late adolescence exhibit enhanced habitual behavior following reward devaluation

The brain undergoes substantial maturation during adolescence, and repeated exposure to ethanol at this time has been shown to result in long-lasting behavioral and neural consequences. During the broad period of adolescence, different neuronal populations and circuits are refined between early and late adolescence, suggesting the possibility that ethanol exposure at these differing times may lead to differential outcomes. The goal of the current study was to evaluate the impact of adolescent intermittent ethanol (AIE) during early and late adolescence on the formation of goal-directed and habitual behavior in adulthood. Male and female Sprague-Dawley rats were exposed to ethanol via intragastric gavage (4.0 g/kg, 25% v/v) every other day from postnatal day (P) 25-45 or P45-65, considered early and late adolescence, respectively. In adulthood (~P70 early or ~ P90 late), rats were gradually food-restricted and began operant training on a fixed ratio 1 schedule. Rats were then transitioned onto random interval schedules and eventually underwent a sensory-specific satiation procedure as a model of reward devaluation. Few differences as a result of adolescent ethanol exposure were found during instrumental training. Following reward devaluation, rats exposed to water and ethanol during early adolescence exhibited reductions in lever pressing, suggestive of a goal-directed response pattern. In contrast, late AIE males and females demonstrated persistent responding following both devalued and non-devalued trials, findings representative of a habitual behavior pattern. The shifts from goal-directed to habitual behavior noted only following late AIE contribute to the growing literature identifying specific behavioral consequences as a result of ethanol exposure during distinct developmental periods within adolescence. More work is needed to determine whether the greater habit formation following late AIE is also associated with elevated habitual ethanol consumption.

Stress reduces both model-based and model-free neural computations during flexible learning

Stressful events are thought to impair the flexible adaptation to changing environments, yet the underlying mechanisms are largely unknown. Here, we combined computational modeling and functional magnetic resonance imaging (fMRI) to elucidate the neurocomputational mechanisms underlying stress-induced deficits in flexible learning. Healthy participants underwent a stress or control manipulation before they completed, in the MRI scanner, a Markov decision task, frequently used to dissociate model-based and model-free contributions to choice, with repeated reversals of reward contingencies. Our results showed that stress attenuated the behavioral sensitivity to reversals in reward contingencies. Computational modeling further indicated that stress specifically affected the use of value computations for subsequent action selection. This reduced application of learned information on subsequent behavior was paralleled by a stress-induced reduction in inferolateral prefrontal cortex activity during model-free computations. For model-based learning, stress decreased specifically posterior, but not anterior, hippocampal activity, pointing to a functional segregation of model-based processing and its modulation by stress along the hippocampal longitudinal axis. Our findings shed light on the mechanisms underlying deficits in flexible learning under stress and indicate that, in highly dynamic environments, stress may hamper both model-based and model-free contributions to adaptive behavior.

Stress drives deliberative tendencies by influencing vicarious trial and error in decision making

Previous studies have reported the effects of stress on decision making. However, the wide range of findings make it difficult to identify the fundamental effects of stress on decision making and, therefore, how stress affects decision making remains unknown. To investigate the influence of stress on decision making, we employed "vicarious trial and error" (VTE), which refers to a rat's behavior of orienting the head toward options at a decision point. VTE is thought to reflect mental simulation for possible options preceding a decision. We examined effects of acute restraint stress on VTE in a T-maze choice task. VTE depended on learning and past reward outcomes. Acute restraint stress before rats ran the T-maze choice task induced VTE, especially in trials with low demand of VTE, and increased the number of head orientations and time spent during each VTE. On the other hand, stress did not affect task performance (probability of advantageous choice) and patterns of behavioral choice (win-stay lose-shift, exploration-exploitation). In addition, stress activated serotonergic and noradrenergic neurons in the dorsal raphe nucleus and locus coeruleus, which are modulators of impulsivity and attentional control in decision making. These results suggest that stress in decision making drives the VTE process, which may lead to deep consideration, over-thinking, and indecisiveness.

Chronic pain impairs cognitive flexibility and engages novel learning strategies in rats

Cognitive flexibility, the ability to adapt behavior to changing outcomes, is critical to survival. The prefrontal cortex is a key site of cognitive control, and chronic pain is known to lead to significant morphological changes to this brain region. Nevertheless, the effects of chronic pain on cognitive flexibility and learning remain uncertain. We used an instrumental paradigm to assess adaptive learning in an experimental model of chronic pain induced by tight ligation of the spinal nerves L5/6 (spinal nerve ligation model). Naive, sham-operated, and spinal nerve ligation (SNL) rats were trained to perform fixed-ratio, variable-ratio, and contingency-shift behaviors for food reward. Although all groups learned an initial lever-reward contingency, learning was slower in SNL animals in a subsequent choice task that reversed reinforcement contingencies. Temporal analysis of lever-press responses across sessions indicated no apparent deficits in memory consolidation or retrieval. However, analysis of learning within sessions revealed that the lever presses of SNL animals occurred in bursts, followed by delays. Unexpectedly, the degree of bursting correlated positively with learning. Under a variable-ratio probabilistic task, SNL rats chose a less profitable behavioral strategy compared with naive and sham-operated animals. After extinction of behavior for learned preferences, SNL animals reverted to their initially preferred (ie, less profitable) behavioral choice. Our data suggest that in the face of uncertainty, chronic pain drives a preference for familiar associations, consistent with reduced cognitive flexibility. The observed burst-like responding may represent a novel learning strategy in animals with chronic pain.

The role of serotonin 1B in the representation of outcomes

Disrupted serotonin neurotransmission has been implicated in the etiology of psychopathic traits. Empirical research has found that people with high levels of psychopathic traits have a deficit in reinforcement learning that is thought to be linked with amygdala dysfunction. Altered serotonin neurotransmission provides a plausible explanation for amygdala dysfunction in psychopathic traits and recent research suggests that this may be associated with serotonin 1B (5-HT_1B) receptor function. This research used an animal model to test the hypothesis that 5-HT_1B receptors are involved in the encoding of the specific features of reinforcing outcomes. An outcome devaluation task was used to test the effect of the systemic administration of a selective 5-HT_1B receptor agonist administered before encoding of “action-outcome” associations. Results showed that while administration of a 5-HT_1B receptor agonist allowed rats to acquire instrumental responding for food, when the content of that learning was further probed using an outcome devaluation task, performance differed from controls. 5-HT_1B agonism impaired learning about the specific sensory qualities of food rewards associated with distinct instrumental responses, required to direct choice performance when the value of one outcome changed. These findings suggest a role for 5-HT_1B receptor function in the encoding of the specific features of reinforcing outcomes.

Peripheral nerve injury impairs the ability to maintain behavioural flexibility following acute stress in the rat

Chronic neuropathic pain often leads to impaired cognition and reduced behavioural flexibility. This study used a rat model to investigate if a peripheral nerve injury, with or without an additional acute psychological stress, alters behavioural flexibility and goal directed behaviour as measured by sensitivity to devaluation. Neuropathic pain was induced by a chronic constriction injury (CCI) of the sciatic nerve. CCI, sham-injury and naïve rats were trained to press two levers for two rewards. In outcome devaluation tests, one of the rewards was devalued by pre-feeding it to satiety, immediately prior to an extinction test measuring responding on the two levers. The ability to preferentially direct responding toward the action earning the currently-valued reward was taken as evidence of goal-directed behaviour. To test the impact of acute stress, rats were subjected to 15min restraint following pre-feeding and prior to the devaluation test. Neither CCI surgery nor acute stress alone altered sensitivity to devaluation, but in combination CCI and acute stress significantly reduced sensitivity to devaluation. This Study demonstrates that relatively mild stressors that are without effect in uninjured populations can markedly impair cognition under conditions of chronic pain. It further suggests that overlapping neural substrates regulated by nerve injury and/or acute stress are having a cumulative effect on behavioural flexibility.

Habit formation

Habits, both good ones and bad ones, are pervasive in animal behavior. Important frameworks have been developed to understand habits through psychological and neurobiological studies. This work has given us a rich understanding of brain networks that promote habits, and has also helped us to understand what constitutes a habitual behavior as opposed to a behavior that is more flexible and prospective. Mounting evidence from studies using neural recording methods suggests that habit formation is not a simple process. We review this evidence and take the position that habits could be sculpted from multiple dissociable changes in neural activity. These changes occur across multiple brain regions and even within single brain regions. This strategy of classifying components of a habit based on different brain signals provides a potentially useful new way to conceive of disorders that involve overly fixed behaviors as arising from different potential dysfunctions within the brain's habit network.

Stress, glucocorticoids and memory: implications for treating fear-related disorders

Glucocorticoid stress hormones are crucially involved in modulating mnemonic processing of emotionally arousing experiences. They enhance the consolidation of new memories, including those that extinguish older memories, but impair the retrieval of information stored in long-term memory. As strong aversive memories lie at the core of several fear-related disorders, including post-traumatic stress disorder and phobias, the memory-modulating properties of glucocorticoids have recently become of considerable translational interest. Clinical trials have provided the first evidence that glucocorticoid-based pharmacotherapies aimed at attenuating aversive memories might be helpful in the treatment of fear-related disorders. Here, we review important advances in the understanding of how glucocorticoids mediate stress effects on memory processes, and discuss the translational potential of these new conceptual insights.

Learning and memory under stress: implications for the classroom

Exams, tight deadlines and interpersonal conflicts are just a few examples of the many events that may result in high levels of stress in both students and teachers. Research over the past two decades identified stress and the hormones and neurotransmitters released during and after a stressful event as major modulators of human learning and memory processes, with critical implications for educational contexts. While stress around the time of learning is thought to enhance memory formation, thus leading to robust memories, stress markedly impairs memory retrieval, bearing, for instance, the risk of underachieving at exams. Recent evidence further indicates that stress may hamper the updating of memories in the light of new information and induce a shift from a flexible, 'cognitive' form of learning towards rather rigid, 'habit'-like behaviour. Together, these stress-induced changes may explain some of the difficulties of learning and remembering under stress in the classroom. Taking these insights from psychology and neuroscience into account could bear the potential to facilitate processes of education for both students and teachers.

Risky choice and brain CRF after adolescent ethanol vapor exposure and social stress in adulthood

Adolescent ethanol exposure increases risky choice and alters corticotropin releasing factor (CRF) systems in adulthood. The impact of stress on risky choice after adolescent intermittent ethanol (AIE) exposure is not known. We investigated time-specific effects of AIE vapor exposure during early adolescence on risky choice after stress or no stress in adulthood. Male Wistar rats were exposed to air or AIE vapor on postnatal days 28-42 (adolescence) and were exposed to 10days of social defeat or no stress on postnatal days 172-181 (adulthood). Risky choice was assessed in the probability discounting task under baseline conditions and after days 1 and 10 of social defeat. CRF and CRF receptor 1 (CRFR1) mRNA levels were assessed in the prefrontal cortex (PFC) and the central nucleus of the amygdala (CeA) 24h post-stress to evaluate persistent effects of stress on the brain. AIE exposure had no effect on risky choice either at baseline or after social defeat. Additionally, neither acute nor chronic social defeat affected risky choice in air-exposed rats. In the PFC, chronic social defeat selectively decreased CRF mRNA levels in air-exposed rats and increased CRFR1 mRNA levels in all rats. AIE exposure increased CRF mRNA levels in the CeA with no effect of social stress. Our results indicate no effect of ethanol exposure via vapor during early adolescence on risky choice, while our previous findings indicated that AIE exposure via gavage affected risky choice. Both AIE exposure and social defeat altered CRF and CRFR1 mRNA levels in the brain.

Stress effects on the neural substrates of motivated behavior

Exposure to stress has profound, but complex, actions on motivated behavior and decision-making. These effects are central to core symptoms of a number of psychiatric disorders that are precipitated or augmented by stress, such as depressive disorders and substance use disorders. Studying the neural substrates of stress's effects on motivation has revealed that stress affects multiple targets on circuits throughout the brain using diverse molecular signaling processes. Moreover, stress does not have unitary effects on motivated behavior, but differences in the intensity, duration, intermittency, controllability and nature of the stressor produce qualitatively and quantitatively different behavioral endpoints. Unsurprisingly, the results of neuroscientific investigations into stress and motivation often open more questions than they resolve. Here we discuss contemporary results pertaining to the neural mechanisms by which stress alters motivation, identify points of contention and highlight integrative areas for continuing research into these multifaceted complexities.

Impaired flexibility in decision making in rats after administration of the pharmacological stressor yohimbine

RATIONALE

Stress-induced disruption of decision making has been hypothesized to contribute to drug-seeking behaviors and addiction. Noradrenergic signaling plays a central role in mediating stress responses. However, the effects of acute stress on decision making, and the role of noradrenergic signaling in regulating these effects, have not been well characterized.

OBJECTIVE

To characterize changes in decision making caused by acute pharmacological stress, the effects of yohimbine (an α2-adrenergic antagonist) were examined in a delay discounting task. Noradrenergic contributions to decision making were further characterized by examining the effects of propranolol (a β antagonist), prazosin (an α1 antagonist), and guanfacine (an α2 agonist).

METHODS

Sprague-Dawley rats were administered drugs prior to performance on a delay discounting task, in which the delay preceding the large reward increased within each session (ascending delays). To dissociate drug-induced changes in delay sensitivity from behavioral inflexibility, drug effects were subsequently tested in a modified version of the discounting task, in which the delay preceding the large reward decreased within each session (descending delays).

RESULTS

Yohimbine increased choice of the large reward when tested with ascending delays but decreased choice of the same large reward when tested with descending delays, suggesting that drug effects could be attributed to perseverative choice of the lever preferred at the beginning of the session. Propranolol increased choice of the large reward when tested with ascending delays. Prazosin and guanfacine had no effect on reward choice.

CONCLUSIONS

The stress-like effects of yohimbine administration may impair decision making by causing inflexible, perseverative behavior.

Glucocorticoids boost stimulus-response memory formation in humans

Stress affects memory beyond hippocampus-dependent spatial or episodic memory processes. In particular, stress may influence also striatum-dependent stimulus-response (S-R) memory processes. Rodent studies point to an important role of glucocorticoids in the modulation of S-R memory. However, whether glucocorticoids influence S-R memory processes in humans is still unknown. Therefore, we examined in the current experiment the impact of glucocorticoids on the formation of S-R memories in humans. For this purpose, healthy men and women received either hydrocortisone or a placebo 45 min before completing an S-R association learning task and an S-R navigation task. In addition, participants performed also a virtual spatial navigation task and a spatial navigation task in a real environment. Memory of all four learning tasks was tested one week later. Our data showed that hydrocortisone before learning enhanced memory of the S-R association learning task. Moreover, hydrocortisone enhanced the memory of the virtual spatial navigation task, mainly in women. Memory performance in the other tasks remained unaffected by hydrocortisone. These findings provide first evidence that glucocorticoids may facilitate S-R memory formation processes in humans.

The suppression of endogenous adrenalin in the prolongation of ketamine anesthesia

This study investigated whether or not the anesthetic effect of ketamine in rats is dependent on adrenal gland hormones. The study was performed on two main rat groups, intact and adrenalectomized. Rat were divided into subgroups and given appropriate doses of ketamine, metyrapone or metyrosine. Durations of anesthesia in the groups were then recorded. Endogenous catecholamine levels were measured in samples taken from peripheral blood. This experimental results showed that ketamine did not induce anesthesia in intact rats at doses of 15 or 30mg/kg, and that at 60mg/kg anesthesia was established for only 11min. However, ketamine induced significant anesthesia even at a dose of 30mg/kg in animals in which production of endogenous catecholamine (adrenalin, noradrenalin dopamine) was inhibited with metyrosine at a level of 45-47%. Ketamine at 60mg/kg in animals in which endogenous catecholamine was inhibited at a level of 45-47% established anesthesia for 47.6min. However, ketamine at 30 and 60mg/kg induced longer anesthesia in adrenalectomized rats with higher noradrenalin and dopamine levels but suppressed adrenalin production. Adrenalin plays an important role in the control of duration of ketamine anesthesia, while noradrenalin, dopamine and corticosterone have no such function. If endogenous adrenalin is suppressed, ketamine can even provide sufficient anesthesia at a 2-fold lower dose. This makes it possible for ketamine to be used in lengthy surgical procedures.