Age differences in appetitive Pavlovian conditioning and extinction in rats

Translating Developmental Neuroscience to Understand Risk for Psychiatric Disorders

The transition from childhood to adulthood represents the developmental time frame in which the majority of psychiatric disorders emerge. Recent efforts to identify risk factors mediating the susceptibility to psychopathology have led to a heightened focus on both typical and atypical trajectories of neural circuit maturation. Mounting evidence has highlighted the immense neural plasticity apparent in the developing brain. Although in many cases adaptive, the capacity for neural circuit alteration also induces a state of vulnerability to environmental perturbations, such that early-life experiences have long-lasting implications for cognitive and emotional functioning in adulthood. The authors outline preclinical and neuroimaging studies of normative human brain circuit development, as well as parallel efforts covered in this issue of the Journal, to identify brain circuit alterations in psychiatric disorders that frequently emerge in developing populations. Continued translational research into the interactive effects of neurobiological development and external factors will be crucial for identifying early-life risk factors that may contribute to the emergence of psychiatric illness and provide the key to optimizing treatments.

Sensitivity and specificity in affective and social learning in adolescence

Adolescence is a period of heightened affective and social sensitivity. In this review we address how this increased sensitivity influences associative learning. Based on recent evidence from human and rodent studies, as well as advances in computational biology, we suggest that, compared to other age groups, adolescents show features of heightened Pavlovian learning but tend to perform worse than adults at instrumental learning. Because Pavlovian learning does not involve decision-making, whereas instrumental learning does, we propose that these developmental differences might be due to heightened sensitivity to rewards and threats in adolescence, coupled with a lower specificity of responding. We discuss the implications of these findings for adolescent mental health and education.

Reward-motivated memories influence new learning across development

Previously rewarding experiences can influence choices in new situations. Past work has demonstrated that existing reward associations can either help or hinder future behaviors and that there is substantial individual variability in the transfer of value across contexts. Developmental changes in reward sensitivity may also modulate the impact of prior reward associations on later goal-directed behavior. The current study aimed to characterize how reward associations formed in the past affected learning in the present from childhood to adulthood. Participants completed a reinforcement learning paradigm using high- and low-reward stimuli from a task completed 24 h earlier, as well as novel stimuli, as choice options. We found that prior high-reward associations impeded learning across all ages. We then assessed how individual differences in the prioritization of high- versus low-reward associations in memory impacted new learning. Greater high-reward memory prioritization was associated with worse learning performance for previously high-reward relative to low-reward stimuli across age. Adolescents also showed impeded early learning regardless of individual differences in high-reward memory prioritization. Detrimental effects of previous reward on choice behavior did not persist beyond learning. These findings indicate that prior reward associations proactively interfere with future learning from childhood to adulthood and that individual differences in reward-related memory prioritization influence new learning across age.

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8-30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants' mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

A Role for Serotonin in Modulating Opposing Drive and Brake Circuits of Impulsivity

Impulsivity generally refers to a deficit in inhibition, with a focus on understanding the neural circuits which constitute the "brake" on actions and gratification. It is likely that increased impulsivity can arise not only from reduced inhibition, but also from a heightened or exaggerated excitatory "drive." For example, an action which has more vigor, or is fueled by either increased incentive salience or a stronger action-outcome association, may be harder to inhibit. From this perspective, this review focuses on impulse control as a competition over behavioral output between an initially learned response-reward outcome association, and a subsequently acquired opposing inhibitory association. Our goal is to present a synthesis of research from humans and animal models that supports this dual-systems approach to understanding the behavioral and neural substrates that contribute to impulsivity, with a focus on the neuromodulatory role of serotonin. We review evidence for the role of serotonin signaling in mediating the balance of the "drive" and "brake" circuits. Additionally, we consider parallels of these competing instrumental systems in impulsivity within classical conditioning processes (e.g., extinction) in order to point us to potential behavioral and neural mechanisms that may modulate the competing instrumental associations. Finally, we consider how the balance of these competing associations might contribute to, or be extracted from, our experimental assessments of impulsivity. A careful understanding of the underlying behavioral and circuit level contributions to impulsivity is important for understanding the pathogenesis of increased impulsivity present in a number of psychiatric disorders. Pathological levels of impulsivity in such disorders are likely subserved by deficits in the balance of motivational and inhibitory processes.

Translations in Stimulus-Stimulus Pairing: Autoshaping of Learner Vocalizations

Stimulus-stimulus pairing (SSP) is a procedure used by behavior analysis practitioners that capitalizes on respondent conditioning processes to elicit vocalizations. These procedures usually are implemented only after other, more customary methods (e.g., standard echoic training via modeling) have been exhausted. Unfortunately, SSP itself has mixed research support, probably because certain as-yet-unidentified procedural variations are more effective than others. Even when SSP produces (or increases) vocalizations, its effects can be short-lived. Although specific features of SSP differ across published accounts, fundamental characteristics include presentation of a vocal stimulus proximal with presentation of a preferred item. In the present article, we draw parallels between SSP procedures and autoshaping, review factors shown to affect autoshaping, and interpret autoshaping research for suggested SSP tests and applications. We then call for extended use and reporting of SSP in behavior-analytic treatments. Finally, three bridges created by this article are identified: basic-applied, respondent-operant, and behavior analysis with other sciences.

Increased Goal Tracking in Adolescent Rats Is Goal-Directed and Not Habit-Like

When a cue is paired with reward in a different location, some animals will approach the site of reward during the cue, a behavior called goal tracking, while other animals will approach and interact with the cue itself: a behavior called sign tracking. Sign tracking is thought to reflect a tendency to transfer incentive salience from the reward to the cue. Adolescence is a time of heightened sensitivity to rewards, including environmental cues that have been associated with rewards, which may account for increased impulsivity and vulnerability to drug abuse. Surprisingly, however, studies have shown that adolescents are actually less likely to interact with the cue (i.e., sign track) than adult animals. We reasoned that adolescents might show decreased sign tracking, accompanied by increased apparent goal tracking, because they tend to attribute incentive salience to a more reward-proximal "cue": the food magazine. On the other hand, adolescence is also a time of enhanced exploratory behavior, novelty-seeking, and behavioral flexibility. Therefore, adolescents might truly express more goal-directed reward-seeking and less inflexible habit-like approach to a reward-associated cue. Using a reward devaluation procedure to distinguish between these two hypotheses, we found that adolescents indeed exhibit more goal tracking, and less sign tracking, than a comparable group of adults. Moreover, adolescents' goal tracking behavior is highly sensitive to reward devaluation and therefore goal-directed and not habit-like.

Adolescent but not adult Sprague-Dawley rats display goal-directed responding after reward devaluation

Alcohol drinking is typically initiated in adolescence, with use sometimes escalating to problematic levels. Escalation of drinking is often associated with a shift in drinking motives, with goal-directed initial use later transitioning to more habitual behavior. This study assessed whether adolescents are more sensitive than adults to habit formation when indexed via insensitivity to reward devaluation in an operant task for food reward. Adolescent and adult Sprague-Dawley rats were trained on either a random ratio (RR) or random interval (RI) schedule before undergoing devaluation. Adolescent animals on both schedules increased the number of lever presses across all training days. In contrast, adults in the RR group increased the number of lever presses across days whereas RI adults remained relatively stable. In response to pellet devaluation, only adolescents exhibited reduced responding, suggestive of goal-directed behavior, whereas no age differences were evident following control (home cage chow) devaluation. Contrary to our hypothesis, adolescents (but not adults) displayed goal-directed responding indexed via sensitivity to reward devaluation. These findings suggest that adolescents are not necessarily more likely to develop habits than adults, and hence other factors may contribute to the greater propensity of adolescents to engage in and escalate alcohol use.

Translating Developmental Neuroscience to Understand Risk for Psychiatric Disorders

The transition from childhood to adulthood represents the developmental time frame in which the majority of psychiatric disorders emerge. Recent efforts to identify risk factors mediating the susceptibility to psychopathology have led to a heightened focus on both typical and atypical trajectories of neural circuit maturation. Mounting evidence has highlighted the immense neural plasticity apparent in the developing brain. Although in many cases adaptive, the capacity for neural circuit alteration also induces a state of vulnerability to environmental perturbations, such that early-life experiences have long-lasting implications for cognitive and emotional functioning in adulthood. The authors outline preclinical and neuroimaging studies of normative human brain circuit development, as well as parallel efforts covered in this issue of the Journal, to identify brain circuit alterations in psychiatric disorders that frequently emerge in developing populations. Continued translational research into the interactive effects of neurobiological development and external factors will be crucial for identifying early-life risk factors that may contribute to the emergence of psychiatric illness and provide the key to optimizing treatments.

Development of Prefrontal Cortical Connectivity and the Enduring Effect of Learned Value on Cognitive Control

Inhibitory control, the capacity to suppress an inappropriate response, is a process employed for guiding action selection in the service of goal-directed behavior. Under neutral circumstances, inhibitory control success improves from childhood to adulthood and has been associated with developmental shifts in functional activation and connectivity of the PFC. However, the ability to exercise inhibitory control is challenged in certain contexts by including appetitive cues, a phenomenon that may be particularly pronounced in youths. Here, we examine the magnitude and temporal persistence of learned value's influence on inhibitory control in a cross-sectional sample of 8- to 25-year-olds. Participants first underwent conditioning of a motor approach response to two initially neutral cues, with one cue reinforced with monetary reward and the other with no monetary outcome. Subsequently, during fMRI, participants reencountered these cues as no-go targets in a nonreinforced go/no-go paradigm. Although the influence of learned value increasingly disrupted inhibitory control with increasing age, in young adults this pattern remitted over the course of the task, whereas during adolescence the impairing effect of reward history persisted. Successful no-go performance to the previously rewarded target was related to greater recruitment of the right inferior frontal gyrus and age-related increase in functional connectivity between the inferior frontal gyrus and the ventromedial PFC for the previously rewarded no-go target over the control target. Together, results indicate the complex influence of value on goals over development relies upon the increased coordination of distinct higher-order regions in the PFC.

Increased sign-tracking behavior in adolescent rats

An autoshaping procedure was used to test the notion that conditioned stimuli (CSs) gain greater incentive salience during adolescence than young adulthood under conditions of social isolation rearing and food restriction. Rats were single-housed and placed on food restriction during 10 daily training sessions in which a lever (CS⁺ ) was presented then followed immediately by a food unconditioned stimulus (US). A second lever (CS^- ) was presented on intermixed trials and was not reinforced. Despite the fact that food delivery was not contingent on the rats' behavior, all rats exhibited behaviors directed towards the lever (i.e., sign-tracking). In the adolescent group, the rate of lever pressing and the percentage of trials with a lever press were higher than in young adults. Initially, group differences were observed when rats were retrained when the adolescents had reached young adulthood. These findings support the hypothesis that cues that come to predict reward become imbued with excessive motivational value in adolescents, perhaps contributing to the hyper-responsiveness to reward-related stimuli typically observed during this period of development.

Setting the occasion for adolescent inhibitory control

During adolescence, individuals experience a broad range of dynamic environments as they strive to establish independence. Learning to respond appropriately in both new and previously encountered environments requires that an individual identify and learn the meaning of cues indicating that a behavior is appropriate, or alternatively, that it should be altered or inhibited. Although the ability to regulate goal-directed behavior continues to develop across adolescence, the specific circumstances under which adolescents experience difficulty with inhibitory control remain unclear. Here we review recent findings in our laboratory that address how adolescents learn to proactively inhibit a response. Much of our research has utilized a negative occasion setting paradigm, in which one cue (a feature) gates the meaning of a second cue (a target). The feature provides information that resolves the ambiguity of the target and indicates the appropriate behavioral response to the target. As such, we have been able to determine how adolescents learn about ambiguous stimuli, such as those whose meaning changes in accordance with other features of the surrounding environment. We consider why adolescents in particular exhibit difficulty in negative occasion setting compared to either pre-adolescents or adults. In addition, we review findings indicating that a balance in neural activity between orbitofrontal cortex and nucleus accumbens is necessary to support normal negative occasion setting. Finally, we consider aspects of associative learning that may contribute to adolescent inhibitory control, as well as provide insight into adolescent behavior as a whole.

Consequences of Adolescent Exposure to the Cannabinoid Receptor Agonist WIN55,212-2 on Working Memory in Female Rats

Marijuana is a prevalent illicit substance used by adolescents, and several studies have indicated that adolescent use can lead to long-term cognitive deficits including problems with attention and memory. However, preclinical animal studies that observe cognitive deficits after cannabinoid exposure during adolescence utilize experimenter administration of doses of cannabinoids that may exceed what an organism would choose to take, suggesting that contingency and dose are critical factors that need to be addressed in translational models of consequences of cannabinoid exposure. Indeed, we recently developed an adolescent cannabinoid self-administration paradigm in male rats, and found that prior adolescent self-administration of the cannabinoid receptor agonist WIN55,212-2 (WIN) resulted in improved working memory performance in adulthood. In addition, the doses self-administered were not as high as those that are found to produce memory deficits. However, given known sex differences in both drug self-administration and learning and memory processes, it is possible that cannabinoid self-administration could have different cognitive consequences in females. Therefore, we aimed to explore the effects of self-administered vs. experimenter-administered WIN in adolescent female rats on adult cognitive function. Female rats were trained to self-administer WIN daily throughout adolescence (postnatal day 34–59). A control group self-administered vehicle solution. The acute effects of adolescent WIN self-administration on memory were determined using a short-term spatial memory test 24 h after final SA session; and the long-term effects on cognitive performance were assessed during protracted abstinence in adulthood using a delayed-match-to-sample working memory task. In a separate experiment, females were given daily intraperitoneal (IP) injections of a low or high dose of WIN, corresponding to self-administered and typical experimenter-administered doses, respectively, or its vehicle during adolescence and working memory was assessed under drug-free conditions in adulthood. While self-administration of WIN in adolescence had no significant effects on short-term spatial memory or adult working memory, experimenter administration of WIN resulted in improved adult working memory performance that was more pronounced in the low dose group. Thus, low-dose adolescent WIN exposure, whether self-administered or experimenter-administered, results in either improvements or no change in adult working memory performance in female rats, similar to previous results found in males.

Sex-specific enhancement of palatability-driven feeding in adolescent rats

It has been hypothesized that brain development during adolescence perturbs reward processing in a way that may ultimately contribute to the risky decision making associated with this stage of life, particularly in young males. To investigate potential reward dysfunction during adolescence, Experiment 1 examined palatable fluid intake in rats as a function of age and sex. During a series of twice-weekly test sessions, non-food-deprived rats were given the opportunity to voluntarily consume a highly palatable sweetened condensed milk (SCM) solution. We found that adolescent male, but not female, rats exhibited a pronounced, transient increase in SCM intake (normalized by body weight) that was centered around puberty. Additionally, adult females consumed more SCM than adult males and adolescent females. Using a well-established analytical framework to parse the influences of reward palatability and satiety on the temporal structure of feeding behavior, we found that palatability-driven intake at the outset of the meal was significantly elevated in adolescent males, relative to the other groups. Furthermore, although we found that there were some group differences in the onset of satiety, they were unlikely to contribute to differences in intake. Experiment 2 confirmed that adolescent male rats exhibit elevated palatable fluid consumption, relative to adult males, even when a non-caloric saccharin solution was used as the taste stimulus, demonstrating that these results were unlikely to be related to age-related differences in metabolic need. These findings suggest that elevated palatable food intake during adolescence is sex specific and driven by a fundamental change in reward processing. As adolescent risk taking has been hypothesized as a potential result of hypersensitivity to and overvaluation of appetitive stimuli, individual differences in reward palatability may factor into individual differences in adolescent risky decision making.