Relapse after intermittent access to cocaine: Discriminative cues more effectively trigger drug seeking than do conditioned cues

RATIONALE

When people with drug addiction encounter cues associated with drug use, this can trigger cravings and relapse. These cues can include conditioned stimuli (CSs) signaling drug delivery and discriminative stimuli (DSs) signaling drug availability. Compared to CS effects, DS effects are less explored in preclinical studies on cue-induced relapse.

OBJECTIVE

We compared CS and DS effects on reward seeking following abstinence from intermittent-access cocaine (or sucrose) self-administration.

METHODS

During 15-20 intermittent-access sessions, rats self-administered i.v. cocaine or sucrose pellets paired with a light-tone CS. Cocaine/sucrose was available for 5-min (signalled by a light; DS+) and unavailable for 25 min (signalled by different lighting conditions; DS-), and this cycled for 4 h/session. Following abstinence, we measured cocaine/sucrose seeking under extinction triggered by CS and DS presentation, and instrumental responding reinforced by these cues.

RESULTS

Across intermittent-access sessions, rats increased lever pressing for cocaine or sucrose during DS+ periods and decreased responding during DS- periods. On days 2 and 21 of abstinence, only presentation of the DS+ or DS+ and CS combined elicited increased cocaine/sucrose-seeking behaviour (i.e., increased active lever presses). Presenting the DS- alongside the DS+ suppressed the increased cocaine-seeking behaviour otherwise produced by the DS+ . Finally, on day 21 of abstinence, rats showed equivalent levels of lever pressing reinforced by the DS+ , CS and by the DS+ and CS combined, suggesting comparable conditioned reinforcing value.

CONCLUSIONS

After intermittent self-administration, cocaine-associated DSs and CSs acquire similar conditioned reinforcing properties, but DSs more effectively trigger increases in drug seeking.

Temporal context effects on suboptimal choice

Choice can be driven both by rewards and stimuli that signal those rewards. Under certain conditions, pigeons will prefer options that lead to less probable reward when the reward is signaled. A recently quantified model, the Signal for Good News (SiGN) model, assumes that in the context of uncertainty, signals for a reduced delay to reward reinforce choice. The SiGN model provides an excellent fit to previous results from pigeons and the current studies are the first to test a priori quantitative predictions. Pigeons chose between a suboptimal alternative that led to signaled 20% food and an optimal alternative that led to 50% food. The duration of the choice period was manipulated across conditions in two experiments. Pigeons strongly preferred the suboptimal alternative at the shorter durations and strongly preferred the optimal alternative at the longer durations. The results from both experiments fit well with predictions from the SiGN model and show that altering the duration of the choice period has a dramatic effect in that it changes which of the two options pigeons prefer. More generally, these results suggest that the relative value of options is not fixed, but instead depends on the temporal context.

The effects of extinction and an explicitly unpaired treatment on the reinforcing properties of a Pavlovian conditioned stimulus

This series of experiments examined the effects of extinction and an explicitly unpaired treatment on the ability of a conditioned stimulus (CS) to function as a reinforcer. Rats were trained to lever press for food, exposed to pairings of a noise CS and food, and, finally, tested for their willingness to lever press for the CS in the absence of the food. Experiment 1 provided a demonstration of conditioned reinforcement (using controls that were only exposed to unpaired presentations of the CS and food) and showed that it was equivalent after one or four sessions of CS-food pairings. Experiments 2 and 3 showed that, after one session of CS-food pairings, repeated presentations of the CS alone reduced its reinforcing properties; but after four sessions of CS-food pairings, repeated presentations of the CS alone had no effect on these properties. Experiment 4 showed that, after four sessions of CS-food pairings, explicitly unpaired presentations of the CS and food completely undermined conditioned reinforcement. Finally, Experiment 5 provided within-experiment evidence that, after four sessions of CS-food pairings, the reinforcing properties of the CS were disrupted by explicitly unpaired presentations of the CS and food but spared by repeated presentations of the CS alone. Together, these findings indicate that the effectiveness of extinction in undermining the reinforcing properties of a CS depends on its level of conditioning; and that, where extinction fails to disrupt these properties, they are successfully undermined by an explicitly unpaired treatment. They are discussed with respect to findings in the literature on Pavlovian-to-instrumental transfer; and the Rescorla-Wagner model, which anticipates that an explicitly unpaired treatment will be more effective than extinction in reversing the effects of conditioning.

Effects of inter-trial interval on sign-tracking and conditioned reinforcer efficacy in female rats

Previous nonhuman studies have reported that sign-tracking to a conditioned stimulus (CS) is increased when the intertrial interval (ITI) duration is increased. Separate studies indicate that individual differences in sign-tracking (vs. goal-tracking) at a fixed ITI (and CS duration) is predictive of the conditioned reinforcer efficacy of the CS. The present study evaluates, for the first time, if increasing the ITI increases rats' sign-tracking and the conditioned reinforcing efficacy of the CS. Forty-five female rats were randomly assigned to one of three groups that completed appetitive Pavlovian training with ITIs of 14, 24, or 96 s. Subsequently, they completed tests of conditioned reinforcement. Replicating previous findings, longer ITIs increased sign-tracking to a lever-CS and, extending the literature, conditioned reinforcer efficacy of that CS was highest at the longest ITI used during Pavlovian training. Implications for behavioral interventions using conditioned reinforcement are discussed.

Optogenetic activation of basolateral amygdala-to-nucleus accumbens core neurons promotes Pavlovian approach responses but not instrumental pursuit of reward cues

Reward-associated conditioned stimuli (CSs) can acquire predictive value, evoking conditioned approach behaviours that prepare animals to engage with forthcoming rewards. Such CSs can also acquire conditioned reinforcing value, becoming attractive and pursued. Through their conditioned effects, CSs can promote adaptive (e.g., locating food) but also maladaptive behaviours (e.g., drug use). Basolateral amygdala neurons projecting to the nucleus accumbens core (BLA→NAc core neurons) mediate the response to appetitive CSs, but the extent to which this involves effects on the predictive and/or conditioned reinforcing properties of CSs is unclear. Thus, we examined the effects of optogenetic stimulation of BLA→NAc core neurons on i) CS-triggered approach to the site of reward delivery, a Pavlovian conditioned approach response and ii) the instrumental pursuit of a CS, a measure of conditioned reinforcement. Water-restricted, adult male rats learned that a light-tone compound cue (the CS) predicts water delivery into a receptacle. Pairing optogenetic stimulation of BLA→NAc core neurons with CS presentation potentiated CS-triggered water receptacle visits. This suggests that activity in BLA→NAc core neurons promotes Pavlovian goal-approach behaviour. Next, rats could lever press for CS presentations, without water delivery. Optogenetic stimulation of BLA→NAc core neurons either during instrumental test sessions or during prior CS-water conditioning did not influence lever responding for the CS. This suggests that activity in BLA→NAc core neurons does not influence the instrumental pursuit of a water-paired CS. We conclude that activation of BLA→NAc core neurons promotes cue-induced control over behaviour by increasing conditioned goal-approach responses, without affecting the operant pursuit of reward cues.

Muscarinic and NMDA Receptors in the Substantia Nigra Play a Role in Reward-Related Learning

BACKGROUND

Reward-related learning, where animals form associations between rewards and stimuli (i.e., conditioned stimuli [CS]) that predict or accompany those rewards, is an essential adaptive function for survival.

METHODS

In this study, we investigated the mechanisms underlying the acquisition and performance of conditioned approach learning with a focus on the role of muscarinic acetylcholine (mACh) and NMDA glutamate receptors in the substantia nigra (SN), a brain region implicated in reward and motor processes.

RESULTS

Using RNAscope in situ hybridization assays, we found that dopamine neurons of the SN express muscarinic (mACh5), NMDA2a, NMDA2b, and NMDA2d receptor mRNA but not mACh4. NMDA, but not mACh5, receptor mRNA was also found on SN GABA neurons. In a conditioned approach paradigm, rats were exposed to 3 or 7 conditioning sessions during which light/tone (CS) presentations were paired with delivery of food pellets, followed by a test session with CS-only presentations. Intra-SN microinjections of scopolamine (a mACh receptor antagonist) or AP-5 (a NMDA receptor antagonist) were made either prior to each conditioning session (to test their effects on acquisition) or prior to the CS-only test (to test their effects on expression of the learned response). Scopolamine and AP-5 produced dose-dependent significant reductions in the acquisition, but not performance, of conditioned approach.

CONCLUSIONS

These results suggest that SN mACh and NMDA receptors are key players in the acquisition, but not the expression, of reward-related learning. Importantly, these findings redefine the role of the SN, which has traditionally been known for its involvement in motor processes, and suggest that the SN possesses attributes consistent with a function as a hub of integration of primary reward and CS signals.

Paradoxical choice and the reinforcing value of information

Signals that reduce uncertainty can be valuable because well-informed decision-makers can better align their preferences to opportunities. However, some birds and mammals display an appetite for informative signals that cannot be used to increase returns. We explore the role that reward-predictive stimuli have in fostering such preferences, aiming at distinguishing between two putative underlying mechanisms. The 'information hypothesis' proposes that reducing uncertainty is reinforcing per se, somewhat consistently with the concept of curiosity: a motivation to know in the absence of tractable extrinsic benefits. In contrast, the 'conditioned reinforcement hypothesis', an associative account, proposes asymmetries in secondarily acquired reinforcement: post-choice stimuli announcing forthcoming rewards (S⁺) reinforce responses more than stimuli signalling no rewards (S^-) inhibit responses. In three treatments, rats faced two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), food or no food was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred Info when (1) both outcomes were explicitly signalled by salient auditory cues, (2) only forthcoming food delivery was explicitly signalled, and (3) only the absence of forthcoming reward was explicitly signalled. Acquisition was slower in (3), when food was not explicitly signalled, showing that signals for positive outcomes have a greater influence on the development of preference than signals for negative ones. Our results are consistent with an elaborated conditioned reinforcement account, and with the conjecture that both uncertainty reduction and conditioned reinforcement jointly act to generate preference.

Good news is better than bad news, but bad news is not worse than no news

Under certain conditions, pigeons will reliably prefer an alternative that leads to a lower probability of food over an alternative that leads to a higher probability of food (i.e., demonstrate suboptimal choice). A critical aspect of the typical procedure is that the alternative associated with less food provides differential stimuli that signal trial outcomes, but the alternative associated with more food does not. Few studies have investigated how partial signaling of an alternative influences preference. In Experiments 1-3, pigeons chose between two alternatives that each led to food 60% of the time with partially signaled trial outcomes. One alternative occasionally provided a stimulus that always preceded food (i.e., "good news") and the other alternative occasionally provided a stimulus that always preceded no food ("bad news"). Experiments 2 and 3 also assessed preference in conditions in which alternatives were either completely unsignaled (provided no differential stimuli) or always led to food. Pigeons consistently preferred the "good news" alternative over the "bad news" alternative and preferred 100% food over the "bad news" alternative. The results from conditions in which pigeons chose between the "bad news" alternative and an unsignaled alternative were inconclusive, but suggestive of a preference for bad news. The results are used to evaluate and distinguish between competing explanations of suboptimal choice.

Forced-exposure trials increase suboptimal choice

The influence of single option or forced-exposure (FE) trials was studied in the suboptimal choice task. Pigeons chose between an optimal alternative that led to food half of the time and a suboptimal alternative that led to food 20% of the time. Choice of the suboptimal alternative was compared across groups of subjects that received different numbers of FE trials during training. In Experiment 1, subjects received 100% FE trials, 67% FE trials, or only choice trials. Pigeons in the two groups that had FE trials developed extreme preference for the signaled suboptimal alternative over the unsignaled optimal alternative, while pigeons that had no FE trials showed pronounced individual differences. Experiment 2 compared 10% and 90% FE trials. When neither alternative signaled trial outcomes, both groups of subjects strongly preferred the optimal alternative. When the suboptimal alternative provided differential signals, the subjects in the 90% FE group developed strong preference for the suboptimal alternative and subjects in the 10% FE group maintained preference for the optimal alternative. The results of both experiments demonstrate that FE trials can have substantial effects on the development of preference in the suboptimal choice task.

The Basolateral Amygdala to Nucleus Accumbens Core Circuit Mediates the Conditioned Reinforcing Effects of Cocaine-Paired Cues on Cocaine Seeking

BACKGROUND

Individuals addicted to cocaine spend much of their time foraging for the drug. Pavlovian drug-associated conditioned stimuli exert a major influence on the initiation and maintenance of drug seeking often long into abstinence, especially when presented response-contingently, acting as conditioned reinforcers that bridge delays to drug use. The acquisition of cue-controlled cocaine seeking has been shown to depend on functional interactions between the basolateral amygdala (BLA) and the nucleus accumbens core (NAcC). However, the precise neuronal circuits underlying the acquisition of cue-controlled cocaine-seeking behavior have not been elucidated.

METHODS

Here, we used a projection-specific Cre-dependent DREADD (designer receptor exclusively activated by designer drugs)-mediated causal approach to test the hypothesis that the direct projections from the BLA to the NAcC are required for the acquisition of cue-controlled cocaine-seeking behavior.

RESULTS

In Sprague Dawley rats with Cre-mediated expression of the inhibitory DREADD hM4D(Gi) in the NAcC-projecting BLA neurons, treatment with clozapine N-oxide, but not vehicle, selectively prevented the impact of cocaine-associated conditioned reinforcers on cocaine seeking under a second-order schedule of reinforcement. This effect was attributable to the chemogenetic inhibition of the NAcC-projecting BLA neurons, as it was reversible, and it was absent in clozapine N-oxide-treated rats expressing an empty control virus. In contrast, chemogenetic inhibition of the anterior insula, which receives collateral projections from NAcC-projecting BLA neurons, was without effect.

CONCLUSIONS

These data demonstrate that the acquisition of cue-controlled cocaine seeking that depends on the conditioned reinforcing effects of cocaine cues requires activity in the direct projections from the BLA to the NAcC.

Neurobiology of reward-related learning

A major goal in psychology is to understand how environmental stimuli associated with primary rewards come to function as conditioned stimuli, acquiring the capacity to elicit similar responses to those elicited by primary rewards. Our neurobiological model is predicated on the Hebbian idea that concurrent synaptic activity on the primary reward neural substrate-proposed to be ventral tegmental area (VTA) dopamine (DA) neurons-strengthens the synapses involved. We propose that VTA DA neurons receive both a strong unconditioned stimulus signal (acetylcholine stimulation of DA cells) from the primary reward capable of unconditionally activating DA cells and a weak stimulus signal (glutamate stimulation of DA cells) from the neutral stimulus. Through joint stimulation the weak signal is potentiated and capable of activating the VTA DA cells, eliciting a conditioned response. The learning occurs when this joint stimulation initiates intracellular second-messenger cascades resulting in enhanced glutamate-DA synapses. In this review we present evidence that led us to propose this model and the most recent evidence supporting it.

Devaluation of a conditioned reinforcer requires its reexposure

A change in motivational state does not guarantee a change in operant behaviour. Only after an organism has had contact with an outcome while in a relevant motivational state does behaviour change, a phenomenon called incentive learning. While ample evidence indicates that this is true for primary reinforcers, it has not been established for conditioned reinforcers. We performed an experiment with rats where lever-presses were reinforced by presentations of an audiovisual stimulus that had previously preceded food delivery; in the critical experimental groups, the audiovisual stimulus was then paired a single time with a strong electric shock. Some animals were reexposed to the audiovisual stimulus. Lever-presses yielding no outcomes were recorded in a subsequent test. Animals that had been reexposed to the audiovisual stimulus after the aversive training responded less than did those that had not received reexposure. Indeed, those animals that were not reexposed did not differ from a control group that received no aversive conditioning of the audiovisual stimulus. Moreover, these results were not mediated by a change in the food's reinforcement value, but instead reflect a change in behaviour with respect to the conditioned reinforcer itself. These are the first data to indicate that the affective value of conditioned stimuli, like that of unconditioned ones, is established when the organism comes into contact with them.

The pharmacological stressor yohimbine, but not U50,488, increases responding for conditioned reinforcers paired with ethanol or sucrose

RATIONALE

Environmental stimuli paired with alcohol can function as conditioned reinforcers (CRfs) and trigger relapse to alcohol-seeking. In animal models, pharmacological stressors can enhance alcohol consumption and reinstate alcohol-seeking, but it is unknown whether stress can potentiate the conditioned reinforcing properties of alcohol-paired stimuli.

OBJECTIVES

We examined whether the pharmacological stressors, the α-2 adrenoreceptor antagonist yohimbine (vehicle, 1.25, 2.5 mg/kg; IP) and the K-opioid receptor agonist U50,488 (vehicle, 1.25, 2.5 mg/kg; SC), increase responding for conditioned reinforcement, and if their effects interact with the nature of the reward (alcohol vs. sucrose). We subsequently examined the effects of yohimbine (vehicle, 1.25, 2.5 mg/kg; IP) on responding for sensory reinforcement.

METHODS

Male Long-Evans underwent Pavlovian conditioning, wherein a tone-light conditioned stimulus (CS) was repeatedly paired with 12% EtOH or 21.7% sucrose. Next, tests of responding for a CRf were conducted. Responding on the CRf lever delivered the CS, whereas responding on the other lever had no consequences. In a separate cohort of rats, the effects of yohimbine on responding just for the sensory reinforcer were examined.

RESULTS

Both doses of yohimbine, but not U50,488, increased responding for conditioned reinforcement. This enhancement occurred independently of the nature of the reward used during Pavlovian conditioning. Yohimbine-enhanced responding for a CRf was reproducible and stable over five tests; it even persisted when the CS was omitted. Both doses of yohimbine also increased responding for sensory reinforcement.

CONCLUSIONS

Yohimbine increases operant responding for a variety of sensory and conditioned reinforcers. This enhancement may be independent of its stress-like effects.

A-learning: A new formulation of associative learning theory

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

Ablation of NMDA receptors in dopamine neurons disrupts attribution of incentive salience to reward-paired stimuli

Midbrain dopamine (DA) neurons play a crucial role in the formation of conditioned associations between environmental cues and appetitive events. Activation of N-methyl-d-aspartate (NMDA) receptors is a key mechanism responsible for the generation of conditioned responses of DA neurons to reward cues. Here, we tested the effects of the cell type-specific inactivation of NMDA receptors in DA neurons in adult mice on stimulus-reward learning. Animals were trained in a Pavlovian learning paradigm in which they had to learn the predictive value of two conditioned stimuli, one of which (CS+) was paired with the delivery of a water reward. Over the course of conditioning, mutant mice learned that the CS+ predicted reward availability, and they approached the reward receptacle more frequently during CS+ trials than CS- trials. However, conditioned responses to the CS+ were weaker in the mutant mice, possibly indicating that they did not attribute incentive salience to the CS+. To further assess whether the attribution of incentive salience was impaired by the mutation, animals were tested in a conditioned reinforcement test. The test revealed that mutant mice made fewer instrumental responses paired with CS+ presentation, confirming that the CS+ had a weaker incentive value. Taken together, these results indicate that reward prediction learning does occur in the absence of NMDA receptors in DA neurons, but the ability of reward-paired cues to invigorate and reinforce behavior is attenuated.

The influence of outcome delay on suboptimal choice

Under certain conditions pigeons will choose an option that provides less probable food over one that provides more probable food. This suboptimal choice behavior occurs when the outcomes are delayed and stimuli during the delay differentially signal the upcoming outcomes on the suboptimal alternative, but not the optimal alternative. The present study assessed whether duration of the outcome delay affects pigeons' suboptimal preference. Pigeons chose between a suboptimal alternative that provided food 20% of the time and an optimal alternative that provided food 80% of the time. Stimuli presented during the delays signaled the outcomes on the suboptimal alternative, but not on the optimal alternative. The outcome delays were 5 s in some conditions and 20 s in others. The results of two experiments demonstrate that behavior is generally more suboptimal when the outcome delays are longer but tends to stay relatively suboptimal if subjects experience the long delay condition before the short delay condition. The finding that behavior is more suboptimal with longer delays to the outcomes is consistent with the view that pigeons' suboptimal choice is influenced by both conditioned and primary reinforcement and is inconsistent with the view that suboptimal choice is influenced solely by signal value.

Cannabinoid receptor-1 signaling contributions to sign-tracking and conditioned reinforcement in rats

RATIONALE

Endocannabinoids (eCBs) are critical gatekeepers of dopaminergic signaling, and disrupting cannabinoid receptor-1 (CB1) signaling alters DA dynamics to attenuate cue-motivated behaviors. Prior studies suggest that dopamine (DA) release plays a critical role in driving sign-tracking.

OBJECTIVES

Here, we determine whether systemic injections of rimonabant, a CB1 receptor inverse agonist, during Pavlovian lever autoshaping impair the expression of sign-tracking. We next examine whether rimonabant blocks the reinforcing properties of the Pavlovian lever cue in a conditioned reinforcement test.

METHODS

In Exp. 1, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1, 3 mg/kg) during early and late Pavlovian lever autoshaping sessions. In Exp. 2, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1 mg/kg) during a conditioned reinforcement test.

RESULTS

Rimonabant dose-dependently decreased lever contact and probability, and increased sign-tracker's latency to approach the lever cue early in Pavlovian training. With extended training, many previously goal-tracking and intermediate rats shifted to lever approach, which remained dose-dependently sensitive to rimonabant. Rimonabant attenuated cue-evoked food cup approach early, but not late, in conditioning, and did not affect pellet retrieval or consumption. The inserted lever cue served as a robust conditioned reinforcer after Pavlovian lever autoshaping, and 1 mg/kg rimonabant blocked conditioned reinforcement.

CONCLUSIONS

Together, our results suggest that CB1 signaling mediates two critical properties of incentive stimuli; their ability to attract (Exp. 1) and their ability to reinforce (Exp. 2) behavior.

Suboptimal choice in pigeons: Does the predictive value of the conditioned reinforcer alone determine choice?

Prior research has found that pigeons are indifferent between an option that always provides a signal for reinforcement and an alternative that provides a signal for reinforcement only 50% of the time (and a signal for the absence of reinforcement 50% of the time). This suboptimal choice suggests that the frequency of the signal for reinforcement plays virtually no role and choice depends only on the predictive value of the signal for reinforcement associated with each alternative. In the present research we tested the hypothesis that if there are two or three signals for reinforcement associated with the suboptimal alternative but each occurs only 25% or 17% of the time, respectively, pigeons would show a greater preference for the suboptimal alternative. Although we found that increasing the number of signals for reinforcement associated with the suboptimal alternative did not increase the preference for the suboptimal alternative (relative to a single signal for reinforcement) extended training on this task resulted in a significant preference for the suboptimal alternative by both groups. This result suggests that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is also responsible for choice of the suboptimal alternative.

Drug Cues, Conditioned Reinforcement, and Drug Seeking: The Sequelae of a Collaborative Venture With Athina Markou

Athina Markou spent a research period in my laboratory, then in the Department of Anatomy in Cambridge University, in 1991 to help us establish a cocaine-seeking procedure. Thus we embarked on developing a second-order schedule of intravenous cocaine reinforcement to investigate the neural basis of the pronounced effects of cocaine-associated conditioned stimuli on cocaine seeking. This brief review summarizes the fundamental aspects of cocaine seeking measured using this approach and the importance of the methodology in enabling us to define the neural mechanisms and circuitry underlying conditioned reinforcement and cocaine, heroin, and alcohol seeking. The shift over time and experience of control over drug seeking from a limbic cortical-ventral striatal circuit underlying goal-directed drug seeking to a dorsal striatal system mediating habitual drug seeking are also summarized. The theoretical implications of these data are discussed, thereby revealing the ways in which the outcomes of a collaboration can endure.

How I Learned to Stop Worrying and Love Replication Failures

Worries about the reproducibility of experiments in the behavioral and social sciences arise from evidence that many published reports contain false positive results. Misunderstanding and misuse of statistical procedures are key sources of false positives. In behavior analysis, however, statistical procedures have not been used much. Instead, the investigator must show that the behavior of an individual is consistent over time within an experimental condition, that the behavior changes systematically across conditions, and that these changes can be reproduced - and then the whole pattern must be shown in additional individuals. These high standards of within- and between-subject replication protect behavior analysis from the publication of false positive findings. When a properly designed and executed experiment fails to replicate a previously published finding, the failure exposes flaws in our understanding of the phenomenon under study - perhaps in recognizing the boundary conditions of the phenomenon, identifying the relevant variables, or bringing the variables under sufficient control. We must accept the contradictory findings as valid and pursue an experimental analysis of the possible reasons. In this way, we resolve the contradiction and advance our science. To illustrate, two research programs are described, each initiated because of a replication failure.

Toward a Unifying Account of Impulsivity and the Development of Self-Control

Impulsivity has traditionally been thought to involve various behavioral traits that can be measured using different laboratory protocols. Whereas some authors regard different measures of impulsivity as reflecting fundamentally distinct and unrelated behavioral tendencies (fragmentation approach), others regard those different indexes as analogue forms of the same behavioral tendency, only superficially different (unification approach). Unifying accounts range from mere intuitions to more sophisticated theoretical systems. Some of the more complete attempts at unifying are intriguing but have validity weaknesses. We propose a new unifying attempt based on theoretical points posed by other authors and supplemented by theory and research on associative learning. We then apply these assumptions to characterize the paradigms used to study impulsivity in laboratory settings and evaluate their scope as an attempt at unification. We argue that our approach possesses a good balance of parsimony and empirical and theoretical grounding, as well as a more encompassing scope, and is more suitable for experimental testing than previous theoretical frameworks. In addition, the proposed approach is capable of generating a new definition of impulsivity and outlines a hypothesis of how self-control can be developed. Finally, we examine the fragmentation approach from a different perspective, emphasizing the importance of finding similarities among seemingly different phenomena.

Facilitating Neurofeedback in Children with Autism and Intellectual Impairments Using TAGteach

Individuals with autism and intellectual impairments tend to be excluded from research due to their difficulties with methodological compliance. This study focuses on using Teaching with Acoustic Guidance-TAGteach-to behaviorally prepare children with autism and a IQ ≤ 80 to participate in a study on neurofeedback training (NFT). Seven children (ages 6-8) learned the prerequisite skills identified in a task analysis in an average of 5 h of TAGteach training, indicating that this is a feasible method of preparing intellectually-impaired children with autism to participate in NFT and task-dependent electroencephalography measures. TAGteach may thus have the potential to augment this population's ability to participate in less accessible treatments and behavioral neuroscientific studies.

Elucidating the role of D4 receptors in mediating attributions of salience to incentive stimuli on Pavlovian conditioned approach and conditioned reinforcement paradigms

The power of drug-associated cues to instigate drug 'wanting' and consequently promote drug seeking is a corner stone of contemporary theories of addiction. Gambling disorder has recently been added to the pantheon of addictive disorders due to the phenomenological similarities between the diseases. However, the neurobiological mechanism that may mediate increased sensitivity towards conditioned stimuli in addictive disorders is unclear. We have previously demonstrated using a rodent analogue of a simple slot machine that the dopamine D4 receptor is critically engaged in controlling animals' attribution of salience to stimuli associated with reward in this paradigm, and consequently may represent a target for the treatment of gambling disorder. Here, we investigated the role of acute administration of a D4 receptor agonist on animals' responsivity to conditioned stimuli on both a Pavlovian conditioned approach (autoshaping) and a conditioned reinforcement paradigm. Following training on one of the two tasks, separate cohorts of rats (male and female) were administered a dose of PD168077 shown to be maximally effective at precipitating errors in reward expectancy on the rat slot machine task (10mg/kg). However, augmenting the activity of the D4 receptors in this manner did not alter behaviour on either task. These data therefore provide novel evidence that the D4 receptor does not alter incentive motivation in response to cues on simple behavioural tasks.

Contributions to drug abuse research of Steven R. Goldberg's behavioral analysis of stimulus-stimulus contingencies

By the mid-1960s, the concept that drugs can function as reinforcing stimuli through response-reinforcer contingencies had created a paradigm shift in drug abuse science. Steve Goldberg's first several publications focused instead on stimulus-stimulus contingencies (respondent conditioning) in examining Abraham Wikler's two-factor hypothesis of relapse involving conditioned withdrawal and reinforcing effects of drugs. Goldberg provided a compelling demonstration that histories of contingencies among stimuli could produce lasting withdrawal reactions in primates formerly dependent on opioids. Other studies conducted by Goldberg extended the analysis of effects of stimulus-stimulus contingencies on behavior maintained by opioid reinforcing effects and showed that withdrawal-inducing antagonist administration can produce conditioned increases in self-administration. Subsequent studies of the effects of stimuli associated with cocaine injection under second-order schedules showed that the maintenance of behavior with drug injections was in most important aspects similar to the maintenance of behavior with more conventional reinforcers when the behavior-disrupting pharmacological effects of the drugs were minimized. Studies on second-order schedules demonstrated a wide array of conditions under which behavior could be maintained by drug injection and further influenced by stimulus-stimulus contingencies. These schedules present opportunities to produce in the laboratory complex situations involving response- and stimulus-stimulus contingencies, which go beyond simplistic pairings of stimuli and more closely approximate those found with human drug abusers. A focus on the response- and stimulus-stimulus contingencies, and resulting quantifiable changes in objective and quantifiable behavioral endpoints exemplified by the studies by Steve Goldberg, remains the most promising way forward for studying problems of drug dependence.

Determinants of conditioned reinforcing effectiveness: Dopamine D2-like receptor agonist-stimulated responding for cocaine-associated stimuli

Environmental stimuli associated with drug use can take on conditioned properties capable of promoting drug-seeking behaviors during abstinence. This study investigated the relative importance of the amount of reinforced responding, number of cocaine-stimulus pairings, total cocaine intake, and reinforcing effectiveness of the self-administered dose of cocaine to the conditioned reinforcing effectiveness of cocaine-associated stimuli (CS). Male rats were trained to self-administer cocaine (0.1 [small] or 1.0mg/kg/inf [large]) under a fixed ratio schedule of reinforcement. A progressive ratio (PR) schedule was used to quantify the reinforcing effectiveness of each dose of cocaine, as well as the conditioned reinforcing effectiveness of the CS following treatment with saline or the dopamine D2-like receptor agonist pramipexole (0.1-3.2mg/kg). The large dose of cocaine maintained larger final ratios and greater levels of cocaine intake, whereas the small dose resulted in more cocaine-CS pairings. The total amount of responding was comparable between groups. During PR tests of conditioned reinforcement, pramipexole increased responding for CS presentations in both groups; however, the final ratio completed was significantly greater in large- as compared to small-dose group. In addition to highlighting a central role for dopamine D2-like receptors in modulating the effectiveness of cocaine-paired stimuli to reinforce behavior, these results suggest that conditioned reinforcing effectiveness is primarily determined by the reinforcing effectiveness of the self-administered dose of cocaine and/or total cocaine intake, and not the total amount of responding or number cocaine-stimulus pairings. These findings have implications for understanding how different patterns of drug-taking might impact vulnerability to relapse.

Inducing Novel Vocalizations by Conditioning Speech Sounds as Reinforcers

Automatic reinforcement by response-produced auditory stimuli has long been hypothesized to play a role in the vocal development of infants. Clinicians and researchers have translated this idea into stimulus pairing interventions intended to increase novel vocalizations of nonverbal children with autism and other developmental disabilities by conditioning speech sounds as reinforcers. A number of studies have demonstrated positive effects of stimulus pairing procedures, but negative results have also been reported in the literature. This article provides a brief review of the existing literature on stimulus pairing procedures and a discussion of alternative procedures that may serve to establish speech sounds as reinforcers. Directions for future research are discussed and recommendations provided to clinicians.

Ethanol self-administration in mice under a second-order schedule

Long Fixed-Interval (FI) schedules, particularly second-order schedules, can engender substantial responding before drug or ethanol delivery that is uninfluenced by the direct effects of the drug or ethanol. Thus, these schedules can be used to study the effects of medications upon drug- or ethanol-seeking, uninfluenced by the direct effects of the self-administered drug or ethanol. Long FI second-order schedules are frequently used in primates and occasionally in rats. Under second-order schedules, completion of one response requirement, e.g., a Fixed Ratio 10 (FR10:S), produces a brief stimulus presentation, e.g., a 1-s 80-dB 4-kHZ tone, and this FR10:S serves as the response unit under another schedule, e.g., an FI 1800-s. Thus, the first FR10 completed after 1800 s would result in delivery both of the tone and of reinforcement, e.g., 10 × 0.01 mL 16% (w/v) ethanol. To examine if such schedules could be effectively used in mice, which have advantages in neurobiological and genetic studies, we trained eight C57BL/6J mice to respond under the schedule just described. This schedule maintained substantial responding. The temporal pattern of behavior was typical of an FI schedule with responding accelerating across the interval. We also examined the effects of acute and chronic administration of fluvoxamine on this responding, and these were modest. Finally, we examined responding when alcohol and/or tone deliveries were withheld, and found that extinction occurred most rapidly when both were withheld. This work demonstrates that long FI schedules of ethanol delivery may be useful in studying ethanol seeking in mice.

Individual variation in the propensity to attribute incentive salience to a food cue: influence of sex

There is considerable individual variation in the propensity of animals to attribute incentive salience to discrete reward cues, but to date most of this research has been conducted in male rats. The purpose of this study was to determine whether sex influences the propensity to attribute incentive salience to a food cue, using rats from two different outbred strains (Sprague-Dawley [SD] and Heterogeneous Stock [HS]). The motivational value of a food cue was assessed in two ways: (i) by the ability of the cue to elicit approach toward it and (ii) by its ability to act as a conditioned reinforcer. We found that female SD rats acquired Pavlovian conditioned approach behavior slightly faster than males, but no sex difference was detected in HS rats, and neither strain showed a sex difference in asymptotic performance of approach behavior. Moreover, female approach behavior did not differ across estrous cycle. Compared to males, females made more active responses during the test for conditioned reinforcement, although they made more inactive responses as well. We conclude that although there are small sex differences in performance on these tasks, these are probably not due to a notable sex difference in the propensity to attribute incentive salience to a food cue.

Responding for a conditioned reinforcer, and its enhancement by nicotine, is blocked by dopamine receptor antagonists and a 5-HT(2C) receptor agonist but not by a 5-HT(2A) receptor antagonist

An aspect of nicotine reinforcement that may contribute to tobacco addiction is the effect of nicotine to enhance the motivational properties of reward-associated cues, or conditioned stimuli (CSs). Several studies have now shown that nicotine enhances responding for a stimulus that has been paired with a natural reinforcer. This effect of nicotine to enhance responding for a conditioned reinforcer is likely due to nicotine-induced enhancements in mesolimbic dopaminergic activity, but this has not been directly assessed. In this study, we assessed roles for dopamine (DA) D1 or D2 receptors, and two serotonin (5-HT) receptor subtypes known to modulate DA activity, the 5-HT2C or 5-HT2A subtypes, on nicotine-enhanced responding for a conditioned reinforcer. Water-restricted rats were exposed to Pavlovian conditioning sessions, where a CS was paired with water delivery. Then, in a second phase, animals were required to perform a novel, lever-pressing response for presentations of the CS as a conditioned reinforcer. Nicotine (0.4 mg/kg) enhanced responding for the conditioned reinforcer. To examine potential roles for dopamine (DA) and serotonin (5-HT) receptors in this effect, separate groups of animals were used to assess the impact of administering the D1 receptor antagonist SCH 23390, D2 receptor antagonist eticlopride, 5-HT2C receptor agonist Ro 60-0175, or 5-HT2A receptor antagonist M100907 on nicotine-enhanced responding for conditioned reinforcement. SCH 23390, eticlopride, and Ro 60-0175 all reduced responding for conditioned reinforcement, and the ability of nicotine to enhance this effect. M100907 did not alter this behavior. Together, these studies indicate that DA D1 and D2 receptors, but not 5-HT2A receptors, contribute to the effect of nicotine to enhance responding for a conditioned reinforcer. This effect can also be modulated by 5-HT2C receptor activation.

Observing of chain-schedule stimuli

A classical-conditioning account of the processes maintaining behavior under chained schedules entails a backward transmission of conditioned-reinforcement effects. Assessing this process in traditional chain schedules is limited because the response maintained by stimulus onset accompanied by each link in a chain schedule may also be maintained by the primary reinforcer. In the present experiment, an observing response was used to measure the conditioned-reinforcing effects of stimuli associated with a three-link chain variable-time (VT) food schedule, and resistance-to-change tests (extinction and prefeeding) were implemented to examine if a backward transmission of reinforcement effects occur. Four pigeons served as subjects. Observing was maintained by the production of stimuli correlated with links of a three-link chain VT schedule with the middle-link stimulus maintaining the highest rate of observing, followed by the initial-link stimulus and the terminal-link stimulus maintaining the lowest observing rate. Results from resistance-to-change tests of extinction and prefeeding were not supportive of a backward transmission of reinforcement effects and in general, the pattern of resistance-to-change was forward. Based on past and current research, it appears that a backward pattern of relative rate decreases in responses maintained by stimuli correlated with a chain schedule due to disruption (i.e., extinction and prefeeding) is not a ubiquitous process that is evident within different chain-schedule arrangements.

Repeated nicotine exposure during adolescence alters reward-related learning in male and female rats

RATIONALE

Repeated nicotine exposure causes neuroadaptations in limbic cortico-striatal circuits involved in learning and motivation. Such alterations are relevant to addiction because they are suggested to mediate the ability of smoking-associated stimuli to control behavior and to enhance nicotine-seeking and -taking behaviors. Female smokers report higher cue reactivity relative to their male counter parts, yet little is known about putative gender-specific effects of adolescent nicotine exposure on reward-related learning. Prior repeated nicotine exposure in adult male rats enhances Pavlovian approach behavior and conditioned reinforcement.

OBJECTIVE

Given that smoking is typically initiated during adolescence, here we assessed the extent to which adolescent nicotine exposure impacts Pavlovian approach and conditioned reinforcement in male and female rats.

METHODS

Rats were injected with nicotine on postnatal days 31-45 prior to training on Pavlovian approach behavior starting on day 51. They were trained to associate a conditioned stimulus (CS), illumination of a magazine light, and tone, with an unconditioned stimulus (US), the delivery of water, for 10-daily sessions, and then were tested on the acquisition of responding with conditioned reinforcement.

RESULTS

Adolescent nicotine exposure selectively increased approach to the magazine during the CS in males but decreased approach to the magazine during the CS in female rats. Vehicle-exposed female rats, however, showed greater magazine approach during the CS than did male control rats. Prior nicotine exposure also enhanced conditioned reinforcement in both male and female rats.

CONCLUSIONS

Repeated exposure to nicotine during adolescence had opposite effects on Pavlovian approach behavior in male and female rats but enhanced acquisition of a new response with conditioned reinforcement. Novel information on how nicotine exposure influences reward-related learning during adolescence may increase our understanding of neurobiological mechanisms involved in the initiation of smoking behavior.