Reward uncertainty enhances incentive salience attribution as sign-tracking

Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors

Rat autoshaping procedures generate two readily measurable conditioned responses: During lever presentations that have previously signaled food, rats approach the food well (called goal-tracking) and interact with the lever itself (called sign-tracking). We investigated how reinforced and nonreinforced trials affect the overall and temporal distributions of these two responses across 10-second lever presentations. In two experiments, reinforced trials generated more goal-tracking than sign-tracking, and nonreinforced trials resulted in a larger reduction in goal-tracking than sign-tracking. The effect of reinforced trials was evident as an increase in goal-tracking and reduction in sign-tracking across the duration of the lever presentations, and nonreinforced trials resulted in this pattern transiently reversing and then becoming less evident with further training. These dissociations are consistent with a recent elaboration of the Rescorla-Wagner model, HeiDI (Honey, R.C., Dwyer, D.M., & Iliescu, A.F. (2020a). HeiDI: A model for Pavlovian learning and performance with reciprocal associations. Psychological Review, 127, 829-852.), a model in which responses related to the nature of the unconditioned stimulus (e.g., goal-tracking) have a different origin than those related to the nature of the conditioned stimulus (e.g., sign-tracking).

Opening new vistas on obsessive-compulsive disorder with the observing response task

Obsessive-compulsive disorder (OCD), a highly prevalent and debilitating disorder, is incompletely understood in terms of underpinning behavioural, psychological, and neural mechanisms. This is attributable to high symptomatic heterogeneity; cardinal features comprise obsessions and compulsions, including clinical subcategories. While obsessive and intrusive thoughts are arguably unique to humans, dysfunctional behaviours analogous to those seen in clinical OCD have been examined in nonhuman animals. Genetic, ethological, pharmacological, and neurobehavioural approaches all contribute to understanding the emergence and persistence of compulsive behaviour. One behaviour of particular interest is maladaptive checking, whereby human patients excessively perform checking rituals despite these serving no purpose. Dysfunctional and excessive checking is the most common symptom associated with OCD and can be readily operationalised in rodents. This review considers animal models of OCD, the neural circuitries associated with impairments in habit-based and goal-directed behaviour, and how these may link to the compulsions observed in OCD. We further review the Observing Response Task (ORT), an appetitive instrumental learning procedure that distinguishes between functional and dysfunctional checking, with translational application in humans and rodents. By shedding light on the psychological and neural bases of compulsive-like checking, the ORT has potential to offer translational insights into the underlying mechanisms of OCD, in addition to being a platform for testing psychological and neurochemical treatment approaches.

Decomposition of Reinforcement Learning Deficits in Disordered Gambling via Drift Diffusion Modeling and Functional Magnetic Resonance Imaging

Gambling disorder is associated with deficits in reward-based learning, but the underlying computational mechanisms are still poorly understood. Here, we examined this issue using a stationary reinforcement learning task in combination with computational modeling and functional resonance imaging (fMRI) in individuals that regular participate in gambling (n = 23, seven fulfilled one to three DSM 5 criteria for gambling disorder, sixteen fulfilled four or more) and matched controls (n = 23). As predicted, the gambling group exhibited substantially reduced accuracy, whereas overall response times (RTs) were not reliably different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance deficit. In both groups, an RLDDM in which both non-decision time and decision threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter and model recovery, and posterior predictive checks revealed that, in both groups, the model accurately reproduced the evolution of accuracies and RTs over time. Modeling revealed that, compared to controls, the learning impairment in the gambling group was linked to a more rapid reduction in decision thresholds over time, and a reduced impact of value-differences on the drift rate. The gambling group also showed shorter non-decision times. FMRI analyses replicated effects of prediction error coding in the ventral striatum and value coding in the ventro-medial prefrontal cortex, but there was no credible evidence for group differences in these effects. Taken together, our findings show that reinforcement learning impairments in disordered gambling are linked to both maladaptive decision threshold adjustments and a reduced consideration of option values in the choice process.

Sign-tracking and goal-tracking in humans: Utilising eye-tracking in clinical and non-clinical populations

BACKGROUND

In Pavlovian conditioning, learned behaviour varies according to the perceived value of environmental cues. For goal-trackers (GT), the cue merely predicts a reward, whilst for sign-trackers (ST), the cue holds incentive value. The sign-tracking/goal-tracking model is well-validated in animals, but translational work is lacking. Despite the model's relevance to several conditions, including attention deficit hyperactivity disorder (ADHD), we are unaware of any studies that have examined the model in clinical populations.

METHODS

The current study used an eye-tracking Pavlovian conditioning paradigm to identify ST and GT in non-clinical (N = 54) and ADHD (N = 57) participants. Eye movements were recorded whilst performing the task. Dwell time was measured for two areas of interest: sign (i.e., cue) and goal (i.e., reward), and an eye-gaze index (EGI) was computed based on the dwell time sign-to-goal ratio. Higher EGI values indicate sign-tracking behaviour. ST and GT were determined using median and tertiary split approaches in both samples.

RESULTS

Despite greater propensity for sign-tracking in those with ADHD, there was no significant difference between groups. The oculomotor conditioned response was reward-specific (CS+) and present, at least partly, from the start of the task indicating dispositional and learned components. There were no differences in externalising behaviours between ST and GT for either sample.

CONCLUSIONS

Sign-tracking is associated with CS+ trials only. There may be both dispositional and learned components to sign-tracking, potentially more common in those with ADHD. This holds translational potential for understanding individual differences in reward-learning.

Effects of predictive and incentive value manipulation on sign- and goal-tracking behavior

When a neutral stimulus is repeatedly paired with an appetitive reward, two different types of conditioned approach responses may develop: a sign-tracking response directed toward the neutral cue, or a goal-tracking response directed toward the location of impending reward delivery. Sign-tracking responses have been postulated to result from attribution of incentive value to conditioned cues, while goal-tracking reflects the assignment of only predictive value to the cue. We therefore hypothesized that sign-tracking rats would be more sensitive to manipulations of incentive value, while goal-tracking rats would be more responsive to changes in the predictive value of the cue. We tested sign- and goal-tracking before and after devaluation of a food reward using lithium chloride, and tested whether either response could be learned under negative contingency conditions that precluded any serendipitous reinforcement of the behavior that might support instrumental learning. We also tested the effects of blocking the predictive value of a cue using simultaneous presentation of a pre-conditioned cue. We found that sign-tracking was sensitive to outcome devaluation, while goal-tracking was not. We also confirmed that both responses are Pavlovian because they can be learned under negative contingency conditions. Goal-tracking was almost completely blocked by a pre-conditioned cue, while sign-tracking was much less sensitive to such interference. These results indicate that sign- and goal-tracking may follow different rules of reinforcement learning and suggest a need to revise current models of associative learning to account for these differences.

Reduction in sugar drink valuation and consumption with gamified executive control training

The overvaluation of high-energy, palatable food cues contributes to unhealthy eating and being overweight. Reducing the valuation of unhealthy food may thus constitute a powerful lever to improve eating habits and conditions characterized by unhealthy eating. We conducted a double-blind, placebo-controlled, randomized intervention trial assessing the efficacy of a five to twenty days online cognitive training intervention to reduce sugary drink perceived palatability and consumption. Our intervention involved a recently identified action-to-valuation mechanism of action, in which the repeated inhibition of prepotent motor responses to hedonic food cues in a Go/NoGo (GNG) and an attentional bias modification (ABM) task eventually reduces their valuation and intake. Confirming our hypotheses, the experimental intervention with consistent (100%) mapping between motor inhibition and the targeted unhealthy sugary drinks cues induced a larger decrease in their valuation than the control intervention with inconsistent (50%) mapping (- 27.6% vs. - 19%), and a larger increase of the (water) items associated with response execution (+ 11% vs + 4.2%). Exploratory analyses suggest that the effect of training on unhealthy items valuation may persist for at least one month. Against our hypothesis, we observed equivalent reductions in self-reported consumption of sugary drinks following the two interventions (exp: - 27% vs. ctrl: - 19%, BF01 = 4.7), suggesting a dose-independent effect of motor inhibition on self-reported consumption. Our collective results corroborate the robustness and large size of the devaluation effects induced by response inhibition on palatable items, but challenge the assumption of a linear relationship between such effects and the actual consumption of the target items. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on 30/03/2021. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/5ESMP .

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

Reactivity to pseudorandom environmental patterns in woodlice: Effects of hydric deprivation and light exposure

Exploratory activity is an essential component of animal behavior, including among invertebrate species. This study examined the effects of hydric deprivation and their possible modulation by light exposure on locomotion and rearing-up behavior in two woodlice species, Porcellio scaber (Latreille 1804) and Armadillidium vulgare (Latreille 1804). It was also an attempt to replicate previous findings on the stimulation of these behaviors in P. scaber, exposed to (pseudo)random vs. regular visual and tactile patterns in a small enclosure. In Experiment 1, two groups of P. scaber and two groups of A. vulgare were exposed to randomly vs. regularly distributed visual (black and white) and tactile (grained and smooth) patterns for approximately 20min. No rewards were present in the environment and the woodlice were tested without preliminary hydric deprivation. In Experiment 2, the same procedure was used but the woodlice were tested following a 20-min hydric deprivation under a bottle cap (darkness). Experiment 3 replicated this procedure with the 20-min hydric deprivation spent in a plastic cup (light exposure). The results of Experiments 1 and 3 provide partial replication with A. vulgare, but not P. scaber, of the previous findings: Random patterns stimulate rearing-up behavior on the apparatus' vertical walls more than regular patterns. Also, a more aversive stimulation in Experiment 3 compared to Experiment 2, increased locomotion, especially in the random environment. The role of hydric deprivation and light exposure in the process of escaping from a hostile environment is discussed, as well as the effects of the treatments used in these experiments.

Flexible control of Pavlovian-instrumental transfer based on expected reward value

The Pavlovian-instrumental transfer (PIT) paradigm is widely used to assay the motivational influence of reward-predictive cues, reflected by their ability to invigorate instrumental behavior. Leading theories assume that a cue's motivational properties are tied to predicted reward value. We outline an alternative view that recognizes that reward-predictive cues may suppress rather than motivate instrumental behavior under certain conditions, an effect termed positive conditioned suppression. We posit that cues signaling imminent reward delivery tend to inhibit instrumental behavior, which is exploratory by nature, in order to facilitate efficient retrieval of the expected reward. According to this view, the motivation to engage in instrumental behavior during a cue should be inversely related to the value of the predicted reward, since there is more to lose by failing to secure a high-value reward than a low-value reward. We tested this hypothesis in rats using a PIT protocol known to induce positive conditioned suppression. In Experiment 1, cues signaling different reward magnitudes elicited distinct response patterns. Whereas the one-pellet cue increased instrumental behavior, cues signaling three or nine pellets suppressed instrumental behavior and elicited high levels of food-port activity. Experiment 2 found that reward-predictive cues suppressed instrumental behavior and increased food-port activity in a flexible manner that was disrupted by post-training reward devaluation. Further analyses suggest that these findings were not driven by overt competition between the instrumental and food-port responses. We discuss how the PIT task may provide a useful tool for studying cognitive control over cue-motivated behavior in rodents. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

No significant effect of frequent online sexual behaviour on Pavlovian-to-instrumental transfer (PIT): Implications for compulsive sexual behaviour disorder

Reward based learning is broadly acknowledged to underpin the development and maintenance of addictive behaviour although the mechanism in sexual compulsivity is less understood. Using a Pavlovian-to-Instrumental Transfer (PIT) task we tested whether the motivational aspect of conditioned Pavlovian conditioned stimulus invigorated instrumental responding in relation to specific compatible monetary rewards. Performance on the task was analysed between two groups of males based on Low (N = 38) and High (N = 41) self-report online sexual behaviour (OSB). Psychometric tests including sexual compulsivity scale and behavioural activation/behavioural inhibition (BIS/BAS) were also administered to determine the relationship between OSB and general reward sensitivity. We show clear evidence of acquisition in the Pavlovian and instrumental conditioning phases. Specific transfer effect was greater in the High-OSB group although the difference compared to the Low-OSB group was non-significant. OSB negatively correlated with both BIS and BAS indicative of introversion and low reward sensitivity. OSB positively correlated with sexual compulsivity although it is unclear whether individuals in the High-OSB group considered their behaviour either excessive or problematic. These findings contribute to the ongoing debate regarding the nature of problematic OSB. Fundamental differences in motivational characteristics and mechanism contributing to compulsive behaviour in relation to high-OSB might indicate incompatibility with behavioural addiction models. PIT was not enhanced in high-OSB by appetitive conditioning, although problematic OSB could stem from failure to inhibit actions. Further research should investigate whether aversive conditioning differentially affects responding in high-OSB individuals, potentially explaining perseverant behaviour despite negative consequences.

More opportunities to peck for identical food availability increases foraging efficiency in pigeons

The opportunity and the information available to secure food resources drives foraging behaviour. We tested how inconsistent hole-food pairings and coverings could alter foraging performance, even when food availability is held constant. In our first experiment, we exposed pigeons (Columba livia) to a board in which each of the 60 covered holes contained one food item and to another board in which only one third of the 180 covered holes randomly contained one food item. In a second experiment, only the 60-hole board was used and the holes were not covered. The pigeons increased their body weight, gave fewer pecks per hole, revisited holes less often, and inspected fewer adjacent holes with 180 rather than 60 covered holes while eating similar amounts. However, their pecks were disproportionately higher near the edges of the board with 60 covered holes. This behaviour was not evident in the second experiment, when the food items were visible and individuals could know where food was available. Thus, the information about food location may drive foraging behaviour more directly than the information about food availability.

Chronic experience with unpredictable food availability promotes food reward, overeating, and weight gain in a novel animal model of food insecurity

Ubiquitous, easy access to food is thought to promote obesity in the modern environment. However, people coping with food insecurity have limited, unpredictable food access and are also prone to obesity. Causal factors linking food insecurity and obesity are not understood. In this study we describe an animal model to investigate biopsychological impacts of the chronic unpredictability inherent in food insecurity. Female rats were maintained on a 'secure' schedule of highly predictable 4x/day feedings of uniform size, or an 'insecure' schedule delivering the same total food over time but frequently unpredictable regarding how much, if any, food would arrive at each scheduled feeding. Subgroups of secure and insecure rats were fed ordinary chow or high-fat/high-sugar (HFHS) chow to identify separate and combined effects of insecurity and diet quality. Insecure chow-fed rats, relative to secure chow-fed rats, were hyperactive and consumed more when provided a palatable liquid diet. Insecure HFHS-fed rats additionally had higher progressive ratio breakpoints for sucrose, increased meal size, and subsequently gained more weight during 8 days of ad libitum HFHS access. Insecurity appeared to maintain a heightened attraction to palatable food that habituated in rats with secure HFHS access. In a second experiment, rats fed ordinary chow on the insecure schedule subsequently gained more weight when provided ad libitum chow, showing that prior insecurity per se promoted short-term weight gain in the absence of HFHS food. We propose this to be a potentially useful animal model for mechanistic research on biopsychological impacts of insecurity, demonstrating that chronic food uncertainty is a factor promoting obesity.

Seeking motivation and reward: roles of dopamine, hippocampus and supramammillo-septal pathway

Reinforcement learning and goal-seeking behavior are thought to be mediated by midbrain dopamine neurons. However, little is known about neural substrates of curiosity and exploratory behavior, which occur in the absence of clear goal or reward. This is despite behavioral scientists having long suggested that curiosity and exploratory behaviors are regulated by an innate drive. We refer to such behavior as information-seeking behavior and propose 1) key neural substrates and 2) the concept of environment prediction error as a framework to understand information-seeking processes. The cognitive aspect of information-seeking behavior, including the perception of salience and uncertainty, involves, in part, the pathways from the posterior hypothalamic supramammillary region to the hippocampal formation. The vigor of such behavior is modulated by the following: supramammillary glutamatergic neurons; their projections to medial septal glutamatergic neurons; and the projections of medial septal glutamatergic neurons to ventral tegmental dopaminergic neurons. Phasic responses of dopaminergic neurons are characterized as signaling potentially important stimuli rather than rewards. This paper describes how novel stimuli and uncertainty trigger seeking motivation and how these neural substrates modulate information-seeking behavior.

Effort-motivated behavior resolves paradoxes in appetitive conditioning

Motivated behavior has long been studied by psychologists, ethologists, and neuroscientists. To date, many scientists agree with the view that cue and reward attraction is the product of a dopamine-dependent unconscious process called incentive salience or "wanting". This process allows the influence of multiple factors such as hunger and odors on motivational attraction. In some cases, however, the resulting motivated behavior differs from what the incentive salience hypothesis would predict. I argue that seeking behavior under reward uncertainty illustrates this situation: Organisms do not just "want" (appetite-based attraction) cues that are inconsistent or associated with reward occasionally, they "hope" that those cues will consistently predict reward procurement in the ongoing trial. Said otherwise, they become motivated to invest time and energy to find consistent cue-reward associations despite no guarantee of success (effort-based attraction). A multi-test comparison of performance between individuals trained under uncertainty and certainty reveals behavioral paradoxes suggesting that the concept of incentive salience cannot fully account for responding to inconsistent cues. A mathematical model explains how appetite-based and effort-based attractions might combine their effects.

Smoke signals: A study of the neurophysiological reaction of smokers and non-smokers to smoking cues inserted into antismoking public service announcements

Tobacco addiction is one of the biggest health emergencies in the world, Antismoking Public Service Announcements (PSAs) represent the main public tool against smoking; however, smoking-related cues (SCs) often included in PSAs can trigger ambiguous cerebral reactions that could impact the persuasiveness and efficacy of the antismoking message. This study aimed to investigate the electroencephalographic (EEG) response in adult smokers and non-smokers during the exposure to SCs presented in antismoking PSAs video, in order to identify eventual neurophysiological features of SCs' 'boomerang effect' elicited in smokers. EEG frontal Alpha asymmetry and frontal Theta were analyzed in 92 adults (30 no smokers, 31 low smokers, 31 high smokers) from EEG recorded during the vision of 3 antismoking PSAs, statistical analysis was conducted using ANOVA. Main results showed a significant interaction between smoking cue condition (Pre and Post) and smoking habit (in particular for female heavy smokers) for the frontal Alpha asymmetry. Since the relative higher right frontal Alpha activity is associated with approach towards a stimulus, it is suggested that the relative left frontal Alpha increase in response to SCs might reflect an appetitive approach in response to it. In the light of the Incentive Sensitization Theory, this pattern can be interpreted as a neurophysiological signal in response to SCs that could undermine the message's effectiveness contributing to the maintenance of the addiction.

Aberrant orbitofrontal cortex reactivity to erotic cues in Compulsive Sexual Behavior Disorder

BACKGROUND AND AIMS

Compulsive Sexual Behavior Disorder (CSBD) is characterized by increased reactivity to erotic reward cues. Cue-encoded reward parameters, such as type (e.g. erotic or monetary) or probability of anticipated reward, shape reward-related motivational processes, increase the attractiveness of cues and therefore might enhance maladaptive behavioral patterns in CSBD. Studies on the neural patterns of cue processing in individuals with CSBD have been limited mainly to ventral striatal responses. Therefore, here we aimed to examine the cue reactivity of multiple key structures in the brain's reward system, taking into account not only the type of predicted reward but also its probability.

METHODS

Twenty Nine men seeking professional help due to CSBD and 24 healthy volunteers took part in an fMRI study with a modified Incentive Delay Task with erotic and monetary rewards preceded by cues indicating a 25%, 50%, or 75% chance of reward. Analyses of functional patterns of activity related to cue type and probability were conducted on the whole-brain and ROI levels.

RESULTS

Increased anticipatory response to cues predictive of erotic rewards was observed among CSBD participants when compared to controls, in the ventral striatum and anterior orbitofrontal cortex (aOFC). The activity in aOFC was modulated by reward probability.

DISCUSSION AND CONCLUSIONS

Type of anticipated reward (erotic vs monetary) affects reward-related behavioral motivation in CSBD more strongly than reward probability. We present evidence of abnormal aOFC function in CSBD by demonstrating the recruitment of additional subsections of this region by erotic reward cues.

Modifying food items valuation and weight with gamified executive control training

Recent lines of research suggest that repeated executive control of motor responses to food items modifies their perceived value and in turn their consumption. Cognitive interventions involving the practice of motor control and attentional tasks have thus been advanced as potential approach to improve eating habits. Yet, their efficacy remains debated, notably due to a lack of proper control for the effects of expectations. We examined whether a one-month intervention combining the practice of Go/NoGo and Cue approach training modified the perceived palatability of food items (i.e. decrease in unhealthy and increase in healthy food items' palatability ratings), and in turn participants' weights. We assessed our hypotheses with a parallel, double-blind, randomized controlled trial. Motivation and adherence to the intervention were maximized by a professional-level gamification of the training tasks. The control intervention differed from the experimental intervention only in the biasing of the stimulus-response mapping rules, enabling to balance expectations between the two groups and thus to conclude on the causal influence of motoric control on items valuation. We found a larger decrease of the unhealthy items' palatability ratings in the experimental (20.6%) than control group (13.1%). However, we did not find any increase of the healthy items' ratings or weight loss. Overall, the present registered report confirms that the repeated inhibition of motor responses to food cues, together with the development of attentional biases away from these cues, reduces their perceived value.

Uncertainty modulates value-driven attentional capture

The majority of previous studies on the value modulation of attention have shown that the magnitude of value-driven attentional bias correlates with the strength of reward association. However, relatively little is known about how uncertainty affects value-based attentional bias. We investigated whether attentional capture by previously rewarded stimuli is modulated by the uncertainty of the learned value without the influence of the strength of reward association. Participants were instructed to identify the line orientation in the target color circle. Importantly, each target color was associated with a different level of uncertainty by tuning the variation in reward delivery (Experiment 1) or reward magnitude (Experiment 2). Attentional interference for uncertainty-related distractors was greater than that for certainty distractors in Experiments 1 and 2. In addition, uncertainty-induced attentional bias disappeared earlier than attentional bias for certainty. The study demonstrated that uncertainty modulates value-based attentional capture in terms of strength and persistence, even when the effect of expected value remains constant.

Individual variation in the vigor and form of Pavlovian conditioned responses: Analysis of a model system

Pavlovian conditioning results in individual variation in the vigor and form of acquired behaviors. Here, we describe a general-process model of associative learning (HeiDI; How excitation and inhibition determine ideo-motion) that provides an analysis for such variation together with a range of other important group-level phenomena. The model takes as its starting point the idea that pairings of a conditioned stimulus (CS) and an unconditioned stimulus (US) result in the formation of reciprocal associations between their central representations. The asymptotic values of these associations and the rate at which these are reached are held to be influenced by the perceived salience of the CS (αCS) and US (βUS). Importantly, whether this associative knowledge is exhibited in behavior that reflects the properties of the CS (e.g., sign-tracking) or US (e.g., goal-tracking) is also influenced by the relative values of αCS and βUS. In this way, HeiDI provides an analysis for both quantitative and qualitative individual differences generated by Pavlovian conditioning procedures.

Effects of partial reinforcement on autoshaping in inbred Roman high- and low-avoidance rats

Individuals trained under partial reinforcement (PR) typically show a greater resistance to extinction than individuals exposed to continuous reinforcement (CR). This phenomenon is referred to as the PR extinction effect (PREE) and is interpreted as a consequence of uncertainty-induced frustration counterconditioning. In this study, we assessed the effects of PR and CR in acquisition and extinction in two strains of rats, the inbred Roman high- and low-avoidance (RHA and RLA, respectively) rats. These two strains mainly differ in the expression of anxiety, the RLA rats showing more anxiety-related behaviors (hence, more sensitive to frustration) than the RHA rats. At a neurobiological level, mild stress is known to elevate corticosterone in RLA rats and dopamine in RHA rats. We tested four groups of rats (RHA/CR, RHA/PR, RLA/CR, and RLA/PR) in two successive acquisition-extinction phases to try to consolidate the behavioral effects. Animals received training in a Pavlovian autoshaping procedure with retractable levers as the conditioned stimulus, food pellets as the unconditioned stimulus, and lever presses as the conditioned response. In Phase 1, we observed a PREE in lever pressing in both strains, but this effect was larger and longer lasting in RHA/PR than in RLA/PR rats. In Phase 2, reacquisition was fast and the PREE persisted in both strains, although the two PR groups no longer differed in lever pressing. The results are discussed in terms of frustration theory and of uncertainty-induced sensitization of dopaminergic neurons.

Directing Technology Addiction Research in Information Systems

Technology-related addictions have become common in many societies. Consequently, IS research has started examining such issues. In these embryonic stages of research, this line of work has already shown some promise in terms of understanding and tackling technology addiction problems. Nevertheless, there is a need to step back and understand the roots of technology-related addictions and how their foundations evolved in reference disciplines in order to be able to conduct more scientifically informed research on such issues. This study, therefore, explains the concept of behavioral addictions (the family of addictions to which technology-related addictions presumably belong), reviews the field's history and evolution, explains the relevant brain circuitry, and discusses similarities and differences between behavioral and substance addictions. A synthesis of this information provides eight key observations and recommendations that should help the field move forward.

Unpredictability as a modulator of drug self-administration: Relevance for substance-use disorders

Drug self-administration has been regarded as a gold-standard preclinical model of addiction and substance-use disorder (SUD). However, investigators are becoming increasingly aware, that certain aspects of addiction or SUDs experienced by humans are not accurately captured in our preclinical self-administration models. The current review will focus on two such aspects of current preclinical drug self-administration models: 1) Predictable vs. unpredictable drug access in terms of the time and effort put into obtaining drugs (i.e., response requirement) and drug quality (i.e., amount) and 2) rich vs. lean access to drugs. Some behavioral and neurobiological mechanisms that could contribute to excessive allocation of behavior toward drug-seeking and drug-taking at the expense of engaging in nondrug-related activities are discussed, and some directions for future research are identified. Based on the experiments reviewed, lean and unpredictable drug access could worsen drug-seeking and drug-taking behavior in individuals with SUDs. Once more fully explored, this area of research will help determine whether and how unpredictable and lean cost requirements affect drug self-administration in preclinical laboratory studies with nonhuman subjects and will help determine whether incorporating these conditions in current self-administration models will increase their predictive validity.

Sex Differences in Demand for Highly Palatable Foods: Role of the Orexin System

BACKGROUND

The prevalence of eating disorders, including binge eating disorder, is significantly higher in women. These findings are mirrored by preclinical studies, which indicate that female rats have a higher preference for palatable food and show greater binge-like eating compared with male rats.

METHODS

Here, we describe a novel within-session behavioral-economic paradigm that allows for the simultaneous measurement of the intake at null cost (Q0) and normalized demand elasticity (α) of 3 types of palatable food (low fat, high fat, and chocolate sucrose pellets) via demand curve analysis. In light of evidence that the orexin (hypocretin) system is critically involved in reward and feeding behaviors, we also examined the role of orexin function in sex differences of economic demand for palatable foods.

RESULTS

The novel within-session behavioral-economic approach revealed that female rats have higher intake (demand) than males for all palatable foods at low cost (normalized to body weight) but no difference in intake at higher prices, indicating sex-dependent differences in the hedonic, but not motivational, aspects of palatable food. Immediately following behavioral-economic testing, we observed more orexin-expressing neurons and Fos expression (measure of recent neural activation) in these neurons in female rats compared with male rats. Moreover, the orexin-1 receptor antagonist SB334867 reduced both low- and high-cost intake for palatable food in both male and female rats.

CONCLUSIONS

These findings provide evidence of higher demand at low prices for palatable food in females and indicate that these behavioral differences may be associated with sexual dimorphism in orexin system function.

Neurobehavioral Effects of Restricted and Unpredictable Environmental Enrichment in Rats

To study how motivational factors modulate experience-dependent neurobehavioral plasticity, we modify a protocol of environmental enrichment (EE) in rats. We assumed that the benefits derived from EE might vary according to the level of incentive salience attributed to it. To enhance the rewarding properties of EE, access to the EE cage varied randomly from 2 to 48 h for 30 days (REE). The REE group was enriched only 50% of the time and was compared to standard housing and continuous EE (CEE) groups. As behavioral readout, we analyzed the spontaneous activity and the ultrasonic vocalizations (USVs) within the EE cage weekly, and in the open field test at the end of the experiment. In the cage, REE increased the utilization of materials, physical activity, and the rate of appetitive USVs. In the OF, the CEE-induced enhancements in novelty habituation and social signaling were equaled by the REE. At the neural level, we measured the expression of genes related to neural plasticity and epigenetic regulations in different brain regions. In the dorsal striatum and hippocampus, REE upregulated the expression of the brain-derived neurotrophic factor, its tropomyosin kinase B receptor, and the DNA methyltransferase 3A. Altogether, our results suggest that the higher activity within the cage and the augmented incentive motivation provoked by the REE boosted its neurobehavioral effects equaling or surpassing those observed in the CEE condition. As constant exposures to treatments or stimulating environments are virtually impossible for humans, restricted EE protocols would have greater translational value than traditional ones.

From sign-tracking to attentional bias: Implications for gambling and substance use disorders

Sign-tracking behavior in Pavlovian autoshaping is known to be a relevant index of the incentive salience attributed to reward-related cues. Evidence has accumulated to suggest that animals that exhibit a sign-tracker phenotype are especially vulnerable to addiction and relapse due to their proneness to attribute incentive salience to drug cues, and their relatively weak cognitive and attentional control over their behavior. Interestingly, sign-tracking is also influenced by reward uncertainty in a way that may promote gambling disorder. Research indicates that reward uncertainty sensitizes sign-tracking responses and favors the development of a sign-tracker phenotype, compatible with the conditioned attractiveness of lights and sounds in casinos for problem gamblers. The study of attentional biases in humans (an effect akin to sign-tracking in animals) leads to similar observations, notably that the propensity to develop attraction for conditioned stimuli (CSs) is predictive of addictive behavior. Here we review the literature on drug addiction and gambling disorder, highlighting the similarities between studies of sign-tracking and attentional biases.

Maladaptive consequences of repeated intermittent exposure to uncertainty

Recently we reported that nucleus accumbens (NAcc) dopamine (DA) tracks uncertainty during operant responding for non-caloric saccharin. We also showed that repeated intermittent exposure to this uncertainty, like exposure to drugs of abuse, leads to sensitization of the locomotor and NAcc DA effects of amphetamine and promotes the subsequent self-administration of the drug. Here we review these findings together with others showing that NAcc glutamate signaling is similarly affected by uncertainty. Extracellular levels of glutamate in this site also track uncertainty in a task in which nose poking for saccharin on an escalating variable ratio schedule of reinforcement is associated with progressively increasing variance between performance of the operant and payout. Furthermore, sensitized behavioral responding to and for amphetamine following exposure to uncertainty is accompanied by increased levels of Ca2+/calmodulin-dependent protein kinase II (CaMKII) and protein kinase C (PKC) phosphorylation as well as altered protein levels of the transcription factor ∆FosB (increased) and glutamate transporter 1 (GLT1; decreased) in NAcc tissues. Notably, phosphorylation by CaMKII and PKC regulates AMPA receptor trafficking and function in this site, is elevated following psychostimulant exposure, and is necessary for the expression of enhanced drug taking. Increased ∆FosB and decreased GLT1 levels are observed following psychostimulant exposure, are associated with increased drug taking and seeking, and are known to modulate AMPA receptors and extracellular glutamate levels respectively. These adaptations in glutamate transmission as well as those observed with DA following repeated intermittent exposure to uncertainty are similar to those produced by exposure to abused drugs. Together, they point to the recruitment of both DA and glutamate signaling pathways in the NAcc in both drug and behavioral addictions. As uncertainty is central to games of chance, these findings have particular relevance for gambling disorders known to exhibit comorbidity with drug abuse.

Reward uncertainty attributes incentive value to reward proximal cues, while amphetamine sensitization reverts attention to more predictive reward distal cues

Slot-machine gambling incorporates numerous audiovisual cues prior to and during reward delivery (e.g. spinning wheels, flashing lights, celebratory sounds). Over time, these cues may motivate playing and even elicit cravings and relapse in those affected by gambling disorder. Animal studies suggest a heightened attraction to these cues despite diminished predictive ability under reward uncertainty, as evidenced by sign-tracking behavior in rats. Repeated amphetamine administration may also enhance the incentive value attributed to cues. Here, we explored the impact of reward uncertainty and prior amphetamine sensitization on the relative attractiveness and conditioned reinforcing properties of serial Pavlovian cues with different degrees of predictive and incentive value in rats. Animals were sensitized through repeated injections of amphetamine (1-4 mg/kg) or saline and then trained in a Pavlovian autoshaping task involving two sequential lever-auditory cue combinations (CS1, CS2) under Certain (100%-1) or Uncertain (50%-1-2-3) reward conditions. Subsequently, we evaluated the impact of acute amphetamine exposure on cue attraction. Our results suggest that Uncertainty alone enhanced attraction towards the reward-proximal CS2. However, combined sensitization and Uncertainty reversed cue preference relative to Uncertainty alone, enhancing attraction towards the more predictive reward-distal CS1. Both cues acquired conditioned reinforcing properties, despite the CS2 being otherwise ignored in all groups besides Uncertainty. However, combined sensitization and Uncertainty increased the reinforcing value of both cues and doubled the amount of interaction with the CS1 lever per presentation. Our results imply competitive mechanisms for attributing incentive value to gambling-related cues between reward uncertainty, prior amphetamine sensitization, and acute amphetamine administration.

Dissociable dopaminergic and pavlovian influences in goal-trackers and sign-trackers on a model of compulsive checking in OCD

RATIONALE

Checking is a functional behaviour that provides information to guide behaviour. However, in obsessive-compulsive disorder (OCD), checking may escalate to dysfunctional levels. The processes underpinning the transition from functional to dysfunctional checking are unclear but may be associated with individual differences that support the development of maladaptive behaviour. We examined one such predisposition, sign-tracking to a pavlovian conditioned stimulus, which we previously found associated with dysfunctional checking. How sign-tracking interacts with another treatment with emerging translational validity for OCD-like checking, chronic administration of the dopamine D₂ receptor agonist quinpirole, is unknown.

OBJECTIVES

We tested how functional and dysfunctional checking in the rat observing response task (ORT) was affected by chronic quinpirole administration in non-autoshaped controls and autoshaped animals classified as sign-trackers or goal-trackers.

METHODS

Sign-trackers or goal-trackers were trained on the ORT before the effects of chronic quinpirole administration on checking were assessed. Subsequently, the effects on checking of different behavioural challenges, including reward omission and the use of unpredictable reinforcement schedules, were tested.

RESULTS

Prior autoshaping increased checking. Sign-trackers and goal-trackers responded differently to quinpirole sensitization, reward omission and reinforcement uncertainty. Sign-trackers showed greater elevations in dysfunctional checking, particularly during uncertainty. By contrast, goal-trackers predominantly increased functional checking responses, possibly in response to reduced discrimination accuracy in the absence of cues signalling which lever was currently active.

CONCLUSIONS

The results are discussed in terms of how pavlovian associations influence behaviour that becomes compulsive in OCD and how this may be dependent on striatal dopamine D₂ receptors.

Learning to lose control: A process-based account of behavioral addiction

Learning psycho(bio)logy has developed a solid corpus of evidence and theory regarding behavior control modes. The present article briefly reviews that literature and its influence on recent models in which the transition from goal-directed to compulsive behavior is identified as the main process underlying substance use disorders. This literature is also relevant to non-substance addictive disorders, and serves as basis to propose a restricted definition of behavioral addiction relying on the presence of behavior-specific compulsivity. Complementarily, we consider whether some activities can become disordered while remaining mostly goal-driven. Based on reinforcement learning models, relative outcome utility computation is proposed as an alternative mechanism through which dysfunctional behaviors (even not qualifying as addictive) can override adaptive ones, causing functional impairment. Beyond issues of conceptual delimitation, recommendations are made regarding the importance of identifying individual etiological pathways to dysregulated behavior, the necessity of accurately profiling at-risk individuals, and the potential hazards of symptom-based diagnosis. In our view, the validity of these recommendations does not depend on the position one takes in the nosological debate.

The gambling craving experience questionnaire: Psychometric properties of a new scale based on the elaborated intrusion theory of desire

Both research and clinical practice acknowledge the importance of craving as a maintenance and relapse factor in gambling disorder. The elaborated intrusion theory (EIT; Kavanagh et al., 2005) of desire has been extensively investigated in relation to psychoactive substance or food cravings but, to date, has scarcely been studied in relation to gambling. In such a context, developing an assessment tool of gambling craving based on the EIT is warranted. To fill this gap in the literature, we aimed to develop and test the psychometric properties of a gambling-adapted version of the Craving Experience Questionnaire (CEQ; May et al., 2014), which is the best established measure of craving theoretically anchored in the EIT. An online survey that included the gambling CEQ (g-CEQ) and a craving induction procedure was administered to 274 community participants involved in gambling at least a few times a year. Concurrent and convergent validity were explored through correlations with a scale that measured gambling urge and with a series of questionnaires that measured disordered gambling symptoms, gambling cognitions, and gambling motives. The confirmatory factor analyses supported the validity of the expected three-factor model of the "strength" and "frequency" forms of the g-CEQ and showed better model fit than a one-factor solution, corroborating the initial structure of the CEQ. Furthermore, the scale has good internal consistency and its validity is supported by correlations with gambling-related constructs. The g-CEQ is thus a theoretically and psychometrically sound instrument to measure gambling craving based on the EIT.

Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event—an occasion setter (OS)—which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Rewards interact with repetition-dependent learning to enhance long-term retention of motor memories

The combination of behavioral experiences that enhance long-term retention remains largely unknown. Informed by neurophysiological lines of work, this study tested the hypothesis that performance-contingent monetary rewards potentiate repetition-dependent forms of learning, as induced by extensive practice at asymptote, to enhance long-term retention of motor memories. To this end, six groups of 14 participants (n = 84) acquired novel motor behaviors by adapting to a gradual visuomotor rotation while these factors were manipulated. Retention was assessed 24 h later. While all groups similarly acquired the novel motor behaviors, results from the retention session revealed an interaction indicating that rewards enhanced long-term retention, but only when practice was extended to asymptote. Specifically, the interaction indicated that this effect selectively occurred when rewards were intermittently available (i.e., 50%), but not when they were absent (i.e., 0%) or continuously available (i.e., 100%) during acquisition. This suggests that the influence of rewards on extensive practice and long-term retention is nonlinear, as continuous rewards did not further enhance retention as compared with intermittent rewards. One possibility is that rewards' intermittent availability allows to maintain their subjective value during acquisition, which may be key to potentiate long-term retention.

Cued for risk: Evidence for an incentive sensitization framework to explain the interplay between stress and anxiety, substance abuse, and reward uncertainty in disordered gambling behavior

Gambling disorder is an impairing condition confounded by psychiatric co-morbidity, particularly with substance use and anxiety disorders. Yet, our knowledge of the mechanisms that cause these disorders to coalesce remains limited. The Incentive Sensitization Theory suggests that sensitization of neural "wanting" pathways, which attribute incentive salience to rewards and their cues, is responsible for the excessive desire for drugs and cue-triggered craving. The resulting hyper-reactivity of the "wanting' system is believed to heavily influence compulsive drug use and relapse. Notably, evidence for sensitization of the mesolimbic dopamine pathway has been seen across gambling and substance use, as well as anxiety and stress-related pathology, with stress playing a major role in relapse. Together, this evidence highlights a phenomenon known as cross-sensitization, whereby sensitization to stress, drugs, or gambling behaviors enhance the sensitivity and dopaminergic response to any of those stimuli. Here, we review the literature on how cue attraction and reward uncertainty may underlie gambling pathology, and examine how this framework may advance our understanding of co-mordidity with substance-use disorders (e.g., alcohol, nicotine) and anxiety disorders. We argue that reward uncertainty, as seen in slot machines and games of chance, increases dopaminergic activity in the mesolimbic pathway and enhances the incentive value of reward cues. We propose that incentive sensitization by reward uncertainty may interact with and predispose individuals to drug abuse and stress, creating a mechanism through which co-mordidity of these disorders may emerge.

Incentive hope: A default psychological response to multiple forms of uncertainty

Our target article proposes that a new concept - incentive hope - is necessary in the behavioral sciences to explain animal foraging under harsh environmental conditions. Incentive hope refers to a specific motivational mechanism in the brain - considered only in mammals and birds. But it can also be understood at a functional level, as an adaptive behavioral strategy that contributes to improve survival. Thus, this concept is an attempt to bridge across different research fields such as behavioral psychology, reward neuroscience, and behavioral ecology. Many commentaries suggest that incentive hope even could help understand phenomena beyond these research fields, including food wasting and food sharing, mental energy conservation, diverse psychopathologies, irrational decisions in invertebrates, and some aspects of evolution by means of sexual selection. We are favorable to such extensions because incentive hope denotes an unconscious process capable of working against many forms of adversity; organisms do not need to hope as a subjective feeling, but to behave as if they had this feeling. In our response, we carefully discuss each suggestion and criticism and reiterate the importance of having a theory accounting for motivation under reward uncertainty.

Encoding prediction signals during appetitive and aversive Pavlovian conditioning in the primate lateral hypothalamus

The lateral hypothalamus (LH), which plays a role in homeostatic functions such as appetite regulation, is also linked to arousal and motivational behavior. However, little is known about how these components are encoded in the LH. Thus cynomolgus monkeys were conditioned with two distinct contexts, i.e., an appetitive context with available rewards and an aversive context with predicted air puffs. Different LH neuron groups encoded different degrees of expectation, predictability, and risks of rewards in a specific manner. A nearly equal number of one-third of the recorded LH neurons showed a positive or negative correlation between their response to visual conditioned stimuli (CS) that predicted the probabilistic delivery of rewards (0%, 50%, and 100%) and the associative values. For another one-third of recorded neurons, a nearly equal number showed a positive or negative correlation between their responses to rewards [appetitive unconditioned stimulus (US)] and reward predictability. Some neurons exhibited their highest or lowest trace-period responses in the 50% reward trials. These response modulations were represented independently and overlaid on a consistent excitatory or inhibitory response across the conditioning events. LH neurons also showed consistent responses in the aversive context. However, the responses to aversive conditioning events depending on the air puff value and predictability were less common. The multifaceted modulation of consistent activity related to outcome predictions may reflect motivational and arousal signals. Furthermore, it may underlie the role the LH plays in the integration and relay of signals to cortices for adaptive and goal-directed physiological and behavioral responses to environmental changes.

NEW & NOTEWORTHY The lateral hypothalamus (LH) is implicated in motivational and arousal behavior; however, the detailed information carried by single LH neurons remains unclear. We demonstrate that primate LH neurons encode multiple combinations of signals concerning different degrees of expectation, appreciation, and uncertainty of rewards in consistent responses across conditioning events and between different contexts. This multifaceted modulation of activity may underlie the role of the LH as a critical node integrating motivational signals with arousal signals.

Exposure to conditions of uncertainty promotes the pursuit of amphetamine

Prior exposure to abused drugs leads to long-lasting neuroadaptations culminating in excessive drug intake. Given the comorbidity between substance use and gambling disorders, surprisingly little is known about the effects of exposure to reinforcement contingencies experienced during games of chance. As it is a central feature of these games, we characterized the effects of exposure to uncertainty on biochemical and behavioral effects normally observed in rats exposed to amphetamine. Rats in different groups were trained to nose-poke for saccharin under certain [fixed-ratio (FR)] or uncertain conditions [variable-ratio (VR)] for 55 1-h sessions. Ratios were escalated on successive sessions and rats maintained on the last ratio (FR/VR 20) for 20-25 days. Two to three weeks later, rats were tested for their locomotor or nucleus accumbens dopamine (NAcc DA) response to amphetamine or self-administration of the drug using a lever press operant. NAcc DA overflow was also assessed in additional rats during the saccharin sessions. Rats exposed to uncertainty subsequently showed a higher locomotor and NAcc DA response to amphetamine and self-administered more drug infusions relative to rats exposed to predictable reinforcement. NAcc DA levels during the saccharin sessions tracked the variance of the scheduled ratios (a measure of uncertainty). VR rats showed escalating DA overflow with increasing ratios. Exposure to uncertainty triggered neuroadaptations similar to those produced by exposure to abused drugs. As these were produced in drug naive rats both during and after exposure to uncertainty, they provide a novel common pathway to drug and behavioral addictions.

Hoarding all of the chips: Slot machine gambling and the foraging for coins

Predictions made by the “incentive hope” hypothesis account for overconsumption in unpredictable food environments. However, when applied to uncertain gambling situations, there are several areas where this theory falls short. Most notably, it has trouble explaining why, in slot machine gambling, players are motivated by extended play to spend time trying to resolve uncertainty, rather than hoarding monetary gains.

Cannabinoid receptor-1 signaling contributions to sign-tracking and conditioned reinforcement in rats

RATIONALE

Endocannabinoids (eCBs) are critical gatekeepers of dopaminergic signaling, and disrupting cannabinoid receptor-1 (CB1) signaling alters DA dynamics to attenuate cue-motivated behaviors. Prior studies suggest that dopamine (DA) release plays a critical role in driving sign-tracking.

OBJECTIVES

Here, we determine whether systemic injections of rimonabant, a CB1 receptor inverse agonist, during Pavlovian lever autoshaping impair the expression of sign-tracking. We next examine whether rimonabant blocks the reinforcing properties of the Pavlovian lever cue in a conditioned reinforcement test.

METHODS

In Exp. 1, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1, 3 mg/kg) during early and late Pavlovian lever autoshaping sessions. In Exp. 2, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1 mg/kg) during a conditioned reinforcement test.

RESULTS

Rimonabant dose-dependently decreased lever contact and probability, and increased sign-tracker's latency to approach the lever cue early in Pavlovian training. With extended training, many previously goal-tracking and intermediate rats shifted to lever approach, which remained dose-dependently sensitive to rimonabant. Rimonabant attenuated cue-evoked food cup approach early, but not late, in conditioning, and did not affect pellet retrieval or consumption. The inserted lever cue served as a robust conditioned reinforcer after Pavlovian lever autoshaping, and 1 mg/kg rimonabant blocked conditioned reinforcement.

CONCLUSIONS

Together, our results suggest that CB1 signaling mediates two critical properties of incentive stimuli; their ability to attract (Exp. 1) and their ability to reinforce (Exp. 2) behavior.

Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Sources of maladaptive behavior in 'normal' organisms

A basic assumption of most researchers is that behavior is generally functional, and indeed, in most instances the function is obvious. But in a number of cases, some behaviors of neurophysiologically 'normal' organisms appear to be maladaptive. Considerable research has been conducted to understand the basis of such behavior as well as how the frequency of such behavior can be reduced. Here we provide a brief panoramic review of the major sources of maladaptive behavior in neurophysiologically 'normal' organisms: a) altered environmental contingencies relative to those faced by ancestral generations in their environment of evolutionary adaptation, b) altered environmental contingencies within the lifespan of the animal, c) linked behaviors in which the dysfunctional behavior is a linked companion of a more valuable beneficial trait, and d) the labeling of some behaviors as 'maladaptive' when more careful examination finds that they provide net benefit. Most of our attention is on the consequences of altered contingencies across and within a generation, with altered contingencies within a generation constituting a form of associative interference. The central issue in these two cases can be framed in terms of insufficient or excessive transfer of training resulting in maladaptive behavior. We discuss the functional basis of successful and unsuccessful near transfer (i.e., stimulus and response generalization) and far transfer (including rule learning and abstraction).

Repeated cocaine exposure dysregulates cognitive control over cue-evoked reward-seeking behavior during Pavlovian-to-instrumental transfer

Drug-paired cues acquire powerful motivational properties, but only lead to active drug-seeking behavior if they are potent enough to overwhelm the cognitive control processes that serve to suppress such urges. Studies using the Pavlovian-to-instrumental transfer (PIT) task have shown that rats pretreated with cocaine or amphetamine exhibit heightened levels of cue-motivated food-seeking behavior, suggesting that exposure to these drugs sensitizes the incentive motivational system. However, the PIT testing protocol can also create conflict between two competing behavioral responses to the reward-paired cue: active reward seeking (e.g., lever pressing) and passive conditioned food-cup approach behavior. We therefore investigated whether repeated cocaine exposure alters the way in which rats use cue-based reward expectations to resolve such conflict. In-depth analysis of previously published and new research confirmed that when drug-naïve rats are given a cue that signals the timing of a delayed noncontingent reward, they adaptively transition from reward seeking to conditioned approach behavior, facilitating efficient collection of the predicted reward. In contrast, cocaine-exposed rats exhibit pronounced behavioral dysregulation, increasing, rather than suppressing, their reward-seeking behavior over time, disrupting their ability to passively collect reward. Such findings speak to the important and sometimes overlooked role that cognitive control plays in determining the motivational impact of cues associated with drug and nondrug rewards.