Appetitive vs. Aversive conditioning in humans

Appetitive conditioning with pornographic stimuli elicits stronger activation in reward regions than monetary and gaming-related stimuli

Appetitive conditioning plays an important role in the development and maintenance of pornography-use and gaming disorders. It is assumed that primary and secondary reinforcers are involved in these processes. Despite the common use of pornography and gaming in the general population appetitive conditioning processes in this context are still not well studied. This study aims to compare appetitive conditioning processes using primary (pornographic) and secondary (monetary and gaming-related) rewards as unconditioned stimuli (UCS) in the general population. Additionally, it investigates the conditioning processes with gaming-related stimuli as this type of UCS was not used in previous studies. Thirty-one subjects participated in a differential conditioning procedure in which four geometric symbols were paired with either pornographic, monetary, or gaming-related rewards or with nothing to become conditioned stimuli (CS + porn, CS + game, CS + money, and CS-) in an functional magnetic resonance imaging study. We observed elevated arousal and valence ratings as well as skin conductance responses for each CS+ condition compared to the CS-. On the neural level, we found activations during the presentation of the CS + porn in the bilateral nucleus accumbens, right medial orbitofrontal cortex, and the right ventral anterior cingulate cortex compared to the CS-, but no significant activations during CS + money and CS + game compared to the CS-. These results indicate that different processes emerge depending on whether primary and secondary rewards are presented separately or together in the same experimental paradigm. Additionally, monetary and gaming-related stimuli seem to have a lower appetitive value than pornographic rewards.

Aversive conditioning is impaired in impulsive individuals: A study on learning asymmetries

BACKGROUND AND OBJECTIVES

Appetitive and aversive conditioning are thought to be involved in the development and maintenance of mental disorders including anxiety, mood, eating, and substance use disorders. However, few studies measure the relative strength of appetitive and aversive associations, and their relevance to the risk of mental disorders. This study aims to address this gap.

METHODS

We tested how readily healthy volunteers acquire appetitive vs. aversive associations. 150 participants associated complex 3D objects with either gain or loss and made decisions to gain or avoid losing points. We investigated the relationship of a learning asymmetry with neuroticism, impulsivity, and anhedonia, to test the hypothesis that a stronger learning asymmetry corresponds to more extreme scores on these traits.

RESULTS

Impulsivity was positively associated with the learning asymmetry (R2 = .10). This resulted from an inverse relation with the strength of aversive associations, indicating that impulsive individuals are worse at aversive learning. However, appetitive associations did not differ significantly. No correlations with neuroticism or anhedonia were found.

LIMITATIONS

Conditioning studies typically use primary reinforcers and a CS-. Lacking these may make these results less comparable to other studies.

CONCLUSIONS

We demonstrate that the learning asymmetry can measure individual differences linked to personality traits, and that impulsivity, normally linked with appetitive learning, also influences aversive learning. These results enable additional studies of learning asymmetry in relation to mental disorders, which could include measurements of mental health symptoms to provide further insight into how appetitive and aversive learning interacts with mental disorders.

Reduced generalization of reward among individuals with subthreshold depression: Behavioral and EEG evidence

Altered stimulus generalization has been well-documented in anxiety disorders; however, there is a paucity of research investigating this phenomenon in the context of depression. Depression is characterized by impaired reward processing and heightened attention to negative stimuli. It is hypothesized that individuals with depression exhibit reduced generalization of reward stimuli and enhanced generalization of loss stimuli. Nevertheless, no study has examined this process and its underlying neural mechanisms. In the present study, we recruited 25 participants with subthreshold depression (SD group) and 24 age-matched healthy controls (HC group). Participants completed an acquisition task, in which they learned to associate three distinct pure tones (conditioned stimuli, CSs) with a reward, a loss, or no outcome. Subsequently, a generalization session was conducted, during which similar tones (generalization stimuli, GSs) were presented, and participants were required to classify them as a reward tone, a loss tone, or neither. The results revealed that the SD group exhibited reduced generalization errors in the early phase of generalization, suggesting a diminished ability to generalize reward-related stimuli. The event-related potential (ERP) results indicated that the SD group exhibited decreased generalization of positive valence to reward-related GSs and heightened generalization of negative valence to loss-related GSs, as reflected by the N1 and P2 components. However, the late positive potential (LPP) was not modulated by depression in reward generalization or loss generalization. These findings suggested that individuals with subthreshold depression may have a blunted or reduced ability to generalize reward stimuli, shedding light on potential treatment strategies targeting this particular process.

Conditioning to true content and artificial intelligence in psychophysiological intention recognition

OBJECTIVE

The objective is to introduce a novel method for classical conditioning to true content (CtTC), and for the first time, apply this approach in the concealed information test (CIT) to effectively discern intentions. During CtTC, participants are trained to exhibit electrodermal responses whenever they recognize true content on a screen. Additionally, the objective is to evaluate a novel CIT-dataset preprocessing algorithm, employed to enhance machine learning (ML) classification performance.

METHODS

A total of 84 participants were evenly divided into four groups. Two groups of participants devised plans for stealing money from a supermarket, while the other two groups did not engage in any planning. One planning group and one non-planning group underwent CIT examination, while the remaining groups were subjected to CtTC.

RESULTS

The CIT accuracy initially stood at 52 % and increased to 71 % after Z-score and ML classification (McNemar test, p < 0.05). Conversely, the CtTC accuracy was 76 % and significantly improved to 93 % following Z-score and 95 % following ML classification (McNemar test, p < 0.05). In the best-performing classifiers, CtTC exhibited significantly superior metrics for guilty/innocent classification compared to CIT (Fisher's exact test, p < 0.05, power 1 - β > 0.90). In the CtTC group, reactivity and sensitivity significantly increased, indicated by higher EDR amplitudes (p < 0.05, two-tailed t-test, power 1 - β = 0.89) and the number of EDRs (p < 0.05, Fisher's exact test, power 1 - β = 0.90). There was no statistically significant difference between the Z-score and ML classification.

CONCLUSIONS

In the assessment of intentions, CtTC enhances both the sensitivity and accuracy of the CIT.

Pupil dilation tracks divergent learning processes in aware versus unaware Pavlovian conditioning

Evidence regarding unaware differential fear conditioning in humans is mixed and even less is known about the effects of contingency awareness on appetitive conditioning. Phasic pupil dilation responses (PDR) might be more sensitive for capturing implicit learning than other measures, such as skin conductance responses (SCR). Here, we report data from two delay conditioning experiments utilizing PDR (alongside SCR and subjective assessments) to investigate the role of contingency awareness in aversive and appetitive conditioning. In both experiments, valence of unconditioned stimuli (UCS) was varied within participants by administering aversive (mild electric shocks) and appetitive UCSs (monetary rewards). Preceding visual stimuli (CSs) predicted either the reward, the shock (65% reinforcement), or neither UCS. In Exp. 1, participants were fully instructed about CS-UCS contingencies, whereas in Exp. 2, no such information was given. PDR and SCR demonstrated successful differential conditioning in Exp. 1 and in (learned) aware participants in Exp. 2. In non-instructed participants who remained fully unaware of contingencies (Exp. 2), differential modulation of early PDR (immediately after CS onset) by appetitive cues emerged. Associations with model-derived learning parameters further suggest that early PDR in unaware participants mainly reflect implicit learning of expected outcome value, whereas early PDR in aware (instructed/learned-aware) participants presumably index attentional processes (related to uncertainty/prediction error processing). Similar, but less clear results emerged for later PDR (preceding UCS onset). Our data argue in favor of a dual-process account of associative learning, suggesting that value-related processing can take place irrespective of mechanisms involved in conscious memory formation.

The relationship between sleep and appetitive conditioning: A systematic review and meta-analysis

This systematic review and meta-analysis (PROSPERO registration animal/human studies: CRD42021234793/CRD42021234790) examined the relationship between sleep and appetitive conditioning. Inclusion criteria included: a) appetitive conditioning paradigm; b) measure of conditioning; c) sleep measurement and/or sleep loss; d) human and/etor non-human animal samples; and e) written in English. Searches of seven databases returned 3777 publications. The final sample consisted of 42 studies using primarily animal samples and involving food- and drug-related conditioning tasks. We found sleep loss disrupted appetitive conditioning of food rewards (p < 0.001) but potentiated appetitive conditioning of drug rewards (p < 0.001). Furthermore, sleep loss negatively impacted extinction learning irrespective of the reward type. Post-learning sleep was associated with increases in REM sleep (p = 0.02). Findings suggest sleep loss potentiates the impact of psychoactive substances in a manner likely to produce an increased risk of problematic substance use. In obese/overweight populations, sleep loss may be associated with deficits in the conditioning and extinction of reward-related behaviours. Further research should assess the relationship between sleep and appetitive conditioning in humans.

Acquisition learning is stronger for aversive than appetitive events

Appetitive and aversive learning are both key building blocks of adaptive behavior, yet knowledge regarding their differences is sparse. Using a capsaicin heat pain model in 36 healthy participants, this study directly compared the acquisition and extinction of conditioned stimuli (CS) predicting pain exacerbation and relief. Valence ratings show stronger acquisition during aversive compared to appetitive learning, but no differences in extinction. Skin conductance responses and contingency ratings confirmed these results. Findings were unrelated to individual differences in pain sensitivity or psychological factors. Our results support the notion of an evolutionarily hardwired preponderance to acquire aversive rather than appetitive cues as is protective for acute aversive states such as pain but may contribute to the development and maintenance of clinical conditions such as chronic pain, depression or anxiety disorders.

In this study, acquisition and extinction of conditioned stimuli are compared in a capsaicin heat pain paradigm in healthy human participants. The results demonstrate a preponderance to acquire aversive rather than appetitive cues, which could initially have a protective function but eventually contribute to the maintenance of chronic pain.

High-throughput automated methods for classical and operant conditioning of Drosophila larvae

Learning which stimuli (classical conditioning) or which actions (operant conditioning) predict rewards or punishments can improve chances of survival. However, the circuit mechanisms that underlie distinct types of associative learning are still not fully understood. Automated, high-throughput paradigms for studying different types of associative learning, combined with manipulation of specific neurons in freely behaving animals, can help advance this field. The Drosophila melanogaster larva is a tractable model system for studying the circuit basis of behaviour, but many forms of associative learning have not yet been demonstrated in this animal. Here, we developed a high-throughput (i.e. multi-larva) training system that combines real-time behaviour detection of freely moving larvae with targeted opto- and thermogenetic stimulation of tracked animals. Both stimuli are controlled in either open- or closed-loop, and delivered with high temporal and spatial precision. Using this tracker, we show for the first time that Drosophila larvae can perform classical conditioning with no overlap between sensory stimuli (i.e. trace conditioning). We also demonstrate that larvae are capable of operant conditioning by inducing a bend direction preference through optogenetic activation of reward-encoding serotonergic neurons. Our results extend the known associative learning capacities of Drosophila larvae. Our automated training rig will facilitate the study of many different forms of associative learning and the identification of the neural circuits that underpin them.

From skinner box to daily life: Sign-tracker phenotype co-segregates with impulsivity, compulsivity, and addiction tendencies in humans

Pavlovian conditioning holds the potential to incentivize environmental cues, leading to approach behavior toward them, even outside our awareness. Animal models suggest that this is particularly true for the so-called sign-tracker (ST) phenotype, which is considered to reflect a predisposition toward developing addiction-related behaviours. Despite its potential clinical relevance, few studies have demonstrated the translational validity of this model, likely due to difficulties in studying Pavlovian processes in humans. To fill this gap, we combined an ecological momentary assessment with ambulatory peripheral autonomic monitoring to test the hypothesis that traits associated with ST in preclinical studies would be associated with attribution of high incentive salience to reward-related cues. Several times for 2 days, participants were asked to rate the attractiveness of several preselected ecological rewards (e.g., coffee) and the preceding cues (the smell of coffee) while their electrocardiogram was recorded. While no absolute difference in subjective and physiological measures of motivational approach to daily cues compared with rewards emerged, individuals with high levels of impulsivity, obsessive-compulsive, and addiction-prone behaviors rated as more attractive and showed a greater increase in sympathetic arousal to cues versus rewards. The opposite pattern emerged for those with low levels in those dispositional traits, who responded more (both subjectively and physiologically) to rewards compared with their preceding cues. This study represents an attempt to answer the call to parcel complex behaviors into smaller constructs, improving the early detection of those who are vulnerable to develop psychopathological disorders, particularly in the domain of impulse control such as addiction.

Reward prediction error in the ERP following unconditioned aversive stimuli

Reinforcement learning in humans and other animals is driven by reward prediction errors: deviations between the amount of reward or punishment initially expected and that which is obtained. Temporal difference methods of reinforcement learning generate this reward prediction error at the earliest time at which a revision in reward or punishment likelihood is signalled, for example by a conditioned stimulus. Midbrain dopamine neurons, believed to compute reward prediction errors, generate this signal in response to both conditioned and unconditioned stimuli, as predicted by temporal difference learning. Electroencephalographic recordings of human participants have suggested that a component named the feedback-related negativity (FRN) is generated when this signal is carried to the cortex. If this is so, the FRN should be expected to respond equivalently to conditioned and unconditioned stimuli. However, very few studies have attempted to measure the FRN's response to unconditioned stimuli. The present study attempted to elicit the FRN in response to a primary aversive stimulus (electric shock) using a design that varied reward prediction error while holding physical intensity constant. The FRN was strongly elicited, but earlier and more transiently than typically seen, suggesting that it may incorporate other processes than the midbrain dopamine system.

Carrots for the donkey: Influence of evaluative conditioning and training on self-paced exercise intensity and delay discounting of exercise in healthy adults

To choose exercise over alternative behaviours, subjective reward evaluation of the potential choices is a principal step in decision making. However, the selection of exercise intensity might integrate acute visceral responses (i.e. pleasant or unpleasant feelings) and motives related to goals (i.e. enjoyment, competition, health). To understand the factors determining the selection of exercise in its intensity and evaluation as a modality, we conducted a study combining exercise training and evaluative conditioning. Evaluative conditioning was performed by using a novel technique using a primary reinforcer (sweetness) as the unconditioned stimulus and physical strain i.e. heart rate elevation as the conditioned stimulus during interval training, using a randomized control design (N = 58). Pre, post-three weeks interval training w/o conditioning, and after 4 weeks follow-up, participants were tested on self-paced speed selection on treadmill measuring heart rate, subjective pleasantness, and effort levels, as well as delay-discounting of exercise and food rewards. Results revealed that the selection of exercise intensity was significantly increased by adaptation to training and evaluative conditioning, revealing the importance of visceral factors as well as learned expected rewards. Delay discounting rates of self-paced exercise were transiently reduced by training but not affected by evaluative conditioning. In conclusion, exercise decisions are suggested to separate the decision-making process into a modality-specific cognitive evaluation of exercise, and an exercise intensity selection based on acute visceral experience integrating effort, pleasantness, and learned rewards.

Pupil dilation as an index of Pavlovian conditioning. A systematic review and meta-analysis

The use of pupillometry to track emotional learning processes in humans is generating an increasing interest. Here, we provide a first systematic review and meta-analysis on the value of pupil dilation as a marker of Pavlovian conditioning, focusing on the roles of UCS valence (aversive vs. appetitive), the time course across trials and response intervals within trials. Based on data from 39 independent samples (total n = 1303), our results revealed strong evidence for the overall validity of conditioned pupil responses, with a trend for larger effects in aversive (average g = 0.73) vs. appetitive conditioning (g = 0.39). Response differentiation increased over the course of acquisition. Substantial differentiation effects were found in both early and late response windows. Moderator analyses revealed a consistent influence of UCS modality on differential conditioning, while evidence for moderation by contingency instructions and length of acquisition phase was mixed. The results highlight pupil dilation as a sensitive and reliable index of Pavlovian conditioning across valence categories and stimulus modalities. Important implications regarding methodological considerations and research goals are discussed.

Cardiac response in aversive and appetitive olfactory conditioning: Evidence for a valence-independent CS-elicited bradycardia

While the examination of conditioned cardiac responses is well established in human fear conditioning research, comparable studies using less-aversive or rather appetitive unconditioned stimuli (UCS) are sparse and results are mixed. Therefore, the aim of this study was a systematic analysis of cardiac reactions in aversive and appetitive conditioning. Olfactory stimuli were used as unconditioned stimuli as they are suitable reinforcers in both an aversive and an appetitive conditioning offering the opportunity for a comparison between conditioned responses. In total, n = 86 participants took part in both an aversive and an appetitive differential conditioning task with a counterbalanced order across participants. Aversive or appetitive odors, respectively, served as UCS and neutral geometrical figures as CS. Subjective ratings, skin conductance response (SCRs), and evoked cardiac reactions were analyzed and compared between tasks. Conditioned responses in subjective ratings could be observed in both aversive conditioning and appetitive conditioning, while SCRs discriminated between CS+ and CS- in aversive conditioning only. Regarding conditioned cardiac responses, the deceleration for the CS+ was longer than for the CS- in both tasks. In addition, a higher deceleration magnitude and a shorter acceleration for the CS+ as compared to the CS- were found in aversive but not in appetitive conditioning. There were medium-size correlations between aversive and appetitive CRs for subjective ratings and none for physiological responses. The results suggest similarities between cardiac response patterns in aversive and appetitive conditioning, which implies that bradycardia in conditioning might not be fear-specific but presents a valence-independent CS-elicited bradycardia.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Amygdala and nucleus accumbens involvement in appetitive extinction

Extinction of appetitive conditioning is regarded as an important model for the treatment of psychiatric disorders like addiction. However, very few studies have investigated its neural correlates. Therefore, we investigated neural correlates of appetitive extinction in a large human sample including all genders (N = 76, 40 females) to replicate and extend results from a previous study. During differential appetitive conditioning, one stimulus (CS+) was paired with the chance to win a monetary reward, whereas another stimulus (CS−) was not. During appetitive extinction on the next day, neither the CS+ nor the CS− were reinforced. After successful acquisition of appetitive conditioning, the extinction phase elicited significant reductions of valence and arousal ratings toward the CS+ and a significant reduction in skin conductance responses to the CS+ from early to late extinction. On a neural level, early extinction showed significant differential (CS+ − CS−) activation in dACC and hippocampus, whereas involvement of the vACC and caudate nucleus did not replicate. The differential activation of amygdala and nucleus accumbens during late extinction was replicated, with the amygdala displaying significantly higher differential activation during the late phase of extinction as compared to the early phase of extinction. We show discernible signals for reward learning and extinction in subregions of amygdala and nucleus accumbens after extinction learning. This successful replication underlines the role of nucleus accumbens and amygdala in neural models of appetitive extinction in humans that was previously only based on animal findings.

Generalization of appetitive conditioned responses

A stimulus (conditioned stimulus, CS) associated with an appetitive unconditioned stimulus (US) acquires positive properties and elicits appetitive conditioned responses (CR). Such associative learning has been examined extensively in animals with food as the US, and results are used to explain psychopathologies (e.g., substance-related disorders or obesity). Human studies on appetitive conditioning exist, too, but we still know little about generalization processes. Understanding these processes may explain why stimuli not associated with a drug, for instance, can elicit craving. Forty-seven hungry participants underwent an appetitive conditioning protocol during which one of two circles with different diameters (CS+) became associated with an appetitive US (chocolate or salty pretzel, according to participants' preference) but never the other circle (CS-). During generalization, US were delivered twice and the two CS were presented again plus four circles (generalization stimuli, GS) with gradually increasing diameters from CS- to CS+. We found successful appetitive conditioning as reflected in appetitive subjective ratings (positive valence, higher contingency) and physiological responses (startle attenuation and larger skin conductance responses) to CS+ versus CS-, and, importantly, both measures confirmed generalization as indicated by generalization gradients. Small changes in CS-US contingency during generalization may have weakened generalization processes on the physiological level. Considering that appetitive conditioned responses can be generalized to non-US-associated stimuli, a next important step would be to investigate risk factors that mediate overgeneralization.

Skin Conductance Responses Indicate Children are Physiologically Aroused by Their Favourite Branded Food and Drink Products

Children's favourite food and beverage brands use various tactics to foster positive associations and loyalty. This brand-consumer dynamic is frequently influenced by the use of implicit techniques and emotional appeals. Few studies have used physiological methods to examine the connections that brands build with children and the influence this has on their automatic responses. These techniques are potentially less prone to bias than behavioural or cognitive methods. This is the first study to explore the implicit response that children have to images of their favourite food and beverage brands using skin conductance responses as a marker of arousal. Australian children aged 8-11 years (n = 48) were recruited. Images of the participants' favourite branded food and beverage products, alongside images of the same products unpackaged, their family and friends, and neutral objects were presented in a randomised order with a standard timed interval between images. Children were significantly more aroused by branded images of their favourite food and beverage products than by their unpackaged counterparts (p < 0.042, d = 0.4). The physiological response to the branded products was similar to the response to the children's family and friends (p = 0.900, d = -0.02). These findings suggest that children may have an implicit connection to their favourite branded products.

Pupil dilation as an implicit measure of appetitive Pavlovian learning

Appetitive Pavlovian conditioning is a learning mechanism of fundamental biological and pathophysiological significance. Nonetheless, its exploration in humans remains sparse, which is partly attributed to the lack of an established psychophysiological parameter that aptly represents conditioned responding. This study evaluated pupil diameter and other ocular response measures (gaze dwelling time, blink duration and count) as indices of conditioning. Additionally, a learning model was used to infer participants' learning progress on the basis of their pupil dilation. Twenty-nine healthy volunteers completed an appetitive differential delay conditioning paradigm with a primary reward, while the ocular response measures along with other psychophysiological (heart rate, electrodermal activity, postauricular and eyeblink reflex) and behavioral (ratings, contingency awareness) parameters were obtained to examine the relation among different measures. A significantly stronger increase in pupil diameter, longer gaze duration and shorter eyeblink duration was observed in response to the reward-predicting cue compared to the control cue. The Pearce-Hall attention model best predicted the trial-by-trial pupil diameter. This conditioned response was corroborated by a pronounced heart rate deceleration to the reward-predicting cue, while no conditioning effect was observed in the electrodermal activity or startle responses. There was no discernible correlation between the psychophysiological response measures. These results highlight the potential value of ocular response measures as sensitive indices for representing appetitive conditioning.

The relationship between neuroticism and appetitive conditioning

Appetitive conditioning is considered a central mechanism for the vulnerability to psychiatric disorders. However, the investigation of individual differences that are related to altered appetitive learning has been almost neglected so far. The aim of this study was to investigate the link between neuroticism and appetitive conditioning processes. 79 subjects participated in a differential conditioning procedure in which a conditioned stimulus (CS+) was paired with a reward (money) after a fast behavioral response, while a second conditioned stimulus (CS-) was never followed by a reward, irrespective of the behavioral response. As a main result, neuroticism correlated negatively with the underlying neural processes of appetitive conditioning in females, but not in males. In detail, higher levels of neuroticism were associated with decreased neural responses in the left (p = .001) and right amygdala (p = .011), left (p = .063) and right (p = .019) nucleus accumbens, and left (p = .002) and right (p = .021) orbitofrontal cortex (all results are family-wise-error-corrected). The present results support previous findings, which also showed an inverse sex-specific effect in the context of neuroticism and emotional processing in females. In addition, the findings suggest that neuroticism is not solely linked to increased amygdala sensitivity during the processing of negative stimuli but also to decreased neural responses when processing rewarding stimuli. Possible explanations for the sex differences and implications are discussed.

Category-based generalization of placebo and nocebo effects

Human beings possess the adaptive ability to apply experiential knowledge to new situations. Although this generalization capability has been demonstrated in fear and reward learning, it remains unclear whether it extends to analgesic and hyperalgesic pain responses. Here, we conducted two experiments (total n = 104) to test the generalization effects of placebo analgesia and nocebo hyperalgesia. The first experiment, using a category-based conditioning paradigm in which two categories of images were used as acquisition stimuli, assessed whether pain perception can be generalized to never-seen pictures of the same category in the generalization phase. The second experiment adopted a single stimulus for each category as CS to further examine the generalization effects after learning a single exemplar. Pain ratings showed that participants reported higher pain or lower pain when the pain was preceded by novel stimuli that were conceptually similar to the previously conditioned stimuli, suggesting a generalization of analgesic and hyperalgesic pain modulation effects. These results provide novel evidence that analgesic and hyperalgesic effects on pain perception can be generalized to conceptually similar new items.

Opposing roles for amygdala and vmPFC in the return of appetitive conditioned responses in humans

Learning accounts of addiction and obesity emphasize the persistent power of Pavlovian reward cues to trigger craving and increase relapse risk. While extinction can reduce conditioned responding, Pavlovian relapse phenomena-the return of conditioned responding following successful extinction-challenge the long-term success of extinction-based treatments. Translational laboratory models of Pavlovian relapse could therefore represent a valuable tool to investigate the mechanisms mediating relapse, although so far human research has mostly focused on return of fear phenomena. To this end we developed an appetitive conditioning paradigm with liquid food rewards in combination with a 3-day design to investigate the return of appetitive Pavlovian responses and the involved neural structures in healthy subjects. Pavlovian conditioning (day 1) was assessed in 62 participants, and a subsample (n = 33) further completed extinction (day 2) and a reinstatement test (day 3). Conditioned responding was assessed on explicit (pleasantness ratings) and implicit measures (reaction time, skin conductance, heart rate, startle response) and reinstatement effects were further evaluated using fMRI. We observed a return of conditioned responding during the reinstatement test, evident by enhanced skin conductance responses, accompanied by enhanced BOLD responses in the amygdala. On an individual level, psychophysiological reinstatement intensity was significantly anticorrelated with ventromedial prefrontal cortex (vmPFC) activation, and marginally anticorrelated with enhanced amygdala-vmPFC connectivity during late reinstatement. Our results extend evidence from return of fear phenomena to the appetitive domain, and highlight the role of the vmPFC and its functional connection with the amygdala in regulating appetitive Pavlovian relapse.

Neural mechanisms of extinguishing drug and pleasant cue associations in human addiction: role of the VMPFC

The neurobiological mechanisms that underlie the resistance of drug cue associations to extinction in addiction remain unknown. Fear extinction critically depends on the ventromedial prefrontal cortex (VMPFC). Here, we tested if this same region plays a role in extinction of non-fear, drug and pleasant cue associations. Eighteen chronic cocaine users and 15 matched controls completed three functional MRI scans. Participants first learned to associate an abstract cue (the conditioned stimulus, CS) with a drug-related (CS_D+ ) or pleasant (CS_P+ ) image. Extinction immediately followed where each CS was repeatedly presented without the corresponding image. Participants underwent a second identical session 24 hours later to assess retention of extinction learning. Results showed that like fear extinction, non-fear-based extinction relies on the VMPFC. However, extinction-related changes in the VMPFC differed by cue valence and diagnosis. In controls, VMPFC activation to the CS_D+ (which was unpleasant for participants) gradually increased as in fear extinction, while it decreased to the CS_P+ , consistent with a more general role of the VMPFC in flexible value updating. Supporting a specific role in extinction retention, we further observed a cross-day association between VMPFC activation and skin conductance, a classic index of conditioned responses. Finally, cocaine users showed VMPFC abnormalities for both CSs, which, in the case of the CS_D+ , correlated with craving. These data suggest a global deficit in extinction learning in this group that may hinder extinction-based treatment efforts. More broadly, these data show that the VMPFC, when functionally intact, supports extinction learning in diverse contexts in humans.

Learned Overeating: Applying Principles of Pavlovian Conditioning to Explain and Treat Overeating

PURPOSE OF REVIEW

This review provides an overview of recent findings relating to the role of Pavlovian conditioning in food cue reactivity, including its application to overeating and weight loss interventions.

RECENT FINDINGS

Both in the laboratory and in real life, cue-elicited appetitive reactivity (e.g., eating desires) can be easily learned, but (long-term) extinction is more difficult. New findings suggest impaired appetitive learning in obesity, which might be causally related to overeating. The clinical analogue of extinction-cue exposure therapy-effectively reduces cue-elicited cravings and overeating. While its working mechanisms are still unclear, some studies suggest that reducing overeating expectancies is important.

SUMMARY

Pavlovian learning theory provides a still undervalued theoretical framework of how cravings and overeating can be learned and how they might be effectively tackled. Future studies should aim to elucidate inter-individual differences in Pavlovian conditioning, study ways to strengthen (long-term) extinction, and investigate the working mechanisms of cue exposure therapy.

Pharmacologically induced amnesia for learned fear is time and sleep dependent

The discovery in animal research that fear memories may change upon retrieval has sparked a wave of interest into whether this phenomenon of reconsolidation also occurs in humans. The critical conditions under which memory reconsolidation can be observed and targeted in humans, however, remain elusive. Here we report that blocking beta-adrenergic receptors in the brain, either before or after reactivation, effectively neutralizes the expression of fear memory. We show a specific time-window during which beta-adrenergic receptors are involved in the reconsolidation of fear memory. Finally, we observe intact fear memory expression 12 h after reactivation and amnesic drug intake when the retention test takes place during the same day as the intervention, but post-reactivation amnesia after a night of sleep (12 h or 24 h later). We conclude that memory reconsolidation is not simply time-dependent, but that sleep is a final and necessary link to fundamentally change the fear memory engram.

Emotional memory can change when retrieved, yet the conditions under which this can occur are not fully described. Here, authors show that taking a pill of propranolol taken during a specific time window can change the expression of fear memory in a person, and that sleep is necessary to forget learned fear.

Measuring Pavlovian appetitive conditioning in humans with the postauricular reflex

Despite its evolutionary and clinical significance, appetitive conditioning has been rarely investigated in humans. It has been proposed that this discrepancy might stem from the difficulty in finding suitable appetitive stimuli that elicit strong physiological responses. However, this might also be due to a possible lack of sensitivity of the psychophysiological measures commonly used to index human appetitive conditioning. Here, we investigated whether the postauricular reflex—a vestigial muscle microreflex that is potentiated by pleasant stimuli relative to neutral and unpleasant stimuli—may provide a valid psychophysiological indicator of appetitive conditioning in humans. To this end, we used a delay differential appetitive conditioning procedure, in which a neutral stimulus was contingently paired with a pleasant odor (CS+), while another neutral stimulus was not associated with any odor (CS−). We measured the postauricular reflex, the startle eyeblink reflex, and skin conductance response (SCR) as learning indices. Taken together, our results indicate that the postauricular reflex was potentiated in response to the CS+ compared with the CS−, whereas this potentiation extinguished when the pleasant odor was no longer delivered. In contrast, we found no evidence for startle eyeblink reflex attenuation in response to the CS+ relative to the CS−, and no effect of appetitive conditioning was observed on SCR. These findings suggest that the postauricular reflex is a sensitive measure of human appetitive conditioning and constitutes a valuable tool for further shedding light on the basic mechanisms underlying emotional learning in humans.

Measuring appetitive conditioned responses in humans

Clinical and preclinical findings suggest that individuals with abnormal responses to reward cues (stimuli associated with reward) may be at risk for maladaptive behaviors including obesity, addiction and depression. Our objective was to develop a new paradigm for producing appetitive conditioning using primary (food) rewards in humans, and investigate the equivalency of several outcomes previously used to measure appetitive responses to conditioned cues. We used an individualized food reward, and multimodal subjective, psychophysiological and behavioral measures of appetitive responses to a conditioned stimulus (CS) that predicted delivery of that food. We tested convergence among these measures of appetitive response, and relationships between these measures and action impulsivity, a putative correlate of appetitive conditioning. 90 healthy young adults participated. Although the paradigm produced robust appetitive conditioning in some measures, particularly psychophysiological ones, there were not strong correlations among measures of appetitive responses to the CS, as would be expected if they indexed a single underlying process. In addition, there was only one measure that related to impulsivity. These results provide important information for translational researchers interested in appetitive conditioning, suggesting that various measures of appetitive conditioning cannot be treated interchangeably.

Startle reflex modulation during threat of shock and "threat" of reward

During threat of shock, the startle reflex is potentiated, suggesting modulation by defensive mobilization. To determine whether startle potentiation is specific to aversive anticipation, startle reflexes were measured in the context of either aversive or appetitive anticipation in a between-subject study. Participants wore a device on the wrist that could deliver electrical shock (n = 49), or vibrotactile stimulation indicating monetary reward (n = 48). Cues signaling "threat" or "safe" periods were presented alone, or accompanied by presentation of affective and neutral pictures on half of the trials. Results indicated that the startle reflex was significantly potentiated when anticipating either shock or reward, compared to safe periods, both when no picture was presented, as well as during picture viewing. The difference between threat and safety in both reflex magnitude and skin conductance changes was larger for those anticipating shock, suggesting that the aversive context was more motivationally engaging. The pattern of reflex modulation as a function of picture valence varied under threat and safety, but was identical in the shock and reward groups, consistent with a hypothesis that anticipation of either aversive or appetitive events prompts heightened perceptual vigilance, potentiating the acoustic startle reflex.

Neural correlates of appetitive extinction in humans

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS−) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS−. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS− after conditioning. Furthermore, fMRI-results (CS+ − CS−) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Altered appetitive conditioning in overweight and obese women

Overweight and obese individuals show increased psychological and physiological reactivity to food cues and many of them have difficulties in achieving long-term weight loss. The current study tests whether abnormalities in the learning and extinction of appetitive responses to food cues might be responsible for this. Overweight/obese and healthy weight women completed a differential appetitive conditioning task using food as rewards, while eating expectancies, eating desires, conditioned stimulus evaluations, salivation, and electrodermal responses were assessed during an acquisition and extinction phase. Results suggested reduced discriminative conditioning in the overweight/obese group, as reflected by a worse acquisition of differential eating desires and no successful acquisition of differential evaluative responses. Some evidence was also found for impaired contingency learning in overweight and obese individuals. No group differences in conditioned salivation and skin conductance responses were found and no compelling evidence for differences in extinction was found as well. In sum, the current findings indicate that overweight and obesity may be characterized by reduced appetitive conditioning. It is suggested that this could be causally related to overeating via stronger context conditioning or a tendency towards overgeneralization in overweight and obese individuals.

Combining D-cycloserine with appetitive extinction learning modulates amygdala activity during recall

Appetitive Pavlovian conditioning plays a crucial role in the pathogenesis of drug addiction and conditioned reward cues can trigger craving and relapse even after long phases of abstinence. Promising preclinical work showed that the NMDA-receptor partial agonist D-cycloserine (DCS) facilitates Pavlovian extinction learning of fear and drug cues. Furthermore, DCS-augmented exposure therapy seems to be beneficial in various anxiety disorders, while the supposed working mechanism of DCS during human appetitive or aversive extinction learning is still not confirmed. To test the hypothesis that DCS administration before extinction training improves extinction learning, healthy adults (n=32) underwent conditioning, extinction, and extinction recall on three successive days in a randomized, double-blind, placebo-controlled fMRI design. Monetary wins and losses served as unconditioned stimuli during conditioning to probe appetitive and aversive learning. An oral dose of 50mg of DCS or placebo was administered 1h before extinction training and DCS effects during extinction recall were evaluated on a behavioral and neuronal level. We found attenuated amygdala activation in the DCS compared to the placebo group during recall of the extinguished appetitive cue, along with evidence for enhanced functional amygdala-vmPFC coupling in the DCS group. While the absence of additional physiological measures of conditioned responses during recall in this study prevent the evaluation of a behavioral DCS effect, our neuronal findings are in accordance with recent theories linking successful extinction recall in humans to modulatory top-down influences from the vmPFC that inhibit amygdala activation. Our results should encourage further translational studies concerning the usefulness of DCS to target maladaptive Pavlovian reward associations.

Appetitive Olfactory Learning and Long-Term Associative Memory in Caenorhabditis elegans

Because of the relative simplicity of its nervous system, Caenorhabditis elegans is a useful model organism to study learning and memory at cellular and molecular levels. For appetitive conditioning in C. elegans, food has exclusively been used as an unconditioned stimulus (US). It may be difficult to analyze neuronal circuits for associative memory since food is a multimodal combination of olfactory, gustatory, and mechanical stimuli. Here, we report classical appetitive conditioning and associative memory in C. elegans, using 1-nonanol as a conditioned stimulus (CS), and potassium chloride (KCl) as a US. Before conditioning, C. elegans innately avoided 1-nonanol, an aversive olfactory stimulus, and was attracted by KCl, an appetitive gustatory stimulus, on assay agar plates. Both massed training without an intertrial interval (ITI) and spaced training with a 10-min ITI induced significant levels of memory of association regarding the two chemicals. Memory induced by massed training decayed within 6 h, while that induced by spaced training was retained for more than 6 h. Animals treated with inhibitors of transcription or translation formed the memory induced by spaced training less efficiently than untreated animals, whereas the memory induced by massed training was not significantly affected by such treatments. By definition, therefore, memories induced by massed training and spaced training are classified as short-term memory (STM) and long-term memory (LTM), respectively. When animals conditioned by spaced training were exposed to 1-nonanol alone, their learning index was lower than that of untreated animals, suggesting that extinction learning occurs in C. elegans. In support of these results, C. elegans mutants defective in nmr-1, encoding an NMDA receptor subunit, formed both STM and LTM less efficiently than wild-type animals, while mutations in crh-1, encoding a ubiquitous transcription factor CREB required for memory consolidation, affected LTM, but not STM. The paradigm established in the present study should allow us to elucidate neuronal circuit plasticity for appetitive learning and memory in C. elegans.

Tackling maladaptive memories through reconsolidation: From neural to clinical science

Behavioral neuroscience has greatly informed how we understand the formation, persistence, and plasticity of memory. Research has demonstrated that memory reactivation can induce a labile period, during which previously consolidated memories are sensitive to change, and in need of restabilization. This process is known as reconsolidation. Such findings have advanced not only our basic understanding of memory processes, but also hint at the prospect of harnessing these insights for the development of a new generation of treatments for disorders of emotional memory. However, even in simple experimental models, the conditions for inducing memory reconsolidation are complex: memory labilization appears to result from the interplay of learning history, reactivation, and also individual differences, posing difficulties for the translation of basic experimental research into effective clinical interventions. In this paper, we review a selection of influential animal and human research on memory reconsolidation to illustrate key insights these studies afford. We then consider how these findings can inform the development of new treatment approaches, with a particular focus on the transition of memory from reactivation, to reconsolidation, to new memory formation, as well as highlighting possible limitations of experimental models. If the challenges of translational research can be overcome, and if reconsolidation-based procedures become a viable treatment option, then they would be one of the first mental health treatments to be directly derived from basic neuroscience research. This would surely be a triumph for the scientific study of mind and brain.

Modeling startle eyeblink electromyogram to assess fear learning

Pavlovian fear conditioning is widely used as a laboratory model of associative learning in human and nonhuman species. In this model, an organism is trained to predict an aversive unconditioned stimulus from initially neutral events (conditioned stimuli, CS). In humans, fear memory is typically measured via conditioned autonomic responses or fear‐potentiated startle. For the latter, various analysis approaches have been developed, but a systematic comparison of competing methodologies is lacking. Here, we investigate the suitability of a model‐based approach to startle eyeblink analysis for assessment of fear memory, and compare this to extant analysis strategies. First, we build a psychophysiological model (PsPM) on a generic startle response. Then, we optimize and validate this PsPM on three independent fear‐conditioning data sets. We demonstrate that our model can robustly distinguish aversive (CS+) from nonaversive stimuli (CS‐, i.e., has high predictive validity). Importantly, our model‐based approach captures fear‐potentiated startle during fear retention as well as fear acquisition. Our results establish a PsPM‐based approach to assessment of fear‐potentiated startle, and qualify previous peak‐scoring methods. Our proposed model represents a generic startle response and can potentially be used beyond fear conditioning, for example, to quantify affective startle modulation or prepulse inhibition of the acoustic startle response.

Treatment for Anhedonia: A Neuroscience Driven Approach

Anhedonia, or loss of interest or pleasure in usual activities, is characteristic of depression, some types of anxiety, as well as substance abuse and schizophrenia. Anhedonia is a predictor of poor long-term outcomes, including suicide, and poor treatment response. Because extant psychological and pharmacological treatments are relatively ineffective for anhedonia, there is an unmet therapeutic need for this high-risk symptom. Current psychological and drug treatments for anxiety and depression focus largely on reducing excesses in negative affect rather than improving deficits in positive affect. Recent advances in affective neuroscience posit that anhedonia is associated with deficits in the appetitive reward system, specifically the anticipation, consumption, and learning of reward. In this paper, we review the evidence for positive affect as a symptom cluster, and its neural underpinnings, and introduce a novel psychological treatment for anxiety and depression that targets appetitive responding. First, we review anhedonia in relation to positive and negative valence systems and current treatment approaches. Second, we discuss the evidence linking anhedonia to biological, experiential, and behavioral deficits in the reward subsystems. Third, we describe the therapeutic approach for Positive Affect Treatment (PAT), an intervention designed to specifically target deficits in reward sensitivity.

Neuroticism and extraversion moderate neural responses and effective connectivity during appetitive conditioning

Classical appetitive conditioning constitutes a basic learning process through which environmental stimuli can be associated with reward. Previous studies showed that individual differences in neuroticism and extraversion influence emotional processing and have been shown to modulate neural activity in subcortical and prefrontal areas in response to emotional stimuli. However, the role of individual differences in appetitive conditioning has so far not been investigated in detail. The aim of this study was to assess the association between neuroticism and extraversion with neural activity and connectivity during appetitive conditioning. The conditioned stimulus (CS) was either a picture of a dish or a cup. One stimulus (CS+) was paired with a monetary reward and the other stimulus (CS-) was associated with its absence while hemodynamic activity was measured by means of functional magnetic resonance imaging. A significant negative correlation of neuroticism scores with amygdala activity was observed during appetitive conditioning. Further, extraversion was positively associated with responses in the hippocampus and the thalamus. In addition, effective connectivity between the amygdala as a seed region and the anterior cingulate cortex, the insula, and the thalamus was negatively correlated with neuroticism scores and positively correlated with extraversion scores. The results may indicate a neural correlate for the deficits in appetitive learning in subjects with high neuroticism scores and point to a facilitating effect of extraversion on reward-related learning. Hum Brain Mapp 37:2992-3002, 2016. © 2016 Wiley Periodicals, Inc.