Scaling relative incentive value in consummatory behavior

Alcohol consumption and preference in female rats induced by reward downshift reveals sex generality of the modulatory role of physical activity

Increased voluntary consumption of alcohol has been demonstrated in male rats exposed to frustrative reward downshift (the emotional self-medication effect). Access to a wheel for voluntary running abolished this effect in male rats, suggesting an attenuating effect of physical exercise on the negative affect induced by reward downshift and its consequences on drug intake. The present study analyzed this effect in female rats. Sixty-four food-deprived female Wistar rats received 32% sucrose [4% (Experiment 1) or 2% (Experiment 2) in controls] during 10, 5-min preshift sessions followed by 4% (Experiment 1) or 2% (Experiment 2) sucrose during 5 postshift sessions. Immediately after each consummatory session, animals were exposed to a 2-h, two-bottle preference test involving 32% alcohol vs. water. Half of the animals also had access to a running wheel during the preference test. The results showed (a) lower sucrose consumption in the downshifted groups (32-4% and 32-2%) compared to the unshifted controls (4-4% and 2-2%, respectively); (b) higher alcohol preference in downshifted groups without access to a wheel compared with downshifted groups with access to the wheel (Experiments 1 and 2); and (c) increased alcohol intake (g/kg) after experiencing reward downshift in animals without access to the wheel (Experiment 1). Voluntary wheel running thus reduced alcohol intake in female rats experiencing reward downshift. These findings are comparable to previous results reported in male rats and support the usefulness of physical exercise to prevent alcohol self-medication induced by frustrative nonreward.

Frustrative nonreward: Detailed c-Fos expression patterns in the amygdala after consummatory successive negative contrast

The amygdala has been implicated in frustrative nonreward induced by unexpected reward downshifts, using paradigms like consummatory successive negative contrast (cSNC). However, existing evidence comes from experiments involving the central and basolateral nuclei on a broad level. Moreover, whether the amygdala's involvement in reward downshift requires a cSNC effect (i.e., greater suppression in downshifted animals than in unshifted controls) or just consummatory suppression without a cSNC effect, remains unclear. Three groups were exposed to (1) a large reward disparity leading to a cSNC effect (32-to-2% sucrose), (2) a small reward disparity involving consummatory suppression in the absence of a cSNC effect (8-to-2% sucrose), and (3) an unshifted control (2% sucrose). Brains obtained after the first reward downshift session were processed for c-Fos expression, a protein often used as a marker for neural activation. c-Fos-positive cells were counted in the anterior, medial, and posterior portions (A/P axis) of ten regions of the rat basolateral, central, and medial amygdala. c-Fos expression was higher in 32-to-2% sucrose downshift animals than in the other two groups in four regions: the anterior and the medial lateral basal amygdala, the medial capsular central amygdala, and the anterior anterio-ventral medial amygdala. None of the areas exhibited differential c-Fos expression between the 8-to-2% sucrose downshift and the unshifted conditions. Thus, amygdala activation requires exposure to a substantial reward disparity. This approach has identified, for the first time, specific amygdala areas relevant to understand the cSNC effect, suggesting follow-up experiments aimed at testing the function of these regions in reward downshift.

Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior

Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior.

Frustrative nonreward and the basal ganglia: Chemogenetic inhibition and excitation of the nucleus accumbens and globus pallidus externus during reward downshift

Frustrative nonreward contributes to anxiety disorders and addiction, and is included in the Research Domain Criteria initiative as a relevant endophenotype. These experiments explored the role of the basal ganglia in consummatory reward downshift (cRD) using inhibitory and excitatory DREADDs (designer receptors exclusively activated by designer drugs) infused in either the nucleus accumbens (NAc) or one of its downstream targets, the globus pallidus externus (GPe). NAc inhibition did not disrupt consummatory suppression during a 32-to-2% (Experiment 1) or 8-to-2% sucrose downshift (Experiment 2). However, NAc excitation enhanced consummatory suppression during a 32-to-2% sucrose downshift (Experiment 1). GPe inhibition caused a trend toward increased consummatory suppression after a 32-to-2% sucrose downshift, whereas GPe excitation eliminated consummatory suppression after an 8-to-2% sucrose downshift (Experiment 3). Chemogenetic manipulations of NAc and GPe had no detectable effects on open field activity. The effects of DREADD activation via clozapine N-oxide (CNO) administration were compared to controls that carried the DREADDs, but received vehicle injections. There was no evidence that CNO or vehicle injections in virus vector control (VVC) animals affected cRD or OF activity after either CNO or vehicle injections. NAc and GPe excitation led to opposite results in the cRD task, providing evidence that the basal ganglia circuit has a function in frustrative nonreward in the absence of detectable motor effects.

Evidence of successive negative contrast in terrestrial toads (Rhinella arenarum): central or peripheral effect?

Prior research with terrestrial toads (Rhinella arenarum) in a water-reinforced instrumental situation indicated a direct relationship between acquisition rate and reward magnitude. However, a reward downshift produced a gradual adjustment of instrumental performance and a rapid adjustment of consummatory performance, rather than the abrupt and transient deterioration of behavior typical of a successive negative contrast effect. In Experiment 1, using a two-chamber box, a downshift from deionized water (which supports maximal rehydration) to 250-mM sodium chloride solution (which supports a lower rehydration), also yielded a gradual adjustment of instrumental behavior. In this experiment, animals received one trial per day and were allowed 300 s of access to the reward in the goal box. Experiment 2 used the same procedure, except that animals were allowed access to the solution in the goal box for 600 s. Under these conditions, reward downshift led to longer latencies (instrumental) and lower rehydration levels (consummatory) than those of unshifted controls, providing evidence for successive negative contrast. Unlike in similar experiments with mammals, the effect was not transient, but persisted relatively unmodified over twelve daily postshift trials. In this case, the possibility of adaptation of the peripheral mechanisms for water uptake is considered. The comparative relevance of these results is discussed in terms of habit formation versus expectancy-guided behavior in vertebrate learning.

Frustrative nonreward and emotional self-medication:Factors modulating alcohol consumption following reward downshift in rats

Life experience involving unexpected incentive loss (e.g., loss of job or a significant other) may result in negative emotional reactions (frustration) and promote alcohol drinking. Similarly, animals exposed to a frustrative 32-to-4% sucrose downshift increase their preference for alcohol (2%) vs. water. This result was interpreted as reflecting emotional self-medication-the consumption of substances that reduce negative emotions. We conducted three experiments examining parametric manipulations of the animal model: (1) effects of a severe reward downshift (32-to-4% sucrose) on consumption of various alcohol concentrations (Experiment 1); (2) effects of different magnitudes of reward downshifts on consumption of 32% alcohol (Experiment 2); and (3) effects of partial reinforcement (an intervention that increases resistance to frustration) on 2% alcohol intake induced by a 32-to-4% sucrose downshift (Experiment 3). The results show that (1) a 32-to-4% sucrose downshift leads to an increase in alcohol intake over a wide range of alcohol concentrations; (2) the greater the reward downshift, the higher the relative increase in alcohol consumption; and (3) a treatment that increases resistance to frustration (partial reinforcement) also attenuates alcohol consumption after a sucrose downshift. These data are discussed in relation to the role of frustrative nonreward in alcohol consumption.

Affective perception of Euro banknotes: cognitive factors and interindividual differences

Money can be a tool to achieve a wide range of goals in everyday life. Different studies have reported that both the mere exposure to money and its use as a reward can determine cognitive and social effects. Nevertheless, little is known about the basic affective perception of Euro banknotes. Thus, in the present study we aim to assess differences in valence, arousal and familiarity evaluations of banknote pictures (from 5 to 500€) by taking into account gender, socioeconomic status and Love of Money (LoM) score, which measures the subjective attitude toward money, in a sample of participants. We found that valence and arousal increase with the nominal value of the banknotes, and that the relationship between these affective scores and the nominal value appears to be logarithmic (Weber's law) rather than linear. High value banknotes were evaluated as pleasant, highly arousing, and less familiar. Low value banknotes instead were evaluated as more familiar, less arousing and neutrally valenced. Finally, we found that valence and arousal evaluations are mainly influenced by the LoM score of our participants. Instead, gender and economic condition influenced only arousal scores. These findings suggest the importance of deepening the study of these variables to shed light on money-related biases and abnormal economic behaviors.

Avian Emotions: Comparative Perspectives on Fear and Frustration

Emotions are complex reactions that allow individuals to cope with significant positive and negative events. Research on emotion was pioneered by Darwin’s work on emotional expressions in humans and animals. But Darwin was concerned mainly with facial and bodily expressions of significance for humans, citing mainly examples from mammals (e.g., apes, dogs, and cats). In birds, emotional expressions are less evident for a human observer, so a different approach is needed. Understanding avian emotions will provide key evolutionary information on the evolution of related behaviors and brain circuitry. Birds and mammals are thought to have evolved from different groups of Mesozoic reptiles, theropod dinosaurs and therapsids, respectively, and therefore, their common ancestor is likely to be a basal reptile living about 300 million years ago, during the Carboniferous or Permian period. Yet, birds and mammals exhibit extensive convergence in terms of relative brain size, high levels of activity, sleep/wakefulness cycles, endothermy, and social behavior, among others. This article focuses on two basic emotions with negative valence: fear and frustration. Fear is related to the anticipation of dangerous or threatening stimuli (e.g., predators or aggressive conspecifics). Frustration is related to unexpected reward omissions or devaluations (e.g., loss of food or sexual resources). These results have implications for an understanding of the conditions that promote fear and frustration and for the evolution of supporting brain circuitry.

Open-field exposure facilitates consummatory extinction

During extinction, the organism learns that a conditioned stimulus or a conditioned response is no longer associated with an unconditioned stimulus, and as a consequence, a decrement in the response is presented. The exposure to novel situations (e.g. exploration of a novel open field) has been used widely to modulate (i.e. either enhance or deteriorate) learning and memory. The aim of the present study was to test whether open-field exposure could modulate consummatory extinction. The results indicated that open-field exposure accelerated the extinction response (i.e. experimental animals provided novelty exposure had lower consummatory behavior than control animals) when applied before - but not after - the first extinction trial, or when applied before the second extinction trial. The results suggest that environmental treatments such as novelty exposure provide a valuable, nonpharmacological alternative to potentially modulate extinction processes.

Complex effects of reward upshift on consummatory behavior

Exposing rats to an upshift from a small reward to a larger reward sometimes yields evidence of consummatory successive positive contrast (cSPC), an effect that could be a suitable animal model of positive emotion. However, cSPC is an unreliable effect. Ten experiments explored the effects of an upshift in sucrose or saccharin concentration on consummatory behavior under several conditions. There was occasional evidence of cSPC, but mostly a combination of increased consummatory behavior relative to preshift reward concentrations and a reduced behavioral level relative to unshifted controls. Such a pattern is consistent with processes causing opposite changes on behavior. Reward upshift may induce processes that suppress behavior, such as taste neophobia (induced by an intense sucrose taste) and generalization decrement (induced by novelty in reward conditions after the upshift). An experiment tested the role of such novelty-related effects by preexposing animals to either the upshift concentration (12% sucrose) or water during three days before the start of the experiment. Sucrose-preexposed animals drank significantly more than water-preexposed animals during the upshift, but just as much as unshifted controls (i.e., no evidence of cSPC). These results suggest that cSPC may be difficult to obtain reliably because reward upshift induces opposing processes. However, they also seriously question the ontological status of cSPC.

Relative reward effects on operant behavior: Incentive contrast, induction and variety effects

Comparing different rewards automatically produces dynamic relative outcome effects on behavior. Each new outcome exposure is to an updated version evaluated relative to alternatives. Relative reward effects include incentive contrast, positive induction and variety effects. The present study utilized a novel behavioral design to examine relative reward effects on a chain of operant behavior using auditory cues. Incentive contrast is the most often examined effect and focuses on increases or decreases in behavioral performance after value upshifts (positive) or downshifts (negative) relative to another outcome. We examined the impact of comparing two reward outcomes in a repeated measures design with three sessions: a single outcome and a mixed outcome and a final single outcome session. Relative reward effects should be apparent when comparing trials for the identical outcome between the single and mixed session types. An auditory cue triggered a series of operant responses (nosepoke-leverpress-food retrieval), and we measured possible contrast effects for different reward magnitude combinations. We found positive contrast for trials with the greatest magnitude differential but positive induction or variety effects in other combinations. This behavioral task could be useful for analyzing environmental or neurobiological factors involved in reward comparisons, decision-making and choice during instrumental, goal-directed action.

Identifying profiles of recovery from reward devaluation in rats

In humans and other mammals, the unexpected loss of a resource can lead to emotional conflict. Consummatory successive negative contrast (cSNC) is a laboratory model of reward devaluation meant to capture that conflict. In this paradigm, animals are exposed to a sharp reduction in the sucrose concentration of a solution after several days of access. This downshift in sucrose content leads to behavioral responses such as the suppression of consumption and physiologic responses including elevation of corticosterone levels. However, response heterogeneity in cSNC has yet to be explored and may be relevant for increasing the validity of this model, as humans demonstrate clinically meaningful heterogeneity in response to resource loss. The current analysis applied latent growth mixture modeling to test for and characterize heterogeneity in recovery from cSNC among rats (N=262). Although most animals exhibited recovery of consummatory behavior after a sharp drop in consumption in the first postshift trial (Recovery class; 83%), two additional classes were identified including animals that did not change their consumption levels after downshift (No Contrast class; 6%), and animals that exhibited an initial response similar to that of the Recovery class but did not recover to preshift consumption levels (No Recovery class; 11%). These results indicate heterogeneity in recovery from reward loss among rats, which may increase the translatability of this animal model to understand diverse responses to loss among humans.

Behavioral neuroscience of psychological pain

Pain is a common word used to refer to a wide range of physical and mental states sharing hedonic aversive value. Three types of pain are distinguished in this article: Physical pain, an aversive state related to actual or potential injury and disease; social pain, an aversive emotion associated to social exclusion; and psychological pain, a negative emotion induced by incentive loss. This review centers on psychological pain as studied in nonhuman animals. After covering issues of terminology, the article briefly discusses the daily-life significance of psychological pain and then centers on a discussion of the results originating from two procedures involving incentive loss: successive negative contrast-the unexpected devaluation of a reward-and appetitive extinction-the unexpected omission of a reward. The evidence reviewed points to substantial commonalities, but also some differences and interactions between physical and psychological pains. This evidence is discussed in relation to behavioral, pharmacological, neurobiological, and genetic factors that contribute to the multidimensional experience of psychological pain.

Propranolol reverses open field effects on frustration

Reactivity to a reward is affected by prior experience with different reinforcer values of that reward, a phenomenon known as incentive relativity. Incentive relativity can be studied via the consummatory successive negative contrast (cSNC) paradigm, in which acceptance of 4% sucrose is assessed in animals that had been exposed to 32% sucrose. These downshifted animals usually exhibit significantly less sucrose acceptance than animals that always received the 4% sucrose solution. In previous work, we found that exploration of a novel open field (OF) before the first trial with the downshifted solution attenuated the contrast effect. The goal of the present experiments was to expand the knowledge on the effects of OF exposure on cSNC. We evaluated the effect OF exposure before the second downshift trial and assessed the mediational role of the adrenergic system in the effects of OF during the first and second trial of cSNC. The results indicate that OF applied before the first or second downshift trials exert opposite effects and that the adrenergic system is involved in the acquisition and consolidation of the OF information.

Weber's law, the magnitude effect and discrimination of sugar concentrations in nectar-feeding animals

Weber's law quantifies the perception of difference between stimuli. For instance, it can explain why we are less likely to detect the removal of three nuts from a bowl if the bowl is full than if it is nearly empty. This is an example of the magnitude effect - the phenomenon that the subjective perception of a linear difference between a pair of stimuli progressively diminishes when the average magnitude of the stimuli increases. Although discrimination performances of both human and animal subjects in various sensory modalities exhibit the magnitude effect, results sometimes systematically deviate from the quantitative predictions based on Weber's law. An attempt to reformulate the law to better fit data from acoustic discrimination tasks has been dubbed the "near-miss to Weber's law". Here, we tested the gustatory discrimination performance of nectar-feeding bats (Glossophaga soricina), in order to investigate whether the original version of Weber's law accurately predicts choice behavior in a two-alternative forced choice task. As expected, bats either preferred the sweeter of the two options or showed no preference. In 4 out of 6 bats the near-miss to Weber's law provided a better fit and Weber's law underestimated the magnitude effect. In order to test the generality of this observation in nectar-feeders, we reviewed previously published data on bats, hummingbirds, honeybees, and bumblebees. In all groups of animals the near-miss to Weber's law provided better fits than Weber's law. Furthermore, whereas the magnitude effect was stronger than predicted by Weber's law in vertebrates, it was weaker than predicted in insects. Thus nectar-feeding vertebrates and insects seem to differ in how their choice behavior changes as sugar concentration is increased. We discuss the ecological and evolutionary implications of the observed patterns of sugar concentration discrimination.

The psychophysics of sugar concentration discrimination and contrast evaluation in bumblebees

The capacity to discriminate between choice options is crucial for a decision-maker to avoid unprofitable options. The physical properties of rewards are presumed to be represented on context-dependent, nonlinear cognitive scales that may systematically influence reward expectation and thus choice behavior. In this study, we investigated the discrimination performance of free-flying bumblebee workers (Bombus impatiens) in a choice between sucrose solutions with different concentrations. We conducted two-alternative free choice experiments on two B. impatiens colonies containing some electronically tagged bumblebees foraging at an array of computer-automated artificial flowers that recorded individual choices. We mimicked natural foraging conditions by allowing uncertainty in the probability of reward delivery while maintaining certainty in reward concentration. We used a Bayesian approach to fit psychometric functions, relating the strength of preference for the higher concentration option to the relative intensity of the presented stimuli. Psychometric analysis was performed on visitation data from individually marked bumblebees and pooled data from unmarked individuals. Bumblebees preferred the more concentrated sugar solutions at high stimulus intensities and showed no preference at low stimulus intensities. The obtained psychometric function is consistent with reward evaluation based on perceived concentration contrast between choices. We found no evidence that bumblebees reduce reward expectations upon experiencing non-rewarded visits. We compare psychometric function parameters between the bumblebee B. impatiens and the flower bat Glossophaga commissarisi and discuss the relevance of psychophysics for pollinator-exerted selection pressures on plants.

Devaluation of low-quality food during early experience by sheep

A ruminant's early experience with low-quality food (LQF) is expected to increase its acceptance and preference in adulthood. Contrarily, we found that experienced sheep (ES) exposed to mature oat hay early in life ate less of this LQF than inexperienced sheep (IS). A possibility is that ES might have devaluated the LQF through continuous comparisons against high-quality food (HQF) supplements (sunflower meal and ground corn) that were simultaneously available during early experience. In this study, we tested the devaluation hypothesis with a successive negative contrast (SNC) procedure. In a consummatory SNC procedure, 'shift' subjects are unexpectedly changed from HQF to LQF, and their consumption is then compared against the consumption of 'unshift' subjects that receive LQF all throughout the SNC procedure. The magnitude of the difference in consumption between preshift and postshift is regarded as a measure of the degree to which both foods (HQF and LQF) are perceived to differ hedonically. When sheep from our previous study were 300 days old, both ES and IS were randomly assigned to either shift (ES-S and IS-S) or unshift conditions (ES-U and IS-U; n = 6 in each group). Groups ES-S and IS-S were fed HQF (alfalfa hay) during the preshift phase, and then suddenly changed to LQF (oat hay) in the postshift phase. Groups ES-U and IS-U (controls) were fed only LQF throughout the SNC procedure. Subjects in ES-S showed a significantly lower intake of LQF than those in ES-U in the first postshift session (i.e. they showed an SNC effect), which was not observed in IS. These results agree with ES subjects having devalued LQF during early experience. We discuss the possibility that high levels of nutrient supplementation can result in devaluation of LQF (i.e. decrease in preference and acceptance), whereas restricted levels of supplementation may promote a positive experience with LQF.

Incentive or habit learning in amphibians?

Toads (Rhinella arenarum) received training with a novel incentive procedure involving access to solutions of different NaCl concentrations. In Experiment 1, instrumental behavior and weight variation data confirmed that such solutions yield incentive values ranging from appetitive (deionized water, DW, leading to weight gain), to neutral (300 mM slightly hypertonic solution, leading to no net weight gain or loss), and aversive (800 mM highly hypertonic solution leading to weight loss). In Experiment 2, a downshift from DW to a 300 mM solution or an upshift from a 300 mM solution to DW led to a gradual adjustment in instrumental behavior. In Experiment 3, extinction was similar after acquisition with access to only DW or with a random mixture of DW and 300 mM. In Experiment 4, a downshift from DW to 225, 212, or 200 mM solutions led again to gradual adjustments. These findings add to a growing body of comparative evidence suggesting that amphibians adjust to incentive shifts on the basis of habit formation and reorganization.