Loss aversion predicts cigarette smoking status across levels of sociodemographic characteristics

Loss aversion (LA) is a tendency to be more sensitive to potential losses relative to similar gains. Low LA is associated with increased risk for cigarette smoking and use of other substances. Previous studies of LA and smoking risk controlled for potentially confounding influences of sociodemographic characteristics associated with smoking risk. The present study replicates these earlier observations while also examining the generality of the association between low LA and smoking risk within different levels of each of the five sociodemographic risk factors for smoking (age, educational attainment, gender, income, race/ethnicity). Parallel analyses were conducted using delay discounting (DD) as a positive control; DD is a decision-making bias regarding the rate at which rewards lose value with increasing delay to receipt. Participants were recruited using standard crowdsourcing methods and completed a sociodemographics questionnaire, a hypothetical gamble task measure of LA, and a monetary choice measure of DD. Low LA was associated with increased risk of cigarette smoking after accounting for the influence of sociodemographic characteristics and DD. Similarly, high DD was associated with increased risk of cigarette smoking after accounting for the influence of sociodemographic characteristics and LA. Further analyses showed that associations of LA with smoking risk or DD with smoking risk generally although not always remained significant within varying levels of the sociodemographic characteristics of interest. These results provide support for low LA as a reliable risk factor for smoking that has generality within and across sociodemographic characteristics and closely parallels associations observed with DD and smoking risk. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Hippocampus and neocortex send and receive predictive information

A recent paper published in Nature by Yadav and colleagues (Nature 608 (7921): 153-160, 2022) illustrates a fundamental aspect on how different brain areas participate in memory storage and retrieval. After identifying neuron activity in the hippocampus CA1 region specific to multi-modal stimuli that predicted appetitive and aversive unconditioned stimuli (conjunctive stimulus encoding), the authors showed that neurons located in the anterior cingulate cortex dynamically encode the discrete sensory features of the outcome-predictive stimuli during training, and these highly specific feature-based projections can excite or inhibit conjunctive-coding neurons in the hippocampus during retrieval.

Partial reinforcement extinction and omission effects in the elimination and recovery of discriminated operant behavior

Three experiments explored how training reinforcement schedules and context influence the elimination and recovery of human operant behavior. In Experiment 1, participants learned a discriminated operant response in Context A before the response was eliminated with extinction in Context B. They then received a final test in each context. Groups were trained with a discriminative stimulus that predicted a reinforced response on either every trial (continuous reinforcement [CRF]) or some of the trials (partial reinforcement [PRF]). Extinction was slower following PRF training (a partial reinforcement extinction effect [PREE]) and extinguished responding increased when tested in Context A ("ABA" renewal). Experiment 2 further confirmed the PREE was obtained equally whether extinction occurred in the training context (Context A) or a new context (Context B) which is consistent with trial-based accounts of the PREE. Experiment 3 used the same design as Experiment 1 to evaluate the influence of training reinforcement on response elimination with an omission contingency. Across the omission training phase in Context B, the decrease in responding occurred more slowly in the PRF-trained group in comparison to the CRF-trained group, perhaps the first demonstration of what might be termed a PRF omission effect. Again, ABA renewal was observed in Context A. Training reinforcement schedule therefore had a similar influence on response elimination with extinction and omission. Elimination and recovery of human instrumental behavior, with extinction or omission, are influenced by training reinforcement schedule and context. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Loss Aversion and Current, Former, and Never-Smoking Status

INTRODUCTION

Loss aversion (LA) is a bias in decision-making wherein potential losses have a greater influence on choices than equivalent gains. Such a bias may protect individuals from harm. Consistent with this idea, lower LA has been measured in individuals who endorse current cigarette smoking as well as other substance use compared to controls matched on important sociodemographic variables (age, gender, and educational attainment). The goal of the present study was to systematically replicate and extend this association between LA and smoking status by comparing those meeting criteria for current-, former-, and never-smoking status.

AIMS AND METHODS

In total, 984 individuals (N = 984) that endorsed current cigarette smoking (past 30-day use; n = 361), former-smoking (no past 30-day use, >100 cigarettes lifetime; n = 317), and never-smoking (no past 30-day use, <100 cigarettes lifetime; n = 306) were recruited using standard crowdsourcing methods and completed measures of LA (50-50 gambles) and delay discounting (DD) (monetary choice questionnaire), an important decision-making bias with an established relationship to cigarette-smoking status.

RESULTS

Lower LA was observed in those endorsing current smoking compared to former smoking (t[952] = -9.57, Bonferroni corrected p < .0001), and never-smoking (t[952] = -3.99, Bonferroni corrected p = .0002). LA was also greater in former- compared to the never-smoking (t[952] = -5.26, Bonferroni corrected p < .0001). This pattern did not change when accounting for DD and sociodemographics. DD results replicated prior findings.

CONCLUSIONS

The results support LA as a decision-making bias related to the risk of cigarette smoking and other substance use. Further research is needed to understand the causal contributions of LA and DD and their potential intersections.

IMPLICATIONS

Low LA is a risk factor for cigarette smoking. This study reports higher LA among individuals that endorsed never-smoking and former-smoking status in comparison to those endorsing current cigarette smoking. LA may influence or be influenced by a change in smoking status.

Intersections between environmental reward availability, loss aversion, and delay discounting as potential risk factors for cigarette smoking and other substance use

Behavioral theory suggests that density of environmental rewarding activities and biases in decision making influence risk for substance use disorder (SUD). To better understand intersections of these potential risk factors, this study examined whether environmental reward predicted smoking status or other drug use and whether such associations were independent of two decision-making biases known to predict SUD risk, namely loss aversion and delay discounting. Individuals that reported current daily cigarette smoking (n = 186; >10 cigarettes/day) and never-smoking (n = 241; <100 cigarettes lifetime) were recruited with standard crowdsourcing methods. Participants answered questions on alcohol and other drug use. Environmental reward was assessed using the Reward Probability Index (RPI), and loss aversion (LA) and delay discounting (DD) using a gamble-acceptance task and monetary choice questionnaire, respectively. Associations of RPI, LA, and DD with cigarette smoking, alcohol use, other drug use, and combinations of co-use were examined with logistic regression controlling for sociodemographic variables (educational attainment, gender, age). Low RPI (odds ratio[OR] = 0.97, p = .006), low LA (OR = 1.22, p < .001), and high DD (OR = 1.12, p = .03), were each independently associated with increased risk for cigarette smoking, as well as other substance use, and use combinations. We saw no evidence that RPI was significantly influencing associations between LA and DD with smoking status or other substance use. Finally, RPI, but not LA or DD, was significantly associated with depressed mood and sleep disturbance. These results provide new evidence on associations of RPI with smoking status and other substance use while further documenting independent associations between LA and DD and those outcomes.

Relapse after incentivized choice treatment in humans: A laboratory model for studying behavior change

Behavior change interventions that incentivize desired behavior are highly effective for improving personal health, but difficult to maintain long term. Relapse is common and examining the mechanisms that contribute to relapse in experimental settings can identify processes relevant to substance abuse treatment. We developed a laboratory task that parallels a recent operant model of relapse after incentivized choice reported in the rodent laboratory. In two experiments, undergraduate participants first learned to make an operant response (keyboard button; R1) to earn a reinforcer consisting of an image of a preferred snack food (O1). In a second phase (Phase 2), R1 was still reinforced, but a new response (R2) was introduced and reinforced with a different reinforcer (a coin; O2). In a test phase, contingent incentives for R2 were removed (extinction) and relapse of R1 was assessed. Experiment 1 found that the O2 contingency suppressed R1 during Phase 2, and R1 relapsed rapidly in the test. Neither effect was consistently related to O2 value. Experiment 2 examined whether noncontingent presentations of O1 or O2 during the test could weaken relapse. Here, we found that noncontingent reinforcers did little to reduce or slow the increase in R1 responding. The present experiments highlight a laboratory approach to studying variables that may influence relapse after incentivized treatment. We identify and discuss areas for development to address differences between the present results and prior observations from animal and clinical studies. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Loss aversion and risk for cigarette smoking and other substance use

BACKGROUND

Cigarette smoking is among the leading preventable causes of global morbidity and mortality. We aimed to determine whether individual differences in loss aversion, a bias in decision-making wherein losses are valued greater than gains, predicts smoking and other addiction risk.

METHODS

We recruited current daily cigarette smokers (n = 181; > 10 cigarettes per day) and never-smokers (n = 237; < 100 cigarettes lifetime) from the United States using Amazon Mechanical Turk. Groups were matched on gender, educational attainment, and age. All completed items related to current cigarette smoking, alcohol use, other drug use, sleep problems, and depressed mood, and task-based measures of loss aversion and delay discounting, a decision-making bias associated with cigarette smoking.

RESULTS

Smokers were less loss averse than never-smokers (F(1, 411) = 24.19, η² = 0.02, p < .0001) even after accounting for delay discounting (F(1, 410) = 20.53, η² = 0.02, p < .0001). Loss aversion was also a significant independent risk factor for alcohol (F(1, 410) = 21.47, η² = 0.02, p < .0001) and other drug use (F(1, 410) = 54.12, η² = 0.04, p < .0001), although not other behavioral-health conditions (i.e., sleep disturbance, depressed mood). Further analyses revealed that co-occurring low loss aversion and high delay discounting were independently associated with greater risk for all patterns of substance use.

CONCLUSIONS

Loss aversion was associated with current cigarette smoking and other substance use patterns independent of delay discounting. Loss aversion may warrant attention as a protective factor and potential target for preventive intervention for substance use and addiction.

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Reinforcer predictability and stimulus salience promote discriminated habit learning

Instrumental (operant) behavior can be goal directed, but after extended practice it can become a habit triggered by environmental stimuli. There is little information, however, about the variables that encourage habit learning, or about the development of discriminated habits that are actually triggered by specific stimuli. (Most studies of habit in animal learning have used free-operant methods.) In the present experiments, rats received training in which a lever press was reinforced only in the presence of a discrete stimulus (S) and the status of the behavior as goal-directed or habitual was determined by reinforcer devaluation tests. Experiment 1 compared lever insertion and an auditory cue (tone) in their ability to support habit learning. Despite prior speculation in the literature, the "salient" lever insertion S was not better than the tone at supporting habit, although the rats learned more rapidly to respond in its presence. Experiment 2 then examined the role of reinforcer predictability with the brief (6-s) tone S. Lever pressing during the tone was reinforced on either every trial or on 50% of trials; habit was observed only with the highly predictable (100%) relationship between S and the reinforcer. Experiment 3 replicated this effect with the tone in a modified procedure and found that lever insertion contrastingly encouraged habit regardless of reinforcer predictability. The results support an interactive role for reinforcer predictability and stimulus salience in discriminated habit learning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Correction of response error versus stimulus error in the extinction of discriminated operant learning

Two experiments with rat subjects separated learning about the discriminative stimulus versus the operant response in the extinction of discriminated operant learning. Each was designed to separate 2 forms of error that could generate extinction learning from an error-correction perspective: Stimulus error, where the discriminative stimulus overpredicts the reinforcer in extinction, and response error, where the response is higher than what the current reinforcer supports. Stimulus error would cause correction of the Pavlovian stimulus-reinforcer association, whereas response error could cause correction of the instrumental response through adjustment of the response-reinforcer association or direct inhibition of the response. Previous research has supported a role for prediction error in instrumental extinction but has confounded these 2 potential sources of error. Using new variations of the concurrent excitor paradigm (Experiment 1) and the overexpectation paradigm (Experiment 2), the present experiments manipulated response error while controlling stimulus error. Both demonstrated that response error plays a role in instrumental extinction. When a discriminated operant response is not reinforced, response error correction may cause the animal to learn to suppress that specific response. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Effects of conditioned stimulus (CS) duration, intertrial interval, and I/T ratio on appetitive Pavlovian conditioning

Pavlovian learning is influenced by at least 2 temporal variables: The time between the onset of the conditioned stimulus (CS) and presentation of the unconditioned stimulus (US), and the time between successive conditioning trials (the intertrial interval [ITI]). Wagner's Sometimes Opponent Process (SOP) model (e.g., 1981) provides a rich account of the effects of varying the absolute durations of CS and ITI. However, other theories have contrastingly emphasized the role of the relative durations of CS (T) and ITI (I). Three experiments with rats used an appetitive conditioning preparation to separate the two approaches. They manipulated absolute values of I and T over a factor of 6 and compared the effect of varying T and I/T by the same factor. Conditioning was indexed by the rate of foodcup entry during training, during common tests conducted later with different combinations of I and T, and with a reinforcers-to-criterion measure. Experiment 1 found that learning with a 10-s CS was superior to that with a 60-s CS when the I/T ratio was the same. There was little evidence of learning with the 60-s CS. Experiment 2 replicated that result and extended it to show that a 60-s CS still supported little conditioned responding when I/T was increased from 12 to 72. Experiment 3 then examined intermediate CS durations between 10-s and 60-s while I/T was controlled. The results support a role for absolute CS duration rather than the I/T ratio. Explanations based on SOP and behavior systems theory are explored. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Unexpected food outcomes can return a habit to goal-directed action

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Inactivation of the prelimbic cortex attenuates operant responding in both physical and behavioral contexts

The present experiments aimed to expand our understanding of the role of the prelimbic cortex (PL) in the contextual control of instrumental behavior. Research has previously shown that the PL is involved when the "physical context," or chamber in which an instrumental behavior is trained, facilitates performance of the instrumental response (Trask, Shipman, Green, & Bouton, 2017). Recently, evidence has suggested that when a sequence of two instrumental behaviors is required to earn a reinforcing outcome, the first response (rather than the physical chamber) can be the "behavioral context" for the second response (Thrailkill, Trott, Zerr, and Bouton, 2016). Could the PL also be involved in this kind of contextual control? Here rats first learned a heterogenous behavior chain in which the first response (i.e., pressing a lever or pulling a chain) was cued by a discriminative stimulus and led to a second stimulus which cued a second response (i.e., pulling a chain or pressing a lever); the second response led to a sucrose reward. When the first and second responses were tested in isolation in the training context, pharmacological inactivation of the PL resulted in a reduction of the first response, but not the second response. When the second response was performed in the "context" of the first response (i.e., as part of the behavior chain) however, PL inactivation reduced the second response. Overall, these results support the idea that the PL is important for mediating the effects of a training context on instrumental responding, whether the context is physical or behavioral.

Relapse of operant behavior after response elimination with an extinction or an omission contingency

The present study compared relapse after responding was eliminated by extinction or omission training in rats. In Experiment 1, lever pressing was reinforced with food pellets in Context A and then eliminated with either extinction or omission training in Context B. The response was then tested in Contexts A and B in either the presence or absence of free food pellets delivered on a random time schedule. All rats showed higher responding when tested in Context A than Context B, and there was little evidence that omission training attenuated this ABA renewal effect. Noncontingent pellets increased responding after extinction but not after omission. However, when responding on the last day of response elimination was compared to responding during the test in the response-elimination context, there was some evidence that omission-trained rats showed a small increase in responding even when tested with free pellets. Results of Experiment 2 suggest this increase was not due to differences in the temporal distribution of pellets during elimination and the test, and that the result might be due to mere removal of the omission contingency, but any such effect is small and difficult to detect statistically. The results provide new information about factors generating relapse after omission training.

Resurgence in humans: Reducing relapse by increasing generalization between treatment and testing

Resurgence is the increase in performance of an extinguished instrumental (operant) response that accompanies the extinction of a response that has been reinforced to replace it. Resurgence may involve processes that are relevant for understanding relapse in applied and clinical settings. While resurgence is known to be a robust phenomenon in human operant extinction, the processes that control it remain unclear. Here we asked whether human resurgence is controlled by processes that are similar to those that have been identified in animals by asking whether two methods that reduce resurgence in animals also reduce it in humans. Participants first learned to make an operant response (R1) for a tangible food reinforcer (O1). In a second phase (Phase 2), R1 was extinguished while a second response (R2) was introduced and reinforced with a virtual monetary reward (USD $0.10 coins; O2). In a test phase, extinction was then introduced for R2 and resurgence of R1 was assessed. In Experiment 1, resurgence that occurred after the treatment just described was attenuated if there had been periodic exposure to R2 extinction during the treatment phase (Phase 2). In Experiment 2, resurgence was prevented when O2, but not O1, was presented noncontingently during the test. The results are among the first to suggest a mechanism underlying resurgence in humans, namely, renewal caused by contextual change. They also provide initial evidence to suggest that resurgence may be the result of common processes in animals and humans. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior

Goal-directed actions are instrumental behaviors whose performance depends on the organism's knowledge of the reinforcing outcome's value. In contrast, habits are instrumental behaviors that are insensitive to the outcome's current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random-interval 30-s schedule in the presence of a discriminative stimulus (S) but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the law of effect) and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Maintaining performance in searching dogs: Evidence from a rat model that training to detect a second (irrelevant) stimulus can maintain search and detection responding

Scent-detecting dogs perform a sequence, or chain, of behaviors that, at minimum, includes searching followed by a detection behavior that signals the presence of a target stimulus to the handler. However, when working, dogs often engage in prolonged periods of searching without encountering a target. It is therefore important for trainers to use methods that promote persistent search behavior and target detection accuracy. Laboratory models can provide insights to the important variables that influence search persistence and accuracy. The present experiments examined a rat model of detection dog behavior. Two experiments assessed the use of practice with a single target stimulus to maintain search and detection of another previously-trained target. In Experiment 1, after learning a search→detection chain with two auditory targets, rats received either brief or extended training with only one of the targets before being tested for detection of both targets in extinction. The results suggest that single-target training strengthened the ability of the other target to control the detection behavior. Experiment 2 found that even infrequent target encounters were still effective at maintaining detection behavior to the other trained target. Importantly, the treatment was effective when the target stimuli were from different sensory modalities. Overall, the results support the utility of the rat model of search-dog behavior for evaluating novel training methods. We suggest several useful procedures for enhancing search persistence and accuracy in detection dogs that can be implemented in training protocols.

Effects of outcome devaluation on instrumental behaviors in a discriminated heterogeneous chain

Operant behavior often takes place in a sequence, or chain, of linked responses that lead to a reinforcer. We have recently studied rats performing a discriminated heterogeneous behavior chain that involves the presentation of a discriminative stimulus (e.g., a panel light) to set the occasion for a procurement behavior (e.g., a lever press) that leads to a second stimulus (e.g., a second panel light) that indicates that a consumption response (e.g., a chain pull) will be reinforced. The present study assessed the role played by a representation of the reinforcer in controlling the performance of the responses in this chain. After acquisition of the chain, rats received a reinforcer devaluation treatment in the form of repeated paired, or unpaired, presentations of the food-pellet outcome and lithium-chloride illness. Once paired rats came to reject the pellets, half the animals in each group were tested on procurement, and the other half were tested on consumption. Neither response was affected by the outcome devaluation treatment, although entries into the food cup were suppressed. Combined with other results, the findings suggest that the "goal" for goal-directed procurement responding in a discriminated heterogeneous chain may be the consumption response rather than the primary reinforcer. (PsycINFO Database Record

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Occasion setting, inhibition, and the contextual control of extinction in Pavlovian and instrumental (operant) learning

An occasion setter is a stimulus that modulates the ability of another stimulus to control behavior. A rich history of experimental investigation has identified several important properties that define occasion setters and the conditions that give rise to occasion setting. In this paper, we first consider the basic hallmarks of occasion setting in Pavlovian conditioning. We then review research that has examined the mechanisms underlying the crucial role of context in Pavlovian and instrumental extinction. In Pavlovian extinction, evidence suggests that the extinction context can function as a negative occasion setter whose role is to disambiguate the current meaning of the conditioned stimulus; the conditioning context can also function as a positive occasion setter. In operant extinction, in contrast, the extinction context may directly inhibit the response, and the conditioning context can directly excite it. We outline and discuss the key results supporting these distinctions.

Contextual control of chained instrumental behaviors

Recent studies suggest a significant role for context in controlling the acquisition and extinction of simple operant responding. The present experiments examined the contextual control of a heterogeneous behavior chain. Rats first learned a chain in which a discriminative stimulus set the occasion for a procurement response (e.g., pulling a chain), which led to a second discriminative stimulus that occasion-set a consumption response (e.g., pressing a lever) that produced a food-pellet reinforcer. Experiment 1 showed that, after separate extinction of procurement and consumption, each response increased when it was returned to the acquisition context (ABA renewal) or was tested in a new context (AAB renewal). In addition, procurement responding, but not consumption responding, was decreased by changing the context after acquisition. Experiment 2 demonstrated ABA and AAB renewal of procurement and consumption following extinction of the whole chain. This time, the context-switch after acquisition weakened both procurement and consumption. Experiment 3 found that return to the context of the behavior chain renewed a consumption response that had been extinguished separately. Finally, in Experiment 4, rats learned 2 different discriminated heterogeneous chains; consumption extinguished outside its chain was only renewed on return to a chain when it was preceded by its associated procurement response. The results suggest a role for context in the extinction of chained behavior. They also support the view that procurement is influenced by the physical context and that consumption is controlled primarily by the response that precedes it in the chain. (PsycINFO Database Record

Extinction and the associative structure of heterogeneous instrumental chains

Drug abuse, overeating, and smoking are all examples of instrumental behaviors that often involve chains or sequences of behavior. A behavior chain is minimally composed of a procurement response that is required in order for a subsequent consumption response to be reinforced. Despite the translational importance of behavior chains, few studies have attempted to understand what binds them together and takes them apart. This article surveys the development of the heterogeneous instrumental chain method and introduces recent findings that have used extinction to analyze the associative content of (what is learned in) the chain. Chained responses that are occasion-set by their own discriminative stimuli may be directly associated; extinction of the procurement response weakens its associated consumption response, and extinction of the consumption response weakens its associated procurement response. Extinction itself involves learning to inhibit the response. Extinguished chained responses are subject to renewal when they are tested either back in the acquisition context or in a new context. In addition, a consumption response that is extinguished outside its chain is renewed when returned to the context of the preceding response in the chain. Research on heterogeneous behavior chains can provide important insights into an important but often overlooked aspect of instrumental learning.

Increasing the persistence of a heterogeneous behavior chain: Studies of extinction in a rat model of search behavior of working dogs

Dogs trained to search for contraband perform a chain of behavior in which they first search for a target and then make a separate response that indicates to the trainer that they have found one. The dogs often conduct multiple searches without encountering a target and receiving the reinforcer (i.e., no contraband is present). Understanding extinction (i.e., the decline in work rate when reinforcers are no longer encountered) may assist in training dogs to work in conditions where targets are rare. We therefore trained rats on a search-target behavior chain modeled on the search behavior of working dogs. A discriminative stimulus signaled that a search response (e.g., chain pull) led to a second stimulus that set the occasion for a target response (e.g., lever press) that was reinforced by a food pellet. In Experiment 1 training with longer search durations and intermittent (partial) reinforcement of searching (i.e. some trials had no target present) both led to more persistent search responding in extinction. The loss of search behavior in extinction was primarily dependent on the number of non-reinforced searches rather than time searching without reinforcement. In Experiments 2 and 3, delivery of non-contingent reinforcers during extinction increased search persistence provided they had also been presented during training. Thus, results with rats suggest that the persistence of working dog performance (or chained behavior generally) may be improved by training with partial reinforcement of searching and non-contingent reinforcement during both training and work (extinction).

Extinction of chained instrumental behaviors: Effects of consumption extinction on procurement responding

Operant behavior is typically organized into sequences of responses that eventually lead to a reinforcer. Response elements can be categorized as those that directly lead to reward consumption (i.e., a consumption response) and those that lead to the opportunity to make the consumption response (i.e., a procurement response). These responses often differ topographically and in terms of the discriminative stimuli that set the occasion for them. We have recently shown that extinction of the procurement response acts to weaken the specific associated consumption response, and that active inhibition of the procurement response is required for this effect. To expand the analysis of the associative structure of chains, in the present experiments we asked the reverse question: whether extinction of consumption behavior results in a decrease in the associated procurement response in a discriminated heterogeneous chain. In Experiment 1, extinction of consumption alone led to an attenuation of the associated procurement response only when rats were allowed to make the consumption response in extinction. Exposure to the consumption stimulus alone was not sufficient to produce weakened procurement responding. In Experiment 2, rats learned two distinct heterogeneous chains, and extinction of one consumption response specifically weakened the procurement response associated with it. The results add to the evidence suggesting that rats learn a highly specific associative structure in behavior chains, emphasizing the role of learning response inhibition in extinction.

Extinction of chained instrumental behaviors: Effects of procurement extinction on consumption responding

Instrumental behavior often consists of sequences or chains of responses that minimally include procurement behaviors that enable subsequent consumption behaviors. In such chains, behavioral units are linked by access to one another and eventually to a primary reinforcer, such as food or a drug. The present experiments examined the effects of extinguishing procurement responding on consumption responding after training of a discriminated heterogeneous instrumental chain. Rats learned to make a procurement response (e.g., pressing a lever) in the presence of a distinctive discriminative stimulus; making that response led to the presentation of a second discriminative stimulus that set the occasion for a consumption response (e.g., pulling a chain), which then produced a food-pellet reinforcer. Experiment 1 showed that extinction of either the full procurement-consumption chain or procurement alone weakened the consumption response tested in isolation. Experiment 2 replicated the procurement extinction effect and further demonstrated that the opportunity to make the procurement response, as opposed to simple exposure to the procurement stimulus alone, was required. In Experiment 3, rats learned 2 distinct discriminated heterogeneous chains; extinction of 1 procurement response specifically weakened the consumption response that had been associated with it. The results suggest that learning to inhibit the procurement response may produce extinction of consumption responding through mediated extinction. The experiments suggest the importance of an associative analysis of instrumental behavior chains. (PsycINFO Database Record

Contextual control of instrumental actions and habits

After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats with a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.

Effects of inter-food interval on the variety effect in an instrumental food-seeking task. Clarifying the role of habituation

Food variety increases consumption and the rate of instrumental behavior that is reinforced by food in humans and animals. The present experiment investigated the relationship between the variety effect and habituation to food by testing the role of the interval between successive food presentations on responding in an operant food-seeking task. Habituation to food was expected at short, but not long, interfood intervals. The effects of variety on food's long-term reinforcing value were also tested. Four groups of rats were trained to lever-press on different random-interval (RI) schedules of reinforcement to earn 45-mg food pellets. Half the rats in each group received an unpredictable mix of grain and sucrose pellets, while the other half consistently received sucrose pellets. Response rate began at a high rate and then decreased within each 30-min session for groups that received short inter-pellet intervals (i.e., RI-3 s and RI-6 s reinforcement schedules) but not in groups that received longer inter-pellet intervals (i.e., RI-12 s and RI-24 s). A variety effect in the form of higher responding in the mix group than the sucrose-only group was also only evident at the shorter intervals. Habituation and variety effects were also most evident with the short intervals when we controlled for the number of reinforcers earned, suggesting that they were not merely due to rapid satiation. The variety effect also appeared quickly when groups trained with longer inter-pellet intervals (RI-12 s and RI-24 s) were transitioned to shorter intervals (RI-3 s and RI-6 s). There was no effect of variety on resistance to extinction or on resistance to the response-suppressing effects of pre-session feeding. The results more clearly link this version of the variety effect to the short-term effect of variety on food habituation.

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich  = VI 30 s; Lean  =  VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.

Temporal context, preference, and resistance to change

According to behavioral momentum theory, preference and relative resistance to change in concurrent-chains schedules are correlated and reflect the relative conditioned value of discriminative stimuli. In the present study, we explore the generality of this relation by manipulating the temporal context within a concurrent-chains procedure through changes in the duration of the initial links. Consistent with previous findings, preference for a richer terminal link was less extreme with longer initial links across three experiments with pigeons. In Experiment 1, relative resistance to change and preference were related inversely when responding was disrupted with response-independent food presentations during initial links, replicating a previous finding with rats. However, more food was presented with longer initial links, confounding the disrupter and initial-link duration. In Experiment 2, presession feeding was used instead and eliminated the negative relation between relative resistance to change and preference, but relative resistance to change was not sensitive to relative terminal-link reinforcement rates. In Experiment 3, with more extreme relative terminal-link reinforcement rates, increasing initial-link duration similarly decreased preference and relative resistance to change for the richer terminal link. Thus, when conditions of disruption are equal and assessed under the appropriate reinforcement conditions, changes in temporal context impact relative resistance to change and preference similarly.

Brief light as a practical aversive stimulus for the albino rat

Bright light was an effective aversive stimulus for Wistar rats in punishment, escape, and avoidance paradigms. Contingent punishment of lever pressing maintained by concurrent schedules of food delivery shifted presses to an alternate lever, and depressed overall response rates. Periodic non-contingent presentation of the light prompted escape responding (head entry into a hole). Unsignaled avoidance contingencies were not effective, but pre-pulse signaling of light supported avoidance behavior. These results demonstrate a possible alternative to foot-shock, one with greater ecological validity, and one that might avoid some of the physiological effects that accompany electric shock.

Progressive ratio schedules of reinforcement

Pigeons' pecks produced grain under progressive ratio (PR) schedules, whose response requirements increased systematically within sessions. Experiment 1 compared arithmetic (AP) and geometric (GP) progressions. Response rates increased as a function of the component ratio requirement, then decreased linearly (AP) or asymptotically (GP). Experiment 2 found the linear decrease in AP rates to be relatively independent of step size. Experiment 3 showed pausing to be controlled by the prior component length, which predicted the differences between PR and regressive ratio schedules found in Experiment 4. When the longest component ratios were signaled by different key colors, rates at moderate ratios increased, demonstrating control by forthcoming context. Models for response rate and pause duration based on Bizo and Killeen (1997) described performance on AP schedules; GP schedules required an additional parameter representing the contextual reinforcement.