Repeated extinction and reversal learning of an approach response supports an arousal-mediated learning model

Randomized controlled trial of a novel lifestyle intervention used with or without meal replacements in work sites

OBJECTIVE

Lifestyle interventions have had limited effectiveness in work sites when evaluated in randomized trials. This study assessed the effectiveness of a novel lifestyle intervention for weight loss (Healthy Weight for Living [HWL]) implemented with or without meal replacements (MR) in work sites. HWL used a new behavioral approach emphasizing reducing hunger and building healthy food preferences, and, unlike traditional lifestyle interventions, it did not require calorie counting.

METHODS

Twelve work sites were randomized to an 18-month intervention (n = 8; randomization within work sites to HWL, HWL + MR) or 6-month wait-listed control (n = 4). Participants were employees with overweight or obesity (N = 335; age = 48 [SD 10] years; BMI = 33 [6] kg/m2 ; 83% female). HWL was group-delivered in person or by videoconference. The primary outcome was 6-month weight change; secondary outcomes included weight and cardiometabolic risk factors measured at 6, 12, and 18 months in intervention groups.

RESULTS

Mean 6-month weight change was -8.8% (95% CI: -11.2% to -6.4%) for enrollees in HWL and -8.0% (-10.4% to -5.5%) for HWL + MR (p < 0.001 for both groups vs. controls), with no difference between interventions (p = 0.40). Clinically meaningful weight loss (≥5%) was maintained at 18 months in both groups (p < 0.001).

CONCLUSIONS

A new lifestyle intervention approach, deliverable by videoconference with or without MR, supported clinically impactful weight loss in employees.

Emergence of complex dynamics of choice due to repeated exposures to extinction learning

Extinction learning, the process of ceasing an acquired behavior in response to altered reinforcement contingencies, is not only essential for survival in a changing environment, but also plays a fundamental role in the treatment of pathological behaviors. During therapy and other forms of training involving extinction, subjects are typically exposed to several sessions with a similar structure. The effects of this repeated exposure are not well understood. Here, we studied the behavior of pigeons across several sessions of a discrimination-learning task in context A, extinction in context B, and a return to context A to test the context-dependent return of the learned responses (ABA renewal). By focusing on individual learning curves across animals, we uncovered a session-dependent variability of behavior: (1) during extinction, pigeons preferred the unrewarded alternative choice in one-third of the sessions, predominantly during the first one. (2) In later sessions, abrupt transitions of behavior at the onset of context B emerged, and (3) the renewal effect decayed as sessions progressed. We show that the observed results can be parsimoniously accounted for by a computational model based only on associative learning between stimuli and actions. Our work thus demonstrates the critical importance of studying the trial-by-trial dynamics of learning in individual sessions, and the power of “simple” associative learning processes.

Reward-enhanced encoding improves relearning of forgotten associations

Research on human memory has shown that monetary incentives can enhance hippocampal memory consolidation and thereby protect memory traces from forgetting. However, it is not known whether initial reward may facilitate the recovery of already forgotten memories weeks after learning. Here, we investigated the influence of monetary reward on later relearning. Nineteen healthy human participants learned object-location associations, for half of which we offered money. Six weeks later, most of these associations had been forgotten as measured by a test of declarative memory. Yet, relearning in the absence of any reward was faster for the originally rewarded associations. Thus, associative memories encoded in a state of monetary reward motivation may persist in a latent form despite the failure to retrieve them explicitly. Alternatively, such facilitation could be analogous to the renewal effect observed in animal conditioning, whereby a reward-associated cue can reinstate anticipatory arousal, which would in turn modulate relearning. This finding has important implications for learning and education, suggesting that even when learned information is no longer accessible via explicit retrieval, the enduring effects of a past prospect of reward could facilitate its recovery.

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.

A bout analysis of operant response disruption

Operant behavior appears to be organized in bouts of responses, whose parameters are differentially sensitive to various manipulations. This study investigated potential differential effects of three forms of operant response disruption-extinction (EXT), non-contingent reinforcement (NCR), and prefeeding (PRE)-on response bouts. In Experiment 1, Wistar Kyoto rats (WKY) were trained on a tandem variable-time (VT) 120s fixed-ratio (FR) 5 schedule of reinforcement; after stability was established, their responding was disrupted for three sessions with one of the three disrupters (EXT, NCR, or PRE). In Experiment 2, Long Evans (LE) rats were trained on a tandem VT 240s FR 5 to stability, and their responding disrupted with EXT or NCR. In EXT and NCR, response rates declined significantly and progressively over the course of the session, primarily due to a declining bout-initiation rate in EXT, and to fewer responses per bout in NCR. In contrast, a session-wide drop in response rate was observed in PRE, primarily due to a reduction in bout-initiation rate at the start of the session. These findings suggest that each form of disruption differentially impacts dissociable aspects of behavior. Theories of behavioral persistence should account for these functional relations, which appear to be obscured in response rate measures.

Extinction learning deficit in a rodent model of attention-deficit hyperactivity disorder

Background

Deficient operant extinction has been hypothesized to be constitutive of ADHD dysfunction. In order to elucidate the behavioral mechanisms underlying this deficit, the performance of an animal model of ADHD, the spontaneously hypertensive rat (SHR), was compared against the performance of a control strain, the Wistar-Kyoto rat (WKY) during extinction.

Method

Following extensive training of lever pressing under variable interval schedules of food reinforcement (reported previously), SHR and WKY rats were exposed to two sessions of extinction training. Extinction data was analyzed using the Dynamic Bi-Exponential Refractory Model (DBERM) of operant performance. DBERM assumes that operant responses are organized in bouts separated by pauses; during extinction, bouts may decline across multiple dimensions, including frequency and length. DBERM parameters were estimated using hierarchical Bayesian modeling.

Results

SHR responded more than WKY during the first extinction session. DBERM parameter estimates revealed that, at the onset of extinction, SHR produced more response bouts than WKY. Over the course of extinction, response bouts progressively shortened for WKY but not for SHR.

Conclusions

Based on prior findings on the sensitivity of DBERM parameters to motivational and schedule manipulations, present data suggests that (1) more frequent response bouts in SHR are likely related to greater incentive motivation, and (2) the persistent length of bouts in SHR are likely related to a slower updating of the response-outcome association. Overall, these findings suggest specific motivational and learning deficits that may explain ADHD-related impairments in operant performance.

Chronic stress impairs prefrontal cortex-dependent response inhibition and spatial working memory

Chronic stress leads to neurochemical and structural alterations in the prefrontal cortex (PFC) that correspond to deficits in PFC-mediated behaviors. The present study examined the effects of chronic restraint stress on response inhibition (using a response-withholding task, the fixed-minimum interval schedule of reinforcement, or FMI), and working memory (using a radial arm water maze, RAWM). Adult male Sprague-Dawley rats were first trained on the RAWM and subsequently trained on FMI. After acquisition of FMI, rats were assigned to a restraint stress (6h/d/28d in wire mesh restrainers) or control condition. Immediately after chronic stress, rats were tested on FMI and subsequently on RAWM. FMI results suggest that chronic stress reduces response inhibition capacity and motivation to initiate the task on selective conditions when sucrose reward was not obtained on the preceding trial. RAWM results suggest that chronic stress produces transient deficits in working memory without altering previously consolidated reference memory. Behavioral measures from FMI failed to correlate with metrics from RAWM except for one in which changes in FMI timing imprecision negatively correlated with changes in RAWM working memory errors for the controls, a finding that was not observed following chronic stress. Fisher's r-to-z transformation revealed no significant differences between control and stress groups with correlation coefficients. These findings are the first to show that chronic stress impairs both response inhibition and working memory, two behaviors that have never been directly compared within the same animals after chronic stress, using FMI, an appetitive task, and RAWM, a nonappetitive task.

Extinction under a behavioral microscope: isolating the sources of decline in operant response rate

Extinction performance is often used to assess underlying psychological processes without the interference of reinforcement. For example, in the extinction/reinstatement paradigm, motivation to seek drug is assessed by measuring responding elicited by drug-associated cues without drug reinforcement. However, extinction performance is governed by several psychological processes that involve motivation, memory, learning, and motoric functions. These processes are confounded when overall response rate is used to measure performance. Based on evidence that operant responding occurs in bouts, this paper proposes an analytic procedure that separates extinction performance into several behavioral components: (1-3) the baseline bout initiation rate, within-bout response rate, and bout length at the onset of extinction; (4-6) their rates of decay during extinction; (7) the time between extinction onset and the decline of responding; (8) the asymptotic response rate at the end of extinction; (9) the refractory period after each response. Data that illustrate the goodness of fit of this analytic model are presented. This paper also describes procedures to isolate behavioral components contributing to extinction performance and make inferences about experimental effects on these components. This microscopic behavioral analysis allows the mapping of different psychological processes to distinct behavioral components implicated in extinction performance, which may further our understanding of the psychological effects of neurobiological treatments.