Operant models of relapse in zebrafish ( Danio rerio ): Resurgence, renewal, and reinstatement

Selective effects of conspecific movement on social preference in zebrafish (Danio rerio) using real-time 3D tracking and 3D animation

Zebrafish show social behavior such as shoaling and schooling, which is a result of complex and interdependent interactions among conspecifics. Zebrafish social behavior is interdependent in the sense that one fish's behavior affects both conspecific behavior and, as a result, their own behavior. Previous research examined effects of the interdependent interactions on the preference for social stimulus but lacked clear evidence that specific conspecific movements were reinforcing. The present research examined whether dependency between individual experimental fish's motion and a social-stimulus fish's motions contributes to preference for the social stimulus. In Experiment 1, a 3D animated stimulus fish either chased individual experimental fish or was motionless, serving as dependent and independent motions, respectively. In Experiment 2, the stimulus fish either chased experimental fish, moved away, or moved independently of the experimental fish. In both experiments, experimental fish spent more time near the stimulus fish showing dependent and interactive movements, indicating preference for dependent motion over independent motion, and chasing over other motions. Implications of these results are discussed including a potential role of operant conditioning in the preference for social stimuli.

ABA, AAB and ABC renewal with Pavlovian Conditioning of Tentacle Lowering procedure in the snail Cornu aspersum

This study assesses the recovery of the conditioned response (CR) due to a contextual change (renewal effect) in the Cornu aspersum, using the appetitive Pavlovian Conditioning of Tentacle Lowering procedure. Snails experienced an odorous conditioned stimulus (CS) paired with food (conditioning), followed by the exposition to the CS without any consequence (extinction). Then, they were exposed to the CS in a different context from the extinction one (renewal test). The contexts were three types of illumination. In Experiment 1a, the conditioning was performed in context A, the extinction was conducted in context B and the renewal test was performed in context A. For Experiment 1b, the conditioning and extinction were conducted in context A and renewal was performed in context B. In Experiment 1c, three dissimilar contexts were used for each experimental phase: context A for the conditioning, context B for the extinction and context C for the renewal. In Experiment 2, the renewal magnitude was compared among the three paradigms (ABA, AAB and ABC). Experiments 1a, 1b and 1c showed a recovery of the CR when subjects experienced a contextual change and Experiment 2 showed equivalent levels of renewal in the three paradigms. Learning processes and theories involved are discussed.

A Comprehensive Systematic Review of Procedures and Analyses Used in Basic and Preclinical Studies of Resurgence, 1970-2020

Resurgence is the return of a previously reinforced response as conditions worsen for an alternative response, such as the introduction of extinction, reductions in reinforcement, or punishment. As a procedure, resurgence has been used to model behavioral treatments and understand behavioral processes contributing both to relapse of problem behavior and flexibility during problem-solving. Identifying existing procedural and analytic methods arranged in basic/preclinical research could be used by basic and preclinical researchers to develop novel approaches to study resurgence, whereas translational and clinical researchers could identify potential approaches to combating relapse during behavioral interventions. Despite the study of resurgence for over half a century, there have been no systematic reviews of the basic/preclinical research on resurgence. To characterize the procedural and analytic methods used in basic/preclinical research on resurgence, we performed a systematic review consistent with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We identified 120 articles consisting of 200 experiments that presented novel empirical research, examined operant behavior, and included standard elements of a resurgence procedure. We reported prevalence and trends in over 60 categories, including participant characteristics (e.g., species, sample size, disability), designs (e.g., single subject, group), procedural characteristics (e.g., responses, reinforcer types, control conditions), criteria defining resurgence (e.g., single test, multiple tests, relative to control), and analytic strategies (e.g., inferential statistics, quantitative analysis, visual inspection). We make some recommendations for future basic, preclinical, and clinical research based on our findings of this expanding literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40614-022-00361-y.

Response effort and resurgence

This study provides an initial translational examination of response effort and resurgence. Eleven typically developing adults and five adolescents with autism served as participants across two experiments. Participants received points for touching moving stimuli on a computer screen. The resurgence evaluation consisted of three phases: establishment wherein R1 was reinforced, elimination wherein R1 was placed on extinction while R2 was reinforced, and extinction wherein R1 and R2 no longer resulted in reinforcement. Rate of R1 during extinction was compared across three conditions: intermediate, easy, and difficult. Disparity in effort was created by manipulations of the size and speed of objects that moved about on a computer screen. In Experiment 2, control stimuli were added to the experimental arrangement. Across the two experiments, the magnitude of resurgence was greater when R1 was easy. In Experiment 2, both R1 and control responding were greater in the extinction phase than in the elimination phase in all conditions with all participants. The present study supports the hypothesis that response effort affects resurgence and that less effortful responses are likely to recur with greater magnitude under conditions that produce resurgence than are their more effortful counterparts.

Assessing human performance during contingency changes and extinction tests in reversal-learning tasks

Serial reversal-learning procedures are simple preparations that allow for a better understanding of how animals learn about environmental changes, including flexibly shifting responding to adapt to changing reinforcement contingencies. The present study examined serial reversal learning with humans by arranging both midsession and variable contingency reversals across two experiments. We also examined the effects of extinction by adding nonreinforced trials at the end of later sessions and provided the first evaluation of effects of win-stay/lose-shift versus counting strategies on accuracy and response latency of humans' reversal-learning performance. In each experiment, responding tracked contingency reversals, primarily with participants using either win-stay/lose-shift or counting strategies. Introducing variable reversal points in the second experiment resulted in near-exclusive win-stay/lose-shift responding among participants and eliminated counting of trials. Each experiment also revealed an immediate shift from S2 to S1 after experiencing extinction during the initial test trial, indicating resurgence of the initial response through a win-stay/lose-shift response pattern. Therefore, the present study replicates and extends prior findings of a win-stay/lose shift response pattern in situations of greater uncertainty. These findings suggest that differences in environmental certainty induce qualitatively different decision-making strategies.

Effects of signaled and unsignaled delays of reinforcement on response maintenance in zebrafish (Danio rerio)

Effects of delays of reinforcement on zebrafish behavior were examined following training with immediate reinforcement. The delay was either signaled by an exteroceptive stimulus present for the entire delay period (fully signaled), signaled briefly only at delay onset (partial signal), or without a stimulus (unsignaled). Unsignaled delays consistently resulted in low response rates. Fully signaled delays resulted in higher response rates than unsignaled delays when the delay was 3 s, but this difference in response rates disappeared at 6-s delays. Partially signaled delays were less effective in maintaining responding than fully signaled delays, but more effective than unsignaled delays, although these results were only suggestive. These results indicate that stimulus changes that occur during delays to reinforcement have similar effects with zebrafish as with other species, but also that responding of zebrafish has a relatively low tolerance to the delay duration. A vast majority of experiments examining zebrafish behavior suggests that the fish have potential to serve as an interface between biological and behavioral science, but this may not be the case in some research areas involving delays, such as delay discounting.

Examining effects of training duration on humans' resurgence and variability using a novel touchscreen procedure

Resurgence occurs when a previously reinforced and then extinguished target response increases due to reducing/eliminating an alternative source of reinforcement or punishing an alternative response. We evaluated whether duration of reinforcement history for a target response (1) affects the degree to which resurgence is observed in humans and (2) produces different gradients of response generalization around target responding during extinction testing. We arranged a novel touchscreen interface in which university students could swipe a 3D soccer ball to spin any direction. In Phase 1, the first direction swiped became the target and produced points exchangeable for money for 3 or 1 min across 2 groups. The first swipe was recorded but had no programmed consequence in a third group. In Phase 2, swipes 180-degrees from the target resulted in points for 3 min in all groups. Point deliveries ceased for 2 min to test for resurgence in Phase 3. Target responses resurged during testing to a relatively greater extent with longer Phase-1 training but gradients of response generalization did not differ among groups. These findings extend prior research on the role of training duration on resurgence. We discuss methodological and conceptual issues surrounding the assessment of response generalization in resurgence.

Recovery-from-extinction effects in an anuran amphibian: renewal effect, but no reinstatement

Recovery-from-extinction effects in which a conditioned response returns after extinction have been shown in mammals, birds and fish. Thus, these effects appear to be conserved among vertebrates; however, they have yet to be investigated in amphibians. Using prey catching conditioning in the fire-bellied toad (Bombina orientalis), we tested if renewal and reinstatement occurred after extinction when subjects were respectively re-exposed to the context or reinforcer used during conditioning. For renewal, a different context was used during extinction and thus renewal tests assessed if external contextual cues associated during conditioning stimulated prey catching performance. For reinstatement, the reinforcer withheld during extinction was simply delivered again prior to a test assessing if internal cues associated with recent prey consumption stimulated prey catching performance. Conditioning followed a fixed ratio 5 schedule of reinforcement, where five attempts to capture a cricket stimulus displayed on a computer screen were reinforced by delivery of a single live cricket. Performance was measured as the time to reach five prey catching attempts. A significant improvement in prey catching performance during conditioning followed by deterioration with extinction was seen in the experiments. Upon return to the context used for conditioning after extinction, toads showed a renewal effect whereby they displayed faster performance during testing compared to the end of extinction. Conversely, toads showed no reinstatement effect because pre-feeding of a cricket did not influence performance during the test that followed extinction. Reinstatement could have been lost in amphibian phylogeny due to secondary simplification of the nervous system.

Zebrafish choice behavior is sensitive to reinforcer rate, immediacy, and magnitude ratios

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.

Evaluating Extinction, Renewal, and Resurgence of Operant Behavior in Humans with Amazon Mechanical Turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace providing researchers with the opportunity to collect behavioral data from remote participants at a low cost. Recent research demonstrated reliable extinction effects, as well as renewal and resurgence of button pressing with MTurk participants. To further examine the generality of these findings, we replicated and extended these methods across six experiments arranging reinforcement and extinction of a target button press. In contrast to previous findings, we did not observe as reliable of decreases in button pressing during extinction (1) after training with VR or VI schedules of reinforcement, (2) in the presence or absence of context changes, or (3) with an added response cost for button pressing. However, we found that that a 1-point response cost for all button presses facilitated extinction to a greater extent than the absence of response cost. Nevertheless, we observed ABA renewal of button pressing when changing background contexts across phases and resurgence when extinguishing presses on an alternative button. Our findings suggest that MTurk could be a viable platform from which to ask and address questions about extinction and relapse processes, but further procedural refinements will be necessary to improve the replicability of control by experimental contingencies.

The devil is in the detail: Zebrafish learn to discriminate visual stimuli only if salient

Due to their unique characteristics, the zebrafish plays a key role in the comprehension of neurobiology of cognition and its pathologies, such as neurodegenerative diseases. More and more molecular tools for this aim are being developed, but our knowledge about the cognitive abilities of zebrafish remains extremely scarce compared to other teleost fish. We aimed to investigate the complex cognitive abilities of zebrafish using a tracking-based automated conditioning chamber that allowed precise experimental control, avoided potential cueing provided by the observer (Clever Hans phenomenon), and was shown to considerably improve learning in other teleosts. A computer presented two visual stimuli in two sectors of the chamber, and zebrafish had to enter the correct sector to obtain a food reward. Zebrafish quickly learned to use the conditioning device and easily performed up to 80 trials per day. In Experiment 1, zebrafish efficiently discriminated between two differently coloured sides, reaching a 75 % accuracy in only 10 training sessions. Surprisingly, zebrafish failed to choose the correct chamber when the stimuli were two shapes, a small circle and a small triangle, even when, in Experiment 2, training on shape discrimination was prolonged for up to 30 sessions. In Experiment 3, we tested the hypothesis that simultaneously learning to use the conditioning chamber and learning discrimination imposes a too-high cognitive load. However, zebrafish that first successfully learned how the conditioning chamber functioned (in the colour discrimination) subsequently failed in the shape discrimination. Conversely, zebrafish that firstly failed the shape discrimination subsequently learned colour discrimination. In Experiment 4, zebrafish showed some evidence of learning when the stimuli were two large shapes, suggesting that zebrafish did not discriminate between the shapes of the previous experiments because they were not salient enough. Altogether, results suggest constraints in the discrimination learning abilities of zebrafish, which should be taken into account when developing cognitive tasks for this species.

An experimental analysis of jackpot reinforcers

Animal trainers and others often recommend the use of jackpot reinforcers, which are disproportionally large and come as a "surprise" to the animals. Because the actual behavioral effects of these jackpots remain uninvestigated, many basic questions about their use and even definition are unanswered. This series of experiments explored the definitions of jackpots using several different behavioral tests with both rats and pigeons. Because the original description of a jackpot resembled the reinstatement of previously reinforced and extinguished responding with response-independent deliveries of reinforcer, reinstatement effects of a jackpot, defined by its quality, were examined with rats in Experiment 1. In Experiment 2, response-potentiating effects of response-independent and -dependent deliveries of a jackpot, defined by its quantity, were assessed with pigeons when responding had nearly ceased. The response-potentiating effects of the frequency of jackpot-reinforcer delivery were investigated when responding of pigeons was maintained in single (Experiment 3) or concurrent (Experiment 4) schedules of reinforcement. Effects of jackpots on resistance to change were assessed with rats in Experiment 5. The effects of jackpots in each experiment were either absent or unsystematic across the subjects, casting doubt on their utility in animal training. Possible factors contributing to the negative results are discussed.

Repeated resurgence with and without a context change

Resurgence is the reoccurrence of a target response when reinforcement for a more recently reinforced alternative response is eliminated or reduced. The present study arranged two successive three-phase procedures to assess whether resurgence decreases with repeated assessments. Moreover, we arranged a contextual change from the first to second assessment for some groups. Phase 1 reinforced a target response on a touchscreen computer with typically developing adults as participants according to either variable-ratio or variable-interval schedules of reinforcement. Phase 2 extinguished target responding and reinforced alternative responding. Phase 3 tested for resurgence by extinguishing alternative responding. Resurgence reliably occurred in all tests and decreased from the first to second exposure to the procedures but there were no effects of context change. Therefore, repeated exposures to resurgence tests reduced those effects but contextual changes had no effect.

A psychophysical approach to measuring the threshold for acoustic stimulation in zebrafish (Danio rerio)

Zebrafish are a popular vertebrate animal model for biomedical research including investigations of the auditory system. Responses to acoustic stimulation have been a challenge to carefully measure in zebrafish. Here, the authors have developed a procedure for measuring hearing sensitivity in adult zebrafish using an appetitive automated Go/No Go task. In this task, a trial is initiated when a fish passes through an observing gate. In a sound trial, the fish is reinforced by an automated food delivery system when it enters the reinforcement compartment. If the fish enters the reinforcement compartment during a no-sound trial, a timeout is implemented. Zebrafish successfully learned this task in a median of about ten days of daily training. Zebrafish were most sensitive at a frequency of 800 Hz, which corresponds well with sensitivity reported from physiological methods. As far as the authors know, the present study is the first to provide hearing thresholds for zebrafish using a conventional combination of operant conditioning and psychophysical procedures. This could open the door to other kinds of tests using acoustic stimuli as are commonly conducted in many other laboratory animals.

Renewal of fear and avoidance in humans to escalating threat: Implications for translational research on anxiety disorders

Exposure-based treatment for threat avoidance in anxiety disorders often results in fear renewal. However, little is known about renewal of avoidance. This multimodal laboratory-based treatment study used an ABA renewal design and an approach-avoidance (AP-AV) task to examine renewal of fear/threat and avoidance in twenty adults. In Context A, 9 visual cues paired with increases in probabilistic money loss (escalating threats) produced increases in ratings of feeling threatened and loss expectancies and skin-conductance responses (SCR). During the AP-AV task, a monetary reinforcer was available concurrently with threats. Approach produced the reinforcer or probabilistic loss, while avoidance prevented loss and forfeited reinforcement. Escalating threat produced increasing avoidance and ratings. In Context B with Pavlovian extinction, threats signaled no money loss and SCR declined. During the AP-AV task, avoidance and ratings also declined. In a return to Context A with Pavlovian threat extinction in effect during the AP-AV task, renewal was observed. Escalating threat was associated with increasing ratings and avoidance in most participants. SCR did not show renewal. These are the first translational findings to highlight renewal of avoidance in humans. Further research should identify individual difference variables and altered neural mechanisms that may confer increased risk of avoidance renewal.

Nonhuman animal models of substance use disorders: Translational value and utility to basic science

BACKGROUND

The National Institute on Drug Abuse (NIDA) recently released a Request for Information (RFI) soliciting comments on nonhuman animal models of substance use disorders (SUD).

METHODS

A literature review was performed to address the four topics outlined in the RFI and one topic inspired by the RFI: (1) animal models that best recapitulate SUD, (2) animal models that best balance the trade-offs between resources and ecological validity, (3) animal models whose translational value are frequently misrepresented or overrepresented by the scientific community, (4) aspects of SUD that are not currently being modeled in animals, and (5) animal models that are optimal for examining the basic mechanisms by which drugs produce their abuse-related effects.

RESULTS

Models that employ response-contingent drug administration, use complex schedules of reinforcement, measure behaviors that mimic the distinguishing features of SUD, and use animals that are phylogenetically similar to humans have the greatest translational value. Models that produce stable and reproducible baselines of behavior, lessen the number of uncontrolled variables, and minimize the influence of extraneous factors are best at examining basic mechanisms contributing to drug reward and reinforcement.

CONCLUSIONS

Nonhuman animal models of SUD have undergone significant refinements to increase their utility for basic science and translational value for SUD. The existing literature describes numerous examples of how these models may best be utilized to answer mechanistic questions of drug reward and identify potential therapeutic interventions for SUD. Progress in the field could be accelerated by further collaborations between researchers using animals versus humans.

Resurgence when challenging alternative behavior with progressive ratios in children and pigeons

Resurgence is defined as the recurrence of a previously reinforced and then extinguished target response when reducing or eliminating a more recently reinforced alternative response. In experiments with children and pigeons, we evaluated patterns of resurgence across and within sessions through decreases in reinforcer availability by challenging alternative responding with extinction and progressive-ratio schedules. In Phase 1, we reinforced only target responding. In Phase 2, we extinguished target responding while reinforcing an alternative response. Finally, Phase 3 assessed resurgence by (a) extinguishing alternative responding versus (b) introducing a progressive-ratio schedule of reinforcement for alternative responding. In both children and pigeons, resurgence of target responding occurred in both conditions but generally was greater when assessed during extinction than with progressive ratios. Importantly, within-session patterns of resurgence did not differ between testing with progressive ratios and extinction. Resurgence with progressive ratios tended to be greater with longer durations between reinforcers but we observed similar findings with only simulated reinforcers during extinction testing. Therefore, the present investigation reveals that the events contributing to instances of resurgence remain to be understood, and presents an approach from which to examine variables influencing within-session patterns of resurgence.

A system for the real-time tracking of operant behavior as an application of 3D camera

The capacity of 3D cameras to measure many different aspects of behavior (e.g., velocity, pattern, and posture) could contribute to the understanding of behavior. The present article describes a system for the real-time tracking of operant behavior, which is applicable to other domains of behavioral science as well. Methods for real-time 3D tracking of animal behavior are described, along with sample C++ programs. A demonstration using one zebrafish as a subject indicated that the present system successfully tracked the 3D motion of the fish. Moreover, the acquisition of a target response (i.e., approach to a corner of the aquarium) was demonstrated with the arrangement of a reinforcement contingency at the corner in the absence of a traditional, salient operandum. The system offers the capacity to characterize more completely ongoing behavior in learning tasks across a range of species than simply performance of discrete operant responses. The system also is capable of tracking multiple individuals simultaneously so it is possible both to study social interactions and arrange contingencies for engaging in social behavior. Other possible applications of 3D cameras are discussed.

Resurgence with and without an alternative response

Resurgence is the reemergence of a previously reinforced response that occurs after the elimination or reduction of reinforcement for an alternative response. Resurgence is problematic in the context of treatment because the reemergence of a previously reinforced destructive response could be detrimental to treatment gains. In the current translational study, we examined a modified resurgence procedure in which the alternative response was either present or absent during extinction. Four participants were exposed to three phases that consisted of (1) reinforcement of a target response, (2) extinction of the target response and differential reinforcement of an alternative response, and (3) extinction of both responses. Results for four out of five assessments showed greater resurgence when the alternative response was absent during Phase 3. Results suggest that more robust resurgence might occur if the alternative response is not available as opposed to the alternative response contacting extinction.

Response acquisition by zebrafish (Danio rerio) with delayed reinforcement

Zebrafish (Danio rerio) is a common vertebrate animal model in biomedical research and is a promising species for studying how genes interact with environmental factors in determining behavior. The present study investigated how reinforcement parameters affect zebrafish behavior by assessing response acquisition with delayed reinforcement, which has been studied with other species (e.g., rats, pigeons, humans, etc.) but not with zebrafish. Twenty-four experimentally naïve subjects were exposed to a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior x-s schedule of reinforcement, where x varied across subjects. There were six different delay-to-reinforcement durations and sets of four fish were assigned to each delay duration. All of the fish assigned to a 0-, 0.5-, or 1-s delay acquired responding. Two fish acquired responding with a 3-s delay and one fish appeared to have acquired it with a 6-s delay although the latter result was less clear. None acquired responding with a 12-s delay. These results suggest that zebrafish behavior is sensitive to delays to reinforcement and the time frame over which reinforcement is effective may be limited approximately to 6 s. This time frame is shorter than that found with other species. Practical and theoretical implications of the present finding are discussed.