Sensitivity to delay is affected by magnitude of reinforcement in rats

Effects of signaled and unsignaled delays of reinforcement on response maintenance in zebrafish (Danio rerio)

Effects of delays of reinforcement on zebrafish behavior were examined following training with immediate reinforcement. The delay was either signaled by an exteroceptive stimulus present for the entire delay period (fully signaled), signaled briefly only at delay onset (partial signal), or without a stimulus (unsignaled). Unsignaled delays consistently resulted in low response rates. Fully signaled delays resulted in higher response rates than unsignaled delays when the delay was 3 s, but this difference in response rates disappeared at 6-s delays. Partially signaled delays were less effective in maintaining responding than fully signaled delays, but more effective than unsignaled delays, although these results were only suggestive. These results indicate that stimulus changes that occur during delays to reinforcement have similar effects with zebrafish as with other species, but also that responding of zebrafish has a relatively low tolerance to the delay duration. A vast majority of experiments examining zebrafish behavior suggests that the fish have potential to serve as an interface between biological and behavioral science, but this may not be the case in some research areas involving delays, such as delay discounting.

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent-chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady-state, single-subject experimental designs can be time-consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent-chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within-session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

Zebrafish choice behavior is sensitive to reinforcer rate, immediacy, and magnitude ratios

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.

Analyzing the magnitude effect in spontaneously hypertensive (SHR) and wistar Kyoto (WKY) rats

We analyzed the magnitude effect in Spontaneously Hypertensive (SHR) and Wistar Kyoto (WKY) rats choosing between a smaller-sooner (SSF) and a larger-later food (LLF) in the initial link of a concurrent-chains procedure. The SSF was delivered immediately in one terminal link and the LLF delayed 0.01, 5, 10, 20, 40, or 80 s in the other terminal link. An ABABA design varied food amount, 1 vs. 4 and 3 vs. 8 food-pellets in conditions A and B, respectively. The SHRs made more impulsive choices than the WKYs. The hyperbolic-decay model and the Generalized Matching Law fitted the data well. Discounting rate (k) and the area under the discounting curve (AUC) for the choices made by the SHRs in conditions A, were like those in conditions B. For the choices that the individual WKYs made, k was slightly higher and AUC smaller in conditions B than in conditions A. For both strains sensitivity to the immediacy of the LLF (s) was slightly higher in conditions A than in conditions B. Thus, we found no conclusive, compelling evidence either supporting or discarding the magnitude effect in the SHRs and scarce evidence supporting an effect opposite to the magnitude effect in the WKYs.

Trait determinants of impulsive behavior: a comprehensive analysis of 188 rats

Impulsivity is a naturally occurring behavior that, when accentuated, can be found in a variety of neuropsychiatric disorders. The expression of trait impulsivity has been shown to change with a variety of factors, such as age and sex, but the existing literature does not reflect widespread consensus regarding the influence of modulating effects. We designed the present study to investigate, in a cohort of significant size (188 rats), the impact of four specific parameters, namely sex, age, strain and phase of estrous cycle, using the variable delay-to-signal (VDS) task. This cohort included (i) control animals from previous experiments; (ii) animals specifically raised for this study; and (iii) animals previously used for breeding purposes. Aging was associated with a general decrease in action impulsivity and an increase in delay tolerance. Females generally performed more impulsive actions than males but no differences were observed regarding delay intolerance. In terms of estrous cycle, no differences in impulsive behavior were observed and regarding strain, Wistar Han animals were, in general, more impulsive than Sprague-Dawley. In addition to further confirming, in a substantial study cohort, the decrease in impulsivity with age, we have demonstrated that both the strain and sex influences modulate different aspects of impulsive behavior manifestations.

Towards validation of delay discounting in the pigeon

A series of procedures were conducted in an attempt to assess various forms of validity related to the use of pigeons in research on delay discounting. In separate experimental arrangements, pigeons pressed a treadle, pecked a lit key, pecked a darkened key, or pecked a lit key with a hold as the required response. First, the obtained results were consistent with what would be necessary if the construct of delay discounting were being measured, which provides evidence of face validity. Second, criterion validity was assessed by comparing individual differences in rates of discounting across procedures. Third, to assess internal validity, each pigeon repeated the Treadle, Key Peck, and Dark Key procedures. Again, at both the aggregate and individual levels, the obtained indifference points did not differ systematically between replications. Finally, to assess external validity, discounting was observed regardless of the procedure, where the patterns of data at the aggregate level, and generally at the individual level, were orderly and well described by a hyperbolic function. In addition, rates of discounting were similar when pigeons pecked a lit key, a dark key, or a key with a hold; and each of those rates of discounting tended to be steeper than when treadle pressing. Generally speaking, pigeons that discounted relatively steeply on one procedure also tended to discount relatively steeply on the other procedures. The procedures evidence some of the necessary elements involved with the use of pigeons in research on delay discounting.

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Valor del Reforzador: ¿Cómo se Usa y Para qué se Usa el Concepto?

<p>La reseña muestra un panorama general de qué es el valor del reforzador, cómo se ha conceptualizado en la literatura y qué investigaciones han utilizado el concepto.  En un sentido general el concepto se utiliza para calificar un reforzador como más o menos efectivo: entre mayor valor del reforzador mayor su eficacia.  Primero se mostrará cómo se ha medido históricamente el valor del reforzador y cómo se ha definido a partir de la literatura sobre economía conductual.  Luego se mostrarán dos usos diferentes del concepto: 1). constructo hipotético, medible indirectamente a través de diferentes procedimientos, 2). variable interviniente, reúne los efectos de un conjunto de operaciones experimentales.  En el segundo conjunto también se incluyen definiciones operacionales, donde no se define exhaustivamente todas las variables dependientes e independientes asociadas, por lo que no es variable interviniente, pero tampoco agrega significado más allá del nivel de observación, por lo que no es constructo hipotético.  Luego se explora la relación entre demora del reforzador y descuento temporal, un tema de gran relevancia en la investigación contemporánea.  En las consideraciones finales se retoma la discusión sobre el valor heurístico y la conveniencia de usar el concepto de valor de reforzador.</p>

Delay discounting: Pigeon, rat, human--does it matter?

Delay discounting refers to the decrease in subjective value of an outcome as the time to its receipt increases. Across species and situations, animals discount delayed rewards, and their discounting is well-described by a hyperboloid function. The current review begins with a comparison of discounting models and the procedures used to assess delay discounting in nonhuman animals. We next discuss the generality of discounting, reviewing the effects of different variables on the degree of discounting delayed reinforcers by nonhuman animals. Despite the many similarities in discounting observed between human and nonhuman animals, several differences have been proposed (e.g., the magnitude effect; nonhuman animals discount over a matter of seconds whereas humans report willing to wait months, if not years before receiving a reward), raising the possibility of fundamental species differences in intertemporal choice. After evaluating these differences, we discuss delay discounting from an adaptationist perspective. The pervasiveness of discounting across species and situations suggests it is a fundamental process underlying decision making.

Impulsivity and sensitivity to amount and delay of reinforcement in an animal model of ADHD

Previous research has been inconclusive about the degree of impulsivity displayed by spontaneously hypertensive rats (SHR), an animal model of Attention Deficit Hyperactivity Disorder (ADHD). In the present set of experiments, concurrent-chains schedules were employed in order to explore SHR's impulsivity, sensitivity to delay, and sensitivity to amount of reinforcement; Wistar rats (WIS) were used as comparison group. In the three experiments - performed with different subjects - non-independent variable interval 30s schedules were presented in the initial links; the difference between experiments was in the terminal links. For exploring impulsivity, one of the terminal links (SS) was associated to a short delay (2s) and a small reinforcer (1 pellet), whereas the other terminal link (LL) was associated to a longer delay (28s) and a larger reinforcer (4 pellets). The results indicated a remarkably higher impulsivity in SHR. Because this impulsivity may have as potential mechanisms an increased sensitivity to delay and/or a decreased sensitivity to the amount of reinforcement, in experiments 2 and 3 these possibilities were examined. For assessing sensitivity to delay, the following pairs of fixed interval (FI) schedules were used in the terminal links in five conditions: 2-28, 6-24, 15-15, 24-6, 28-2s; the magnitude of reinforcement was 1 pellet in all conditions for both alternatives. For assessing sensitivity to amount, in five conditions the alternatives were associated with different magnitudes of reinforcement: 1-5 pellets, 2-4, 3-3, 4-2 and 5-1 in left-right alternatives, respectively; the delay to reinforcement was controlled by a FI 15s in all conditions and for both alternatives. The sensitivity to delay and the sensitivity to amount were calculated according to the Generalized Matching Law. The results indicated a higher sensitivity to delay in SHR, and the same sensitivity to amount in SHR and WIS rats. These results suggest that the increased sensitivity to delay influences the high level of impulsivity observed in SHR.

Rats are optimal in a choice task in which pigeons are not

In an extensive list of studies, it has been found that pigeons prefer an alternative associated with discriminative stimuli over another associated with non-discriminative stimuli, even when the probability of reinforcement is higher in the latter. This behavior has been named "suboptimal choice". In the present experiment, we evaluated whether rats, another widely studied species within the Experimental Analysis of Behavior, also shows this behavior. We systematically replicated the procedure employed with pigeons, and found that rats are not suboptimal, i.e., they prefer the non-discriminative alternative associated with .5 probability of reinforcement, over the discriminative alternative associated with .2 probability of reinforcement. This effect occurred even though rats discriminated the contingencies of reinforcement associated with each stimulus, suggesting that rats' optimal choice was driven by the overall probability of reinforcement of each alternative. Different procedural details are offered as possibilities for explaining this apparent inter-species difference.

Delay-specific stimuli and genotype interact to determine temporal discounting in a rapid-acquisition procedure

The importance of delay discounting to many socially important behavior problems has stimulated investigations of biological and environmental mechanisms responsible for variations in the form of the discount function. The extant experimental research, however, has yielded disparate results, raising important questions regarding Gene X Environment interactions. The present study determined the influence of stimuli that uniquely signal delays to reinforcement on delay discounting in two inbred mouse strains using a rapid-acquisition procedure. BALB/c and C57BL/6 mice responded under a six-component, concurrent-chained schedule in which the terminal-link delays preceding the larger-reinforcer were presented randomly across components of an individual session. Across conditions, components were presented either with or without delay-specific auditory stimuli, i.e., as multiple or mixed schedules. A generalized matching-based model was used to incorporate the impact of current and previous component reinforcer-delay ratios on current component response allocation. Sensitivity to reinforcer magnitude and delay were higher for BALB/c mice, but within-component preference reached final levels faster for C57Bl/6 mice. For BALB/c mice, acquisition of preference across blocks of a component was faster under the multiple than the mixed schedule, but final levels of sensitivity to reinforcement were unaffected by schedule. The speed of acquisition of preference was not different across schedules for C57Bl/6 mice, but sensitivity to reinforcement was higher under the multiple than the mixed schedule. Overall, differences in the acquisition and final form of the discount function were determined by a Gene X Environment interaction, but the presence of delay-specific stimuli attenuated genotype-dependent differences in magnitude and delay sensitivity.

Relatively high motivation for context-evoked reward produces the magnitude effect in rats

Using a concurrent-chain schedule, we demonstrated the effect of absolute reinforcement (i.e., the magnitude effect) on choice behavior in rats. In general, animals' simultaneous choices conform to a relative reinforcement ratio between alternatives. However, studies in pigeons and rats have found that on a concurrent-chain schedule, the overall reinforcement ratio, or absolute amount, also influences choice. The effect of reinforcement amount has also been studied in inter-temporal choice situations, and this effect has been referred to as the magnitude effect. The magnitude effect has been observed in humans under various conditions, but little research has assessed it in animals (e.g., pigeons and rats). The present study confirmed the effect of reinforcement amount in rats during simultaneous and inter-temporal choice situations. We used a concurrent-chain procedure to examine the cause of the magnitude effect during inter-temporal choice. Our results suggest that rats can use differences in reinforcement amount as a contextual cue during choice, and the direction of the magnitude effect in rats might be similar to humans when using the present procedure. Furthermore, our results indicate that the magnitude effect was caused by the initial-link effect when the reinforcement amount was relatively small, while a loss aversion tendency was observed when the reinforcement amount changed within a session. The emergence of the initial-link effect and loss aversion suggests that rats make choices through cognitive processes predicted by prospect theory.

Pigeons' delay discounting functions established using a concurrent-chains procedure

Several studies have examined discounting by pigeons and rats using concurrent-chains procedures, but the results have been inconsistent. None of these studies, however, has established that discounting functions derived from estimates of indifference points can be obtained with a concurrent-chains procedure, so their validity remains in doubt. The present study used a concurrent-chains procedure within sessions combined with an adjusting-amount procedure across sessions to determine the present, subjective values of food reinforcers to be obtained after a delay. Discounting was well described by the hyperbolic discounting function, suggesting that the concurrent-chains procedure and the more typical adjusting-amount procedure are measuring the same process. Consistent with previous studies with rats and pigeons using adjusting-amount procedures, no significant effect of the amount of the delayed reinforcer on the degree of discounting was observed, suggesting that the amount effect may be unique to humans although consistent with the view that animals' choices are controlled by the relative, rather than the absolute, value of reinforcers.