Every reinforcer counts: reinforcer magnitude and local preference

A contextualized reinforcer pathology approach to addiction

Behavioural economic accounts of addiction conceptualize harmful drug use as an operant reinforcer pathology, emphasizing that a drug is consumed because of overvaluation of smaller immediate rewards relative to larger delayed rewards (delay discounting) and high drug reinforcing value (drug demand). These motivational processes are within-individual determinants of behaviour. A third element of learning theory posits that harmful drug use depends on the relative constraints on access to other available activities and commodities in the choice context (alternative reinforcers), reflecting the substantial influence of environmental factors. In this Perspective, we integrate alternative reinforcers into the contemporary behavioural economic account of harmful drug use - the contextualized reinforcer pathology model - and review empirical literature across the translational spectrum in support of this model. Furthermore, we consider how increases in drug-related mortality and health disparities in addiction can be understood and potentially ameliorated via a contextualized reinforcer pathology model in which lack of alternative reinforcement is a major risk factor for addiction.

Choice and rate-amount independence in a titration procedure

The generalized matching law or Law of Allocation proposed by Baum (2018a, 2018b) potentially provides a broad conceptual framework within which to understand the allocation of time among activities. In its simplest form, the law incorporates power-function induction of activities by variables such as rate and amount of delivered inducers. Whether these variables affect allocation independently of one another is a central issue, because independence of the variables would allow simple multiplication of power functions and would make quantitative prediction simple too. The present experiment used a titration procedure to test the independence of rate and amount of food in determining pigeons' allocation of pecking between two keys. Amount ratio was varied within sessions to engender different peck ratios. Rate ratio was varied across two series of conditions. The results conformed to the predictions of the simple version of the Law of Allocation by strongly supporting independence of rate and amount. The Law of Allocation may have broad application for understanding activities in natural settings and everyday life.

Machine learning with a snapshot of data: Spiking neural network 'predicts' reinforcement histories of pigeons' choice behavior

An accumulated body of choice research has demonstrated that choice behavior can be understood within the context of its history of reinforcement by measuring response patterns. Traditionally, work on predicting choice behaviors has been based on the relationship between the history of reinforcement—the reinforcer arrangement used in training conditions—and choice behavior. We suggest an alternative method that treats the reinforcement history as unknown and focuses only on operant choices to accurately predict (more precisely, retrodict) reinforcement histories. We trained machine learning models known as artificial spiking neural networks (SNNs) on previously published pigeon datasets to detect patterns in choices with specific reinforcement histories—seven arranged concurrent variable‐interval schedules in effect for nine reinforcers. Notably, SNN extracted information from a small ‘window’ of observational data to predict reinforcer arrangements. The models' generalization ability was then tested with new choices of the same pigeons to predict the type of schedule used in training. We examined whether the amount of the data provided affected the prediction accuracy and our results demonstrated that choices made by the pigeons immediately after the delivery of reinforcers provided sufficient information for the model to determine the reinforcement history. These results support the idea that SNNs can process small sets of behavioral data for pattern detection, when the reinforcement history is unknown. This novel approach can influence our decisions to determine appropriate interventions; it can be a valuable addition to our toolbox, for both therapy design and research.

Transitive inference after minimal training in rhesus macaques (Macaca mulatta)

Rhesus macaques, when trained for several hundred trials on adjacent items in an ordered list (e.g., A > B, B > C, C > D), are able to make accurate transitive inferences (TI) about previously untrained pairs (e.g., A > C, B > D). How that learning unfolds during training, however, is not well understood. We sought to measure the relationship between the amount of TI training and the resulting response accuracy in 4 rhesus macaques using seven-item lists. The training conditions included the absolute minimal case of presenting each of the six adjacent pairs only once prior to testing. We also tested transfer to nonadjacent pairs with 24 and 114 training trials. Because performance during and after small amounts of training is expected to be near chance levels, we developed a descriptive statistical model to estimate potentially subtle learning effects in the presence of much larger random response variability and systematic bias. These results suggest that subjects learned serial order in an incremental fashion. Thus, rather than performing transitive inference by a logical process, serial learning in rhesus macaques proceeds in a manner more akin to a statistical inference, with an initial uncertainty about list position that gradually becomes more accurate as evidence accumulates. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Analyzing the magnitude effect in spontaneously hypertensive (SHR) and wistar Kyoto (WKY) rats

We analyzed the magnitude effect in Spontaneously Hypertensive (SHR) and Wistar Kyoto (WKY) rats choosing between a smaller-sooner (SSF) and a larger-later food (LLF) in the initial link of a concurrent-chains procedure. The SSF was delivered immediately in one terminal link and the LLF delayed 0.01, 5, 10, 20, 40, or 80 s in the other terminal link. An ABABA design varied food amount, 1 vs. 4 and 3 vs. 8 food-pellets in conditions A and B, respectively. The SHRs made more impulsive choices than the WKYs. The hyperbolic-decay model and the Generalized Matching Law fitted the data well. Discounting rate (k) and the area under the discounting curve (AUC) for the choices made by the SHRs in conditions A, were like those in conditions B. For the choices that the individual WKYs made, k was slightly higher and AUC smaller in conditions B than in conditions A. For both strains sensitivity to the immediacy of the LLF (s) was slightly higher in conditions A than in conditions B. Thus, we found no conclusive, compelling evidence either supporting or discarding the magnitude effect in the SHRs and scarce evidence supporting an effect opposite to the magnitude effect in the WKYs.

Measurement of substance-free reinforcement in addiction: A systematic review

A robust body of theoretical and experimental work highlights the influence of alternative, substance-free rewards on decisions to use alcohol and other drugs. However, translational applications have been limited in part by the lack of consensus on how to measure substance-free reinforcement in applied and clinical settings. The current study summarizes extant research utilizing self-report reinforcement or reward methodologies, and critically reviews the psychometric properties of the available measures. These studies (N = 50) fell into three categories: measures of recent substance-related and substance-free activity participation and enjoyment (n = 32), measures of time or monetary resource allocation (n = 15), and rating scale measures of reward availability and experience (n = 8). The available research suggests that, consistent with experimental laboratory research and with behavioral economic predictions, there is an inverse relation between substance-free reinforcement and substance use. These studies also support the clinical utility of these measures in predicting substance use severity and course. Reinforcement measures could be improved by enhancing content validity, multimethod convergent validity, and generalizability.

Spontaneously hypertensive (SHR) rats choose more impulsively than Wistar-Kyoto (WKY) rats on a delay discounting task

Indications of Attention Deficit Hyperactivity Disorder (ADHD) in the spontaneously hypertensive rat (SHR) are not consistent across different tests of impulsivity, questioning the SHR's validity as a rodent model of ADHD. This study used a concurrent-chains procedure to examine possible differences in impulsive choice between SHRs and control-normotensive Wistar Kyoto (WKY) rats. The aim was to extend the generality of findings showing regularities between the hyperbolic-decay model and the generalized matching law fitting delay discounting data from nonhuman animals. The objectives were to: (1) examine differences in impulsive choice between SHRs and WKYs; (2) add evidence suggesting that the SHR is a suitable model of ADHD; (3) demonstrate that concurrent-chains procedures requiring locomotion detect differences in impulsive choice between SHRs and WKYs; (4) support the idea that impulsivity in nonhuman animals increases with training. The initial link used two non-independent random interval schedules arranging entries to the terminal links, where one fixed-time (FT) schedule delayed 1-food pellet and the other FT 4-food pellets. The FT delaying the former was kept constant at 0.1s and that delaying the latter changed after every 10 food deliveries, defining six delay components (0.1, 5, 10, 20, 40, and 80s) presented in random order each session. Results showed that the SHRs choose more impulsively than the WKYs, adding to the body of evidence suggesting that the SHR may be a suitable model of ADHD. Both models of choice fitted the impulsive choices of the SHRs and WKYs well; positive correlations between estimates of parameters k and s suggested compatibility between models of choice showing that impulsivity increases with training.

Delay-specific stimuli and genotype interact to determine temporal discounting in a rapid-acquisition procedure

The importance of delay discounting to many socially important behavior problems has stimulated investigations of biological and environmental mechanisms responsible for variations in the form of the discount function. The extant experimental research, however, has yielded disparate results, raising important questions regarding Gene X Environment interactions. The present study determined the influence of stimuli that uniquely signal delays to reinforcement on delay discounting in two inbred mouse strains using a rapid-acquisition procedure. BALB/c and C57BL/6 mice responded under a six-component, concurrent-chained schedule in which the terminal-link delays preceding the larger-reinforcer were presented randomly across components of an individual session. Across conditions, components were presented either with or without delay-specific auditory stimuli, i.e., as multiple or mixed schedules. A generalized matching-based model was used to incorporate the impact of current and previous component reinforcer-delay ratios on current component response allocation. Sensitivity to reinforcer magnitude and delay were higher for BALB/c mice, but within-component preference reached final levels faster for C57Bl/6 mice. For BALB/c mice, acquisition of preference across blocks of a component was faster under the multiple than the mixed schedule, but final levels of sensitivity to reinforcement were unaffected by schedule. The speed of acquisition of preference was not different across schedules for C57Bl/6 mice, but sensitivity to reinforcement was higher under the multiple than the mixed schedule. Overall, differences in the acquisition and final form of the discount function were determined by a Gene X Environment interaction, but the presence of delay-specific stimuli attenuated genotype-dependent differences in magnitude and delay sensitivity.

Pigeons' use of cues in a repeated five-trial-sequence, single-reversal task

We studied behavioral flexibility, or the ability to modify one's behavior in accordance with the changing environment, in pigeons using a reversal-learning paradigm. In two experiments, each session consisted of a series of five-trial sequences involving a simple simultaneous color discrimination in which a reversal could occur during each sequence. The ideal strategy would be to start each sequence with a choice of S1 (the first correct stimulus) until it was no longer correct, and then to switch to S2 (the second correct stimulus), thus utilizing cues provided by local reinforcement (feedback from the preceding trial). In both experiments, subjects showed little evidence of using local reinforcement cues, but instead used the mean probabilities of reinforcement for S1 and S2 on each trial within each sequence. That is, subjects showed remarkably similar behavior, regardless of where (or, in Exp. 2, whether) a reversal occurred during a given sequence. Therefore, subjects appeared to be relatively insensitive to the consequences of responses (local feedback) and were not able to maximize reinforcement. The fact that pigeons did not use the more optimal feedback afforded by recent reinforcement contingencies to maximize their reinforcement has implications for their use of flexible response strategies under reversal-learning conditions.

Information: theory, brain, and behavior

In the 65 years since its formal specification, information theory has become an established statistical paradigm, providing powerful tools for quantifying probabilistic relationships. Behavior analysis has begun to adopt these tools as a novel means of measuring the interrelations between behavior, stimuli, and contingent outcomes. This approach holds great promise for making more precise determinations about the causes of behavior and the forms in which conditioning may be encoded by organisms. In addition to providing an introduction to the basics of information theory, we review some of the ways that information theory has informed the studies of Pavlovian conditioning, operant conditioning, and behavioral neuroscience. In addition to enriching each of these empirical domains, information theory has the potential to act as a common statistical framework by which results from different domains may be integrated, compared, and ultimately unified.

Computational model of selection by consequences: patterns of preference change on concurrent schedules

The computational model of selection by consequences is an ontogenetic dynamic account of adaptive behavior based on the Darwinian principle of selection by consequences. The model is a virtual organism based on a genetic algorithm, a class of computational algorithms that instantiate the principles of selection, fitness, reproduction and mutation. The computational model has been thoroughly tested in experiments with a variety of single alternative and concurrent schedules. A number of published reports demonstrate that the model generates patterns of behavior that are quantitatively equivalent to the findings from live organisms. The experiments and analyses in this study assess the behavior of the computational model for evidence of preference change phenomena in environments with rapidly changing reinforcement rate ratios. Molar and molecular effects of behavioral adjustment were consistent with those observed in live organisms. The results of this study provide strong evidence supporting the selectionist account of adaptive behavior.

Selection dynamics in joint matching to rate and magnitude of reinforcement

Virtual organisms animated by a selectionist theory of behavior dynamics worked on concurrent random interval schedules where both the rate and magnitude of reinforcement were varied. The selectionist theory consists of a set of simple rules of selection, recombination, and mutation that act on a population of potential behaviors by means of a genetic algorithm. An extension of the power function matching equation, which expresses behavior allocation as a joint function of exponentiated reinforcement rate and reinforcer magnitude ratios, was fitted to the virtual organisms' data, and over a range of moderate mutation rates was found to provide an excellent description of their behavior without residual trends. The mean exponents in this range of mutation rates were 0.83 for the reinforcement rate ratio and 0.68 for the reinforcer magnitude ratio, which are values that are comparable to those obtained in experiments with live organisms. These findings add to the evidence supporting the selectionist theory, which asserts that the world of behavior we observe and measure is created by evolutionary dynamics.

Rapid acquisition of bias in signal detection: dynamics of effective reinforcement allocation

We investigated changes in bias (preference for one response alternative) in signal detection when relative reinforcer frequency for correct responses varied across sessions. In Experiment 1, 4 rats responded in a two-stimulus, two-response identification procedure employing temporal stimuli (short vs. long houselight presentations). Relative reinforcer frequency varied according to a 31-step pseudorandom binary sequence and stimulus duration difference varied over two values across conditions. In Experiment 2, 3 rats responded in a five-stimulus, two-response classification procedure employing temporal stimuli. Relative reinforcer frequency was varied according to a 36-step pseudorandom ternary sequence. Results of both experiments were analyzed according to a behavioral model of detection. The model was extended to incorporate the effects of current and previous session reinforcer frequency ratios on current-session performance. Similar to findings with concurrent schedules, effects on bias of relative reinforcer frequency were highest for the current session. However, carryover from reinforcer ratios of previous sessions was evident. Generally, the results indicate that bias can come under control of frequent changes in relative reinforcer frequency in both identification and classification procedures.

Development and maintenance of choice in a dynamic environment

Four pigeons were exposed to a concurrent procedure similar to that used by Davison, Baum, and colleagues (e.g., Davison & Baum, 2000, 2006) in which seven components were arranged in a mixed schedule, and each programmed a different left∶right reinforcer ratio (1∶27, 1∶9, 1∶3, 1∶1, 3∶1, 9∶1, 27∶1). Components within each session were presented randomly, lasted for 10 reinforcers each, and were separated by 10-s blackouts. These conditions were in effect for 100 sessions. When data were aggregated over Sessions 16-50, the present results were similar to those reported by Davison, Baum, and colleagues: (a) preference adjusted rapidly (i.e., sensitivity to reinforcement increased) within components; (b) preference for a given alternative increased with successive reinforcers delivered via that alternative (continuations), but was substantially attenuated following a reinforcer on the other alternative (a discontinuation); and (c) food deliveries produced preference pulses (immediate, local, increases in preference for the just-reinforced alternative). The same analyses were conducted across 10-session blocks for Sessions 1-100. In general, the basic structure of choice revealed by analyses of data from Sessions 16-50 was preserved at a smaller level of aggregation (10 sessions), and it developed rapidly (within the first 10 sessions). Some characteristics of choice, however, changed systematically across sessions. For example, effects of successive reinforcers within a component tended to increase across sessions, as did the magnitude and length of the preference pulses. Thus, models of choice under these conditions may need to take into account variations in behavior allocation that are not captured completely when data are aggregated over large numbers of sessions.

Dynamics of choice: a tutorial

Choice may be defined as the allocation of behavior among activities. Since all activities take up time, choice is conveniently thought of as the allocation of time among activities, even if activities like pecking are most easily measured by counting. Since dynamics refers to change through time, the dynamics of choice refers to change of allocation through time. In the dynamics of choice, as in other dynamical systems that include feedback, change is away from perturbation and toward a steady state. Steady state or equilibrium is assessed on a longer time scale than change because change is only visible on a smaller time scale. When we compare laws of equilibrium, such as the matching law with laws of dynamics, two possibilities emerge. Self-similarity occurs when the same law can be seen across smaller time scales, with the result that the law at longer time scales may be understood as the expression of its application at smaller time scales. Reduction occurs when the dynamics at a small time scale are incommensurate with the dynamics at longer time scales. Then the process at the longer time scale is reduced to a qualitatively different process at the smaller time scale, as when choice is reduced to switching patterns. When reduction occurs, the dynamics at the longer time scale may be derived from the process at the smaller time scale, but not the other way around. Research at different time scales is facilitated by the molar view of behavior.

Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid

Behavior such as depression of a lever or perception of a stimulus may be strengthened by consequent behaviorally significant events (BSEs), such as reinforcers. This is the Law of Effect. As time passes since its emission, the ability for the behavior to be reinforced decreases. This is trace decay. It is upon decayed traces that subsequent BSEs operate. If the trace comes from a response, it constitutes primary reinforcement; if from perception of an extended stimulus, it is classical conditioning. This paper develops simple models of these processes. It premises exponentially decaying traces related to the richness of the environment, and conditioned reinforcement as the average of such traces over the extended stimulus, yielding an almost-hyperbolic function of duration. The models account for some data, and reinforce the theories of other analysts by providing a sufficient account of the provenance of these effects. It leads to a linear relation between sooner and later isopreference delays whose slope depends on sensitivity to reinforcement, and intercept on that and the steepness of the delay gradient. Unlike human prospective judgments, all control is vested in either primary or secondary reinforcement processes; therefore the use of the term discounting, appropriate for humans, may be less descriptive of the behavior of nonverbal organisms.

Stimulus effects on local preference: stimulus-response contingencies, stimulus-food pairing, and stimulus-food correlation

Four pigeons were trained in a procedure in which concurrent-schedule food ratios changed unpredictably across seven unsignaled components after 10 food deliveries. Additional green-key stimulus presentations also occurred on the two alternatives, sometimes in the same ratio as the component food ratio, and sometimes in the inverse ratio. In eight experimental conditions, we varied the contingencies surrounding these additional stimuli: In two conditions, stimulus onset and offset were noncontingent; in another two, stimulus onset was noncontingent, and offset was response contingent. In four conditions, both stimulus onset and offset were contingent, and in two of these conditions the stimulus was simultaneously paired with food delivery. Sensitivity to component food ratios was significantly higher when stimulus onset was response contingent compared to when it was noncontingent. Choice changes following food delivery were similar in all eight conditions. Choice changes following stimuli were smaller than those following food, and directionally were completely determined by the food-ratio:stimulus-ratio correlation, not by the stimulus contingency nor by whether the stimulus was paired with food or not. These results support the idea that conditional reinforcers may best be viewed as signals for next-food location rather than as stimuli that have acquired hedonic value, at least when the signals are differential with respect to future conditions.

The matching relation and situation-specific bias modulation in professional football play selection

The utility of a quantitative model depends on the extent to which its fitted parameters vary systematically with environmental events of interest. Professional football statistics were analyzed to determine whether play selection (passing versus rushing plays) could be accounted for with the generalized matching equation, and in particular whether variations in play selection across game situations would manifest as changes in the equation's fitted parameters. Statistically significant changes in bias were found for each of five types of game situations; no systematic changes in sensitivity were observed. Further analyses suggested relationships between play selection bias and both turnover probability (which can be described in terms of punishment) and yards-gained variance (which can be described in terms of variable-magnitude reinforcement schedules). The present investigation provides a useful demonstration of association between face-valid, situation-specific effects in a domain of everyday interest, and a theoretically important term of a quantitative model of behavior. Such associations, we argue, are an essential focus in translational extensions of quantitative models.

Dynamics of choice: relative rate and amount affect local preference at three different time scales

To examine extended control over local choice, the present study investigated preference in transition as food-rate ratio provided by two levers changed across seven components within daily sessions, and food-amount ratio changed across phases. Phase 1 arranged a food-amount ratio of 4:1 (i.e., the left lever delivered four pellets and the right lever one pellet); Phase 2 reversed the food-amount ratio to 1:4, and in Phase 3 the food-amount ratio was 3:2. At a relatively extended time scale, preference was described well by a linear relation between log response ratio and log rate ratio (the generalized matching law). A small amount of carryover occurred from one rate ratio to the next but disappeared after four food deliveries. Estimates of sensitivity to food-amount ratio were around 1.0 and were independent of rate ratio. Analysis across food deliveries within rate-ratio components showed that the effect of a small amount was diminished by the presence of a large amount-that is, when a larger amount was present in the situation (three or four pellets), the value of a small amount (one or two pellets) became paltry. More local analysis of visits to the levers between food deliveries showed that postfood visits following a large amount were disproportionately longer than following a small amount. Continuing food deliveries from the same source tended to make visits less dependent on relative amount, but a discontinuation (i.e., food from the other lever) reinstated dependence on relative amount. Analysis at a still smaller time scale revealed preference pulses following food deliveries that confirmed the tendency toward dependence on absolute amount with continuing deliveries, and toward dependence on relative amount following discontinuations. A mathematical model based on a linear-operator equation accounts for many of the results. The larger and longer preference following a switch to a larger amount is consistent with the idea that local preference depends on relatively extended variables even on short time scales.

The dynamics of the law of effect: a comparison of models

Dynamical models based on three steady-state equations for the law of effect were constructed under the assumption that behavior changes in proportion to the difference between current behavior and the equilibrium implied by current reinforcer rates. A comparison of dynamical models showed that a model based on Navakatikyan's (2007) two-component functions law-of-effect equations performed better than models based on Herrnstein's (1970) and Davison and Hunter's (1976) equations. Navakatikyan's model successfully described the behavioral dynamics in schedules with negative-slope feedback functions, concurrent variable-ratio schedules, Vaughan's (1981) melioration experiment, and experiments that arranged equal, and constant-ratio unequal, local reinforcer rates.

Modeling the dynamics of choice

A simple linear-operator model both describes and predicts the dynamics of choice that may underlie the matching relation. We measured inter-food choice within components of a schedule that presented seven different pairs of concurrent variable-interval schedules for 12 food deliveries each with no signals indicating which pair was in force. This measure of local choice was accurately described and predicted as obtained reinforcer sequences shifted it to favor one alternative or the other. The effect of a changeover delay was reflected in one parameter, the asymptote, whereas the effect of a difference in overall rate of food delivery was reflected in the other parameter, rate of approach to the asymptote. The model takes choice as a primary dependent variable, not derived by comparison between alternatives-an approach that agrees with the molar view of behaviour.

Relative reinforcer rates and magnitudes do not control concurrent choice independently

One assumption of the matching approach to choice is that different independent variables control choice independently of each other. We tested this assumption for reinforcer rate and magnitude in an extensive parametric experiment. Five pigeons responded for food reinforcement on switching-key concurrent variable-interval variable-interval schedules. Across conditions, the ratios of reinforcer rates and of reinforcer magnitudes on the two alternatives were both manipulated. Control by each independent variable, as measured by generalized-matching sensitivity, changed significantly with the ratio of the other independent variable. Analyses taking the model-comparison approach, which weighs improvement in goodness-of-fit against increasing number of free parameters, were inconclusive. These analyses compared a model assuming constant sensitivity to magnitude across all reinforcer-rate ratios with two alternative models. One of those alternatives allowed sensitivity to magnitude to vary freely across reinforcer-rate ratios, and was less efficient than the common-sensitivity model for all pigeons, according to the Schwarz-Bayes information criterion. The second alternative model constrained sensitivity to magnitude to be equal for pairs of reinforcer-rate ratios that deviated from unity by proportionately equal amounts but in opposite directions. This model was more efficient than the common-magnitude-sensitivity model for 2 of the pigeons, but not for the other 3. An analysis of variance, carried out independently of the generalized-matching analysis, also showed a significant interaction between the effects of reinforcer rate and reinforcer magnitude on choice. On balance, these results suggest that the assumption of independence inherent in the matching approach cannot be maintained. Relative reinforcer rates and magnitudes do not control choice independently.

Sensitivity of conditional-discrimination performance to within-session variation of reinforcer frequency

The present experiment developed a methodology for assessing sensitivity of conditional-discrimination performance to within-session variation of reinforcer frequency. Four pigeons responded under a multiple schedule of matching-to-sample components in which the ratio of reinforcers for correct S1 and S2 responses was varied across components within session. Initially, five components, each arranging a different reinforcer-frequency ratio (from 1:9 to 9:1), were presented randomly within a session. Under this condition, sensitivity to reinforcer frequency was low. Sensitivity failed to improve after extended exposure to this condition, and under a condition in which only three reinforcer-frequency ratios were varied within session. In a later condition, three reinforcer-frequency ratios were varied within session, but the reinforcer-frequency ratio in effect was differentially signaled within each component. Under this condition, values of sensitivity were similar to those traditionally obtained when reinforcer-frequency ratios for correct responses are varied across conditions. The effects of signaled vs. unsignaled reinforcer-frequency ratios were replicated in two subsequent conditions. The present procedure could provide a practical alternative to parametric variation of reinforcer frequency across conditions and may be useful in characterizing the effects of a variety of manipulations on steady-state sensitivity to reinforcer frequency.

Basolateral amygdala lesions and sensitivity to reinforcer magnitude in concurrent chains schedules

Previous studies show that the basolateral amygdala (BLA) is required for behavior to adjust when the value of a reinforcer decreases after satiation or pairing with gastric distress. This study evaluated the effect of pre- or post-training excitotoxic lesions of the BLA on changes in preference with another type of contingency change, reinforcer magnitude reversal. Rats were trained to press left and right levers during a variable-interval choice phase for 50 microl or 150 microl sucrose delivered to consistent locations after a 16-s delay. Tones were presented during the first and last 2s of the delay to reinforcement. The tone frequency predicted the magnitude of sucrose reinforcement in baseline conditions. All groups acquired stable preference for the lever on the large (150 microl) reinforcer side. However, nose poking during the delay to large reinforcement was highly accurate (i.e., to the reinforced side) for all groups except the rats with BLA lesions induced before training, suggesting impaired control of behavior by the tone. After the acquisition of stable preference, the locations of the reinforcer magnitudes were unpredictably reversed for a single session. Pre-training lesions blunted changes in preference when the reinforcer magnitudes were reversed. Lesions induced after stable preference was acquired, but prior to reversal, did not disrupt changes in preference. The data suggest that the BLA contributes to the adaptation of choice behavior following changes in reinforcer magnitude. Impaired learning about the tone-reinforcer magnitude relationships may have disrupted discrimination of the reinforcer magnitude reversal.

Quantification of drug choice with the generalized matching law in rhesus monkeys

The generalized matching law provides precise descriptions of choice, but has not been used to characterize choice between different doses of drugs or different classes of drugs. The current study examined rhesus monkeys' drug self-administration choices between identical drug doses, different doses, different drugs (cocaine, remifentanil, and methohexital), and between drug and drug-paired stimuli. The bias parameter of the generalized matching law was used to quantify preference for one reinforcer over another. Choice between identical drug doses yielded undermatching. Choices between 0.3 microg/kg/injection remifentanil and either 0.1 microg/kg/injection remifentanil or saline plus drug-paired stimuli revealed bias for the 0.3 microg/kg/injection dose. Choice was relatively insensitive to differences in random interval schedule value when one reinforcer was replaced with drug-paired stimulus presentations. Bias for 0.3 microg/kg/injection remifentanil over 10 microg/kg/injection cocaine was seen in one subject, and indifference was generally observed between 0.1 microg/kg/injection remifentanil and 56 microg/kg/injection cocaine and between 30 microg/kg/injection cocaine and 320 microg/kg/injection methohexital. These findings suggest the bias parameter may be useful in quantitatively measuring level of preference, which would be an advantage over concurrent FR procedures that often result in exclusive choice.

The stay/switch model describes choice among magnitudes of reinforcers

The stay/switch model is an alternative to the generalized matching law for describing choice in concurrent procedures. The purpose of the present experiment was to extend this model to choice among magnitudes of reinforcers. Rats were exposed to conditions in which the magnitude of reinforcers (number of food pellets) varied for staying at alternative 1, switching from alternative 1, staying at alternative 2 and switching from alternative 2. A changeover delay was not used. The results showed that the stay/switch model provided a good account of the data overall, and deviations from fits of the generalized matching law to response allocation data were in the direction predicted by the stay/switch model. In addition, comparisons among specific conditions suggested that varying the ratio of obtained reinforcers, as in the generalized matching law, was not necessary to change the response and time allocations. Other comparisons suggested that varying the ratio of obtained reinforcers was not sufficient to change response allocation. Taken together these results provide additional support for the stay/switch model of concurrent choice.

Local effects of delayed food

Five pigeons were trained on a procedure in which seven concurrent variable-interval schedules arranged seven different food-rate ratios in random sequence in each session. Each of these components lasted for 10 response-produced food deliveries, and components were separated by 10-s blackouts. We varied delays to food (signaled by blackout) between the two response alternatives in an experiment with three phases: In Phase 1, the delay on one alternative was 0 s, and the other was varied between 0 and 8 s; in Phase 2, both delays were equal and were varied from 0 to 4 s; in Phase 3, the two delays summed to 8 s, and each was varied from 1 to 7 s. The results showed that increasing delay affected local choice, measured by a pulse in preference, in the same way as decreasing magnitude, but we found also that increasing the delay at the other alternative increased local preference. This result casts doubt on the traditional view that a reinforcer strengthens a response depending only on the reinforcer's value discounted by any response-reinforcer delay. The results suggest that food guides, rather than strengthens, behavior.

Maternal nutrition and four-alternative choice

Two groups of 10 male rats were trained to nose poke for food pellets at four alternatives that provided differing rates of pellet delivery on aperiodic schedules. After a fixed number of pellets had been delivered, 5, 10 or 20 in different conditions of the experiment, a 10-s blackout occurred, and the locations of the differing rates of pellet delivery were randomized for the next component. Two groups of rats were used: The AD group consisted of 10 rats born to dams that had normal (ad libitum) nutrition during pregnancy, whereas the 10 rats in the UN group were from dams exposed to reduced food availability during pregnancy. All pups received normal nutrition after birth. Choice between the nose-poke alternatives quickly adapted when the rates of pellet delivery were changed in both groups, but there were no consistent differences in the speed of adaptation between the two groups. The generalized matching relation failed to describe the allocation of responses among alternatives, but the contingency-discriminability model provided a precise description of performance.

Haloperidol, dynamics of choice, and the parameters of the matching law

The idea that dopamine mediates the reinforcing effects of stimuli persists in the field of neurosciences. The present study shows that haloperidol, a dopamine antagonist, does not eliminate the reinforcing value of food reinforcers. The ratio of reinforcers changed seven times across two levers within sessions, modeling a dynamic environment. The magnitude of the reinforcer was manipulated independently of the reinforcer ratio. Four doses of intraperitoneal haloperidol were assessed over periods of 12 daily sessions. Haloperidol did not impair the discrimination that the rats established between rich and lean levers; the response distributions favored the lever associated with the higher probability of reinforcement and the larger pellets. The parameters of the generalized matching law (bias and sensitivity) were used to estimate effects of haloperidol upon the motor system and upon the rats' motivation for food reinforcers.