Delay-specific stimuli and genotype interact to determine temporal discounting in a rapid-acquisition procedure

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent-chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady-state, single-subject experimental designs can be time-consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent-chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within-session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

Neuronal activity associated with cocaine preference: Effects of differential cocaine intake

Differences in overall cocaine intake can directly affect neuroadaptations, and this relationship can make it difficult to interpret neurobiological changes seen in drug-choice studies, since drug intake varies between subjects. Herein, a choice procedure that controls for cocaine intake was utilized to explore if neuronal activity, measured as cFos expression in the orbitofrontal cortex (OFC) and nucleus accumbens (NAc), was reflective of preference. Results demonstrated that cFos expression, in both the OFC and NAc, was independent of cocaine preference when cocaine intake was kept constant across individuals. However, when cocaine intake was systematically varied, the expression of cFos associated with cocaine preference was related to overall cocaine intake in the OFC, but not the NAc. Altogether, these results demonstrate that cocaine intake during choice can affect neurobiological outcome measures; thus, the neurobehavioral mechanisms underlying cocaine preference may be better isolated when controlling for cocaine frequency and intake. In all, some caution is warranted when interpreting results from choice studies evaluating the neurobehavioral mechanisms that underlie drug preference when drug frequency and intake are uncontrolled, and future research is needed to determine the role of drug frequency and intake on neurobiological measures associated with drug choice.

Pair housing, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effect of probability presentation order on amphetamine-induced changes in risky choice

Probability discounting is often measured with independent schedules. Independent schedules have several limitations, such as confounding preference for one alternative with frequency of reward presentation and generating ceiling/floor effects at certain probabilities. To address this potential caveat, a controlled reinforcer frequency schedule can be used, in which the manipulandum that leads to reinforcement is pseudo-randomly determined before each trial. This schedule ensures subjects receive equal presentations of the small and large magnitude reinforcers across each block of trials. A total of 24 pair-housed and 11 individually housed female Sprague Dawley rats were tested in a controlled reinforcer frequency procedure. For half of the rats, the odds against (OA) receiving the large magnitude reinforcer increased across the session (ascending schedule); the OA decreased across the session for half of the rats (descending schedule). Following training, rats received treatments of amphetamine (AMPH; 0, 0.25, 0.5, 1.0 mg/kg; s.c.). For pair-housed rats, AMPH (0.5 mg/kg) increased risky choice, regardless of probability presentation order, whereas a higher dose of AMPH (1.0 mg/kg) decreased discriminability of reinforcer magnitude for rats trained on the descending schedule only. For individually housed rats, probability presentation order modulated the effects of AMPH on probability discounting, as AMPH (0.25 and 0.5 mg/kg) increased risky choice in rats trained on the ascending schedule but not on the descending schedule. These results show that pair-housing animals, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effects of probability presentation order on drug effects on risky choice.

Using a dependent schedule to measure risky choice in male rats: Effects of d-amphetamine, methylphenidate, and methamphetamine

Risky choice is the tendency to choose a large, uncertain reward over a small, certain reward, and is typically measured with probability discounting, in which the probability of obtaining the large reinforcer decreases across blocks of trials. One caveat to traditional procedures is that independent schedules are used, in which subjects can show exclusive preference for one alternative relative to the other. For example, some rats show exclusive preference for the small, certain reinforcer as soon as delivery of the large reinforcer becomes probabilistic. Therefore, determining if a drug increases risk aversion (i.e., decreases responding for the probabilistic alternative) is difficult (due to floor effects). The overall goal of this experiment was to use a concurrent-chains procedure that incorporated a dependent schedule during the initial link, thus preventing animals from showing exclusive preference for one alternative relative to the other. To determine how pharmacological manipulations alter performance in this task, male Sprague-Dawley rats (n = 8) received injections of amphetamine (0, 0.25, 0.5, 1.0 mg/kg), methylphenidate (0, 0.3, 1.0, 3.0 mg/kg), and methamphetamine (0, 0.5, 1.0, 2.0 mg/kg). Amphetamine (0.25 mg/kg) and methylphenidate (3.0 mg/kg) selectively increased risky choice, whereas higher doses of amphetamine (0.5 and 1.0 kg/mg) and each dose of methamphetamine impaired stimulus control (i.e., flattened the discounting function). These results show that dependent schedules can be used to measure risk-taking behavior and that psychostimulants promote suboptimal choice when this schedule is used. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Delay discounting of different outcomes: Review and theory

Steep delay discounting is characterized by a preference for small immediate outcomes relative to larger delayed outcomes and is predictive of drug abuse, risky sexual behaviors, and other maladaptive behaviors. Nancy M. Petry was a pioneer in delay discounting research who demonstrated that people discount delayed monetary gains less steeply than they discount substances with abuse liability. Subsequent research found steep discounting for not only drugs, but other nonmonetary outcomes such as food, sex, and health. In this systematic review, we evaluate the hypotheses proposed to explain differences in discounting as a function of the type of outcome and explore the trait- and state-like nature of delay discounting. We found overwhelming evidence for the state-like quality of delay discounting: Consistent with Petry and others' work, nonmonetary outcomes are discounted more steeply than monetary outcomes. We propose two hypotheses that together may account for this effect: Decreasing Future Preference and Decreasing Future Worth. We also found clear evidence that delay discounting has trait-like qualities: People who steeply discount monetary outcomes steeply discount nonmonetary outcomes as well. The implication is that changing delay discounting for one outcome could change discounting for other outcomes.

Cocaine-associated decision-making: Toward isolating preference

Ever-increasing evidence suggests that substance use disorder is mediated by decision-making processes, and as such, providing nondrug alternatives can shift maladaptive preferences away from drug reinforcers, such as cocaine. Of note, a recent hypothesis suggests that preference for cocaine is simply a byproduct of cocaine intake, such that the 'direct' effects of cocaine weaken the impact of non-drug alternatives while measuring choice. Conversely, existing quantitative theories of decision-making suggest preference is determined by various dimensions of concurrent reinforcers that in turn determine the relative value of available alternatives. Toward teasing apart the conflicting theories above, we developed a novel drug-choice procedure to control for reinforcer frequency and magnitude (two reinforcer dimensions well known to influence preference) that consequently controls for overall cocaine intake. As predicted by quantitative choice theory, results suggest that cocaine intake and preference are dissociable while measuring choice, with reinforcer frequency and magnitude having independent influence on the relative value of choice alternatives. Furthermore, we demonstrate that the choice procedure is sensitive to various manipulations known to alter cocaine reinforcement, all while keeping cocaine intake constant. Finally, the results point to the process of economic substitution as an important avenue of future neurobehavioral investigation toward the improvement of behavioral and pharmacological therapies for substance use disorders. Overall, the proposed choice procedure will allow for improved isolation of the neurobehavioral processes that mediate drug-associated decision-making in future studies.

Mercury Involvement in Neuronal Damage and in Neurodegenerative Diseases

Neurodegenerative diseases such as Alzheimer's disease, Parkinson's disease, amyotrophic lateral sclerosis, and multiple sclerosis are characterized by a chronic and selective process of neuronal cell death. Although the causes of neurodegenerative diseases remain still unknown, it is now a well-established idea that more factors, such as genetic, endogenous, and environmental, are involved. Among environmental causes, the accumulation of mercury, a heavy metal considered a toxic agent, was largely studied as a probable factor involved in neurodegenerative disease course. Mercury exists in three main forms: elemental mercury, inorganic mercury, and organic mercury (methylmercury and ethylmercury). Sources of elemental mercury can be natural (volcanic emission) or anthropogenic (coal-fired electric utilities, waste combustion, hazardous-waste incinerators, and gold extraction). Moreover, mercury is still used as an antiseptic, as a medical preservative, and as a fungicide. Dental amalgam can emit mercury vapor. Mercury vapor, being highly volatile and lipid soluble, can cross the blood-brain barrier and the lipid cell membranes and can be accumulated into the cells in its inorganic forms. Also, methylmercury can pass through blood-brain and placental barriers, causing serious damage in the central nervous system. This review describes the toxic effects of mercury in cell cultures, in animal models, and in patients with neurodegenerative diseases. In vitro experiments showed that mercury exposure was principally involved in oxidative stress and apoptotic processes. Moreover, motor and cognitive impairment and neural loss have been confirmed in various studies performed in animal models. Finally, observational studies on patients with neurodegenerative diseases showed discordant data about a possible mercury involvement.

Dissecting drug effects in preclinical models of impulsive choice: emphasis on glutamatergic compounds

RATIONALE

Impulsive choice is often measured with delay discounting paradigms. Because there are multiple discounting procedures, as well as different statistical analyses that can be applied to data generated from these paradigms, there are some inconsistencies in the literature regarding drug effects on impulsive choice.

OBJECTIVES

The goal of the current paper is to review the methodological and analytic approaches used to measure discounting and to discuss how these differences can account for differential drug effects observed across studies.

RESULTS

Because some procedures/analyses use a single data point as the dependent variable, changes in this value following pharmacological treatment may be interpreted as alterations in sensitivity to delayed reinforcement, but when other procedures/analyses are used, no changes in behavior are observed. Even when multiple data points are included, some studies show that the statistical analysis (e.g., ANOVA on raw proportion of responses vs. using hyperbolic/exponential functions) can lead to different interpretations. Finally, procedural differences (e.g., delay presentation order, signaling the delay to reinforcement, etc.) in the same discounting paradigm can alter how drugs affect sensitivity to delayed reinforcement.

CONCLUSIONS

Future studies should utilize paradigms that allow one to observe alterations in responding at each delay (e.g., concurrent-chains schedules). Concerning statistical analyses, using parameter estimates derived from nonlinear functions or incorporating the generalized matching law can allow one to determine if drugs affect sensitivity to delayed reinforcement or impair discrimination of the large and small magnitude reinforcers. Using these approaches can help further our understanding of the neurochemical underpinnings of delay discounting.

Adolescent methylmercury exposure affects choice and delay discounting in mice

The developing fetus is vulnerable to low-level exposure to methylmercury (MeHg), an environmental neurotoxicant, but the consequences of exposure during the adolescent period remain virtually unknown. The current experiments were designed to assess the effects of low-level MeHg exposure during adolescence on delay discounting, preference for small, immediate reinforcers over large, delayed ones, using a mouse model. Thirty-six male C57BL/6n mice were exposed to 0, 0.3, or 3.0ppm mercury (as MeHg) via drinking water from postnatal day 21 through 59, encompassing the murine adolescent period. As adults, mice lever pressed for a 0.01-cc droplet of milk solution delivered immediately or four 0.01-cc droplets delivered after a delay. Delays ranged from 1.26 to 70.79s, and all were presented within a session. A model based on the Generalized Matching Law indicated that sensitivity to reinforcer magnitude was lower for MeHg-exposed mice relative to controls, indicating that responding in MeHg-exposed mice was relatively indifferent to the larger reinforcer. Sensitivity to reinforcer delay was reduced (delay discounting was decreased) in the 0.3-ppm group, but not in the 3.0-ppm group, compared to controls. Adolescence is a developmental period during which the brain and behavior may be vulnerable to MeHg exposure. As with gestational MeHg exposure, the effects are reflected in the impact of reinforcing stimuli.

Determinants of choice, and vulnerability and recovery in addiction

Addiction may be viewed as choice governed by competing contingencies. One factor impacting choice, particularly as it relates to addiction, is sensitivity to delayed rewards. Discounting of delayed rewards influences addiction vulnerability because of competition between relatively immediate gains of drug use, e.g. intoxication, versus relatively remote gains of abstinence, e.g. family stability. Factors modifying delay sensitivity can be modeled in the laboratory. For instance, increased delay sensitivity can be similarly observed in adolescent humans and non-human animals. Similarly, genetic factors influence delay sensitivity in humans and animals. Recovery from addiction may also be viewed as choice behavior. Thus, reinforcing alternative behavior facilitates recovery because reinforcing alternative behavior decreases the frequency of using drugs. How reinforcing alternative behavior influences recovery can also be modeled in the laboratory. For instance, relapse risk decreases as abstinence duration increases, and this decreasing risk can be modeled in animals using choice procedures. In summary, addiction in many respects can be conceptualized as a problem of choice. Animal models of choice disorders stand to increase our understanding of the core processes that establish and maintain addiction and serve as a proving ground for development of novel treatments.

Minor Functional Deficits in Basic Response Patterns for Reinforcement after Frontal Traumatic Brain Injury in Rats

Traumatic brain injury (TBI) is a major contributor to numerous psychiatric conditions and chronic behavioral dysfunction. Recent studies in experimental brain injury have begun to adopt operant methodologies to assess these deficits, all of which rely on the process of reinforcement. No studies have directly examined how reinforced behaviors are affected by TBI, however. The current study assessed performance under the four most common schedules of reinforcement (fixed ratio, variable ratio, fixed interval, variable interval) and one higher order schedule assessing motivation (progressive ratio) after bilateral, pre-frontal controlled cortical impact injury. TBI-induced differences on the basic schedules were minor, with the exception of the variable ratio, where increased efficacy (more reinforcers, higher response rates, lower interresponse times) at higher requirements was observed as a result of brain injury. Performance on the progressive ratio schedule showed some gross differences between the groups, in that sham rats became more efficient under this schedule while injured rats perseverated in lever pressing. Further, injured rats were specifically impaired at lower response requirements on the progressive ratio. Taken together, these findings indicate that simple reinforced behaviors are mostly unaffected after TBI, except in the case of variable ratio schedules, but the altered performance on the higher-order progressive ratio schedule suggests changes involving motivation or potentially perseveration. These findings validate operant measures of more complex behaviors for brain injury, all of which rely on reinforcement and can be taken into consideration when adapting and developing novel functional assessments.

The effects of physical activity on impulsive choice: Influence of sensitivity to reinforcement amount and delay

Impulsive choice is a diagnostic feature and/or complicating factor for several psychological disorders and may be examined in the laboratory using delay-discounting procedures. Recent investigators have proposed using quantitative measures of analysis to examine the behavioral processes contributing to impulsive choice. The purpose of this study was to examine the effects of physical activity (i.e., wheel running) on impulsive choice in a single-response, discrete-trial procedure using two quantitative methods of analysis. To this end, rats were assigned to physical activity or sedentary groups and trained to respond in a delay-discounting procedure. In this procedure, one lever always produced one food pellet immediately, whereas a second lever produced three food pellets after a 0, 10, 20, 40, or 80-s delay. Estimates of sensitivity to reinforcement amount and sensitivity to reinforcement delay were determined using (1) a simple linear analysis and (2) an analysis of logarithmically transformed response ratios. Both analyses revealed that physical activity decreased sensitivity to reinforcement amount and sensitivity to reinforcement delay. These findings indicate that (1) physical activity has significant but functionally opposing effects on the behavioral processes that contribute to impulsive choice and (2) both quantitative methods of analysis are appropriate for use in single-response, discrete-trial procedures.