A quantitative analysis of the effects of qualitatively different reinforcers on fixed ratio responding in inbred strains of mice

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Assessing reward preference using operant behavior in male and female mice

Many different solid food pellets are available as reinforcers for rodents in operant behavior tests. Different reward formulations have not been compared, so it is unclear whether mice show strong preferences for different rewards and whether such preferences are consistent within or across sex and background strain. Here we show that mice have strong preferences for two balanced diet food rewards over sucrose pellets, and preference for one balanced diet pellet formulation over another, in a simultaneous choice test using a low effort fixed ratio operant test. All mice, of both sexes and both CD1 and C57 background strains, showed the same strong preferences among these three types of reinforcers. In contrast, flavorings added to the reward pellets had relatively small and more variable effects on preference. The preference for balanced diet pellets over sucrose pellets was seen also in the total numbers of rewards consumed in low effort tests with food pellets or only sucrose pellets available. However, progressive ratio testing showed that mice worked harder for sucrose pellets than for the preferred balanced diet pellets. These findings indicate that reinforcers with similar and very different preference profiles are readily available and that testing with different rewards can produce different, and sometimes unexpected, results.

Reinforcer value moderates the effects of prenatal alcohol exposure on learning and reversal

Introduction

Fetal Alcohol Spectrum Disorders (FASD) are the leading cause of preventable developmental disability and are commonly characterized by alterations in executive function. Reversal learning tasks are reliable, cross-species methods for testing a frequently impaired aspect of executive control, behavioral flexibility. Pre-clinical studies commonly require the use of reinforcers to motivate animals to learn and perform the task. While there are several reinforcers available, the most commonly employed are solid (food pellets) and liquid (sweetened milk) rewards. Previous studies have examined the effects of different solid rewards or liquid dietary content on learning in instrumental responding and found that rodents on liquid reward with higher caloric content performed better with increased response and task acquisition rate. The influence of reinforcer type on reversal learning and how this interacts with developmental insults such as prenatal alcohol exposure (PAE) has not been explored.

Methods

We tested whether reinforcer type during learning or reversal would impact an established deficit in PAE mice.

Results

We found that all male and female mice on liquid reward, regardless of prenatal exposure were better motivated to learn task behaviors during pre-training. Consistent with previous findings, both male and female PAE mice and Saccharine control mice were able to learn the initial stimulus reward associations irrespective of the reinforcer type. During the initial reversal phase, male PAE mice that received pellet rewards exhibited maladaptive perseverative responding whereas male mice that received liquid rewards performed comparable to their control counterparts. Female PAE mice that received either reinforcer types did not exhibit any deficits on behavioral flexibility. Female saccharine control mice that received liquid, but not pellet, rewards showed increased perseverative responding during the early reversal phase.

Discussion

These data suggest that reinforcer type can have a major impact on motivation, and therefore performance, during reversal learning. Highly motivating rewards may mask behavioral deficits seen with more moderately sought rewards and gestational exposure to the non-caloric sweetener, saccharine, can impact behavior motivated by those reinforcers in a sex-dependent manner.

Something to Snack on: Can Dietary Modulators Boost Mind and Body?

The last decades have shown that maintaining a healthy and balanced diet can support brain integrity and functionality, while an inadequate diet can compromise it. However, still little is known about the effects and utility of so-called healthy snacks or drinks and their immediate short-term effects on cognition and physical performance. Here, we prepared dietary modulators comprising the essential macronutrients at different ratios and a controlled balanced dietary modulator. We assessed, in healthy adult mice, the short-term effects of these modulators when consumed shortly prior to tests with different cognitive and physical demands. A high-fat dietary modulator sustained increased motivation compared to a carbohydrate-rich dietary modulator (p = 0.041) which had a diminishing effect on motivation (p = 0.018). In contrast, a high-carbohydrate modulator had an initial beneficial effect on cognitive flexibility (p = 0.031). No apparent effects of any of the dietary modulators were observed on physical exercise. There is increasing public demand for acute cognitive and motor function enhancers that can improve mental and intellectual performance in daily life, such as in the workplace, studies, or sports activities. Our findings suggest such enhancers should be tailored to the cognitive demand of the task undertaken, as different dietary modulators will have distinct effects when consumed shortly prior to the task.

Adolescence as a sensitive period for neurotoxicity: Lifespan developmental effects of methylmercury

Neurotoxicity resulting from the environmental contaminant, methylmercury (MeHg), is a source of concern for many human populations that rely heavily on the consumption of fish and rice as stable ingredients in the diet. The developmental period of exposure is important both to the qualitative effects of MeHg and to the dose required to produce those effects. MeHg exposure during the sensitive prenatal period causes deleterious and long-lasting changes in neurodevelopment at particularly low doses. The effects include a wide host of cognitive and behavioral outcomes expressed in adulthood and sometimes not until aging. However, neurotoxic outcomes of methylmercury when exposure occurs during adolescence are only recently revealing impacts on human populations and animal models. This review examines the current body of work and showcases the sensitivity of adolescence, a period that straddles early development and adulthood, to methylmercury neurotoxicity and the implications such toxicity has in our understanding of methylmercury's effects in human populations and animal models.

Dissociating the effects of dopamine D2 receptors on effort-based versus value-based decision making using a novel behavioral approach

Cost-benefit decision making is essential for organisms to adapt to their ever-changing environment. Most studies of cost-benefit decision making involve choice conditions in which effort and value are varied simultaneously. This prevents identification of the aspects of cost-benefit decision making that are affected by experimental manipulations. We developed operant assays to isolate the individual impacts of effort and value manipulations on cost-benefit decision making. In the concurrent effort choice (CEC) task, mice choose between exerting two distinct types of effort: the number of responses and the duration of a response, to earn the same reward. By parametrically varying response cost, psychometric functions are obtained that reflect how the two types of effort scale against one another. Direct manipulations of effort shift the functions. Because reward value is held constant in this task, differences in scaling of the two response types must be related to the effort manipulations. In the concurrent value choice (CVC) task, mice make the same type of response to earn rewards of different value (e.g., pellets vs. sucrose solutions). Here the effort required to earn one reward type is parametrically varied to obtain the psychometric function that scales the value of the two rewards into the number of responses subjects will pay to earn one reward over the other. Direct value manipulations shift these functions. We tested the effect of the dopamine D2 receptor antagonist, haloperidol, on performance in the CEC and CVC assays and found that D2R signaling is important for effort-based, but not value-based decision making. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Adolescent methylmercury exposure: Behavioral mechanisms and effects of sodium butyrate in mice

Methylmercury (MeHg), an environmental neurotoxicant primarily found in fish, produces neurobehavioral impairment when exposure occurs during gestation. Whether other developmental periods, such as adolescence, display enhanced vulnerability to the behavioral effects of MeHg exposure is only beginning to be explored. Further, little is known about the effects of repeated administration of lysine deacetylase inhibitors, such as sodium butyrate (NaB), on operant behavior. In Experiment 1, male C57BL6/n mice were exposed to 0, 0.3, and 3.0 ppm MeHg (n = 12 each) via drinking water from postnatal days 21 to 60 (murine adolescence). As adults, mice were trained to lever press under an ascending series of fixed-ratio schedules of milk reinforcement selected to enable the analysis of three important parameters of operant behavior using the framework provided by Mathematical Principles of Reinforcement. Adolescent MeHg exposure dose-dependently increased saturation rate, a measure of the retroactive reach of a reinforcer, and decreased minimum response time relative to controls. In Experiment 2, the behavioral effects of repeated NaB administration both alone and following adolescent MeHg exposure were examined. Male C57BL6/n mice were given either 0 or 3.0 ppm MeHg during adolescence and, before behavioral testing, two weeks of once daily i.p. injections of saline or 0.6 g/kg NaB (n = 12 in each cell). Adolescent MeHg exposure again increased saturation rate but did not significantly alter minimum response time. NaB also increased saturation rate in both MeHg exposure groups. These data suggest that the behavioral mechanisms of adolescent MeHg exposure and NaB may be related to the impact of reinforcement on prior responses. Specifically, MeHg and NaB concentrated the effects of reinforcers onto the most recent responses.

Optimisation of cognitive performance in rodent operant (touchscreen) testing: Evaluation and effects of reinforcer strength

Operant testing is a widely used and highly effective method of studying cognition in rodents. Performance on such tasks is sensitive to reinforcer strength. It is therefore advantageous to select effective reinforcers to minimize training times and maximize experimental throughput. To quantitatively investigate the control of behavior by different reinforcers, performance of mice was tested with either strawberry milkshake or a known powerful reinforcer, super saccharin (1.5% or 2% (w/v) saccharin/1.5% (w/v) glucose/water mixture). Mice were tested on fixed (FR)- and progressive-ratio (PR) schedules in the touchscreen-operant testing system. Under an FR schedule, both the rate of responding and number of trials completed were higher in animals responding for strawberry milkshake versus super saccharin. Under a PR schedule, mice were willing to emit similar numbers of responses for strawberry milkshake and super saccharin; however, analysis of the rate of responding revealed a significantly higher rate of responding by animals reinforced with milkshake versus super saccharin. To determine the impact of reinforcer strength on cognitive performance, strawberry milkshake and super saccharin-reinforced animals were compared on a touchscreen visual discrimination task. Animals reinforced by strawberry milkshake were significantly faster to acquire the discrimination than animals reinforced by super saccharin. Taken together, these results suggest that strawberry milkshake is superior to super saccharin for operant behavioral testing and further confirms that the application of response rate analysis to multiple ratio tasks is a highly sensitive method for the detection of behavioral differences relevant to learning and motivation.

Pavlovian influences on learning differ between rats and mice in a counter-balanced Go/NoGo judgement bias task

Judgement bias tests of animal affect and hence welfare assume that the animal's responses to ambiguous stimuli, which may herald positive or negative outcomes, are under instrumental control and reflect 'optimism' or 'pessimism' about what will happen. However, Pavlovian control favours responses (e.g. approach or withdrawal) according to the valence associated with a stimulus, rather than the anticipated response outcomes. Typically, positive contexts promote action and approach whilst negative contexts promote inhibition or withdrawal. The prevalence of Go-for-reward (Go-pos) and NoGo-to-avoid-punishment (NoGo-neg) judgement bias tasks reflects this Pavlovian influence. A Pavlovian increase or decrease in activity or vigour has also been argued to accompany positive or negative affective states, and this may interfere with instrumental Go or NoGo decisions under ambiguity based on anticipated decision outcomes. One approach to these issues is to develop counter-balanced Go-pos/NoGo-neg and Go-neg/NoGo-pos tasks. Here we implement such tasks in Sprague Dawley rats and C57BL/6J mice using food and air-puff as decision outcomes. We find striking species/strain differences with rats achieving criterion performance on the Go-pos/NoGo-neg task but failing to learn the Go-neg/NoGo-pos task, in line with predictions, whilst mice do exactly the opposite. Pavlovian predispositions may thus differ between species, for example reflecting foraging and predation ecology and/or baseline activity rates. Learning failures are restricted to cues predicting a negative outcome; use of a more powerful air-puff stimulus may thus allow implementation of a fully counter-balanced task. Rats and mice achieve criterion faster than in comparable automated tasks and also show the expected generalisation of responses across ambiguous tones. A fully counter-balanced task thus offers a potentially rapidly implemented and automated method for assessing animal welfare, identifying welfare problems and areas for welfare improvement and 3Rs Refinement, and assessing the effectiveness of refinements.

Optimizing reproducibility of operant testing through reinforcer standardization: identification of key nutritional constituents determining reward strength in touchscreens

Reliable and reproducible assessment of animal learning and behavior is a central aim of basic and translational neuroscience research. Recent developments in automated operant chamber technology have led to the possibility of universal standard protocols, in addition to increased translational potential, reliability and accuracy. However, the impact of regional and national differences in the supplies of available reinforcers in this system on behavioural performance and inter-laboratory variability is an unknown and at present uncontrolled variable. Therefore, we aimed to identify which constituent(s) of the reward determines reinforcer strength to enable improved standardization of this parameter across laboratories. Male C57BL/6 mice were examined in the touchscreen-based fixed ratio (FR) and progressive ratio (PR) schedules, reinforced with different kinds of milk-based reinforcers to directly compare the incentive values of plain milk (PM, high-calorie: high-fat/low-sugar), strawberry-flavored milk (SM, high-calorie: low-fat/high-sugar), and semi-skimmed low-fat milk (LM, low-calorie: low-fat/low-sugar) on the basis of differences in caloric content, sugar/fat content, and flavor. Use of a higher caloric content reward was effective in increasing operant training acquisition rate. Total trial number completed in FR and breakpoint in PR were higher using the two isocaloric milk products (PM and SM) than the lower caloric LM, with comparable outcomes between PM and SM conditions, suggesting that total caloric content determines reward strength. Analysis of within-session changes in response rate revealed that overall outputs in FR and PR primarily depend on the response rate at the initial phase of a session, which itself was dependent on reinforcer caloric content. Interestingly, the rate of satiation, indicated by decay in response rate within a FR session, was highest when reinforced with SM, suggesting a rapid satiating effect of sugar. The key contribution of reward caloric content to operant performance was confirmed in a multi-laboratory study using the touchscreen 5-choice serial reaction time task (5-CSRTT) reinforced by two isocaloric milk-based liquid rewards with different countries of origin, which yielded consistent performance parameters across sites. Our results indicate that milk-based liquid reinforcer standardization can be facilitated by matching caloric content across laboratories despite regional or national differences in other non-caloric aspects of the reinforcers.

Mouse strain differences in punished ethanol self-administration

Determining the neural factors contributing to compulsive behaviors such as alcohol-use disorders (AUDs) has become a significant focus of current preclinical research. Comparison of phenotypic differences across genetically distinct mouse strains provides one approach to identify molecular and genetic factors contributing to compulsive-like behaviors. Here we examine a rodent assay for punished ethanol self-administration in four widely used inbred strains known to differ on ethanol-related behaviors: C57BL/6J (B6), DBA/2J (D2), 129S1/SvImJ (S1), and BALB/cJ (BALB). Mice were trained in an operant task (FR1) to reliably lever-press for 10% ethanol using a sucrose-fading procedure. Once trained, mice received a punishment session in which lever pressing resulted in alternating ethanol reward and footshock, followed by tests to probe the effects of punishment on ethanol self-administration. Results indicated significant strain differences in training performance and punished attenuation of ethanol self-administration. S1 and BALB showed robust attenuation of ethanol self-administration after punishment, whereas behavior in B6 was attenuated only when the punishment and probe tests were conducted in the same contexts. By contrast, D2 were insensitive to punishment regardless of context, despite receiving more shocks during punishment and exhibiting normal footshock reactivity. Additionally, B6, but not D2, reduced operant self-administration when ethanol was devalued with a bitter tastant. B6 and D2 showed devaluation of sucrose self-administration, and punished suppression of sucrose seeking was context dependent in both the strains. While previous studies have demonstrated avoidance of ethanol in D2, particularly when ethanol is orally available from a bottle, current findings suggest this strain may exhibit heightened compulsive-like self-administration of ethanol, although there are credible alternative explanations for the phenotype of this strain. In sum, these findings offer a foundation for future studies examining the neural and genetic factors underlying AUDs.

The development and biological characteristics of a novel potentially radioresistant inbred mouse strain

The growth of biomedical research over the previous decades has been accompanied by an increase in the number, complexity and diversity of experimental animals developed as research tools, and inbred mice are some of the most widely used. However, thus far, no inbred mice have exhibited strong radioresistance for use in radiation‑damage research. To develop a radioresistant mouse model, a female Japanese outbreeding strain ICR/JCL mouse was mated with a male Chinese inbred strain 615 mouse. From the F1 generation, the mouse line was maintained by brother‑to‑sister mating. A novel mouse strain was established over >20 continuous generations and designated the Institute of Radiation Medicine-2 (IRM-2) mouse. The biological characteristics, genetic characteristics and susceptibility to radiation of these mice were determined. The IRM‑2 mice inherited traits from the parents, including strong reproductive capacity, stable physiological and biochemical indices and few differences among individuals. According to the genetic results, the IRM‑2 mice exhibited homozygosity, isogenicity and consistency, in agreement with international standards for inbred strains. Radiosensitivity studies have previously suggested that the lethal dose (LD)50 values for IRM‑2 mice were 7.17 Gy (male) and 7.5 Gy (female), resulting in a dose reduction factor value of 1.39 (male) and 1.37 (female). The mortality of IRM‑2 mice irradiated with 8 Gy total body irradiation was 15% at day 9 and 90% at day 15 after radiation. The number of nucleated cells in bone marrow, DNA content and colony‑forming unit‑spleen counts in IRM‑2 mice after exposure to γ‑ray irradiation were markedly higher than the corresponding values for the parental strains, suggesting that the IRM‑2 mice exhibit high resistance to ionizing radiation. Thus, it is suggested that this novel inbred mouse strain may be developed as an animal model of radioresistance for future use in radiation research.

Adolescent methylmercury exposure affects choice and delay discounting in mice

The developing fetus is vulnerable to low-level exposure to methylmercury (MeHg), an environmental neurotoxicant, but the consequences of exposure during the adolescent period remain virtually unknown. The current experiments were designed to assess the effects of low-level MeHg exposure during adolescence on delay discounting, preference for small, immediate reinforcers over large, delayed ones, using a mouse model. Thirty-six male C57BL/6n mice were exposed to 0, 0.3, or 3.0ppm mercury (as MeHg) via drinking water from postnatal day 21 through 59, encompassing the murine adolescent period. As adults, mice lever pressed for a 0.01-cc droplet of milk solution delivered immediately or four 0.01-cc droplets delivered after a delay. Delays ranged from 1.26 to 70.79s, and all were presented within a session. A model based on the Generalized Matching Law indicated that sensitivity to reinforcer magnitude was lower for MeHg-exposed mice relative to controls, indicating that responding in MeHg-exposed mice was relatively indifferent to the larger reinforcer. Sensitivity to reinforcer delay was reduced (delay discounting was decreased) in the 0.3-ppm group, but not in the 3.0-ppm group, compared to controls. Adolescence is a developmental period during which the brain and behavior may be vulnerable to MeHg exposure. As with gestational MeHg exposure, the effects are reflected in the impact of reinforcing stimuli.

Single and repeated exposures to the volatile anesthetic isoflurane do not impair operant performance in aged rats

Postoperative Cognitive Dysfunction (POCD) is a complication that can occur in the elderly after anesthesia and surgery and is characterized by impairments in information processing, memory, and executive function. Currently, it is unclear whether POCD is due to the effects of surgery, anesthesia, or perhaps some interaction between these or other perioperative variables. Studies in rodents suggest that the development of POCD may be related directly to anesthesia-induced neuroactivity. Volatile anesthetics have been shown to increase cellular inflammation and apoptosis within the hippocampus of aged rodents, while producing corresponding impairments in hippocampal-dependent brain functions. However, it is unclear whether volatile anesthetics can affect additional aspects of cognition that do not primarily depend upon the hippocampus. The purpose of this study was to use established operant tests to examine the effects of isoflurane on aspects of behavioral inhibition, learning, and motivation in aged rats. Twenty-one adult Sprague-Dawley rats (11 male, 10 female) were trained to perform fixed consecutive number (FCN), incremental repeated acquisition (IRA), and progressive ratio (PR) tasks for a minimum of 15 months prior to receiving anesthesia. At 23 months of age, rats were exposed to 1.3% isoflurane or medical grade air for 2h. Initial results revealed that a 2h exposure to isoflurane had no effect on IRA, FCN, or PR performance. Thus, rats received 3 additional exposures to 1.3% isoflurane or medical grade air: 2, 4 and 6h exposures with 2 weeks elapsing before exposure two, 3 weeks elapsing between exposures two and three, and 2 weeks elapsing between exposures three and four. These additional exposures had no observable effects on performance of any operant task. These results suggest that single and repeated exposures to isoflurane do not impair the performance of aged rats in tasks designed to measure behavioral inhibition, learning, and motivation. This lack of significant effect suggests that the impairments associated with isoflurane exposure may not generalize to all aspects of cognition, but may be selective to tasks that primarily measure spatial memory processes.

Strain commonalities and differences in response-outcome decision making in mice

The ability to select between actions that are more vs. less likely to be reinforced is necessary for survival and navigation of a changing environment. A task termed "response-outcome contingency degradation" can be used in the laboratory to determine whether rodents behave according to such goal-directed response strategies. In one iteration of this task, rodents are trained to perform two food-reinforced behaviors, then the predictive relationship between one instrumental response and the associated outcome is modified by providing the reinforcer associated with that response non-contingently. During a subsequent probe test, animals can select between the two trained responses. Preferential engagement of the behavior most likely to be reinforced is considered goal-directed, while non-selective responding is considered a failure in response-outcome conditioning, or "habitual." This test has largely been used with rats, and less so with mice. Here we compiled data collected from several cohorts of mice tested in our lab between 2012 and 2015. Mice were bred on either a C57BL/6 or predominantly BALB/c strain background. We report that both strains of mice can use information acquired as a result of instrumental contingency degradation training to select amongst multiple response options the response most likely to be reinforced. Mice differ, however, during the training sessions when the familiar response-outcome contingency is being violated. BALB/c mice readily generate perseverative or habit-like response strategies when the only available response is unlikely to be reinforced, while C57BL/6 mice more readily inhibit responding. These findings provide evidence of strain differences in response strategies when an anticipated reinforcer is unlikely to be delivered.

Delay-specific stimuli and genotype interact to determine temporal discounting in a rapid-acquisition procedure

The importance of delay discounting to many socially important behavior problems has stimulated investigations of biological and environmental mechanisms responsible for variations in the form of the discount function. The extant experimental research, however, has yielded disparate results, raising important questions regarding Gene X Environment interactions. The present study determined the influence of stimuli that uniquely signal delays to reinforcement on delay discounting in two inbred mouse strains using a rapid-acquisition procedure. BALB/c and C57BL/6 mice responded under a six-component, concurrent-chained schedule in which the terminal-link delays preceding the larger-reinforcer were presented randomly across components of an individual session. Across conditions, components were presented either with or without delay-specific auditory stimuli, i.e., as multiple or mixed schedules. A generalized matching-based model was used to incorporate the impact of current and previous component reinforcer-delay ratios on current component response allocation. Sensitivity to reinforcer magnitude and delay were higher for BALB/c mice, but within-component preference reached final levels faster for C57Bl/6 mice. For BALB/c mice, acquisition of preference across blocks of a component was faster under the multiple than the mixed schedule, but final levels of sensitivity to reinforcement were unaffected by schedule. The speed of acquisition of preference was not different across schedules for C57Bl/6 mice, but sensitivity to reinforcement was higher under the multiple than the mixed schedule. Overall, differences in the acquisition and final form of the discount function were determined by a Gene X Environment interaction, but the presence of delay-specific stimuli attenuated genotype-dependent differences in magnitude and delay sensitivity.