Effort as a dimension of spontaneous activity in rats

Effort-reward balance and work motivation in rats: Effects of context and order of experience

Being motivated means exerting effort toward a goal. The 'law of least work' emphasizes a preference for exerting relatively less effort. The law crosses boundaries among species and between physical and mental work. Organisms should be highly sensitive to shifts in effort-reward balance (ERB) in order to make optimal choices. We used a free operant-foraging task to investigate changes in ERB on choice between options requiring more or less effort. Results showed a consistent preference for the option with less effort and insensitivity to shifts in ERB. A second aim explored the influence of order of experience on effort choice. Choice for the more effortful option significantly increased after experiencing an equal effort-reward relationship during the initial free operant-foraging session. This relative increase in choice for the effortful option persisted even after effort-reward imbalance. The findings highlight the importance of contextual factors such as order of experience when examining the impact of shifting effort-reward associations. Instead of ignoring or reducing order effects, the sequence of experience (e.g. for shifts in ERB) could be manipulated to enhance or reduce value of outcomes or effort itself.

Fatigue modulates dopamine availability and promotes flexible choice reversals during decision making

During decisions, animals balance goal achievement and effort management. Despite physical exercise and fatigue significantly affecting the levels of effort that an animal exerts to obtain a reward, their role in effort-based choice and the underlying neurochemistry are incompletely known. In particular, it is unclear whether fatigue influences decision (cost-benefit) strategies flexibly or only post-decision action execution and learning. To answer this question, we trained mice on a T-maze task in which they chose between a high-cost, high-reward arm (HR), which included a barrier, and a low-cost, low-reward arm (LR), with no barrier. The animals were parametrically fatigued immediately before the behavioural tasks by running on a treadmill. We report a sharp choice reversal, from the HR to LR arm, at 80% of their peak workload (PW), which was temporary and specific, as the mice returned to choose the HC when the animals were successively tested at 60% PW or in a two-barrier task. These rapid reversals are signatures of flexible choice. We also observed increased subcortical dopamine levels in fatigued mice: a marker of individual bias to use model-based control in humans. Our results indicate that fatigue levels can be incorporated in flexible cost-benefits computations that improve foraging efficiency.

Dopamine and effort-based decision making

Motivational theories of choice focus on the influence of goal values and strength of reinforcement to explain behavior. By contrast relatively little is known concerning how the cost of an action, such as effort expended, contributes to a decision to act. Effort-based decision making addresses how we make an action choice based on an integration of action and goal values. Here we review behavioral and neurobiological data regarding the representation of effort as action cost, and how this impacts on decision making. Although organisms expend effort to obtain a desired reward there is a striking sensitivity to the amount of effort required, such that the net preference for an action decreases as effort cost increases. We discuss the contribution of the neurotransmitter dopamine (DA) toward overcoming response costs and in enhancing an animal's motivation toward effortful actions. We also consider the contribution of brain structures, including the basal ganglia and anterior cingulate cortex, in the internal generation of action involving a translation of reward expectation into effortful action.

Central mechanisms of HPA axis regulation by voluntary exercise

Stress exerts complex effects on the brain and periphery, dependent on the temporal profile and intensity of the stressor. The consequences of a stressful event can also be determined by other characteristics of the stressor, such as whether it is predictable and controllable. While the traditional view has focused primarily on the negative effects of stress on a variety of somatic systems, emerging data support the idea that certain forms of stress can enhance cellular function. Here we review the current literature on the hypothalamic-pituitary-adrenal (HPA) axis regulation by wheel running, a voluntary and controllable stressor with a distinct temporal profile. While running indeed activates a number of systems related to the stress response, other mechanisms exist to reduce the reactivity to this stressor, with possible crosstalk between running and other forms of stress.

A brief opportunity to run does not function as a reinforcer for mice selected for high daily wheel-running rates

Mice from replicate lines, selectively bred based on high daily wheel-running rates, run more total revolutions and at higher average speeds than do mice from nonselected control lines. Based on this difference it was assumed that selected mice would find the opportunity to run in a wheel a more efficacious consequence. To assess this assumption within an operant paradigm, mice must be trained to make a response to produce the opportunity to run as a consequence. In the present study an autoshaping procedure was used to compare the acquisition of lever pressing reinforced by the opportunity to run for a brief opportunity (i.e., 90 s) between selected and control mice and then, using an operant procedure, the effect of the duration of the opportunity to run on lever pressing was assessed by varying reinforcer duration over values of 90 s, 30 min, and 90 s. The reinforcement schedule was a ratio schedule (FR 1 or VR 3). Results from the autoshaping phase showed that more control mice met a criterion of responses on 50% of trials. During the operant phase, when reinforcer duration was 90 s, almost all control, but few selected mice completed a session of 20 reinforcers; however, when reinforcer duration was increased to 30 min almost all selected and control mice completed a session of 20 reinforcers. Taken together, these results suggest that selective breeding based on wheel-running rates over 24 hr may have altered the motivational system in a way that reduces the reinforcing value of shorter running durations. The implications of this finding for these mice as a model for attention deficit hyperactivity disorder (ADHD) are discussed. It also is proposed that there may be an inherent trade-off in the motivational system for activities of short versus long duration.

Wheel access duration in rats: I. Effects on feeding and running

The effects of 0-, 2-, and 24-hr wheel access on the pattern of running, feeding, and weight were explored over 24 days in 3 groups of 8 male rats. Both 2 and 24 hr of wheel access suppressed feeding by about 15% for about 8 days before feeding gradually returned to normal. Weight in these 2 groups was similar and was suppressed for the 24 days. Like the pattern seen with drug self-administration (S. H. Ahmed & G. F. Koob, 1998, 1999), running levels stayed low with short, 2-hr daytime wheel access, but with long, 24-hr access, rats' running escalated over days to chronically high levels. These results may have relevance for the understanding of addiction and anorexia nervosa.

Increased effort requirements and risk sensitivity: a comparison of delay and magnitude manipulations

Reward magnitude and delay to reward were independently manipulated in two separate experiments examining risk-sensitive choice in rats. A dual-running wheel apparatus was used and the tangential force resistance required to displace both wheels was low (50g) for half of the subjects, and high (120g) for the remaining subjects. Concurrent FI30-s and FI60-s schedules delivered equivalent amounts of food reward per unit time (i.e. 5 and 10 pellets of food, respectively), and these conditions served as the baseline treatment for all subjects. Variability, either in reward magnitude or delay, was introduced on the long-delay (60s) schedule during the second phase. All subjects were returned to the baseline condition in the third phase, and variability was introduced on the short-delay (30s) interval schedule during phase four. The subjects were again returned to the baseline condition in the fifth and final phase, ultimately yielding a five-phase ABACA design. Original baseline performance was characterized by a slight short-delay interval preference, and this pattern of performance was recovered with each subsequent presentation of the baseline condition. Overall, the data obtained from the reward magnitude and delay-to-reward manipulations were indistinguishable; subjects experiencing low-response effort requirement behaved in a risk-indifferent manner and subjects experiencing high-response effort requirement preferred the variable schedule. Implications for the daily energy budget rule on risk-sensitive foraging are discussed in light of these findings.

Determinants affecting physical activity levels in animal models

Weight control is dependent on energy balance. Reduced energy expenditure (EE) associated with decreased physical activity is suggested to be a major underlying cause in the increasing prevalence of weight gain and obesity. Therefore, a better understanding of the biological determinants involved in the regulation of physical activity is essential. To facilitate interpretation in humans, it is helpful to consider the evidence from animal studies. This review focuses on animal studies examining the biological determinants influencing activity and potential implications to human. It appears that physical activity is influenced by a number of parameters. However, regardless of the parameter involved, body weight appears to play an underlying role in the regulation of activity. Furthermore, the regulation of activity associated with body weight appears to occur only after the animal achieves a critical weight. This suggests that activity levels are a consequence rather than a contributor to weight control. However, the existence of an inverse weight-activity relationship remains inconclusive. Confounding the results are the multifactorial nature of physical activity and the lack of appropriate measuring devices. Furthermore, many determinants of body weight are closely interlocked, making it difficult to determine whether a single, combination, or interaction of factors is important for the regulation of activity. For example, diet-induced obesity, aging, lesions to the ventral medial hypothalamus, and genetics all produce hypoactivity. Providing a better understanding of the biological determinants involved in the regulation of activity has important implications for the development of strategies for the prevention of weight gain leading to obesity and subsequent morbidity and mortality in the human population.

The currency of procurement cost

As the number of instrtumental responses required to procure access to food is increased, animals decrease the frequency of initiating meals and increase meal size, conserving total intake while limiting the increase in the overall cost of feeding. In two studies, one using wheel turns and one using bar presses as the instrumental response, we asked whether freely feeding laboratory rats measure cost according to the energy or the time they expend. In each study we varied both the price (i.e., number of wheel turns or bar presses) and the force required to make a response (i.e., torque on the wheel or weight of the bar). Price affected both procurement time (from the first to the last procurement response) and procurement work, whereas torque and bar weight affected work without altering time in most cases. Meal patterns were altered by all manipulations of price, but changes in torque and bar weight had little effect on meal patterns, except in the conditions in which they altered procurement time. These results suggest that time is a critical currency of procurement cost in rats.

Habituation contributes to within-session changes in free wheel running

Three experiments tested the hypothesis that habituation contributes to the regulation of wheel running. Rats ran in a wheel for 30-min sessions. Experiment 1 demonstrated spontaneous recovery. Rats ran more and the within-session decreases in running were smaller after 2 days of wheel deprivation than after 1 day. Experiment 2 demonstrated dishabituation. Running rate increased immediately after the termination of a brief extra event (application of the brake or flashing of the houselight). Experiment 3 demonstrated stimulus specificity. Rats completed the second half of the session in either the same wheel as the first half, or a different wheel. Second-half running was faster in the latter case. Within-session patterns of running were well described by equations that describe data from the habituation, motivation, and operant literatures. These results suggest that habituation contributes to the regulation of running. In fact, habituation provides a better explanation for the termination of wheel running than fatigue, the variable to which this termination is usually attributed. Overall, the present findings are consistent with the proposition that habituation and sensitization contribute to the regulation of several forms of motivated behavior.

Risk-sensitive foraging in rats: the effects of response-effort and reward-amount manipulations on choice behavior

The literature on risk-sensitive foraging theory provides several accounts of species that fluctuate between risk-averse and risk-prone strategies. The daily energy budget rule suggests that shifts in foraging strategy are precipitated by changes in the forager's energy budget. Researchers have attempted to alter the organism's energy budget using a variety of techniques such as food deprivation, manipulation of ambient temperatures, and delays to food reward; however, response-effort manipulations have been relatively neglected. A choice preparation using a wheel-running response and rats examined risk-sensitive preferences when both response effort and reward amounts were manipulated. Concurrently available reinforcement schedules (FI/60 and VI/60) yielded equivalent food amounts per unit time in all treatments. Two levels of response effort (20 or 120 g tangential resistance) and two levels of reward amount (three or nine pellets) were combined to form four distinct response-effort/reward-amount pairings. Increasing reward amounts significantly shifted choice toward the FI schedule in both response-effort conditions. The incidence of choice preference and the magnitude of shifts in choice were greater for the high response-effort conditions than for the low response-effort conditions. Implications of the significant interaction between response effort and reward amount are discussed in terms of a general energy-budget model.

Voluntary wheel running: a review and novel interpretation

Voluntary wheel running by animals is an activity that has been observed and recorded in great detail for almost a century. This review shows that it is performed, often with startling intensity and coordination, by a wide variety of wild, laboratory and domestic species with diverse evolutionary histories. However, despite the plethora of published studies on wheel running, there is considerable disagreement between many findings, thus leading to a lack of consensus on explanations of the causality and function. In the initial part of this review, I discuss the internal and external factors that may be involved in the causality of this behaviour, with an emphasis on disparities in both the factual and theoretical development of the subject. I then address the various proposed functions of wheel running, again highlighting evidence to the contrary. This leads to the conclusion that any single theory on the basis of wheel running is likely to be simplistic with little generality. I then present a novel, behaviour-based interpretation in which it is argued that wheel running has no directly analogous naturally occurring behaviour, it is (sometimes) performed for its own sake per se rather than as a redirected or substitute activity, and studies on motivation show that wheel running is self-reinforcing and perceived by animals as 'important'. This review proposes that wheel running may be an artefact of captive environments or of the running-wheel itself, possibly resulting from feedback dysfunction. I also discuss the ubiquity and intensity of its performance, along with its great plasticity and maladaptiveness, all indicating that if it is an artefact, it is nevertheless one of great interest to behavioural science. Copyright 1998 The Association for the Study of Animal Behaviour.

Hippocampal serotonin mediates hypoactivity in dietarily obese hamsters: a possible manifestation of aging?

To determine whether endogenous opiates mediate hyperactivity in food restricted hamsters and serotonergic fibers innervating the hippocampus mediate hypoactivity in obese hamsters, food restriction and high-fat diet supplementation were used to produce significant body fat changes (8 vs. 21%). The levels and pattern of spontaneous running were examined after IP saline or naloxone HCl (20 mg/kg) and following the infusion of vehicle and 5,7-dihydroxytryptamine creatine sulfate (4 micrograms/2 microliters) into rostromedial septum of mature female hamsters. Septum-medial preoptic area (POA), hippocampus, hypothalamus, and cortex were dissected from the three groups as well as from two additional groups of hamsters receiving vehicle or neurotoxin. Concentrations of serotonin, norepinephrine, and dopamine were measured in these tissues by HPLC method. Fat-fed hamsters were hypoactive relative to food-restricted hamsters. Naloxone had no significant effect on running behavior. Serotonin neurotoxin increased the running activity of fat-fed hamsters to the level displayed by control hamsters by increasing the number of runs, the total activity level, the speed of running and by decreasing the duration of pauses. Neurotoxin led to selective deletion of serotonin in the hippocampus (77%) and parietal cortex (50%). Serotonergic fibers innervating the hippocampus thus appear to mediate the hypoactivity that is induced by dietary obesity in mature hamsters. Since serotonin mediates some other manifestations of aging, and slow weight increases characterize mid-portion of hamster life span, we hypothesize that serotonergic mediation of hypoactivity is another manifestation of aging.

Time budgets in growing chicks

The manner in which rapidly growing chicks distribute their time among diurnal activities was measured in two studies in which the amount of available light was varied to match seasonal extremes. The effectiveness of the chick's time budgets was assessed in terms of the impact on growth. In Experiment 1, 24-hr patterns of feeding, drinking, nesting, and wheel-running were recorded during successive photoperiods with 12, 6, 18, and 12 hr of light when access to a social partner was concurrently available. In Experiment 2, access to a social partner was an exclusive activity. In both studies, chicks' 24-hr behavioral patterns in response to temporal constraints on their diurnal activities were surprisingly plastic, permitting them to defend a normal and high rate of growth. This was accomplished by changes in feeding rate and by nocturnal feeding rather than by systematic elimination of other diurnal activities. When sociality and feeding were competing activities, feeding time decreased and feeding rate increased such that contact with a conspecific as well as rapid growth were defended. The absolute amount of time spent in measured activities was invariant within subjects irrespective of age, photoperiod, or the number and type of activities recorded, providing evidence of time budgets in immature organisms. In both studies, buffer time (the amount of time spent in no activity) emerged as a significant aspect of the daily time budget that is defended through a variety of environmental challenges.

Basis for the hypoactivity that accompanies rapid weight gain in hamsters

The pattern of hypoactivity that accompanies rapid weight gain following septal lesions in hamsters was characterized. Lesioned hamsters displayed reduced levels of running, shorter and slower running bouts, and longer pauses. We examined whether this hypoactivity was due to reassignment of metabolic fuels from supporting physical activity to anabolism, or due to reduced capacity of running to induce psychomotor arousal and mobilize metabolic fuels. Septal lesions were associated with increased rate of ponderal growth and higher titers of circulating growth hormone and insulin. No difference in concentrations of muscle and liver glycogen, percentage of body fat, or the capacity of muscle homogenates to oxidize substrates were identified. Lesioned hamsters ran as fast and as long as control animals on electrical-shock reinforced treadmill, but were unable to generate as much heat in response to injection (0.8 mg/kg) of norepinephrine. We concluded that hypoactivity that accompanies rapid weight gain in hamsters results either from a reduced capacity of running to induce psychomotor arousal and provide incentives that normally motivate that behavior, or from a failure of running to mobilize metabolic fuels at a rate necessary to sustain normal running speed and duration, and not from reduced availability of metabolic fuels or reduced muscle capacity to oxidize metabolic substrates.

Interrelationships among activity, food intake and weight gain in genetically obese and lean Zucker rats

In Experiment 1, food deprivation resulting in a 30% reduction in body weight produced significant increases in wheel running in both obese and lean female Zucker rats. In Experiment 2, a new technique, food contingent activity (FR, VI), dramatically increased wheel running in both obese and lean female Zucker rats. This increase in activity was achieved primarily during the dark period. Regardless of changes in activity levels, food intake and body weight gain remained similar to controls. When food was again available ad lib, activity levels rapidly decreased for obese but not lean rats. These results indicate that behavioral interventions alone are not sufficient to correct the obesity of the genetically obese rat.