Mice optimize timed decisions about probabilistic outcomes under deadlines

Mice are Near Optimal Timers

Many conventional interval timing tasks do not contain asymmetric cost (loss) functions and thereby favor high temporal accuracy. On the other hand, asymmetric cost functions that differentially penalize/reinforce the early or late responses result in adaptive biases (shift) in timed responses due to timing uncertainty. Consequently, optimal performance in these tasks entails the normative parametrization of adaptive timing biases by the level of timing uncertainty. Differential reinforcement of response duration (DRRD) is one of these tasks that require mice to actively respond (e.g., continuously depressing a lever) for a minimum amount of time to be reinforced. The active production of a time interval by mice in DRRD differentiates this task from the differential reinforcement of low rates of responding (DRL) task as a passive waiting task that was used in earlier studies to investigate the optimality of adaptive biases in timing behavior. We tested 21 Th-Cre male mice (9 weeks old) in a DRRD task with a minimum requirement of 2 s. Mean response durations were positively biased (longer than the minimum requirement), and the extent of bias was predicted by the level of endogenous timing uncertainty. Mice nearly maximized the reward rate in this task. These results contribute to the accumulating evidence supporting optimal temporal risk assessment in non-human animals.

Numerical averaging in mice

Rodents can be trained to associate different durations with different stimuli (e.g., light/sound). When the associated stimuli are presented together, maximal responding is observed around the average of individual durations (akin to averaging). The current study investigated whether mice can also average independently trained numerosities. Mice were initially trained to make 10 or 20 lever presses on a single (run) lever to obtain a reward and each fixed-ratio schedule was signaled either with an auditory or visual stimulus. Then, mice were trained to press another lever to obtain the reward after they responded on the run lever for the minimum number of presses [Fixed Consecutive Number (FCN)-10 or -20 trials] signaled by the corresponding discriminative stimulus. Following this training, FCN trials with the compound stimulus were introduced to test the counting behavior of mice when they encountered conflicting information regarding the number of responses required to obtain the reward. Our results showed that the numbers of responses on these compound test trials were around the average of the number of responses in FCN-10 and FCN-20 trials particularly when the auditory stimulus was associated with a fewer number of required responses. The counting strategy explained the behavior of the majority of the mice in the FCN-Compound test trials (as opposed to the timing strategy). The number of responses in FCN-Compound trials was accounted for equally well by the arithmetic, geometric, and Bayesian averages of the number of responses observed in FCN-10 and FCN-20 trials.

Integrating Models of Interval Timing and Reinforcement Learning

We present an integrated view of interval timing and reinforcement learning (RL) in the brain. The computational goal of RL is to maximize future rewards, and this depends crucially on a representation of time. Different RL systems in the brain process time in distinct ways. A model-based system learns 'what happens when', employing this internal model to generate action plans, while a model-free system learns to predict reward directly from a set of temporal basis functions. We describe how these systems are subserved by a computational division of labor between several brain regions, with a focus on the basal ganglia and the hippocampus, as well as how these regions are influenced by the neuromodulator dopamine.

Interval timing is disrupted in female 5xFAD mice: An indication of altered memory processes

Temporal information processing in the seconds-to-minutes range is disrupted in patients with Alzheimer's disease (AD). In this study, we investigated the timing behavior of the 5xFAD mouse model of AD in the peak interval (PI) procedure. Nine-month-old female mice were trained with sucrose solution reinforcement for their first response after a fixed-interval (FI) and tested in the inter-mixed non-reinforced PI trials that lasted longer than FI. Timing performance indices were estimated from steady-state timed anticipatory nose-poking responses in the PI trials. We found that the time of maximal reward expectancy (peak time) of the 5xFAD mice was significantly earlier than that of the wild-type (WT) controls with no differences in other indices of timing performance. These behavioral differences corroborate the findings of previous studies on the disruption of temporal associative memory abilities of 5xFAD mice and can be accounted for by the scalar timing theory based on altered long-term memory consolidation of temporal information in the 5xFAD mice. This is the first study to directly show an interval timing phenotype in a genetic mouse model of AD.

Recalibrating timing behavior via expected covariance between temporal cues

Individuals must predict future events to proactively guide their behavior. Predicting when events will occur is a critical component of these expectations. Temporal expectations are often generated based on individual cue-duration relationships. However, the durations associated with different environmental cues will often co-vary due to a common cause. We show that timing behavior may be calibrated based on this expected covariance, which we refer to as the 'common cause hypothesis'. In five experiments using rats, we found that when the duration associated with one temporal cue changes, timed-responding to other cues shift in the same direction. Furthermore, training subjects that expecting covariance is not appropriate in a given situation blocks this effect. Finally, we confirmed that this transfer is context-dependent. These results reveal a novel principle that modulates timing behavior, which we predict will apply across a variety of magnitude-expectations.

Striatal dopamine and the temporal control of behavior

Striatal dopamine strongly regulates how individuals use time to guide behavior. Dopamine acts on D1- and D2- dopamine receptors in the striatum. However, the relative role of these receptors in the temporal control of behavior is unclear. To assess this, we trained rats on a task in which they decided to start and stop a series of responses based on the passage of time and evaluated how blocking D1 or D2-dopamine receptors in the dorsomedial or dorsolateral striatum impacted performance. D2 blockade delayed the decision to start and stop responding in both regions, and this effect was larger in the dorsomedial striatum. By contrast, dorsomedial D1 blockade delayed stop times, without significantly delaying start times, whereas dorsolateral D1 blockade produced no detectable effects. These findings suggest that striatal dopamine may tune decision thresholds during timing tasks. Furthermore, our data indicate that the dorsomedial striatum plays a key role in temporal control, which may be useful for localizing neural circuits that mediate the temporal control of action.

Spontaneous integration of temporal information: implications for representational/computational capacity of animals

How do animals adapt their behaviors to changing conditions? This question relates to the debate between associative versus representational/computational approaches in cognitive science. An influential line of research that has significantly shaped the conceptual development of animal learning over decades has primarily focused on the role of associative dynamics with little-to-no ascription of representational/combinatorial capacities. The common assumption of these models is that behavioral adjustments are incremental and they result from updating of associations based on actions and their outcomes, without encoding the critical information serving as the determinant(s) of such contingencies (e.g., time in interval schedules, number in ratio schedules). On the other hand, an independent line of research provides evidence for behavioral phenomena that cannot be readily accounted for by the conventional associationist approach. In this paper, we will review different sets of findings particularly in the area of interval timing that suggest the ability of animals to make swift spontaneous computations on subjective quantities and incorporate them into their behavior. Findings of these studies constitute empirical challenges for the associationist approaches to behavioral flexibility. We argue that interval timing is a fertile ground for the formulation of critical tests of different theoretical approaches to animal behavior.