Adapting the flow of time with dopamine

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

Interaction between spatial perception and temporal perception enables preservation of cause-effect relationship: Visual psychophysics and neuronal dynamics

INTRODUCTION

Visual perception of moving objects is integral to our day-to-day life, integrating visual spatial and temporal perception. Most research studies have focused on finding the brain regions activated during motion perception. However, an empirically validated general mathematical model is required to understand the modulation of the motion perception. Here, we develop a mathematical formulation of the modulation of the perception of a moving object due to a change in speed, under the formulation of the invariance of causality.

METHODS

We formulated the perception of a moving object as the coordinate transformation from a retinotopic space onto perceptual space and derived a quantitative relationship between spatiotemporal coordinates. To validate our model, we undertook the analysis of two experiments: (i) the perceived length of the moving arc, and (ii) the perceived time while observing moving stimuli. We performed a magnetic resonance imaging (MRI) tractography investigation of subjects to demarcate the anatomical correlation of the modulation of the perception of moving objects.

RESULTS

Our theoretical model shows that the interaction between visual-spatial and temporal perception, during the perception of moving object is described by coupled linear equations; and experimental observations validate our model. We observed that cerebral area V5 may be an anatomical correlate for this interaction. The physiological basis of interaction is shown by a Lotka-Volterra system delineating interplay between acetylcholine and dopamine neurotransmitters, whose concentrations vary periodically with the orthogonal phase shift between them, occurring at the axodendritic synapse of complex cells at area V5.

CONCLUSION

Under the invariance of causality in the representation of events in retinotopic space and perceptual space, the speed modulates the perception of a moving object. This modulation may be due to variations of the tuning properties of complex cells at area V5 due to the dynamic interaction between acetylcholine and dopamine. Our analysis is the first significant study, to our knowledge, that establishes a mathematical linkage between motion perception and causality invariance.

Revealing human sensitivity to a latent temporal structure of changes

Precisely timed behavior and accurate time perception plays a critical role in our everyday lives, as our wellbeing and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modeling, we found that human subjects' beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in the active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry.

Dopamine mediates the bidirectional update of interval timing

The role of dopamine (DA) as a reward prediction error (RPE) signal in reinforcement learning (RL) tasks has been well-established over the past decades. Recent work has shown that the RPE interpretation can also account for the effects of DA on interval timing by controlling the speed of subjective time. According to this theory, the timing of the dopamine signal relative to reward delivery dictates whether subjective time speeds up or slows down: Early DA signals speed up subjective time and late signals slow it down. To test this bidirectional prediction, we reanalyzed measurements of dopaminergic neurons in the substantia nigra pars compacta of mice performing a self-timed movement task. Using the slope of ramping dopamine activity as a readout of subjective time speed, we found that trial-by-trial changes in the slope could be predicted from the timing of dopamine activity on the previous trial. This result provides a key piece of evidence supporting a unified computational theory of RL and interval timing. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The neural bases for timing of durations

Durations are defined by a beginning and an end, and a major distinction is drawn between durations that start in the present and end in the future ('prospective timing') and durations that start in the past and end either in the past or the present ('retrospective timing'). Different psychological processes are thought to be engaged in each of these cases. The former is thought to engage a clock-like mechanism that accurately tracks the continuing passage of time, whereas the latter is thought to engage a reconstructive process that utilizes both temporal and non-temporal information from the memory of past events. We propose that, from a biological perspective, these two forms of duration 'estimation' are supported by computational processes that are both reliant on population state dynamics but are nevertheless distinct. Prospective timing is effectively carried out in a single step where the ongoing dynamics of population activity directly serve as the computation of duration, whereas retrospective timing is carried out in two steps: the initial generation of population state dynamics through the process of event segmentation and the subsequent computation of duration utilizing the memory of those dynamics.

Review: Subjective Time Perception, Dopamine Signaling, and Parkinsonian Slowness

The association between idiopathic Parkinson's disease, a paradigmatic dopamine-deficiency syndrome, and problems in the estimation of time has been studied experimentally for decades. I review that literature, which raises a question about whether and if dopamine deficiency relates not only to the motor slowness that is an objective and cardinal parkinsonian sign, but also to a compromised neural substrate for time perception. Why does a clinically (motorically) significant deficiency in dopamine play a role in the subjective perception of time's passage? After a discussion of a classical conception of basal ganglionic control of movement under the influence of dopamine, I describe recent work in healthy mice using optogenetics; the methodology visualizes dopaminergic neuronal firing in very short time intervals, then allows for correlation with motor behaviors in trained tasks. Moment-to-moment neuronal activity is both highly dynamic and variable, as assessed by photometry of genetically defined dopaminergic neurons. I use those animal data as context to review a large experimental experience in humans, spanning decades, that has examined subjective time perception mainly in Parkinson's disease, but also in other movement disorders. Although the human data are mixed in their findings, I argue that loss of dynamic variability in dopaminergic neuronal activity over very short intervals may be a fundamental sensory aspect in the pathophysiology of parkinsonism. An important implication is that therapeutic response in Parkinson's disease needs to be understood in terms of short-term alterations in dynamic neuronal firing, as has already been examined in novel ways—for example, in the study of real-time changes in neuronal network oscillations across very short time intervals. A finer analysis of a treatment's network effects might aid in any effort to augment clinical response to either medications or functional neurosurgical interventions in Parkinson's disease.

Performance-gated deliberation: A context-adapted strategy in which urgency is opportunity cost

Finding the right amount of deliberation, between insufficient and excessive, is a hard decision making problem that depends on the value we place on our time. Average-reward, putatively encoded by tonic dopamine, serves in existing reinforcement learning theory as the opportunity cost of time, including deliberation time. Importantly, this cost can itself vary with the environmental context and is not trivial to estimate. Here, we propose how the opportunity cost of deliberation can be estimated adaptively on multiple timescales to account for non-stationary contextual factors. We use it in a simple decision-making heuristic based on average-reward reinforcement learning (AR-RL) that we call Performance-Gated Deliberation (PGD). We propose PGD as a strategy used by animals wherein deliberation cost is implemented directly as urgency, a previously characterized neural signal effectively controlling the speed of the decision-making process. We show PGD outperforms AR-RL solutions in explaining behaviour and urgency of non-human primates in a context-varying random walk prediction task and is consistent with relative performance and urgency in a context-varying random dot motion task. We make readily testable predictions for both neural activity and behaviour.

A proxy measure of striatal dopamine predicts individual differences in temporal precision

The perception of time is characterized by pronounced variability across individuals, with implications for a diverse array of psychological functions. The neurocognitive sources of this variability are poorly understood, but accumulating evidence suggests a role for inter-individual differences in striatal dopamine levels. Here we present a pre-registered study that tested the predictions that spontaneous eyeblink rates, which provide a proxy measure of striatal dopamine availability, would be associated with aberrant interval timing (lower temporal precision or overestimation bias). Neurotypical adults (N = 69) underwent resting state eye tracking and completed visual psychophysical interval timing and control tasks. Elevated spontaneous eyeblink rates were associated with poorer temporal precision but not with inter-individual differences in perceived duration or performance on the control task. These results signify a role for striatal dopamine in variability in human time perception and can help explain deficient temporal precision in psychiatric populations characterized by elevated dopamine levels.

Slowly evolving dopaminergic activity modulates the moment-to-moment probability of reward-related self-timed movements

Clues from human movement disorders have long suggested that the neurotransmitter dopamine plays a role in motor control, but how the endogenous dopaminergic system influences movement is unknown. Here we examined the relationship between dopaminergic signaling and the timing of reward-related movements in mice. Animals were trained to initiate licking after a self-timed interval following a start-timing cue; reward was delivered in response to movements initiated after a criterion time. The movement time was variable from trial-to-trial, as expected from previous studies. Surprisingly, dopaminergic signals ramped-up over seconds between the start-timing cue and the self-timed movement, with variable dynamics that predicted the movement/reward time on single trials. Steeply rising signals preceded early lick-initiation, whereas slowly rising signals preceded later initiation. Higher baseline signals also predicted earlier self-timed movements. Optogenetic activation of dopamine neurons during self-timing did not trigger immediate movements, but rather caused systematic early-shifting of movement initiation, whereas inhibition caused late-shifting, as if modulating the probability of movement. Consistent with this view, the dynamics of the endogenous dopaminergic signals quantitatively predicted the moment-by-moment probability of movement initiation on single trials. We propose that ramping dopaminergic signals, likely encoding dynamic reward expectation, can modulate the decision of when to move.

Dopamine and the interdependency of time perception and reward

Time is a fundamental dimension of our perception of the world and is therefore of critical importance to the organization of human behavior. A corpus of work - including recent optogenetic evidence - implicates striatal dopamine as a crucial factor influencing the perception of time. Another stream of literature implicates dopamine in reward and motivation processes. However, these two domains of research have remained largely separated, despite neurobiological overlap and the apothegmatic notion that "time flies when you're having fun". This article constitutes a review of the literature linking time perception and reward, including neurobiological and behavioral studies. Together, these provide compelling support for the idea that time perception and reward processing interact via a common dopaminergic mechanism.

Bidirectional role of microtubule dynamics in the acquisition and maintenance of temporal information in dorsolateral striatum

Accurate and precise timing is crucial for complex and purposeful behaviors, such as foraging for food or playing a musical instrument. The brain is capable of processing temporal information in a coordinated manner, as if it contains an 'internal clock'. Similar to the need for the brain to orient itself in space in order to understand its surroundings, temporal orientation and tracking is an essential component of cognition as well. While there have been multiple models explaining the neural correlates of timing, independent lines of research appear to converge on the conclusion that populations of neurons in the dorsal striatum encode information relating to where a subject is in time relative to an anticipated goal. Similar to other learning processes, acquisition and maintenance of this temporal information is dependent on synaptic plasticity. Microtubules are cytoskeletal proteins that have been implicated in synaptic plasticity mechanisms and therefore are considered key elements in learning and memory. In this study, we investigated the role of microtubule dynamics in temporal learning by local infusions of microtubule stabilizing and destabilizing agents into the dorsolateral striatum. Our results suggested a bidirectional role for microtubules in timing, such that microtubule stabilization improves the maintenance of learned target durations, but impairs the acquisition of a novel duration. On the other hand, microtubule destabilization enhances the acquisition of novel target durations, while compromising the maintenance of previously learned durations. These findings suggest that microtubule dynamics plays an important role in synaptic plasticity mechanisms in the dorsolateral striatum, which in turn modulates temporal learning and time perception.

Awareness of errors and feedback in human time estimation

Behavioral and electrophysiology studies have shown that humans possess a certain self-awareness of their individual timing ability. However, conflicting reports raise concerns about whether humans can discern the direction of their timing error, calling into question the extent of this timing awareness. To understand the depth of this ability, the impact of nondirectional feedback and reinforcement learning on time perception were examined in a unique temporal reproduction paradigm that involved a mixed set of interval durations and the opportunity to repeat every trial immediately after receiving feedback, essentially allowing a “redo.” Within this task, we tested two groups of participants on versions where nondirectional feedback was provided after every response, or not provided at all. Participants in both groups demonstrated reduced central tendency and exhibited significantly greater accuracy in the redo trial temporal estimates, showcasing metacognitive ability, and an inherent capacity to adjust temporal responses despite the lack of directional information or any feedback at all. Additionally, the feedback group also exhibited an increase in the precision of responses on the redo trials, an effect not observed in the no-feedback group, suggesting that feedback may specifically reduce noise when making a temporal estimate. These findings enhance our understanding of timing self-awareness and can provide insight into what may transpire when this is disrupted.

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.