The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions

Mental time travel in animals: the 'when' of mental time travel

While many aspects of cognition have been shown to be shared between humans and non-human animals, there remains controversy regarding whether the capacity to mentally time travel is a uniquely human one. In this paper, we argue that there are four ways of representing when some event happened: four kinds of temporal representation. Distinguishing these four kinds of temporal representation has five benefits. First, it puts us in a position to determine the particular benefits these distinct temporal representations afford an organism. Second, it provides the conceptual resources to foster a discussion about which of these representations is necessary for an organism to count as having the capacity to mentally time travel. Third, it enables us to distinguish stricter from more liberal views of mental time travel that differ regarding which kind(s) of temporal representation is taken to be necessary for mental time travel. Fourth, it allows us to determine the benefits of taking a stricter or more liberal view of mental time travel. Finally, it ensures that disagreement about whether some species can mentally time travel is not merely the product of unrecognized disagreement about which temporal representation is necessary for mental time travel. We argue for a more liberal view, on the grounds that it allows us to view mental time travel as an evolutionarily continuous phenomenon and to recognize that differences in the ways that organisms mentally time travel might reflect different temporal representations, or combinations thereof, that they employ. Our ultimate aim, however, is to create a conceptual framework for further discussion regarding what sorts of temporal representations are required for mental time travel.This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Deep Brain Stimulation and Neuropsychiatric Anthropology - The "Prosthetisability" of the Lifeworld

Deep Brain Stimulation (DBS) represents a key area of neuromodulation that has gained wide adoption for the treatment of neurological and experimental testing for psychiatric disorders. It is associated with specific therapeutic effects based on the precision of an evolving mechanistic neuroscientific understanding. At the same time, there are obstacles to achieving symptom relief because of the incompleteness of such an understanding. These obstacles are at least in part based on the complexity of neuropsychiatric disorders and the incompleteness of DBS devices to represent prosthetics that modulate the breadth of pathological processes implicated in these disorders. Neuroprostheses, such as an implanted DBS system, can have vast effects on subjects in addition to the specific neuropsychiatric changes they are intended to produce. These effects largely represent blind spots in the current debate on neuromodulation. Anthropological accounts can illustrate the broad existential dimensions of patients' illness and responses to neural implants. In combination with current neuroscientific understanding, neuropsychiatric anthropology may illuminate the possibilities and limits of neurodevices as technical "world enablers".

Complementary cognitive roles for D2-MSNs and D1-MSNs during interval timing

The role of striatal pathways in cognitive processing is unclear. We studied dorsomedial striatal cognitive processing during interval timing, an elementary cognitive task that requires mice to estimate intervals of several seconds and involves working memory for temporal rules as well as attention to the passage of time. We harnessed optogenetic tagging to record from striatal D2-dopamine receptor-expressing medium spiny neurons (D2-MSNs) in the indirect pathway and from D1-dopamine receptor-expressing MSNs (D1-MSNs) in the direct pathway. We found that D2-MSNs and D1-MSNs exhibited distinct dynamics over temporal intervals as quantified by principal component analyses and trial-by-trial generalized linear models. MSN recordings helped construct and constrain a four-parameter drift-diffusion computational model. This model predicted that disrupting either D2-MSNs or D1-MSNs would increase interval timing response times and alter MSN firing. In line with this prediction, we found that optogenetic inhibition or pharmacological disruption of either D2-MSNs or D1-MSNs increased interval timing response times. Pharmacologically disrupting D2-MSNs or D1-MSNs also changed MSN dynamics and degraded trial-by-trial temporal decoding. Together, our findings demonstrate that D2-MSNs and D1-MSNs make complementary contributions to interval timing despite opposing dynamics, implying that striatal direct and indirect pathways work together to shape temporal control of action. These data provide novel insight into basal ganglia cognitive operations beyond movement and have implications for human striatal diseases and therapies targeting striatal pathways.

Analysis of fast calcium dynamics of honey bee olfactory coding

Odour processing exhibits multiple parallels between vertebrate and invertebrate olfactory systems. Insects, in particular, have emerged as relevant models for olfactory studies because of the tractability of their olfactory circuits. Here, we used fast calcium imaging to track the activity of projection neurons in the honey bee antennal lobe (AL) during olfactory stimulation at high temporal resolution. We observed a heterogeneity of response profiles and an abundance of inhibitory activities, resulting in various response latencies and stimulus-specific post-odour neural signatures. Recorded calcium signals were fed to a mushroom body (MB) model constructed implementing the fundamental features of connectivity between olfactory projection neurons, Kenyon cells (KC), and MB output neurons (MBON). The model accounts for the increase of odorant discrimination in the MB compared to the AL and reveals the recruitment of two distinct KC populations that represent odorants and their aftersmell as two separate but temporally coherent neural objects. Finally, we showed that the learning-induced modulation of KC-to-MBON synapses can explain both the variations in associative learning scores across different conditioning protocols used in bees and the bees' response latency. Thus, it provides a simple explanation of how the time contingency between the stimulus and the reward can be encoded without the need for time tracking. This study broadens our understanding of olfactory coding and learning in honey bees. It demonstrates that a model based on simple MB connectivity rules and fed with real physiological data can explain fundamental aspects of odour processing and associative learning.

Insights into the interaction between time and reward prediction on the activity of striatal tonically active neurons: A pilot study in rhesus monkeys

Prior studies have documented the role of the striatum and its dopaminergic input in time processing, but the contribution of local striatal cholinergic innervation has not been specifically investigated. To address this issue, we recorded the activity of tonically active neurons (TANs), thought to be cholinergic interneurons in the striatum, in two male macaques performing self-initiated movements after specified intervals in the seconds range have elapsed. The behavioral data showed that movement timing was adjusted according to the temporal requirements. About one-third of all recorded TANs displayed brief depressions in firing in response to the cue that indicates the interval duration, and the strength of these modulations was, in some instances, related to the timing of movement. The rewarding outcome of actions also impacted TAN activity, as reflected by stronger responses to the cue paralleled by weaker responses to reward when monkeys performed correctly timed movements over consecutive trials. It therefore appears that TAN responses may act as a start signal for keeping track of time and reward prediction could be incorporated in this signaling function. We conclude that the role of the striatal cholinergic TAN system in time processing is embedded in predicting rewarding outcomes during timing behavior.

Mechanically Induced Motor Tremors Disrupt the Perception of Time

Contemporary research has begun to show a strong relationship between movements and the perception of time. More specifically, concurrent movements serve to both bias and enhance time estimates. To explain these effects, we recently proposed a mechanism by which movements provide a secondary channel for estimating duration that is combined optimally with sensory estimates. However, a critical test of this framework is that by introducing "noise" into movements, sensory estimates of time should similarly become noisier. To accomplish this, we had human participants move a robotic arm while estimating intervals of time in either auditory or visual modalities (n = 24, ea.). Crucially, we introduced an artificial "tremor" in the arm while subjects were moving, that varied across three levels of amplitude (1-3 N) or frequency (4-12 Hz). The results of both experiments revealed that increasing the frequency of the tremor led to noisier estimates of duration. Further, the effect of noise varied with the base precision of the interval, such that a naturally less precise timing (i.e., visual) was more influenced by the tremor than a naturally more precise modality (i.e., auditory). To explain these findings, we fit the data with a recently developed drift-diffusion model of perceptual decision-making, in which the momentary, within-trial variance was allowed to vary across conditions. Here, we found that the model could recapitulate the observed findings, further supporting the theory that movements influence perception directly. Overall, our findings support the proposed framework, and demonstrate the utility of inducing motor noise via artificial tremors.

Crossmodal to unimodal transfer of temporal perceptual learning

Subsecond temporal processing is crucial for activities requiring precise timing. Here, we investigated perceptual learning of crossmodal (auditory-visual or visual-auditory) temporal interval discrimination (TID) and its impacts on unimodal (visual or auditory) TID performance. The research purpose was to test whether learning is based on a more abstract and conceptual representation of subsecond time, which would predict crossmodal to unimodal learning transfer. The experiments revealed that learning to discriminate a 200-ms crossmodal temporal interval, defined by a pair of visual and auditory stimuli, significantly reduced crossmodal TID thresholds. Moreover, the crossmodal TID training also minimized unimodal TID thresholds with a pair of visual or auditory stimuli at the same interval, even if crossmodal TID thresholds are multiple times higher than unimodal TID thresholds. Subsequent training on unimodal TID failed to reduce unimodal TID thresholds further. These results indicate that learning of high-threshold crossmodal TID tasks can benefit low-threshold unimodal temporal processing, which may be achieved through training-induced improvement of a conceptual representation of subsecond time in the brain.

Recall tempo of Hebbian sequences depends on the interplay of Hebbian kernel with tutor signal timing

Understanding how neural circuits generate sequential activity is a longstanding challenge. While foundational theoretical models have shown how sequences can be stored as memories in neural networks with Hebbian plasticity rules, these models considered only a narrow range of Hebbian rules. Here, we introduce a model for arbitrary Hebbian plasticity rules, capturing the diversity of spike-timing-dependent synaptic plasticity seen in experiments, and show how the choice of these rules and of neural activity patterns influences sequence memory formation and retrieval. In particular, we derive a general theory that predicts the tempo of sequence replay. This theory lays a foundation for explaining how cortical tutor signals might give rise to motor actions that eventually become "automatic." Our theory also captures the impact of changing the tempo of the tutor signal. Beyond shedding light on biological circuits, this theory has relevance in artificial intelligence by laying a foundation for frameworks whereby slow and computationally expensive deliberation can be stored as memories and eventually replaced by inexpensive recall.

Integrator dynamics in the cortico-basal ganglia loop underlie flexible motor timing

Flexible control of motor timing is crucial for behavior. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviors, distinguishing their individual roles in this timer function remains challenging. To address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings suggest the striatum is a part of the network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.

Temporal information in the anterior cingulate cortex relates to accumulated experiences

Anterior cingulate cortex (ACC) activity is important for operations that require the ability to integrate multiple experiences over time, such as rule learning, cognitive flexibility, working memory, and long-term memory recall. To shed light on this, we analyzed neuronal activity while rats repeated the same behaviors during hour-long sessions to investigate how activity changed over time. We recorded neuronal ensembles as rats performed a decision-free operant task with varying reward likelihoods at three different response ports (n = 5). Neuronal state space analysis revealed that each repetition of a behavior was distinct, with more recent behaviors more similar than those further apart in time. ACC activity was dominated by a slow, gradual change in low-dimensional representations of neural state space aligning with the pace of behavior. Temporal progression, or drift, was apparent on the top principal component for every session and was driven by the accumulation of experiences and not an internal clock. Notably, these signals were consistent across subjects, allowing us to accurately predict trial numbers based on a model trained on data from a different animal. We observed that non-continuous ramping firing rates over extended durations (tens of minutes) drove the low-dimensional ensemble representations. 40% of ACC neurons' firing ramped over a range of trial lengths and combinations of shorter duration ramping neurons created ensembles that tracked longer durations. These findings provide valuable insights into how the ACC, at an ensemble level, conveys temporal information by reflecting the accumulation of experiences over extended periods.

Sustained behaviour: Encoding of cumulative experience in the anterior cingulate

Time is a ubiquitous dimension of behaviour. A new study demonstrates that low-dimensional temporal drift in rodent anterior cingulate ensembles encodes cumulative experience. These data provide fresh insight into how neurons encode extended periods of time to guide high-level behaviours.

Cortical neurite microstructural correlates of time perception in healthy older adults

The human experience is significantly impacted by timing as it structures how information is processed. Nevertheless, the neurological foundation of time perception remains largely unresolved. Understanding cortical microstructure related to timing is crucial for gaining insight into healthy aging and recognizing structural alterations that are typical of neurodegenerative diseases associated with age. Given the importance, this study aimed to determine the brain regions that are accountable for predicting time perception in older adults using microstructural measures of the brain. In this study, elderly healthy adults performed the Time-Wall Estimation task to measure time perception through average error time. We used support vector regression (SVR) analyses to predict the average error time using cortical neurite microstructures derived from orientation dispersion and density imaging based on multi-shell diffusion magnetic resonance imaging (dMRI). We found significant correlations between observed and predicted average error times for neurite arborization (ODI) and free water (FISO). Neurite arborization and free water properties in specific regions in the medial and lateral prefrontal, superior parietal, and medial and lateral temporal lobes were among the most significant predictors of timing ability in older adults. Further, our results revealed that greater branching along with lower free water in cortical structures result in shorter average error times. Future studies should assess whether these same networks are contributing to time perception in older adults with mild cognitive impairment (MCI) and whether degeneration of these networks contribute to early diagnosis or detection of dementia.

Goal-Directed Learning is Multidimensional and Accompanied by Diverse and Widespread Changes in Neocortical Signaling

New tasks are often learned in stages with each stage reflecting a different learning challenge. Accordingly, each learning stage is likely mediated by distinct neuronal processes. And yet, most rodent studies of the neuronal correlates of goal-directed learning focus on individual outcome measures and individual brain regions. Here, we longitudinally studied mice from naïve to expert performance in a head-fixed, operant conditioning whisker discrimination task. In addition to tracking the primary behavioral outcome of stimulus discrimination, we tracked and compared an array of object-based and temporal-based behavioral measures. These behavioral analyses identify multiple, partially overlapping learning stages in this task, consistent with initial response implementation, early stimulus-response generalization, and late response inhibition. To begin to understand the neuronal foundations of these learning processes, we performed widefield Ca2+ imaging of dorsal neocortex throughout learning and correlated behavioral measures with neuronal activity. We found distinct and widespread correlations between neocortical activation patterns and various behavioral measures. For example, improvements in sensory discrimination correlated with target stimulus evoked activations of licking-related cortices along with distractor stimulus evoked global cortical suppression. Our study reveals multidimensional learning for a simple goal-directed learning task and generates hypotheses for the neuronal modulations underlying these various learning processes.

Effects of caffeine on temporal perception in Rattus norvegicus

We report two studies that tested the effects of caffeine, the world's most widely used psychoactive drug, on temporal perception. We trained Wistar rats using the Bisection Procedure (Experiment 1) or the Stubbs' Procedure (Experiment 2) to discriminate between short and long light stimuli. Once training finished, we administered caffeine orally (0, 9.6, and 96.0 mg/kg for Experiment 1 and 0, 9.6, 19.2, and 38.4 mg/kg for Experiment 2) 15 minutes prior to testing. Relative to the control condition, the 9.6 mg/kg condition (Experiments 1 and 2) and the 19.2 mg/kg condition (Experiment 2) resulted in an increase in proportion of choosing the long response. Meanwhile, overall accuracy was not affected by any condition in both experiments. Taken together, these results are consistent with the notion that caffeine, at some doses, speeds up temporal perception. However, it is not clear why the effect disappears at higher doses.

Ex Vivo Cortical Circuits Learn to Predict and Spontaneously Replay Temporal Patterns

It has been proposed that prediction and timing are computational primitives of neocortical microcircuits, specifically, that neural mechanisms are in place to allow neocortical circuits to autonomously learn the temporal structure of external stimuli and generate internal predictions. To test this hypothesis, we trained cortical organotypic slices on two specific temporal patterns using dual-optical stimulation. After 24-hours of training, whole-cell recordings revealed network dynamics consistent with training-specific timed prediction. Unexpectedly, there was replay of the learned temporal structure during spontaneous activity. Furthermore, some neurons exhibited timed prediction errors. Mechanistically our results indicate that learning relied in part on asymmetric connectivity between distinct neuronal ensembles with temporally-ordered activation. These findings further suggest that local cortical microcircuits are intrinsically capable of learning temporal information and generating predictions, and that the learning rules underlying temporal learning and spontaneous replay can be intrinsic to local cortical microcircuits and not necessarily dependent on top-down interactions.

Exploring the correlation and causation between alpha oscillations and one-second time perception through EEG and tACS

Alpha oscillations have been implicated in time perception, yet a consensus on their precise role remains elusive. This study directly investigates this relationship by examining the impact of alpha oscillations on time perception. Resting-state EEG recordings were used to extract peak alpha frequency (PAF) and peak alpha power (PAP) characteristics. Participants then performed a time generalization task under transcranial alternating current stimulation (tACS) at frequencies of PAF-2, PAF, and PAF+2, as well as a sham condition. Results revealed a significant correlation between PAP and accuracy, and between PAF and precision of one-second time perception in the sham condition. This suggests that alpha oscillations may influence one-second time perception by modulating their frequency and power. Interestingly, these correlations weakened with real tACS stimulations, particularly at higher frequencies. A second analysis aimed to establish a causal relationship between alpha peak modulation by tACS and time perception using repeated measures ANOVAs, but no significant effect was observed. Results were interpreted according to the state-dependent networks and internal clock model.

Prenatal exposure to metal mixtures and childhood temporal processing in the PROGRESS Birth Cohort Study: Modification by childhood obesity

Children are frequently exposed to various biological trace metals, some essential for their development, while others can be potent neurotoxicants. Furthermore, the inflammatory and metabolic conditions associated with obesity may interact with and amplify the impact of metal exposure on neurodevelopment. However, few studies have assessed the potential modification effect of body mass index (BMI). As a result, we investigated the role of child BMI phenotype on the relationship between prenatal exposure to metal mixtures and temporal processing. Leveraging the PROGRESS birth cohort in Mexico City, children (N = 563) aged 6-9 years completed a Temporal Response Differentiation (TRD) task where they had to hold a lever down for 10-14 s. Blood and urinary metal (As, Pb, Cd, and Mn) measurements were collected from mothers in the 2nd and 3rd trimesters. Child BMI z-scores were dichotomized to normal (between -2 and +0.99) and high (≥1.00). Covariate-adjusted weighted quantile sum (WQS) regression models were used to estimate and examine the combined effect of metal biomarkers (i.e., blood and urine) on TRD measures. Effect modification by the child's BMI was evaluated using 2-way interaction terms. Children with a high BMI and greater exposure to the metal mixture during prenatal development exhibited significant temporal processing deficits compared to children with a normal BMI. Notably, children with increased exposure to the metal mixture and higher BMI had a decrease in the percent of tasks completed (β = -10.13; 95 % CI: -19.84, -0.42), number of average holds (β = -2.15; 95 % CI: -3.88, -0.41), longer latency (β = 0.78; 95 % CI: 0.13, 1.44), and greater variability in the standard deviation of the total hold time (β = 2.08; 95 % CI: 0.34, 3.82) compared to normal BMI children. These findings implicate that high BMI may amplify the effect of metals on children's temporal processing. Understanding the relationship between metal exposures, temporal processing, and childhood obesity can provide valuable insights for developing targeted environmental interventions.

Temporal scaling of motor cortical dynamics reveals hierarchical control of vocal production

Neocortical activity is thought to mediate voluntary control over vocal production, but the underlying neural mechanisms remain unclear. In a highly vocal rodent, the male Alston's singing mouse, we investigate neural dynamics in the orofacial motor cortex (OMC), a structure critical for vocal behavior. We first describe neural activity that is modulated by component notes (~100 ms), probably representing sensory feedback. At longer timescales, however, OMC neurons exhibit diverse and often persistent premotor firing patterns that stretch or compress with song duration (~10 s). Using computational modeling, we demonstrate that such temporal scaling, acting through downstream motor production circuits, can enable vocal flexibility. These results provide a framework for studying hierarchical control circuits, a common design principle across many natural and artificial systems.

The perceptual timescape: Perceptual history on the sub-second scale

There is a high-capacity store of brief time span (∼1000 ms) which information enters from perceptual processing, often called iconic memory or sensory memory. It is proposed that a main function of this store is to hold recent perceptual information in a temporally segregated representation, named the perceptual timescape. The perceptual timescape is a continually active representation of change and continuity over time that endows the perceived present with a perceived history. This is accomplished primarily by two kinds of time marking information: time distance information, which marks all items of information in the perceptual timescape according to how far in the past they occurred, and ordinal temporal information, which organises items of information in terms of their temporal order. Added to that is information about connectivity of perceptual objects over time. These kinds of information connect individual items over a brief span of time so as to represent change, persistence, and continuity over time. It is argued that there is a one-way street of information flow from perceptual processing either to the perceived present or directly into the perceptual timescape, and thence to working memory. Consistent with that, the information structure of the perceptual timescape supports postdictive reinterpretations of recent perceptual information. Temporal integration on a time scale of hundreds of milliseconds takes place in perceptual processing and does not draw on information in the perceptual timescape, which is concerned with temporal segregation, not integration.

Embodying Time in the Brain: A Multi-Dimensional Neuroimaging Meta-Analysis of 95 Duration Processing Studies

Time is an omnipresent aspect of almost everything we experience internally or in the external world. The experience of time occurs through such an extensive set of contextual factors that, after decades of research, a unified understanding of its neural substrates is still elusive. In this study, following the recent best-practice guidelines, we conducted a coordinate-based meta-analysis of 95 carefully-selected neuroimaging papers of duration processing. We categorized the included papers into 14 classes of temporal features according to six categorical dimensions. Then, using the activation likelihood estimation (ALE) technique we investigated the convergent activation patterns of each class with a cluster-level family-wise error correction at p < 0.05. The regions most consistently activated across the various timing contexts were the pre-SMA and bilateral insula, consistent with an embodied theory of timing in which abstract representations of duration are rooted in sensorimotor and interoceptive experience, respectively. Moreover, class-specific patterns of activation could be roughly divided according to whether participants were timing auditory sequential stimuli, which additionally activated the dorsal striatum and SMA-proper, or visual single interval stimuli, which additionally activated the right middle frontal and inferior parietal cortices. We conclude that temporal cognition is so entangled with our everyday experience that timing stereotypically common combinations of stimulus characteristics reactivates the sensorimotor systems with which they were first experienced.

Direct contribution of the sensory cortex to the judgment of stimulus duration

Decision making frequently depends on monitoring the duration of sensory events. To determine whether, and how, the perception of elapsed time derives from the neuronal representation of the stimulus itself, we recorded and optogenetically modulated vibrissal somatosensory cortical activity as male rats judged vibration duration. Perceived duration was dilated by optogenetic excitation. A second set of rats judged vibration intensity; here, optogenetic excitation amplified the intensity percept, demonstrating sensory cortex to be the common gateway both to time and to stimulus feature processing. A model beginning with the membrane currents evoked by vibrissal and optogenetic drive and culminating in the representation of perceived time successfully replicated rats' choices. Time perception is thus as deeply intermeshed within the sensory processing pathway as is the sense of touch itself, suggesting that the experience of time may be further investigated with the toolbox of sensory coding.

Sub-second and multi-second dopamine dynamics underlie variability in human time perception

Timing behaviour and the perception of time are fundamental to cognitive and emotional processes in humans. In non-human model organisms, the neuromodulator dopamine has been associated with variations in timing behaviour, but the connection between variations in dopamine levels and the human experience of time has not been directly assessed. Here, we report how dopamine levels in human striatum, measured with sub-second temporal resolution during awake deep brain stimulation surgery, relate to participants' perceptual judgements of time intervals. Fast, phasic, dopaminergic signals were associated with underestimation of temporal intervals, whereas slower, tonic, decreases in dopamine were associated with poorer temporal precision. Our findings suggest a delicate and complex role for the dynamics and tone of dopaminergic signals in the conscious experience of time in humans.

Keeping time and rhythm by internal simulation of sensory stimuli and behavioral actions

Effective behavior often requires synchronizing our actions with changes in the environment. Rhythmic changes in the environment are easy to predict, and we can readily time our actions to them. Yet, how the brain encodes and maintains rhythms is not known. Here, we trained primates to internally maintain rhythms of different tempos and performed large-scale recordings of neuronal activity across the sensory-motor hierarchy. Results show that maintaining rhythms engages multiple brain areas, including visual, parietal, premotor, prefrontal, and hippocampal regions. Each recorded area displayed oscillations in firing rates and oscillations in broadband local field potential power that reflected the temporal and spatial characteristics of an internal metronome, which flexibly encoded fast, medium, and slow tempos. The presence of widespread metronome-related activity, in the absence of stimuli and motor activity, suggests that internal simulation of stimuli and actions underlies timekeeping and rhythm maintenance.

Brain mechanisms of temporal processing in impulsivity: Relevance to attention-deficit hyperactivity disorder

In this article, we critique the hypothesis that different varieties of impulsivity, including impulsiveness present in attention-deficit hyperactivity disorder, encompass an accelerated perception of time. This conceptualisation provides insights into how individuals with attention-deficit hyperactivity disorder have the capacity to maximise cognitive capabilities by more closely aligning themselves with appropriate environmental contexts (e.g. fast paced tasks that prevent boredom). We discuss the evidence for altered time perception in attention-deficit hyperactivity disorder alongside putative underlying neurobiological substrates, including a distributed brain network mediating time perception over multiple timescales. In particular, we explore the importance of temporal representations across the brain for time perception and symptom manifestation in attention-deficit hyperactivity disorder, including a prominent role of the hippocampus and other temporal lobe regions. We also reflect on how abnormalities in the perception of time may be relevant for understanding the aetiology of attention-deficit hyperactivity disorder and mechanism of action of existing medications.

Neural Sequences and the Encoding of Time

Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.

Estimating Time and Rhythm by Predicting External Stimuli

In this chapter, we present recent findings from our group showing that elapsed time, interval timing, and rhythm maintenance might be achieved by the well-known ability of the brain to predict the future states of the world. The difference between predictions and actual sensory evidence is used to generate perceptual and behavioral adjustments that help subjects achieve desired behavioral goals. Concretely, we show that (1) accumulating prediction errors is a plausible strategy humans could use to determine whether a train of consecutive stimuli arrives at regular or irregular intervals. By analyzing the behavior of human and non-human primate subjects performing rhythm perception tasks, we demonstrate that (2) the ability to estimate elapsed time and internally maintain rhythms is shared across primates and humans. Neurophysiological recordings show that (3) the medial premotor cortex engages in rhythm entrainment and maintains oscillatory activity that reveals an internal metronome's spatial and temporal characteristics. Finally, we demonstrate that (4) the amplitude of gamma oscillations within this cortex increases proportionally to the total elapsed time. In conjunction with our most recent experiments, our results suggest that timing might be achieved by an internal simulation of the sensory stimuli and the motor commands that define the timing task that needs to be performed.

The neural basis of somatosensory temporal discrimination threshold as a paradigm for time processing in the sub-second range: An updated review

BACKGROUND AND OBJECTIVE

The temporal aspect of somesthesia is a feature of any somatosensory process and a pre-requisite for the elaboration of proper behavior. Time processing in the milliseconds range is crucial for most of behaviors in everyday life. The somatosensory temporal discrimination threshold (STDT) is the ability to perceive two successive stimuli as separate in time, and deals with time processing in this temporal range. Herein, we focus on the physiology of STDT, on a background of the anatomophysiology of somesthesia and the neurobiological substrates of timing.

METHODS

A review of the literature through PubMed & Cochrane databases until March 2023 was performed with inclusion and exclusion criteria following PRISMA recommendations.

RESULTS

1151 abstracts were identified. 4 duplicate records were discarded before screening. 957 abstracts were excluded because of redundancy, less relevant content or not English-written. 4 were added after revision. Eventually, 194 articles were included.

CONCLUSIONS

STDT encoding relies on intracortical inhibitory S1 function and is modulated by the basal ganglia-thalamic-cortical interplay through circuits involving the nigrostriatal dopaminergic pathway and probably the superior colliculus.

Coordinate-Based Meta-Analyses of the Time Perception Network

The study of time perception has advanced over the past three decades to include numerous neuroimaging studies, most notably including the use of functional Magnetic Resonance Imaging (fMRI). Yet, with this increase in studies, there comes the desire to draw broader conclusions across datasets about the nature and instantiation of time in the human brain. In the absence of collating individual studies together, the field has employed the use of Coordinate-Based Meta-Analyses (CBMA), in which foci from individual studies are modeled as probability distributions within the brain, from which common areas of activation-likelihood are determined. This chapter provides an overview of these CBMA studies, the methods they employ, the conclusions drawn by them, and where future areas of inquiry lie. The result of this survey suggests the existence of a domain-general "timing network" that can be used both as a guide for individual neuroimaging studies and as a template for future meta-analyses.

Cognition of Time and Thinking Beyond

A common research protocol in cognitive neuroscience is to train subjects to perform deliberately designed experiments while recording brain activity, with the aim of understanding the brain mechanisms underlying cognition. However, how the results of this protocol of research can be applied in technology is seldom discussed. Here, I review the studies on time processing of the brain as examples of this research protocol, as well as two main application areas of neuroscience (neuroengineering and brain-inspired artificial intelligence). Time processing is a fundamental dimension of cognition, and time is also an indispensable dimension of any real-world signal to be processed in technology. Therefore, one may expect that the studies of time processing in cognition profoundly influence brain-related technology. Surprisingly, I found that the results from cognitive studies on timing processing are hardly helpful in solving practical problems. This awkward situation may be due to the lack of generalizability of the results of cognitive studies, which are under well-controlled laboratory conditions, to real-life situations. This lack of generalizability may be rooted in the fundamental unknowability of the world (including cognition). Overall, this paper questions and criticizes the usefulness and prospect of the abovementioned research protocol of cognitive neuroscience. I then give three suggestions for future research. First, to improve the generalizability of research, it is better to study brain activity under real-life conditions instead of in well-controlled laboratory experiments. Second, to overcome the unknowability of the world, we can engineer an easily accessible surrogate of the object under investigation, so that we can predict the behavior of the object under investigation by experimenting on the surrogate. Third, the paper calls for technology-oriented research, with the aim of technology creation instead of knowledge discovery.

Neural circuit models for evidence accumulation through choice-selective sequences

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.

Core clock gene, Bmal1, is required for optimal second-level interval production

Perception and production of second-level temporal intervals are critical in several behavioral and cognitive processes, including adaptive anticipation, motor control, and social communication. These processes are impaired in several neurological and psychological disorders, such as Parkinson's disease and attention-deficit hyperactivity disorder. Although evidence indicates that second-level interval timing exhibit circadian patterns, it remains unclear whether the core clock machinery controls the circadian pattern of interval timing. To investigate the role of core clock molecules in interval timing capacity, we devised a behavioral assay called the interval timing task to examine prospective motor interval timing ability. In this task, the mouse produces two separate nose pokes in a pretrained second-level interval to obtain a sucrose solution as a reward. We discovered that interval perception in wild-type mice displayed a circadian pattern, with the best performance observed during the late active phase. To investigate whether the core molecular clock is involved in the circadian control of interval timing, we employed Bmal1 knockout mice (BKO) in the interval timing task. The interval production of BKO did not display any difference between early and late active phase, without reaching the optimal interval production level observed in wild-type. In summary, we report that the core clock gene Bmal1 is required for the optimal performance of prospective motor timing typically observed during the late part of the active period.

Influences of temporal order in temporal reproduction

Despite the crucial role of complex temporal sequences, such as speech and music, in our everyday lives, our ability to acquire and reproduce these patterns is prone to various contextual biases. In this study, we examined how the temporal order of auditory sequences affects temporal reproduction. Participants were asked to reproduce accelerating, decelerating or random sequences, each consisting of four intervals, by tapping their fingers. Our results showed that the reproduction and the reproduction variability were influenced by the sequential structure and interval orders. The mean reproduced interval was assimilated by the first interval of the sequence, with the lowest mean for decelerating and the highest for accelerating sequences. Additionally, the central tendency bias was affected by the volatility and the last interval of the sequence, resulting in a stronger central tendency in the random and decelerating sequences than the accelerating sequence. Using Bayesian integration between the ensemble mean of the sequence and individual durations and considering the perceptual uncertainty associated with the sequential structure and position, we were able to accurately predict the behavioral results. The findings highlight the critical role of the temporal order of a sequence in temporal pattern reproduction, with the first interval exerting greater influence on mean reproduction and the volatility and the last interval contributing to the perceptual uncertainty of individual intervals and the central tendency bias.

Effective connectivity in a duration selective cortico-cerebellar network

How the human brain represents millisecond unit of time is far from clear. A recent neuroimaging study revealed the existence in the human premotor cortex of a topographic representation of time i.e., neuronal units selectively responsive to specific durations and topographically organized on the cortical surface. By using high resolution functional Magnetic Resonance Images here, we go beyond this previous work, showing duration preferences across a wide network of cortical and subcortical brain areas: from cerebellum to primary visual, parietal, premotor and prefrontal cortices. Most importantly, we identify the effective connectivity structure between these different brain areas and their duration selective neural units. The results highlight the role of the cerebellum as the network hub and that of medial premotor cortex as the final stage of duration recognition. Interestingly, when a specific duration is presented, only the communication strength between the units selective to that specific duration and to the neighboring durations is affected. These findings link for the first time, duration preferences within single brain region with connectivity dynamics between regions, suggesting a communication mode that is partially duration specific.

GABAergic signaling shapes multiple aspects of Drosophila courtship motor behavior

Inhibitory neurons are essential for orchestrating and structuring behavior. We use one of the best studied behaviors in Drosophila, male courtship, to analyze how inhibitory, GABAergic neurons shape the different steps of this multifaceted motor sequence. RNAi-mediated knockdown of the GABA-producing enzyme GAD1 and the ionotropic receptor Rdl in sex specific, fruitless expressing neurons in the ventral nerve cord causes uncoordinated and futile copulation attempts, defects in wing extension choice and severe alterations of courtship song. Altered song of GABA depleted males fails to stimulate female receptivity, but rescue of song patterning alone is not sufficient to rescue male mating success. Knockdown of GAD1 and Rdl in male brain circuits abolishes courtship conditioning. We characterize the around 220 neurons coexpressing GAD1 and Fruitless in the Drosophila male nervous system and propose inhibitory circuit motifs underlying key features of courtship behavior based on the observed phenotypes.

Time for Memories

The ability to store information about the past to dynamically predict and prepare for the future is among the most fundamental tasks the brain performs. To date, the problems of understanding how the brain stores and organizes information about the past (memory) and how the brain represents and processes temporal information for adaptive behavior have generally been studied as distinct cognitive functions. This Symposium explores the inherent link between memory and temporal cognition, as well as the potential shared neural mechanisms between them. We suggest that working memory and implicit timing are interconnected and may share overlapping neural mechanisms. Additionally, we explore how temporal structure is encoded in associative and episodic memory and, conversely, the influences of episodic memory on subsequent temporal anticipation and the perception of time. We suggest that neural sequences provide a general computational motif that contributes to timing and working memory, as well as the spatiotemporal coding and recall of episodes.

Lost in time: Relocating the perception of duration outside the brain

It is well-accepted in neuroscience that animals process time internally to estimate the duration of intervals lasting between one and several seconds. More than 100 years ago, Henri Bergson nevertheless remarked that, because animals have memory, their inner experience of time is ever-changing, making duration impossible to measure internally and time a source of change. Bergson proposed that quantifying the inner experience of time requires its externalization in movements (observed or self-generated), as their unfolding leaves measurable traces in space. Here, studies across species are reviewed and collectively suggest that, in line with Bergson's ideas, animals spontaneously solve time estimation tasks through a movement-based spatialization of time. Moreover, the well-known scalable anticipatory responses of animals to regularly spaced rewards can be explained by the variable pressure of time on reward-oriented actions. Finally, the brain regions linked with time perception overlap with those implicated in motor control, spatial navigation and motivation. Thus, instead of considering time as static information processed by the brain, it might be fruitful to conceptualize it as a kind of force to which animals are more or less sensitive depending on their internal state and environment.

Context-specific and context-invariant computations of interval timing

Introduction

An accurate sense of time is crucial in flexible sensorimotor control and other cognitive functions. However, it remains unknown how multiple timing computations in different contexts interact to shape our behavior.

Methods

We asked 41 healthy human subjects to perform timing tasks that differed in the sensorimotor domain (sensory timing vs. motor timing) and effector (hand vs. saccadic eye movement). To understand how these different behavioral contexts contribute to timing behavior, we applied a three-stage Bayesian model to behavioral data.

Results

Our results demonstrate that the Bayesian model for each effector could not describe bias in the other effector. Similarly, in each task the model-predicted data could not describe bias in the other task. These findings suggest that the measurement stage of interval timing is context-specific in the sensorimotor and effector domains. We also showed that temporal precision is context-invariant in the effector domain, unlike temporal accuracy.

Discussion

This combination of context-specific and context-invariant computations across sensorimotor and effector domains suggests overlapping and distributed computations as the underlying mechanism of timing in different contexts.

Emergence of belief-like representations through reinforcement learning

To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.

Keep your finger on the pulse: Better rate perception and gap detection with vibrotactile compared to visual stimuli

Important characteristics of the environment can be represented in the temporal pattern of sensory stimulation. In two experiments, we compared accuracy of temporal processing by different modalities. Experiment 1 examined binary categorization of rate for visual (V) or vibrotactile (T) stimulus pulses presented at either 4 or 6 Hz. Inter-pulse intervals were either constant or variable, perturbed by random Gaussian variates. Subjects categorized the rate of T pulse sequences more accurately than V sequences. In V conditions only, subjects disproportionately tended to mis-categorize 4-Hz pulse rates, for all but the most variable sequences. In Experiment 2, we compared gap detection thresholds across modalities, using the same V and T pulses from Experiment 1, as well as with bimodal (VT) pulses. Visual gap detection thresholds were larger (3[Formula: see text]) than tactile thresholds. Additionally, performance with VT stimuli seemed to be nearly completely dominated by their T components. Together, these results suggest (i) that vibrotactile temporal acuity surpasses visual temporal acuity, and (ii) that vibrotactile stimulation has considerable, untapped potential to convey temporal information like that needed for eyes-free alerting signals.

Using temperature to analyze the neural basis of a time-based decision

The basal ganglia are thought to contribute to decision-making and motor control. These functions are critically dependent on timing information, which can be extracted from the evolving state of neural populations in their main input structure, the striatum. However, it is debated whether striatal activity underlies latent, dynamic decision processes or kinematics of overt movement. Here, we measured the impact of temperature on striatal population activity and the behavior of rats, and compared the observed effects with neural activity and behavior collected in multiple versions of a temporal categorization task. Cooling caused dilation, and warming contraction, of both neural activity and patterns of judgment in time, mimicking endogenous decision-related variability in striatal activity. However, temperature did not similarly affect movement kinematics. These data provide compelling evidence that the timecourse of evolving striatal activity dictates the speed of a latent process that is used to guide choices, but not continuous motor control. More broadly, they establish temporal scaling of population activity as a likely neural basis for variability in timing behavior.

Neural substrates of top-down processing during perceptual duration-based timing and beat-based timing

Temporal context is a crucial factor in timing. Previous studies have revealed that the timing of regular stimuli, such as isochronous beats or rhythmic sequences (termed beat-based timing), activated the basal ganglia, whereas the timing of single intervals or irregular stimuli (termed duration-based timing) activated the cerebellum. We conducted a functional magnetic resonance imaging (fMRI) experiment to determine whether top-down processing of perceptual duration-based and beat-based timings affected brain activation patterns. Our participants listened to auditory sequences containing both single intervals and isochronous beats and judged either the duration of the intervals or the tempo of the beats. Whole-brain analysis revealed that both duration judgments and tempo judgments activated similar areas, including the basal ganglia and cerebellum, with no significant difference in the activated regions between the two conditions. In addition, an analysis of the regions of interest revealed no significant differences between the activation levels measured for the two tasks in the basal ganglia as well as the cerebellum. These results suggested that a set of common brain areas were involved in top-down processing of both duration judgments and tempo judgments. Our findings indicate that perceptual duration-based timing and beat-based timing are driven by stimulus regularity irrespective of top-down processing.

Multiplexing working memory and time in the trajectories of neural networks

Working memory (WM) and timing are generally considered distinct cognitive functions, but similar neural signatures have been implicated in both. To explore the hypothesis that WM and timing may rely on shared neural mechanisms, we used psychophysical tasks that contained either task-irrelevant timing or WM components. In both cases, the task-irrelevant component influenced performance. We then developed recurrent neural network (RNN) simulations that revealed that cue-specific neural sequences, which multiplexed WM and time, emerged as the dominant regime that captured the behavioural findings. During training, RNN dynamics transitioned from low-dimensional ramps to high-dimensional neural sequences, and depending on task requirements, steady-state or ramping activity was also observed. Analysis of RNN structure revealed that neural sequences relied primarily on inhibitory connections, and could survive the deletion of all excitatory-to-excitatory connections. Our results indicate that in some instances WM is encoded in time-varying neural activity because of the importance of predicting when WM will be used.

Multimodal Temporal Pattern Discrimination Is Encoded in Visual Cortical Dynamics

Discriminating between temporal features in sensory stimuli is critical to complex behavior and decision-making. However, how sensory cortical circuit mechanisms contribute to discrimination between subsecond temporal components in sensory events is unclear. To elucidate the mechanistic underpinnings of timing in primary visual cortex (V1), we recorded from V1 using two-photon calcium imaging in awake-behaving mice performing a go/no-go discrimination timing task, which was composed of patterns of subsecond audiovisual stimuli. In both conditions, activity during the early stimulus period was temporally coordinated with the preferred stimulus. However, while network activity increased in the preferred condition, network activity was increasingly suppressed in the nonpreferred condition over the stimulus period. Multiple levels of analyses suggest that discrimination between subsecond intervals that are contained in rhythmic patterns can be accomplished by local neural dynamics in V1.

The Self-Simulational Theory of temporal extension

Subjective experience is experience in time. Unfolding in a continuous river of moments, our experience, however, consists not only in the changing phenomenological content per se but, further, in additional retrodiction and prospection of the moments that immediately preceded and followed it. It is in this way that William James's 'specious present' presents itself as extending between the past and future. While the phenomenology of temporality always happens, in normal waking states, to someone, and the notions of self-representation and temporal experience have continuously been associated with each other, there has not yet been an explicit account of their relationship. In this paper, the emergence of the subjective experience of temporal extension will be conceived of as arising out of a difference-relation between counterfactual and actual self-representations. After presenting the proposed relationship on both a conceptual level and a formalized and neuronally realistic level of description using information theory, convergent empirical evidence from general findings about temporal experience and inference, altered states of consciousness, and mental illness is examined. The Self-Simulational Theory of temporal extension is able to explain systematic variations in the subjectively experienced length of the temporal Now across numerous domains and holds potentially wide implications for the neuroscience of consciousness, as well as for a deeper understanding of different forms of mental illness.

Optogenetic manipulation of inhibitory interneurons can be used to validate a model of spatiotemporal sequence learning

The brain uses temporal information to link discrete events into memory structures supporting recognition, prediction, and a wide variety of complex behaviors. It is still an open question how experience-dependent synaptic plasticity creates memories including temporal and ordinal information. Various models have been proposed to explain how this could work, but these are often difficult to validate in a living brain. A recent model developed to explain sequence learning in the visual cortex encodes intervals in recurrent excitatory synapses and uses a learned offset between excitation and inhibition to generate precisely timed "messenger" cells that signal the end of an instance of time. This mechanism suggests that the recall of stored temporal intervals should be particularly sensitive to the activity of inhibitory interneurons that can be easily targeted in vivo with standard optogenetic tools. In this work we examined how simulated optogenetic manipulations of inhibitory cells modifies temporal learning and recall based on these mechanisms. We show that disinhibition and excess inhibition during learning or testing cause characteristic errors in recalled timing that could be used to validate the model in vivo using either physiological or behavioral measurements.

Musical Training Facilitates Exogenous Temporal Attention via Delta Phase Entrainment within a Sensorimotor Network

Temporal orienting of attention plays an important role in our day-to-day lives and can use timing information from exogenous or endogenous sources. Yet, it is unclear what neural mechanisms give rise to temporal attention, and it is debated whether both exogenous and endogenous forms of temporal attention share a common neural source. Here, older adult nonmusicians (N = 47, 24 female) were randomized to undergo 8 weeks of either rhythm training, which places demands on exogenous temporal attention, or word search training as a control. The goal was to assess (1) the neural basis of exogenous temporal attention and (2) whether training-induced improvements in exogenous temporal attention can transfer to enhanced endogenous temporal attention abilities, thereby providing support for a common neural mechanism of temporal attention. Before and after training, exogenous temporal attention was assessed using a rhythmic synchronization paradigm, whereas endogenous temporal attention was evaluated via a temporally cued visual discrimination task. Results showed that rhythm training improved performance on the exogenous temporal attention task, which was associated with increased intertrial coherence within the δ (1-4 Hz) band as assessed by EEG recordings. Source localization revealed increased δ-band intertrial coherence arose from a sensorimotor network, including premotor cortex, anterior cingulate cortex, postcentral gyrus, and the inferior parietal lobule. Despite these improvements in exogenous temporal attention, such benefits were not transferred to endogenous attentional ability. These results support the notion that exogenous and endogenous temporal attention uses independent neural sources, with exogenous temporal attention relying on the precise timing of δ band oscillations within a sensorimotor network.SIGNIFICANCE STATEMENT Allocating attention to specific points in time is known as temporal attention, and may arise from external (exogenous) or internal (endogenous) sources. Despite its importance to our daily lives, it is unclear how the brain gives rise to temporal attention and whether exogenous- or endogenous-based sources for temporal attention rely on shared brain regions. Here, we demonstrate that musical rhythm training improves exogenous temporal attention, which was associated with more consistent timing of neural activity in sensory and motor processing brain regions. However, these benefits did not extend to endogenous temporal attention, indicating that temporal attention relies on different brain regions depending on the source of timing information.

Space is a late heuristic of elapsing time: New evidence from the STEARC effect

To get a concrete representation of its intangible flow, culture frames elapsing time along spatially oriented mental or graphical lines, which are organised according to reading habits, from left to right in western cultures. One of the strongest evidence for this spatial representation of time is the STEARC effect (Spatial-Temporal Association of Response Codes), which consists of faster coding of "short" durations with motor responses in the left side of space and of "long" durations with responses in the right side. Here, we investigated the STEARC as a function of response speed in two different experiments in healthy participants. Surprisingly, in both sub- and supra-second ranges, we found the STEARC only when decisions on time durations were slow, while no spatial representation of time was present with fast decisions. This first demonstrates that space slowly takes over faster non-spatial processing of time flow and that it is possible to empirically separate the behavioural manifestations of the non-spatial and the nurtured spatial mechanisms of time coding.

Learning long-term motor timing/patterns on an orthogonal basis in random neural networks

The ability of the brain to generate complex spatiotemporal patterns with specific timings is essential for motor learning and temporal processing. An approach that can model this function, using the spontaneous activity of a random neural network (RNN), is associated with orbital instability. We propose a simple system that learns an arbitrary time series as the linear sum of stable trajectories produced by several small network modules. New finding in computer experiments is that the trajectories of the module outputs are orthogonal to each other. They created a dynamic orthogonal basis acquiring a high representational capacity, which enabled the system to learn the timing of extremely long intervals, such as tens of seconds for a millisecond computation unit, and also the complex time series of Lorenz attractors. This self-sustained system satisfies the stability and orthogonality requirements and thus provides a new neurocomputing framework and perspective for the neural mechanisms of motor learning.

Emergence of belief-like representations through reinforcement learning

To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming "beliefs"-optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN's learned representation encodes belief information, but only when the RNN's capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.

Author Summary

Natural environments are full of uncertainty. For example, just because my fridge had food in it yesterday does not mean it will have food today. Despite such uncertainty, animals can estimate which states and actions are the most valuable. Previous work suggests that animals estimate value using a brain area called the basal ganglia, using a process resembling a reinforcement learning algorithm called TD learning. However, traditional reinforcement learning algorithms cannot accurately estimate value in environments with state uncertainty (e.g., when my fridge's contents are unknown). One way around this problem is if agents form "beliefs," a probabilistic estimate of how likely each state is, given any observations so far. However, estimating beliefs is a demanding process that may not be possible for animals in more complex environments. Here we show that an artificial recurrent neural network (RNN) trained with TD learning can estimate value from observations, without explicitly estimating beliefs. The trained RNN's error signals resembled the neural activity of dopamine neurons measured during the same task. Importantly, the RNN's activity resembled beliefs, but only when the RNN had enough capacity. This work illustrates how animals could estimate value in uncertain environments without needing to first form beliefs, which may be useful in environments where computing the true beliefs is too costly.