Decoding of temporal intervals from cortical ensemble activity

Diverse Time Encoding Strategies Within the Medial Premotor Areas of the Primate

The measurement of time in the subsecond scale is critical for many sophisticated behaviors, yet its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the neural activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing. During single interval categorization, we found that preSMA encodes a subjective category limit by reaching a peak of activity at a time that divides the set of test intervals into short and long. We also observed neural signals associated with the category selected by the subjects and the reward outcomes of the perceptual decision. On the other hand, we have studied the behavioral and neurophysiological basis of rhythmic timing. First, we have shown in different tapping tasks that macaques are able to produce predictively and accurately intervals that are cued by auditory or visual metronomes or when intervals are produced internally without sensory guidance. In addition, we found that the rhythmic timing mechanism in MPC is governed by different layers of neural clocks. Next, the instantaneous activity of single cells shows ramping activity that encodes the elapsed or remaining time for a tapping movement. In addition, we found MPC neurons that build neural sequences, forming dynamic patterns of activation that flexibly cover all the produced interval depending on the tapping tempo. This rhythmic neural clock resets on every interval providing an internal representation of pulse. Furthermore, the MPC cells show mixed selectivity, encoding not only elapsed time, but also the tempo of the tapping and the serial order element in the rhythmic sequence. Hence, MPC can map different task parameters, including the passage of time, using different cell populations. Finally, the projection of the time varying activity of MPC hundreds of cells into a low dimensional state space showed circular neural trajectories whose geometry represented the internal pulse and the tapping tempo. Overall, these findings support the notion that MPC is part of the core timing mechanism for both single interval and rhythmic timing, using neural clocks with different encoding principles, probably to flexibly encode and mix the timing representation with other task parameters.

Decoding hand kinetics and kinematics using somatosensory cortex activity in active and passive movement

Area 2 of the primary somatosensory cortex (S1), encodes proprioceptive information of limbs. Several studies investigated the encoding of movement parameters in this area. However, the single-trial decoding of these parameters, which can provide additional knowledge about the amount of information available in sub-regions of this area about instantaneous limb movement, has not been well investigated. We decoded kinematic and kinetic parameters of active and passive hand movement during center-out task using conventional and state-based decoders. Our results show that this area can be used to accurately decode position, velocity, force, moment, and joint angles of hand. Kinematics had higher accuracies compared to kinetics and active trials were decoded more accurately than passive trials. Although the state-based decoder outperformed the conventional decoder in the active task, it was the opposite in the passive task. These results can be used in intracortical micro-stimulation procedures to provide proprioceptive feedback to BCI subjects.

Lost in time: Relocating the perception of duration outside the brain

It is well-accepted in neuroscience that animals process time internally to estimate the duration of intervals lasting between one and several seconds. More than 100 years ago, Henri Bergson nevertheless remarked that, because animals have memory, their inner experience of time is ever-changing, making duration impossible to measure internally and time a source of change. Bergson proposed that quantifying the inner experience of time requires its externalization in movements (observed or self-generated), as their unfolding leaves measurable traces in space. Here, studies across species are reviewed and collectively suggest that, in line with Bergson's ideas, animals spontaneously solve time estimation tasks through a movement-based spatialization of time. Moreover, the well-known scalable anticipatory responses of animals to regularly spaced rewards can be explained by the variable pressure of time on reward-oriented actions. Finally, the brain regions linked with time perception overlap with those implicated in motor control, spatial navigation and motivation. Thus, instead of considering time as static information processed by the brain, it might be fruitful to conceptualize it as a kind of force to which animals are more or less sensitive depending on their internal state and environment.

Intrinsic hippocampal connectivity is associated with individual differences in retrospective duration processing

The estimation of incidentally encoded durations of time intervals (retrospective duration processing) is thought to rely on the retrieval of contextual information associated with a sequence of events, automatically encoded in medial temporal lobe regions. "Time cells" have been described in the hippocampus (HC), encoding the temporal progression of events and their duration. However, whether the HC supports explicit retrospective duration judgments in humans, and which neural dynamics are involved, is still poorly understood. Here we used resting-state fMRI to test the relation between variations in intrinsic connectivity patterns of the HC, and individual differences in retrospective duration processing, assessed using a novel task involving the presentation of ecological stimuli. Results showed that retrospective duration discrimination performance predicted variations in the intrinsic connectivity of the bilateral HC with the right precentral gyrus; follow-up exploratory analyses suggested a role of the CA1 and CA4/DG subfields in driving the observed pattern. Findings provide insights on neural networks associated with implicit processing of durations in the second range.

Time encoding migrates from prefrontal cortex to dorsal striatum during learning of a self-timed response duration task

Although time is a fundamental dimension of life, we do not know how brain areas cooperate to keep track and process time intervals. Notably, analyses of neural activity during learning are rare, mainly because timing tasks usually require training over many days. We investigated how the time encoding evolves when animals learn to time a 1.5 s interval. We designed a novel training protocol where rats go from naive- to proficient-level timing performance within a single session, allowing us to investigate neuronal activity from very early learning stages. We used pharmacological experiments and machine-learning algorithms to evaluate the level of time encoding in the medial prefrontal cortex and the dorsal striatum. Our results show a double dissociation between the medial prefrontal cortex and the dorsal striatum during temporal learning, where the former commits to early learning stages while the latter engages as animals become proficient in the task.

The neural bases for timing of durations

Durations are defined by a beginning and an end, and a major distinction is drawn between durations that start in the present and end in the future ('prospective timing') and durations that start in the past and end either in the past or the present ('retrospective timing'). Different psychological processes are thought to be engaged in each of these cases. The former is thought to engage a clock-like mechanism that accurately tracks the continuing passage of time, whereas the latter is thought to engage a reconstructive process that utilizes both temporal and non-temporal information from the memory of past events. We propose that, from a biological perspective, these two forms of duration 'estimation' are supported by computational processes that are both reliant on population state dynamics but are nevertheless distinct. Prospective timing is effectively carried out in a single step where the ongoing dynamics of population activity directly serve as the computation of duration, whereas retrospective timing is carried out in two steps: the initial generation of population state dynamics through the process of event segmentation and the subsequent computation of duration utilizing the memory of those dynamics.

Bromazepam increases the error of the time interval judgments and modulates the EEG alpha asymmetry during time estimation

AIM

This study investigated the bromazepam effects in male subjects during the time estimation performance and EEG alpha asymmetry in electrodes associated with the frontal and motor cortex.

MATERIAL AND METHODS

This is a double-blind, crossover study with a sample of 32 healthy adults under control (placebo) vs. experimental (bromazepam) during visual time-estimation task in combination with electroencephalographic analysis.

RESULTS

The results demonstrated that the bromazepam increased the relative error in the 4 s, 7 s, and 9 s intervals (p = 0.001). In addition, oral bromazepam modulated the EEG alpha asymmetry in cortical areas during the time judgment (p ≤ 0.025).

CONCLUSION

The bromazepam decreases the precision of time estimation judgments and modulates the EEG alpha asymmetry, with greater left hemispheric dominance during time perception. Our findings suggest that bromazepam influences internal clock synchronization via the modulation of GABAergic receptors, strongly relating to attention, conscious perception, and behavioral performance.

Dissociable neural indices for time and space estimates during virtual distance reproduction

The perception and measurement of spatial and temporal dimensions have been widely studied. Yet, whether these two dimensions are processed independently is still being debated. Additionally, whether EEG components are uniquely associated with time or space, or whether they reflect a more general measure of magnitude quantity remains unknown. While undergoing EEG, subjects performed a virtual distance reproduction task, in which they were required to first walk forward for an unknown distance or time, and then reproduce that distance or time. Walking speed was varied between estimation and reproduction phases, to prevent interference between distance or time in each estimate. Behaviorally, subject performance was more variable when reproducing time than when reproducing distance, but with similar patterns of accuracy. During estimation, EEG data revealed the contingent negative variation (CNV), a measure previously associated with timing and expectation, tracked the probability of the upcoming interval, for both time and distance. However, during reproduction, the CNV exclusively oriented to the upcoming temporal interval at the start of reproduction, with no change across spatial distances. Our findings indicate that time and space are neurally separable dimensions, with the CNV both serving a supramodal role in temporal and spatial expectation, yet an exclusive role in preparing duration reproduction.

Decoding Movement From Electrocorticographic Activity: A Review

Electrocorticography (ECoG) holds promise to provide efficient neuroprosthetic solutions for people suffering from neurological disabilities. This recording technique combines adequate temporal and spatial resolution with the lower risks of medical complications compared to the other invasive methods. ECoG is routinely used in clinical practice for preoperative cortical mapping in epileptic patients. During the last two decades, research utilizing ECoG has considerably grown, including the paradigms where behaviorally relevant information is extracted from ECoG activity with decoding algorithms of different complexity. Several research groups have advanced toward the development of assistive devices driven by brain-computer interfaces (BCIs) that decode motor commands from multichannel ECoG recordings. Here we review the evolution of this field and its recent tendencies, and discuss the potential areas for future development.

Rats can learn a temporal task in a single session

Fixed interval, peak interval, and temporal bisection procedures have been used to assess cognitive functions and address questions such as how animals perceive, represent, and reproduce time intervals. They have also been extensively used to test the effects of drugs on behavior, and to describe the neural correlates of interval timing. However, those procedures usually require several weeks of training for behavior to stabilize. Here, we investigated a variation of the Differential Reinforcement of Response Duration (DRRD) task with a target time of 1.2 s. We compared three types of training protocols and reported a procedure in which performance by the end of the very first session nearly matches the performance of long-term training. We also showed that the initial distribution of the responses is uni-modal and, as training evolves (and rats improve their performance), a second peak emerges and progressively shifts toward longer times. This one-day training protocol can be used to investigate temporal learning and may be especially useful to electrophysiological and neuropharmacological studies.

Modulation of Beta Oscillations for Implicit Motor Timing in Primate Sensorimotor Cortex during Movement Preparation

Motor timing is an important part of sensorimotor control. Previous studies have shown that beta oscillations embody the process of temporal perception in explicit timing tasks. In contrast, studies focusing on beta oscillations in implicit timing tasks are lacking. In this study, we set up an implicit motor timing task and found a modulation pattern of beta oscillations with temporal perception during movement preparation. We trained two macaques in a repetitive visually-guided reach-to-grasp task with different holding intervals. Spikes and local field potentials were recorded from microelectrode arrays in the primary motor cortex, primary somatosensory cortex, and posterior parietal cortex. We analyzed the association between beta oscillations and temporal interval in fixed-duration experiments (500 ms as the Short Group and 1500 ms as the Long Group) and random-duration experiments (500 ms to 1500 ms). The results showed that the peak beta frequencies in both experiments ranged from 15 Hz to 25 Hz. The beta power was higher during the hold period than the movement (reach and grasp) period. Further, in the fixed-duration experiments, the mean power as well as the maximum rate of change of beta power in the first 300 ms were higher in the Short Group than in the Long Group when aligned with the Center Hit event. In contrast, in the random-duration experiments, the corresponding values showed no statistical differences among groups. The peak latency of beta power was shorter in the Short Group than in the Long Group in the fixed-duration experiments, while no consistent modulation pattern was found in the random-duration experiments. These results indicate that beta oscillations can modulate with temporal interval in their power mode. The synchronization period of beta power could reflect the cognitive set maintaining working memory of the temporal structure and attention.

Increase in Mutual Information During Interaction with the Environment Contributes to Perception

Perception and motor interaction with physical surroundings can be analyzed by the changes in probability laws governing two possible outcomes of neuronal activity, namely the presence or absence of spikes (binary states). Perception and motor interaction with the physical environment are partly accounted for by a reduction in entropy within the probability distributions of binary states of neurons in distributed neural circuits, given the knowledge about the characteristics of stimuli in physical surroundings. This reduction in the total entropy of multiple pairs of circuits in networks, by an amount equal to the increase of mutual information, occurs as sensory information is processed successively from lower to higher cortical areas or between different areas at the same hierarchical level, but belonging to different networks. The increase in mutual information is partly accounted for by temporal coupling as well as synaptic connections as proposed by Bahmer and Gupta (Front. Neurosci. 2018). We propose that robust increases in mutual information, measuring the association between the characteristics of sensory inputs' and neural circuits' connectivity patterns, are partly responsible for perception and successful motor interactions with physical surroundings. The increase in mutual information, given the knowledge about environmental sensory stimuli and the type of motor response produced, is responsible for the coupling between action and perception. In addition, the processing of sensory inputs within neural circuits, with no prior knowledge of the occurrence of a sensory stimulus, increases Shannon information. Consequently, the increase in surprise serves to increase the evidence of the sensory model of physical surroundings.

Neuroinflammation, oxidative stress, and blood-brain barrier (BBB) disruption in acute Utah electrode array implants and the effect of deferoxamine as an iron chelator on acute foreign body response

The use of intracortical microelectrode arrays has gained significant attention in being able to help restore function in paralysis patients and study the brain in various neurological disorders. Electrode implantation in the cortex causes vasculature or blood-brain barrier (BBB) disruption and thus elicits a foreign body response (FBR) that results in chronic inflammation and may lead to poor electrode performance. In this study, a comprehensive insight into the acute molecular mechanisms occurring at the Utah electrode array-tissue interface is provided to understand the oxidative stress, neuroinflammation, and neurovascular unit (astrocytes, pericytes, and endothelial cells) disruption that occurs following microelectrode implantation. Quantitative real time polymerase chain reaction (qRT-PCR) was used to quantify the gene expression at acute time-points of 48-hr, 72-hr, and 7-days for factors mediating oxidative stress, inflammation, and BBB disruption in rats implanted with a non-functional 4 × 4 Utah array in the somatosensory cortex. During vascular disruption, free iron released into the brain parenchyma can exacerbate the FBR, leading to oxidative stress and thus further contributing to BBB degradation. To reduce the free iron released into the brain tissue, the effects of an iron chelator, deferoxamine mesylate (DFX), was also evaluated.

Role of Oscillations in Auditory Temporal Processing: A General Model for Temporal Processing of Sensory Information in the Brain?

We review the role of oscillations in the brain and in the auditory system showing that the ability of humans to distinguish changes in pitch can be explained as a precise analysis of temporal information in auditory signals by neural oscillations. The connections between auditory brain stem chopper neurons construct neural oscillators, which discharge spikes at various constant intervals that are integer multiples of 0.4 ms, contributing to the temporal processing of auditory cochlear output. This is subsequently spatially mapped in the inferior colliculus. Electrophysiological measurements of auditory chopper neurons in different species show oscillations with periods which are integer multiples of 0.4 ms. The constant intervals of 0.4 ms can be attributed to the smallest synaptic delay between interconnected simulated chopper neurons. We also note the patterns of similarities between microcircuits in the brain stem and other parts of the brain (e.g., the pallidum, reticular formation, locus coeruleus, oculomotor nuclei, limbic system, amygdala, hippocampus, basal ganglia and substantia nigra), dedicated to the processing of temporal information. Similarities in microcircuits across the brain reflect the importance of one of the key mechanisms in the information processing in the brain, namely the temporal coupling of different neural events via coincidence detection.

A model of temporal scaling correctly predicts that motor timing improves with speed

Timing is fundamental to complex motor behaviors: from tying a knot to playing the piano. A general feature of motor timing is temporal scaling: the ability to produce motor patterns at different speeds. One theory of temporal processing proposes that the brain encodes time in dynamic patterns of neural activity (population clocks), here we first examine whether recurrent neural network (RNN) models can account for temporal scaling. Appropriately trained RNNs exhibit temporal scaling over a range similar to that of humans and capture a signature of motor timing, Weber's law, but predict that temporal precision improves at faster speeds. Human psychophysics experiments confirm this prediction: the variability of responses in absolute time are lower at faster speeds. These results establish that RNNs can account for temporal scaling and suggest a novel psychophysical principle: the Weber-Speed effect.

More than Just a "Motor": Recent Surprises from the Frontal Cortex

Motor and premotor cortices are crucial for the control of movements. However, we still know little about how these areas contribute to higher-order motor control, such as deciding which movements to make and when to make them. Here we focus on rodent studies and review recent findings, which suggest that-in addition to motor control-neurons in motor cortices play a role in sensory integration, behavioral strategizing, working memory, and decision-making. We suggest that these seemingly disparate functions may subserve an evolutionarily conserved role in sensorimotor cognition and that further study of rodent motor cortices could make a major contribution to our understanding of the evolution and function of the mammalian frontal cortex.

Space and Time: The Hippocampus as a Sequence Generator

Neural computations are often compared to instrument-measured distance or duration, and such relationships are interpreted by a human observer. However, neural circuits do not depend on human-made instruments but perform computations relative to an internally defined rate-of-change. While neuronal correlations with external measures, such as distance or duration, can be observed in spike rates or other measures of neuronal activity, what matters for the brain is how such activity patterns are utilized by downstream neural observers. We suggest that hippocampal operations can be described by the sequential activity of neuronal assemblies and their internally defined rate of change without resorting to the concept of space or time.

Metastable States of Multiscale Brain Networks Are Keys to Crack the Timing Problem

The dynamics of the environment where we live in and the interaction with it, predicting events, provided strong evolutionary pressures for the brain functioning to process temporal information and generate timed responses. As a result, the human brain is able to process temporal information and generate temporal patterns. Despite the clear importance of temporal processing to cognition, learning, communication and sensory, motor and emotional processing, the basal mechanisms of how animals differentiate simple intervals or provide timed responses are still under debate. The lesson we learned from the last decade of research in neuroscience is that functional and structural brain connectivity matter. Specifically, it has been accepted that the organization of the brain in interacting segregated networks enables its function. In this paper we delineate the route to a promising approach for investigating timing mechanisms. We illustrate how novel insight into timing mechanisms can come by investigating brain functioning as a multi-layer dynamical network whose clustered dynamics is bound to report the presence of metastable states. We anticipate that metastable dynamics underlie the real-time coordination necessary for the brain's dynamic functioning associated to time perception. This new point of view will help further clarifying mechanisms of neuropsychiatric disorders.

Distinct Dynamics of Striatal and Prefrontal Neural Activity During Temporal Discrimination

The frontal cortex-basal ganglia circuit plays an important role in interval timing. We examined neuronal discharges in the dorsomedial and dorsolateral striatum (DMS and DLS) in rats performing a temporal categorization task and compared them with previously recorded neuronal activity in the medial prefrontal cortex (mPFC). All three structures conveyed significant temporal information, but striatal neurons seldom showed the prolonged, full-interval spanning ramping activity frequently observed in the mPFC. Instead, the majority fired briefly during sample intervals. Also, the precision of neural time decoding became progressively worse with increasing time duration in the mPFC, but not in the striatum. With the caveat that mPFC and striatal units were recorded from different animals, our results suggest that the striatum and mPFC convey temporal information via distinct neural processes.

Oscillation patterns of local field potentials in the dorsal striatum and sensorimotor cortex during the encoding, maintenance, and decision stages for the ordinal comparison of sub- and supra-second signal durations

Ordinal comparison of successively presented signal durations requires (a) the encoding of the first signal duration (standard), (b) maintenance of temporal information specific to the standard duration in memory, and (c) timing of the second signal duration (comparison) during which a comparison is made of the first and second durations. Rats were first trained to make ordinal comparisons of signal durations within three time ranges using 0.5, 1.0, and 3.0-s standard durations. Local field potentials were then recorded from the dorsal striatum and sensorimotor cortex in order to investigate the pattern of neural oscillations during each phase of the ordinal-comparison process. Increased power in delta and theta frequency ranges was observed during both the encoding and comparison stages. Active maintenance of a selected response, "shorter" or "longer" (counter-balanced across left and right levers), was represented by an increase of theta and delta oscillations in the contralateral striatum and cortex. Taken together, these data suggest that neural oscillations in the delta-theta range play an important role in the encoding, maintenance, and comparison of signal durations.

Object size determines the spatial spread of visual time

A key question for temporal processing research is how the nervous system extracts event duration, despite a notable lack of neural structures dedicated to duration encoding. This is in stark contrast with the orderly arrangement of neurons tasked with spatial processing. In this study, we examine the linkage between the spatial and temporal domains. We use sensory adaptation techniques to generate after-effects where perceived duration is either compressed or expanded in the opposite direction to the adapting stimulus' duration. Our results indicate that these after-effects are broadly tuned, extending over an area approximately five times the size of the stimulus. This region is directly related to the size of the adapting stimulus-the larger the adapting stimulus the greater the spatial spread of the after-effect. We construct a simple model to test predictions based on overlapping adapted versus non-adapted neuronal populations and show that our effects cannot be explained by any single, fixed-scale neural filtering. Rather, our effects are best explained by a self-scaled mechanism underpinned by duration selective neurons that also pool spatial information across earlier stages of visual processing.

Interference between Space and Time Estimations: From Behavior to Neurons

Influences between time and space can be found in our daily life in which we are surrounded by numerous spatial metaphors to refer to time. For instance, when we move files from one folder to another in our computer a horizontal line that grows from left to right informs us about the elapsed and remaining time to finish the procedure and, similarly, in our communication we use several spatial terms to refer to time. Although with some differences in the degree of interference, not only space has an influence on time but both magnitudes influence each other. Indeed, since our childhood our estimations of time are influenced by space even when space should be irrelevant and the same occurs when estimating space with time as distractor. Such interference between magnitudes has also been observed in monkeys even if they do not use language or computers, suggesting that the two magnitudes are tightly coupled beyond communication and technology. Imaging and lesion studies have indicated that same brain areas are involved during the processing of both magnitudes and have suggested that rather than coding the specific magnitude itself the brain represents them as abstract concepts. Recent neurophysiological studies in prefrontal cortex, however, have shown that the coding of absolute and relative space and time in this area is realized by independent groups of neurons. Interestingly, instead, a high overlap was observed in this same area in the coding of goal choices across tasks. These results suggest that rather than during perception or estimation of space and time the interference between the two magnitudes might occur, at least in the prefrontal cortex, in a subsequent phase in which the goal has to be chosen or the response provided.

Posterior parietal cortex estimates the relationship between object and body location during locomotion

We test the hypothesis that the posterior parietal cortex (PPC) contributes to the control of visually guided locomotor gait modifications by constructing an estimation of object location relative to body state, and in particular the changing gap between them. To test this hypothesis, we recorded neuronal activity from areas 5b and 7 of the PPC of cats walking on a treadmill and stepping over a moving obstacle whose speed of advance was varied (slowed or accelerated with respect to the speed of the cat). We found distinct populations of neurons in the PPC, primarily in area 5b, that signaled distance- or time-to-contact with the obstacle, regardless of which limb was the first to step over the obstacle. We propose that these cells are involved in a sensorimotor transformation whereby information on the location of an obstacle with respect to the body is used to initiate the gait modification.

Imagine crossing the street and having to step up onto a sidewalk, or running up to kick a moving soccer ball. How does the brain allow you to accomplish these deceptively simple tasks? You might say that you look at the target and then adjust where you place your feet in order to achieve your goal. That would be correct, but to make that adjustment you have to determine where you are with respect to the curb or the soccer ball. A key aspect of both of these activities is the ability to determine where your target is with respect to your current location, even if that target is moving. One way to do that is to determine the distance or the time required to reach that target. The brain can then use this information to adjust your foot placement and limb movement to fulfill your goal.

Despite the fact that we constantly use vision to examine our environment as we walk, we have little understanding as to how the brain uses vision to plan where to step next. Marigold and Drew have now determined whether one specific part of the brain called the posterior parietal cortex, which is known to be involved in integrating vision and movement, is involved in this planning. Specifically, can it estimate the relative location of a moving object with respect to the body?

Marigold and Drew recorded from neurons in the posterior parietal cortex of cats while they walked on a treadmill and stepped over an obstacle that moved towards them. On some tests, the obstacle was either slowed or accelerated quickly as it approached the cat. Regardless of these manipulations, some neurons always became active when the obstacle was at a specific distance from the cat. By contrast, other neurons always became active at a specific time before the cat met the obstacle. Animals use this information to adjust their gait to step over an obstacle without hitting it.

Overall, the results presented by Marigold and Drew provide new insights into how animals use vision to modify their stepping pattern. This information could potentially be used to devise rehabilitation techniques, perhaps using virtual reality, to aid patients with damage to the posterior parietal cortex. Equally, the results from this research could help to design brain-controlled devices that help patients to walk – or even intelligent walking robots.

Cortical neurons multiplex reward-related signals along with sensory and motor information

Rewards are known to influence neural activity associated with both motor preparation and execution. This influence can be exerted directly upon the primary motor (M1) and somatosensory (S1) cortical areas via the projections from reward-sensitive dopaminergic neurons of the midbrain ventral tegmental areas. However, the neurophysiological manifestation of reward-related signals in M1 and S1 are not well understood. Particularly, it is unclear how the neurons in these cortical areas multiplex their traditional functions related to the control of spatial and temporal characteristics of movements with the representation of rewards. To clarify this issue, we trained rhesus monkeys to perform a center-out task in which arm movement direction, reward timing, and magnitude were manipulated independently. Activity of several hundred cortical neurons was simultaneously recorded using chronically implanted microelectrode arrays. Many neurons (9-27%) in both M1 and S1 exhibited activity related to reward anticipation. Additionally, neurons in these areas responded to a mismatch between the reward amount given to the monkeys and the amount they expected: A lower-than-expected reward caused a transient increase in firing rate in 60-80% of the total neuronal sample, whereas a larger-than-expected reward resulted in a decreased firing rate in 20-35% of the neurons. Moreover, responses of M1 and S1 neurons to reward omission depended on the direction of movements that led to those rewards. These observations suggest that sensorimotor cortical neurons corepresent rewards and movement-related activity, presumably to enable reward-based learning.

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links between living brains and artificial actuators. Although theoretical propositions and some proof of concept experiments on directly linking the brains with machines date back to the early 1960s, BMI research only took off in earnest at the end of the 1990s, when this approach became intimately linked to new neurophysiological methods for sampling large-scale brain activity. The classic goals of BMIs are 1) to unveil and utilize principles of operation and plastic properties of the distributed and dynamic circuits of the brain and 2) to create new therapies to restore mobility and sensations to severely disabled patients. Over the past decade, a wide range of BMI applications have emerged, which considerably expanded these original goals. BMI studies have shown neural control over the movements of robotic and virtual actuators that enact both upper and lower limb functions. Furthermore, BMIs have also incorporated ways to deliver sensory feedback, generated from external actuators, back to the brain. BMI research has been at the forefront of many neurophysiological discoveries, including the demonstration that, through continuous use, artificial tools can be assimilated by the primate brain's body schema. Work on BMIs has also led to the introduction of novel neurorehabilitation strategies. As a result of these efforts, long-term continuous BMI use has been recently implicated with the induction of partial neurological recovery in spinal cord injury patients.

Monkeys Share the Human Ability to Internally Maintain a Temporal Rhythm

Timing is a fundamental variable for behavior. However, the mechanisms allowing human and non-human primates to synchronize their actions with periodic events are not yet completely understood. Here we characterize the ability of rhesus monkeys and humans to perceive and maintain rhythms of different paces in the absence of sensory cues or motor actions. In our rhythm task subjects had to observe and then internally follow a visual stimulus that periodically changed its location along a circular perimeter. Crucially, they had to maintain this visuospatial tempo in the absence of movements. Our results show that the probability of remaining in synchrony with the rhythm decreased, and the variability in the timing estimates increased, as a function of elapsed time, and these trends were well described by the generalized law of Weber. Additionally, the pattern of errors shows that human subjects tended to lag behind fast rhythms and to get ahead of slow ones, suggesting that a mean tempo might be incorporated as prior information. Overall, our results demonstrate that rhythm perception and maintenance are cognitive abilities that we share with rhesus monkeys, and these abilities do not depend on overt motor commands.

Interfacing to the brain's motor decisions

It has been long known that neural activity, recorded with electrophysiological methods, contains rich information about a subject's motor intentions, sensory experiences, allocation of attention, action planning, and even abstract thoughts. All these functions have been the subject of neurophysiological investigations, with the goal of understanding how neuronal activity represents behavioral parameters, sensory inputs, and cognitive functions. The field of brain-machine interfaces (BMIs) strives for a somewhat different goal: it endeavors to extract information from neural modulations to create a communication link between the brain and external devices. Although many remarkable successes have been already achieved in the BMI field, questions remain regarding the possibility of decoding high-order neural representations, such as decision making. Could BMIs be employed to decode the neural representations of decisions underlying goal-directed actions? In this review we lay out a framework that describes the computations underlying goal-directed actions as a multistep process performed by multiple cortical and subcortical areas. We then discuss how BMIs could connect to different decision-making steps and decode the neural processing ongoing before movements are initiated. Such decision-making BMIs could operate as a system with prediction that offers many advantages, such as shorter reaction time, better error processing, and improved unsupervised learning. To present the current state of the art, we review several recent BMIs incorporating decision-making components.

Time to Move Again: Does the Bereitschaftspotential Covary with Demands on Internal Timing?

When Bereitschaftspotentials (BPs) are measured, participants are required to voluntarily perform a predefined number of identical movements, with varying intervals between movements, exceeding some obligatory minimum interval. Participants might cope with these demands on timing by installing a slow, broadly tuned rhythm of activation, serving as an internal trigger for executing movements in time. The BP might reflect the rising phase of this activation, culminating at the movement. If so (i) not only should BP amplitudes become larger, but BPs should also have their onsets earlier before movements when longer minimum intervals are required between movements (Experiment 1). Further, (ii) BP amplitudes should covary with demands on internal timing: decrease when internal timing is less necessary and increase in the other case. Variation of timing demands was realized by requiring participants to count vs. not to count the seconds between movements (Experiment 1) and by regular vs. irregular vs. no ticking of a clock (Experiment 2). Prediction (i) was confirmed while prediction (ii) was not. Thus, BP onsets did vary in accordance with the temporal constraints about when the movements should be performed, suggesting some relation to timing mechanisms, but we could not provide evidence for the notion that the process reflected by BPs is this timing mechanism.

Independent coding of absolute duration and distance magnitudes in the prefrontal cortex

The estimation of space and time can interfere with each other, and neuroimaging studies have shown overlapping activation in the parietal and prefrontal cortical areas. We used duration and distance discrimination tasks to determine whether space and time share resources in prefrontal cortex (PF) neurons. Monkeys were required to report which of two stimuli, a red circle or blue square, presented sequentially, were longer and farther, respectively, in the duration and distance tasks. In a previous study, we showed that relative duration and distance are coded by different populations of neurons and that the only common representation is related to goal coding. Here, we examined the coding of absolute duration and distance. Our results support a model of independent coding of absolute duration and distance metrics by demonstrating that not only relative magnitude but also absolute magnitude are independently coded in the PF.

NEW & NOTEWORTHY

Human behavioral studies have shown that spatial and duration judgments can interfere with each other. We investigated the neural representation of such magnitudes in the prefrontal cortex. We found that the two magnitudes are independently coded by prefrontal neurons. We suggest that the interference among magnitude judgments might depend on the goal rather than the perceptual resource sharing.

Recording extracellular neural activity in the behaving monkey using a semichronic and high-density electrode system

We describe a technique to semichronically record the cortical extracellular neural activity in the behaving monkey employing commercial high-density electrodes. After the design and construction of low cost microdrives that allow varying the depth of the recording locations after the implantation surgery, we recorded the extracellular unit activity from pools of neurons at different depths in the presupplementary motor cortex (pre-SMA) of a rhesus monkey trained in a tapping task. The collected data were processed to classify cells as putative pyramidal cells or interneurons on the basis of their waveform features. We also demonstrate that short time cross-correlogram occasionally yields unit pairs with high short latency (<5 ms), narrow bin (<3 ms) peaks, indicative of monosynaptic spike transmission from pre- to postsynaptic neurons. These methods have been verified extensively in rodents. Finally, we observed that the pattern of population activity was repetitive over distinct trials of the tapping task. These results show that the semichronic technique is a viable option for the large-scale parallel recording of local circuit activity at different depths in the cortex of the macaque monkey and other large species.

Contribution of Cerebellar Loops to Action Timing

Recent studies of sensorimotor processing have benefited from decision-making paradigms that emphasize the selection of appropriate movements. Selecting when to make those responses, or action timing, is important as well. Although the cerebellum is commonly viewed as a controller of movement dynamics, its role in action timing is also firmly supported. Several lines of research have now extended this idea. Anatomical findings have revealed connections between the cerebellum and broader timing circuits, neurophysiological results have suggested mechanisms for timing within its microcircuitry, and theoretical work has indicated how temporal signals are processed through it and decoded by its targets. These developments are inspiring renewed studies of the role of the cerebellar loops in action timing.

Striatal dynamics explain duration judgments

The striatum is an input structure of the basal ganglia implicated in several time-dependent functions including reinforcement learning, decision making, and interval timing. To determine whether striatal ensembles drive subjects' judgments of duration, we manipulated and recorded from striatal neurons in rats performing a duration categorization psychophysical task. We found that the dynamics of striatal neurons predicted duration judgments, and that simultaneously recorded ensembles could judge duration as well as the animal. Furthermore, striatal neurons were necessary for duration judgments, as muscimol infusions produced a specific impairment in animals' duration sensitivity. Lastly, we show that time as encoded by striatal populations ran faster or slower when rats judged a duration as longer or shorter, respectively. These results demonstrate that the speed with which striatal population state changes supports the fundamental ability of animals to judge the passage of time.

DOI:http://dx.doi.org/10.7554/eLife.11386.001

You know someone is a good cook from their rice - grains must be well cooked, but not to the point of being mushy. Despite consistently using the same pot and stove, we, however, will sometimes overcook it. It is as if our inner sense of time itself is variable. What is it about the brain that explains this variability in time estimation and indeed our ability to estimate time in the first place?

One issue the brain must confront in order to estimate time is that individual brain cells typically fire in bursts that last for tens of milliseconds. So how does the brain use this short-lived activity to track minutes and hours? One possibility is that individual neurons in a given brain region are programmed to fire at different points in time. The overall firing pattern of a group of neurons will therefore change in a predictable way as time passes.

Gouvêa, Monteiro et al. found such predictably changing patterns of activity in the striatum of rats trained to estimate and categorize the duration of time intervals as longer or shorter than 1.5 seconds. Interestingly, when rats mistakenly categorized a short interval as a long one, population activity had travelled farther down its path than it would normally (and vice-versa for long intervals incorrectly categorized as short), suggesting that variability in subjective estimates of the passage of time might arise from variability in the speed of a changing pattern of activity across groups of neurons.

As further evidence for the involvement of the striatum, inactivating the structure impaired the rats’ ability to correctly classify even the longest and shortest interval durations.

The next challenge is to determine exactly how the striatum generates these time-keeping signals, at which stage variability originates, and how the brain regions that the striatum signals to use them to control an animal’s behavior.

DOI:http://dx.doi.org/10.7554/eLife.11386.002

Context-Dependent Duration Signals in the Primate Prefrontal Cortex

The activity of some prefrontal (PF) cortex neurons distinguishes short from long time intervals. Here, we examined whether this property reflected a general timing mechanism or one dependent on behavioral context. In one task, monkeys discriminated the relative duration of 2 stimuli; in the other, they discriminated the relative distance of 2 stimuli from a fixed reference point. Both tasks had a pre-cue period (interval 1) and a delay period (interval 2) with no discriminant stimulus. Interval 1 elapsed before the presentation of the first discriminant stimulus, and interval 2 began after that stimulus. Both intervals had durations of either 400 or 800 ms. Most PF neurons distinguished short from long durations in one task or interval, but not in the others. When neurons did signal something about duration for both intervals, they did so in an uncorrelated or weakly correlated manner. These results demonstrate a high degree of context dependency in PF time processing. The PF, therefore, does not appear to signal durations abstractedly, as would be expected of a general temporal encoder, but instead does so in a highly context-dependent manner, both within and between tasks.

Signs of timing in motor cortex during movement preparation and cue anticipation

The capacity to accurately anticipate the timing of predictable events is essential for sensorimotor behavior. Motor cortex holds an established role in movement preparation and execution. In this chapter we review the different ways in which motor cortical activity is modulated by event timing in sensorimotor delay tasks. During movement preparation, both single neuron and population responses reflect the temporal constraints of the task. Anticipatory modulations prior to sensory cues are also observed in motor cortex when the cue timing is predictable. We propose that the motor cortical activity during cue anticipation and movement preparation is embedded in a timing network that facilitates sensorimotor processing. In this context, the pre-cue and post-cue activity may reflect a presetting mechanism, complementing processing during movement execution, while prohibiting premature responses in situations requiring delayed motor output.

Context-dependent urgency influences speed-accuracy trade-offs in decision-making and movement execution

Speed-accuracy tradeoffs (SATs) exist in both decision-making and movement control, and are generally studied separately. However, in natural behavior animals are free to adjust the time invested in deciding and moving so as to maximize their reward rate. Here, we investigate whether shared mechanisms exist for SAT adjustment in both decisions and actions. Two monkeys performed a reach decision task in which they watched 15 tokens jump, one every 200 ms, from a central circle to one of two peripheral targets, and had to guess which target would ultimately receive the majority of tokens. The monkeys could decide at any time, and once a target was reached, the remaining token movements accelerated to either 50 ms ("fast" block) or 150 ms ("slow" block). Decisions were generally earlier and less accurate in fast than slow blocks, and in both blocks, the criterion of accuracy decreased over time within each trial. This could be explained by a simple model in which sensory information is combined with a linearly growing urgency signal. Remarkably, the duration of the reaching movements produced after the decision decreased over time in a similar block-dependent manner as the criterion of accuracy estimated by the model. This suggests that SATs for deciding and acting are influenced by a shared urgency/vigor signal. Consistent with this, we observed that the vigor of saccades performed during the decision process was higher in fast than in slow blocks, suggesting the influence of a context-dependent global arousal.

Neuronal representation of duration discrimination in the monkey striatum

Functional imaging and lesion studies in humans and animals suggest that the basal ganglia are crucial for temporal information processing. To elucidate neuronal mechanisms of interval timing in the basal ganglia, we recorded single-unit activity from the striatum of two monkeys while they performed a visual duration discrimination task. In the task, blue and red cues of different durations (0.2-2.0 sec) were successively presented. Each of the two cues was followed by a 1.0 sec delay period. The animals were instructed to choose the longer presented colored stimulus after the second delay period. A total of 498 phasically active neurons were recorded from the striatum, and 269 neurons were defined as task related. Two types of neuronal activity were distinguished during the delay periods. First, the activity gradually changed depending on the duration of the cue presented just before. This activity may represent the signal duration for later comparison between two cue durations. The activity during the second cue period also represented duration of the first cue. Second, the activity changed differently depending on whether the first or second cue was presented longer. This activity may represent discrimination results after the comparison between the two cue durations. These findings support the assumption that striatal neurons represent timing information of sensory signals for duration discrimination.

Preparing and selecting actions with neural populations: toward cortical circuit mechanisms

How the brain selects one action among multiple alternatives is a central question of neuroscience. An influential model is that action preparation and selection arise from subthreshold activation of the very neurons encoding the action. Recent work, however, shows a much greater diversity of decision-related and action-related signals coexisting with other signals in populations of motor and parietal cortical neurons. We discuss how such distributed signals might be decoded by biologically plausible mechanisms. We also discuss how neurons within cortical circuits might interact with each other during action selection and preparation and how recurrent network models can help to reveal dynamical principles underlying cortical computation.

Dissociating movement from movement timing in the rat primary motor cortex

Neural encoding of the passage of time to produce temporally precise movements remains an open question. Neurons in several brain regions across different experimental contexts encode estimates of temporal intervals by scaling their activity in proportion to the interval duration. In motor cortex the degree to which this scaled activity relies upon afferent feedback and is guided by motor output remains unclear. Using a neural reward paradigm to dissociate neural activity from motor output before and after complete spinal transection, we show that temporally scaled activity occurs in the rat hindlimb motor cortex in the absence of motor output and after transection. Context-dependent changes in the encoding are plastic, reversible, and re-established following injury. Therefore, in the absence of motor output and despite a loss of afferent feedback, thought necessary for timed movements, the rat motor cortex displays scaled activity during a broad range of temporally demanding tasks similar to that identified in other brain regions.

Oscillatory multiplexing of neural population codes for interval timing and working memory

Interval timing and working memory are critical components of cognition that are supported by neural oscillations in prefrontal-striatal-hippocampal circuits. In this review, the properties of interval timing and working memory are explored in terms of behavioral, anatomical, pharmacological, and neurophysiological findings. We then describe the various neurobiological theories that have been developed to explain these cognitive processes - largely independent of each other. Following this, a coupled excitatory - inhibitory oscillation (EIO) model of temporal processing is proposed to address the shared oscillatory properties of interval timing and working memory. Using this integrative approach, we describe a hybrid model explaining how interval timing and working memory can originate from the same oscillatory processes, but differ in terms of which dimension of the neural oscillation is utilized for the extraction of item, temporal order, and duration information. This extension of the striatal beat-frequency (SBF) model of interval timing (Matell and Meck, 2000, 2004) is based on prefrontal-striatal-hippocampal circuit dynamics and has direct relevance to the pathophysiological distortions observed in time perception and working memory in a variety of psychiatric and neurological conditions.

Neural antecedents of self-initiated actions in secondary motor cortex

The neural origins of spontaneous or self-initiated actions are not well understood and their interpretation is controversial. To address these issues, we used a task in which rats decide when to abort waiting for a delayed tone. We recorded neurons in the secondary motor cortex (M2) and interpreted our findings in light of an integration-to-bound decision model. A first population of M2 neurons ramped to a constant threshold at rates proportional to waiting time, strongly resembling integrator output. A second population, which we propose provide input to the integrator, fired in sequences and showed trial-to-trial rate fluctuations correlated with waiting times. An integration model fit to these data also quantitatively predicted the observed inter-neuronal correlations. Together, these results reinforce the generality of the integration-to-bound model of decision-making. These models identify the initial intention to act as the moment of threshold crossing while explaining how antecedent subthreshold neural activity can influence an action without implying a decision.

Joint cross-correlation analysis reveals complex, time-dependent functional relationship between cortical neurons and arm electromyograms

Correlation between cortical activity and electromyographic (EMG) activity of limb muscles has long been a subject of neurophysiological studies, especially in terms of corticospinal connectivity. Interest in this issue has recently increased due to the development of brain-machine interfaces with output signals that mimic muscle force. For this study, three monkeys were implanted with multielectrode arrays in multiple cortical areas. One monkey performed self-timed touch pad presses, whereas the other two executed arm reaching movements. We analyzed the dynamic relationship between cortical neuronal activity and arm EMGs using a joint cross-correlation (JCC) analysis that evaluated trial-by-trial correlation as a function of time intervals within a trial. JCCs revealed transient correlations between the EMGs of multiple muscles and neural activity in motor, premotor and somatosensory cortical areas. Matching results were obtained using spike-triggered averages corrected by subtracting trial-shuffled data. Compared with spike-triggered averages, JCCs more readily revealed dynamic changes in cortico-EMG correlations. JCCs showed that correlation peaks often sharpened around movement times and broadened during delay intervals. Furthermore, JCC patterns were directionally selective for the arm-reaching task. We propose that such highly dynamic, task-dependent and distributed relationships between cortical activity and EMGs should be taken into consideration for future brain-machine interfaces that generate EMG-like signals.

Attention and working memory: two basic mechanisms for constructing temporal experiences

Various kinds of observations show that the ability of human beings to both consciously relive past events – episodic memory – and conceive future events, entails an active process of construction. This construction process also underpins many other important aspects of conscious human life, such as perceptions, language, and conscious thinking. This article provides an explanation of what makes the constructive process possible and how it works. The process mainly relies on attentional activity, which has a discrete and periodic nature, and working memory, which allows for the combination of discrete attentional operations. An explanation is also provided of how past and future events are constructed.

Processing of sub- and supra-second intervals in the primate brain results from the calibration of neuronal oscillators via sensory, motor, and feedback processes

The processing of time intervals in the sub- to supra-second range by the brain is critical for the interaction of primates with their surroundings in activities, such as foraging and hunting. For an accurate processing of time intervals by the brain, representation of physical time within neuronal circuits is necessary. I propose that time dimension of the physical surrounding is represented in the brain by different types of neuronal oscillators, generating spikes or spike bursts at regular intervals. The proposed oscillators include the pacemaker neurons, tonic inputs, and synchronized excitation and inhibition of inter-connected neurons. Oscillators, which are built inside various circuits of brain, help to form modular clocks, processing time intervals or other temporal characteristics specific to functions of a circuit. Relative or absolute duration is represented within neuronal oscillators by "neural temporal unit," defined as the interval between regularly occurring spikes or spike bursts. Oscillator output is processed to produce changes in activities of neurons, named frequency modulator neuron, wired within a separate module, represented by the rate of change in frequency, and frequency of activities, proposed to encode time intervals. Inbuilt oscillators are calibrated by (a) feedback processes, (b) input of time intervals resulting from rhythmic external sensory stimulation, and (c) synchronous effects of feedback processes and evoked sensory activity. A single active clock is proposed per circuit, which is calibrated by one or more mechanisms. Multiple calibration mechanisms, inbuilt oscillators, and the presence of modular connections prevent a complete loss of interval timing functions of the brain.

Linking perception, cognition, and action: psychophysical observations and neural network modelling

It has been argued that perception, decision making, and movement planning are in reality tightly interwoven brain processes. However, how they are implemented in neural circuits is still a matter of debate. We tested human subjects in a temporal categorization task in which intervals had to be categorized as short or long. Subjects communicated their decision by moving a cursor into one of two possible targets, which appeared separated by different angles from trial to trial. Even though there was a 1 second-long delay between interval presentation and decision communication, categorization difficulty affected subjects’ performance, reaction (RT) and movement time (MT). In addition, reaction and movement times were also influenced by the distance between the targets. This implies that not only perceptual, but also movement-related considerations were incorporated into the decision process. Therefore, we searched for a model that could use categorization difficulty and target separation to describe subjects’ performance, RT, and MT. We developed a network consisting of two mutually inhibiting neural populations, each tuned to one of the possible categories and composed of an accumulation and a memory node. This network sequentially acquired interval information, maintained it in working memory and was then attracted to one of two possible states, corresponding to a categorical decision. It faithfully replicated subjects’ RT and MT as a function of categorization difficulty and target distance; it also replicated performance as a function of categorization difficulty. Furthermore, this model was used to make new predictions about the effect of untested durations, target distances and delay durations. To our knowledge, this is the first biologically plausible model that has been proposed to account for decision making and communication by integrating both sensory and motor planning information.