Covert representation of second-next movement in the pre-supplementary motor area of monkeys

The Neural Correlates of Individual Differences in Reinforcement Learning during Pain Avoidance and Reward Seeking

Organisms learn to gain reward and avoid punishment through action-outcome associations. Reinforcement learning (RL) offers a critical framework to understand individual differences in this associative learning by assessing learning rate, action bias, pavlovian factor (i.e., the extent to which action values are influenced by stimulus values), and subjective impact of outcomes (i.e., motivation to seek reward and avoid punishment). Nevertheless, how these individual-level metrics are represented in the brain remains unclear. The current study leveraged fMRI in healthy humans and a probabilistic learning go/no-go task to characterize the neural correlates involved in learning to seek reward and avoid pain. Behaviorally, participants showed a higher learning rate during pain avoidance relative to reward seeking. Additionally, the subjective impact of outcomes was greater for reward trials and associated with lower response randomness. Our imaging findings showed that individual differences in learning rate and performance accuracy during avoidance learning were positively associated with activities of the dorsal anterior cingulate cortex, midcingulate cortex, and postcentral gyrus. In contrast, the pavlovian factor was represented in the precentral gyrus and superior frontal gyrus (SFG) during pain avoidance and reward seeking, respectively. Individual variation of the subjective impact of outcomes was positively predicted by activation of the left posterior cingulate cortex. Finally, action bias was represented by the supplementary motor area (SMA) and pre-SMA whereas the SFG played a role in restraining this action tendency. Together, these findings highlight for the first time the neural substrates of individual differences in the computational processes during RL.

Premotor Cortex Provides a Substrate for the Temporal Transformation of Information During the Planning of Gait Modifications

We tested the hypothesis that the premotor cortex (PMC) in the cat contributes to the planning and execution of visually guided gait modifications. We analyzed single unit activity from 136 cells localized within layer V of cytoarchitectonic areas 6iffu and that part of 4δ within the ventral bank of the cruciate sulcus while cats walked on a treadmill and stepped over an obstacle that advanced toward them. We found a rich variety of discharge patterns, ranging from limb-independent cells that discharged several steps in front of the obstacle to step-related cells that discharged either during steps over the obstacle or in the steps leading up to that step. We propose that this population of task-related cells within this region of the PMC contributes to the temporal evolution of a planning process that transforms global information of the presence of an obstacle into the precise spatio-temporal limb adjustment required to negotiate that obstacle.

Probabilistic associative learning suffices for learning the temporal structure of multiple sequences

From memorizing a musical tune to navigating a well known route, many of our underlying behaviors have a strong temporal component. While the mechanisms behind the sequential nature of the underlying brain activity are likely multifarious and multi-scale, in this work we attempt to characterize to what degree some of this properties can be explained as a consequence of simple associative learning. To this end, we employ a parsimonious firing-rate attractor network equipped with the Hebbian-like Bayesian Confidence Propagating Neural Network (BCPNN) learning rule relying on synaptic traces with asymmetric temporal characteristics. The proposed network model is able to encode and reproduce temporal aspects of the input, and offers internal control of the recall dynamics by gain modulation. We provide an analytical characterisation of the relationship between the structure of the weight matrix, the dynamical network parameters and the temporal aspects of sequence recall. We also present a computational study of the performance of the system under the effects of noise for an extensive region of the parameter space. Finally, we show how the inclusion of modularity in our network structure facilitates the learning and recall of multiple overlapping sequences even in a noisy regime.

Cytoarchitecture, probability maps, and functions of the human supplementary and pre-supplementary motor areas

The dorsal  mesial frontal cortex contains the supplementary motor area (SMA) and the pre-supplementary motor area (pre-SMA), which play an important role in action and cognition. Evidence from cytoarchitectonic, stimulation, and functional studies suggests structural and functional divergence between the two subregions. However, a microstructural map of these areas obtained in a representative sample of brains in a stereotaxic reference space is still lacking. In the present study we show that the dorsal mesial frontal motor cortex comprises two microstructurally different brain regions: area SMA and area pre-SMA. Area-specific cytoarchitectonic patterns were studied in serial histological sections stained for cell bodies of ten human postmortem brains. Borders of the two cortical areas were identified using image analysis and statistical features. The 3D reconstructed areas were transferred to a common reference space, and probabilistic maps were calculated by superimposing the individual maps. A coordinate-based meta-analysis of functional imaging data was subsequently performed using the two probabilistic maps as microstructurally defined seed regions. It revealed that areas SMA and pre-SMA were strongly co-activated with areas in precentral, supramarginal and superior frontal gyri, Rolandic operculum, thalamus, putamen and cerebellum. Both areas were related to motor functions, but area pre-SMA was involved in more complex processes such as learning, cognitive processes and perception. The here described subsequent analyses led to converging evidence supporting the microstructural, and functional segregation of areas SMA and pre-SMA, and maps will be made available to the scientific community to further elucidate the microstructural substrates of motor and cognitive control.

Oscillation patterns of local field potentials in the dorsal striatum and sensorimotor cortex during the encoding, maintenance, and decision stages for the ordinal comparison of sub- and supra-second signal durations

Ordinal comparison of successively presented signal durations requires (a) the encoding of the first signal duration (standard), (b) maintenance of temporal information specific to the standard duration in memory, and (c) timing of the second signal duration (comparison) during which a comparison is made of the first and second durations. Rats were first trained to make ordinal comparisons of signal durations within three time ranges using 0.5, 1.0, and 3.0-s standard durations. Local field potentials were then recorded from the dorsal striatum and sensorimotor cortex in order to investigate the pattern of neural oscillations during each phase of the ordinal-comparison process. Increased power in delta and theta frequency ranges was observed during both the encoding and comparison stages. Active maintenance of a selected response, "shorter" or "longer" (counter-balanced across left and right levers), was represented by an increase of theta and delta oscillations in the contralateral striatum and cortex. Taken together, these data suggest that neural oscillations in the delta-theta range play an important role in the encoding, maintenance, and comparison of signal durations.

Reading positional codes with fMRI: Problems and solutions

Neural mechanisms which bind items into sequences have been investigated in a large body of research in animal neurophysiology and human neuroimaging. However, a major problem in interpreting this data arises from a fact that several unrelated processes, such as memory load, sensory adaptation, and reward expectation, also change in a consistent manner as the sequence unfolds. In this paper we use computational simulations and data from two fMRI experiments to show that a host of unrelated neural processes can masquerade as sequence representations. We show that dissociating such unrelated processes from a dedicated sequence representation is an especially difficult problem for fMRI data, which is almost exclusively the modality used in human experiments. We suggest that such fMRI results must be treated with caution and in many cases the assumed neural representation might actually reflect unrelated processes.

A shared representation of order between encoding and recognition in visual short-term memory

Many complex tasks require people to bind individual events into a sequence that can be held in short term memory (STM). For this purpose information about the order of the individual events in the sequence needs to be maintained in an active and accessible form in STM over a period of few seconds. Here we investigated how the temporal order information is shared between the presentation and response phases of an STM task. We trained a classification algorithm on the fMRI activity patterns from the presentation phase of the STM task to predict the order of the items during the subsequent recognition phase. While voxels in a number of brain regions represented positional information during either presentation and recognition phases, only voxels in the lateral prefrontal cortex (PFC) and the anterior temporal lobe (ATL) represented position consistently across task phases. A shared positional code in the ATL might reflect verbal recoding of visual sequences to facilitate the maintenance of order information over several seconds.

Extending the Cortical Grasping Network: Pre-supplementary Motor Neuron Activity During Vision and Grasping of Objects

Grasping relies on a network of parieto-frontal areas lying on the dorsolateral and dorsomedial parts of the hemispheres. However, the initiation and sequencing of voluntary actions also requires the contribution of mesial premotor regions, particularly the pre-supplementary motor area F6. We recorded 233 F6 neurons from 2 monkeys with chronic linear multishank neural probes during reaching–grasping visuomotor tasks. We showed that F6 neurons play a role in the control of forelimb movements and some of them (26%) exhibit visual and/or motor specificity for the target object. Interestingly, area F6 neurons form 2 functionally distinct populations, showing either visually-triggered or movement-related bursts of activity, in contrast to the sustained visual-to-motor activity displayed by ventral premotor area F5 neurons recorded in the same animals and with the same task during previous studies. These findings suggest that F6 plays a role in object grasping and extend existing models of the cortical grasping network.

Spike-Based Bayesian-Hebbian Learning of Temporal Sequences

Many cognitive and motor functions are enabled by the temporal representation and processing of stimuli, but it remains an open issue how neocortical microcircuits can reliably encode and replay such sequences of information. To better understand this, a modular attractor memory network is proposed in which meta-stable sequential attractor transitions are learned through changes to synaptic weights and intrinsic excitabilities via the spike-based Bayesian Confidence Propagation Neural Network (BCPNN) learning rule. We find that the formation of distributed memories, embodied by increased periods of firing in pools of excitatory neurons, together with asymmetrical associations between these distinct network states, can be acquired through plasticity. The model’s feasibility is demonstrated using simulations of adaptive exponential integrate-and-fire model neurons (AdEx). We show that the learning and speed of sequence replay depends on a confluence of biophysically relevant parameters including stimulus duration, level of background noise, ratio of synaptic currents, and strengths of short-term depression and adaptation. Moreover, sequence elements are shown to flexibly participate multiple times in the sequence, suggesting that spiking attractor networks of this type can support an efficient combinatorial code. The model provides a principled approach towards understanding how multiple interacting plasticity mechanisms can coordinate hetero-associative learning in unison.

From one moment to the next, in an ever-changing world, and awash in a deluge of sensory data, the brain fluidly guides our actions throughout an astonishing variety of tasks. Processing this ongoing bombardment of information is a fundamental problem faced by its underlying neural circuits. Given that the structure of our actions along with the organization of the environment in which they are performed can be intuitively decomposed into sequences of simpler patterns, an encoding strategy reflecting the temporal nature of these patterns should offer an efficient approach for assembling more complex memories and behaviors. We present a model that demonstrates how activity could propagate through recurrent cortical microcircuits as a result of a learning rule based on neurobiologically plausible time courses and dynamics. The model predicts that the interaction between several learning and dynamical processes constitute a compound mnemonic engram that can flexibly generate sequential step-wise increases of activity within neural populations.

Anticipating conflict: Neural correlates of a Bayesian belief and its motor consequence

Previous studies have examined the neural correlates of proactive control using a variety of behavioral paradigms; however, the neural network relating the control process to its behavioral consequence remains unclear. Here, we applied a dynamic Bayesian model to a large fMRI data set of the stop signal task to address this issue. By estimating the probability of the stop signal - p(Stop) - trial by trial, we showed that higher p(Stop) is associated with prolonged go trial reaction time (RT), indicating proactive control of motor response. In modeling fMRI signals at trial and target onsets, we distinguished activities of proactive control, prediction error, and RT slowing. We showed that the anterior pre-supplementary motor area (pre-SMA) responds specifically to increased stop signal likelihood, and its activity is correlated with activations of the posterior pre-SMA and bilateral anterior insula during prolonged response times. This directional link is also supported by Granger causality analysis. Furthermore, proactive control, prediction error, and time-on-task are each mapped to distinct areas in the medial prefrontal cortex. Together, these findings dissect regional functions of the medial prefrontal cortex in cognitive control and provide system level evidence associating conflict anticipation with its motor consequence.

Parkinson-related changes of activation in visuomotor brain regions during perceived forward self-motion

Radial expanding optic flow is a visual consequence of forward locomotion. Presented on screen, it generates illusionary forward self-motion, pointing at a close vision-gait interrelation. As particularly parkinsonian gait is vulnerable to external stimuli, effects of optic flow on motor-related cerebral circuitry were explored with functional magnetic resonance imaging in healthy controls (HC) and patients with Parkinson’s disease (PD). Fifteen HC and 22 PD patients, of which 7 experienced freezing of gait (FOG), watched wide-field flow, interruptions by narrowing or deceleration and equivalent control conditions with static dots. Statistical parametric mapping revealed that wide-field flow interruption evoked activation of the (pre-)supplementary motor area (SMA) in HC, which was decreased in PD. During wide-field flow, dorsal occipito-parietal activations were reduced in PD relative to HC, with stronger functional connectivity between right visual motion area V5, pre-SMA and cerebellum (in PD without FOG). Non-specific ‘changes’ in stimulus patterns activated dorsolateral fronto-parietal regions and the fusiform gyrus. This attention-associated network was stronger activated in HC than in PD. PD patients thus appeared compromised in recruiting medial frontal regions facilitating internally generated virtual locomotion when visual motion support falls away. Reduced dorsal visual and parietal activations during wide-field optic flow in PD were explained by impaired feedforward visual and visuomotor processing within a magnocellular (visual motion) functional chain. Compensation of impaired feedforward processing by distant fronto-cerebellar circuitry in PD is consistent with motor responses to visual motion stimuli being either too strong or too weak. The ‘change’-related activations pointed at covert (stimulus-driven) attention.

Comparison of abstract decision encoding in the monkey prefrontal cortex, the presupplementary, and cingulate motor areas

Deciding between alternatives is a critical element of flexible behavior. Perceptual decisions have been studied extensively in an action-based framework. Recently, we have shown that abstract perceptual decisions are encoded in prefrontal cortex (PFC) neurons ( Merten and Nieder 2012 ). However, the role of other frontal cortex areas remained elusive. Here, we trained monkeys to perform a rule-based visual detection task that disentangled abstract perceptual decisions from motor preparation. We recorded the single-neuron activity in the presupplementary (preSMA) and the rostral part of the cingulate motor area (CMAr) and compared it to the results previously found in the PFC. Neurons in both areas traditionally identified with motor planning process the abstract decision independently of any motor preparatory activity by similar mechanisms as the PFC. A larger proportion of decision neurons and a higher strength of decision encoding was found in the preSMA than in the PFC. Neurons in both areas reliably predicted the monkeys' decisions. The fraction of CMAr decision neurons and their strength of the decision encoding were comparable to the PFC. Our findings highlight the role of both preSMA and CMAr in abstract cognitive processing and emphasize that both frontal areas encode decisions prior to the preparation of a motor output.

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device

Animal behavior often involves a temporally ordered sequence of actions learned from experience. Here we describe simulations of interconnected networks of spiking neurons that learn to generate patterns of activity in correct temporal order. The simulation consists of large-scale networks of thousands of excitatory and inhibitory neurons that exhibit short-term synaptic plasticity and spike-timing dependent synaptic plasticity. The neural architecture within each area is arranged to evoke winner-take-all (WTA) patterns of neural activity that persist for tens of milliseconds. In order to generate and switch between consecutive firing patterns in correct temporal order, a reentrant exchange of signals between these areas was necessary. To demonstrate the capacity of this arrangement, we used the simulation to train a brain-based device responding to visual input by autonomously generating temporal sequences of motor actions.

Neural population partitioning and a concurrent brain-machine interface for sequential motor function

While brain-machine interfaces (BMIs) have largely focused on performing single-targeted movements, many natural tasks involve planning a complete sequence of such movements before execution. For these tasks, a BMI that can concurrently decode the full planned sequence prior to its execution may also consider the higher-level goal of the task to reformulate and perform it more effectively. Here, we show that concurrent BMI decoding is possible. Using population-wide modeling, we discover two distinct subpopulations of neurons in the rhesus monkey premotor cortex that allow two planned targets of a sequential movement to be simultaneously held in working memory without degradation. Such surprising stability occurred because each subpopulation encoded either only currently held or only newly added target information irrespective of the exact sequence. Based on these findings, we develop a BMI that concurrently decodes a full motor sequence in advance of movement and then can accurately execute it as desired.

Influence of anti-Nogo-A antibody treatment on the reorganization of callosal connectivity of the premotor cortical areas following unilateral lesion of primary motor cortex (M1) in adult macaque monkeys

Following unilateral lesion of the primary motor cortex, the reorganization of callosal projections from the intact hemisphere to the ipsilesional premotor cortex (PM) was investigated in 7 adult macaque monkeys, in absence of treatment (control; n = 4) or treated with function blocking antibodies against the neurite growth inhibitory protein Nogo-A (n = 3). After functional recovery, though incomplete, the tracer biotinylated dextran amine (BDA) was injected in the ipsilesional PM. Retrogradely labelled neurons were plotted in the intact hemisphere and their number was normalized with respect to the volume of the core of BDA injection sites. (1) The callosal projections to PM in the controls originate mainly from homotypic PM areas and, but to a somewhat lesser extent, from the mesial cortex (cingulate and supplementary motor areas). (2) In the lesioned anti-Nogo-A antibody-treated monkeys, the normalized number of callosal retrogradely labelled neurons was up to several folds higher than in controls, especially in the homotypic PM areas. (3) Except one control with a small lesion and a limited, transient deficit, the anti-Nogo-A antibody-treated monkeys recovered to nearly baseline levels of performance (73–90 %), in contrast to persistent deficits in the control monkeys. These results are consistent with a sprouting and/or sparing of callosal axons promoted by the anti-Nogo-A antibody treatment after lesion of the primary motor cortex, as compared to untreated monkeys.

Suppression of proprioceptive feedback control in movement sequences through intermediate targets

Simple movements can be seen as building blocks for complex action sequences, and neural control of an action sequence can be expected to preserve some control features of its constituent blocks. It was previously found that during single-joint elbow movements to a single target, the proprioceptive feedback control is initially suppressed, and we tested this feedback suppression in a two-segment sequence during which subjects momentarily slowed down at an intermediate target at a 30° distance (first segment) and then immediately moved another 30° to the final target (second segment). Either the first or second segment was unexpectedly perturbed; the latency of the earliest response to the perturbation in the muscle surface electromyogram was analyzed. The perturbations were delivered either at the onset of each segment or about 0.1 s later. We found that in both segments, the response latency to the late perturbation was shorter than the latency to the early perturbation, which suggests that the proprioceptive feedback control is suppressed in the beginning of each segment. Next, we determined the latency of the response to unexpected perturbations in 30° movements to a single target. We found that the response latency was not significantly different in the movement to a single target and in each segment in the sequence. This result suggests that the initial suppression of the proprioceptive feedback control in movements to single targets is preserved in movements through intermediate targets and supports the idea of modular organization of neural control of movement sequences.

Neuronal activity reflecting progression of trials in the pre-supplementary motor area of macaque monkey: an expression of neuronal flexibility

We studied the activity of single neurons in the pre-supplementary motor area (pre-SMA) of macaque monkeys as they performed two visuomotor tasks, called the visual fixation task and the visual fixation-blink task. Both tasks involved a sequence of three visual stimuli, red followed by yellow and green. The tasks differed in that the latter one had a gap within the period of the red stimulus, called a "blink". The tasks were performed in two modes, one of which included movements of both the arm and eye and the other of which involved only eye movements. In the arm-eye mode, the monkeys had to press a bar and fixate the red stimulus that appeared after bar press. To receive a reward, both the bar press and visual fixation had to be maintained until the green stimulus triggered bar release. In the eye mode, bar press and bar release were eliminated from the task. Of the 42 neurons active during the visual fixation task, 15 showed task-related activity in both arm-eye and eye modes, and our analysis focused on these cells. We found that the introduction of the blink in visual fixation-blink task abolished the task-related activity of these cells over the course of 2-4 trials. This finding suggests a role for the pre-SMA in reflecting progression of trials as an updating of motor instruction.

Neural modulation of temporal encoding, maintenance, and decision processes

Time perception emerges from an interaction among multiple processes that are normally intertwined. Therefore, a challenge has been to disentangle timekeeping from other processes. Though the striatum has been implicated in interval timing, it also modulates nontemporal processes such as working memory. To distinguish these processes, we separated neural activation associated with encoding, working-memory maintenance, and decision phases of a time-perception task. We also asked whether neuronal processing of duration (i.e., pure tone) was distinct from the processing of identity (i.e., pitch perception) or sensorimotor features (i.e., control task). Striatal activation was greater when encoding the duration than the pitch or basic sensory features, which did not differentially engage the striatum. During the maintenance phase, striatal activation was similar for duration and pitch but at baseline in the control task. In the decision phase, a stepwise reduction in striatal activation was found across the 3 tasks, with activation greatest in the timing task and weakest in the control task. Task-related striatal activations in different cognitive phases were distinguished from those of the supplementary motor area, inferior frontal gyrus, thalamus, frontoparietal cortices, and the cerebellum. Our results were consistent with a model in which timing emerges from context-dependent corticostriatal interactions.

Rank signals in four areas of macaque frontal cortex during selection of actions and objects in serial order

Neurons in several areas of monkey frontal cortex exhibit ordinal position (rank) selectivity during the performance of serial order tasks. It has been unclear whether rank selectivity or the dependence of rank selectivity on task context varies across the areas of frontal cortex. To resolve this issue, we recorded from neurons in the supplementary motor area (SMA), presupplementary motor area (pre-SMA), supplementary eye field (SEF), and dorsolateral prefrontal cortex (dlPFC) as monkeys performed two oculomotor tasks, one requiring the selection of three actions in sequence and the other requiring the selection of three objects in sequence. We found that neurons representing all ranks were present in all areas. Only to a moderate degree did the prevalence and nature of rank selectivity vary from area to area. The two most prominent inter-area differences involved a lower prevalence of rank selectivity in the dlPFC than in the other areas and a higher proportion of neurons preferring late ranks in the SMA and SEF than in the other areas. Neurons in all four areas are rank generalists in the sense of favoring the same rank in both the serial action task and the serial object task.

Activation of the pre-supplementary motor area but not inferior prefrontal cortex in association with short stop signal reaction time--an intra-subject analysis

BACKGROUND

Our previous work described the neural processes of motor response inhibition during a stop signal task (SST). Employing the race model, we computed the stop signal reaction time (SSRT) to index individuals' ability in inhibitory control. The pre-supplementary motor area (preSMA), which shows greater activity in individuals with short as compared to those with long SSRT, plays a role in mediating response inhibition. In contrast, the right inferior prefrontal cortex (rIFC) showed greater activity during stop success as compared to stop error. Here we further pursued this functional differentiation of preSMA and rIFC on the basis of an intra-subject approach.

RESULTS

Of 65 subjects who participated in four sessions of the SST, we identified 30 individuals who showed a difference in SSRT but were identical in other aspects of stop signal performance between the first ("early") and last two ("late") sessions. By comparing regional brain activation between the two sessions, we confirmed greater preSMA but not rIFC activity during short as compared to long SSRT session within individuals. Furthermore, putamen, anterior cerebellum and middle/posterior cingulate cortex also showed greater activity in association with short SSRT.

CONCLUSION

These results are consistent with a role of medial prefrontal cortex in controlled action and inferior frontal cortex in orienting attention. We discussed these findings with respect to the process of attentional monitoring and inhibitory motor control during stop signal inhibition.