Cortical populations and behaviour: Hebb's thread

This paper discusses work on the function of the motor cortex as revealed by single cell recordings in monkeys and artificial neural network modelling. Our key conceptual approach both in behavioural neuroscience and neural network modeling of motor cortical function relies on reconstructing, visualizing, and modelling the activity in neuronal populations, indeed a key concept advanced by Hebb (1949). The behaviour investigated ranges from exertion of isometric force to pointing movements to complex cognitive processing. The functional properties of single cells with respect to the direction of movement in space are described as well as a population code which provides a unique measure for this direction. Finally, the results of modeling studies are discussed in which directional population activity is used as an input to an artificial neural network to drive a simulated arm.

Drawing under visuomotor incongruence

Six human subjects were asked to draw ellipses presented on a screen by moving a manipulandum that controlled the position of a cursor. Six visual templates were used, which comprised three different ellipses displayed either horizontally or vertically; the ratio between the major and minor axes was 2, 4, or 5. For each visual template, the gains were set such that the movement trajectories required to trace the template with the cursor corresponded to one of six ellipses. Thus these movement ellipses were horizontal or vertical with a ratio between major and minor axes of 2, 4, or 5. All 36 combinations of six visual ellipses and six required movement ellipses were used. Therefore, in some conditions the required movement ellipse had a different orientation (with respect to the major axis) than the visual template. These conditions were called orientation incongruent, whereas, when the orientation of the required movement ellipse matched the orientation of the visual template, the conditions were called orientation congruent. Similarly, eccentricity incongruent referred to conditions where the eccentricities of the visual ellipse and the required movement ellipse were different, as opposed to eccentricity congruent. The main results were as follows: (a) The perimeter of the traced ellipse always tended to be larger than that of the visual template. In addition, it was significantly larger in the orientation incongruent conditions than in the orientation congruent conditions. Nevertheless, the perimeter of the traced figure increased with the template in both orientation congruent and incongruent conditions. (b) The shape of the traced figure varied appropriately with the visual template, but differed significantly between the orientation congruent and incongruent conditions. It was closer to the one of the template in the orientation congruent than in the incongruent conditions. Finally, (c) the instantaneous speed was significantly correlated with curvature but more tightly so in the orientation congruent than in the orientation incongruent conditions. The parameters defining the relation between speed and curvature were affected by the required movement ellipse, but not by the particular visuomotor condition. These results showed that although spatial motor performance was affected by changes in the correspondence between visual and movement coordinates, the relation between the speed and curvature of the movement trajectory was stable despite drastic changes in this correspondence.

Online visual control of the arm

The psychophysical and cerebrocortical mechanisms in visually guided reaching movements and isometric force pulses are discussed. The results of psychophysical studies of pointing movements have demonstrated a tight coupling between the visual information and the direction of the movement, and those of studies of directed isometric force pulses have documented the sensitive dependence of the motor system on the continuous availability of visual information for the ongoing correction of directional deviations from the instructed direction. Recordings of the activity of single cells in the motor cortex and parietal areas 2 and 5 have revealed the same tight, online coupling between visual information and cell discharge, and have partially elucidated the neural mechanisms underlying this function at the cortical level.

Interview with Apostolos P. Georgopoulos

Apostolos P. Georgopoulos studied Medicine and Physiology at the University of Athens in Greece where he obtained his M.D. and Ph.D. degrees. He was trained in neurophysiology by Vernon B. Mountcastle at Johns Hopkins and, after a brief return to Athens, he came back to Johns Hopkins. He ascended the faculty ranks and promoted to Professor of Neuroscience in 1986. He was a member of the Philip Bard Laboratories of Neurophysiology at the Department of Neuroscience until 1991 when he moved to Minnesota as the American Legion Brain Sciences Chair at the Minneapolis Veterans Affairs Medical Center and the University of Minnesota.

Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex

We analyzed the magnitude and interneuronal correlation of the variability in the activity of single neurons that were recorded simultaneously using a multielectrode array in the primary motor cortex and parietal areas 2/5 in rhesus monkeys. The animals were trained to move their arms in one of eight directions as instructed by a visual target. The relationship between variability (SD) and mean of the discharge rate was described by a power function with a similar exponent ( approximately 0.57), regardless of the cortical area or the behavioral condition. We examined whether the deviation from mean activity between target onset and the end of the movement was correlated on a trial-by-trial basis with variability in activity during the hold period before target onset. In both cortical areas, for about a quarter of the neurons, the neuronal noise of these two periods was positively correlated, whereas significant negative correlations were seldom observed. Overall, neurons with higher signal correlation (i.e., similar directional pattern) showed higher noise correlation in both cortical areas. On the other hand, when the data were divided according to the distance between the electrode tips from which the neurons were recorded, a consistent relationship between the signal and noise correlations was found only for pairs of neurons recorded through the same electrode. These results suggest that nearby neurons with similar directional tuning carry primarily redundant messages, whereas neurons in separate cortical columns perform more independent processing.

Manual interception of moving targets. II. On-line control of overlapping submovements

We studied the kinematic characteristics of arm movements and their relation to a stimulus moving with a wide range of velocity and acceleration. The target traveled at constant acceleration, constant deceleration, or constant velocity for 0.5-2.0 s, until it arrived at a location where it was required to be intercepted. For fast moving targets, subjects produced single movements with symmetrical, bell-shaped velocity profiles. In contrast, for slowly moving targets, hand velocity profiles displayed multiple peaks, which suggests a control mechanism that produces a series of discrete submovements according to characteristics of target motion. To analyze how temporal and spatial aspects of these submovements are influenced by target motion, we decomposed the vertical hand velocity profiles into bell-shaped velocity pulses according to the minimum-jerk model. The number of submovements was roughly proportional to the movement time, resulting in a relatively constant submovement frequency (approximately 2.5 Hz). On the other hand, the submovement onset asynchrony showed significantly more variability than the intersubmovement interval, indicating that the submovement onset was delayed more following a submovement with a longer duration. Examination of submovement amplitude and its relation to target motion revealed that the subjects achieved interception mainly by producing a series of submovements that would keep the displacement of the hand proportional to the first-order estimate of target position at the end of each submovement along the axis of hand movement. Finally, we did not find any evidence that information regarding target acceleration is properly utilized in the production of submovements.

Manual interception of moving targets. I. Performance and movement initiation

We investigated the capacities of human subjects to intercept moving targets in a two-dimensional (2D) space. Subjects were instructed to intercept moving targets on a computer screen using a cursor controlled by an articulated 2D manipulandum. A target was presented in 1 of 18 combinations of three acceleration types (constant acceleration, constant deceleration, and constant velocity) and six target motion times, from 0.5 to 2.0 s. First, subjects held the cursor in a start zone located at the bottom of the screen along the vertical meridian. After a pseudorandom hold period, the target appeared in the lower left or right corner of the screen and traveled at 45 degrees toward an interception zone located on the vertical meridian 12.5 cm above the start zone. For a trial to be considered successful, the subject's cursor had to enter the interception zone within 100 ms of the target's arrival at the center of the interception zone and stay inside a slightly larger hold zone. Trials in which the cursor arrived more than 100 ms before the target were classified as "early errors," whereas trials in which the cursor arrived more than 100 ms after the target were classified as "late errors." Given the criteria above, the task proved to be difficult for the subjects. Only 41.3% (1080 out of 2614) of the movements were successful, whereas the remaining 58.7% were temporal (i.e., early or late) errors. A large majority of the early errors occurred in trials with decelerating targets, and their percentage tended to increase with longer target motion times. In contrast, late errors occurred in relation to all three target acceleration types, and their percentage tended to decrease with longer target motion times. Three models of movement initiation were investigated. First, the threshold-distance model, originally proposed for optokinetic eye movements to constant-velocity visual stimuli, maintains that response time is composed of two parts, a constant processing time and the time required for the stimulus to travel a threshold distance. This model only partially fit our data. Second, the threshold-tau model, originally proposed as a strategy for movement initiation, assumes that the subject uses the first-order estimate of time-to-contact (tau) to determine when to initiate the interception movement. Similar to the threshold distance model, the threshold-tau model only partially fit the data. Finally, a dual-strategy model was developed which allowed for the adoption of either of the two strategies for movement initiation; namely, a strategy based on the threshold-distance model ("reactive" strategy) and another based on the threshold-tau model ("predictive" strategy). This model provided a good fit to the data. In fact, individual subjects preferred to use one or the other strategy. This preference was allowed to be manifested at long target motion times, whereas shorter target motion times (i.e., 0.5 s and 0.8 s) forced the subjects to use only the reactive strategy.

Sequential activity in human motor areas during a delayed cued finger movement task studied by time-resolved fMRI

Activity in the human primary motor cortex, the premotor cortex and the supplementary motor area during a delayed cued finger movement task was measured by time-resolved functional magnetic resonance imaging. Activity during movement preparation can be resolved from activity during movement execution in a single trial. All three areas were active during both movement preparation and movement execution. Activity in the primary motor cortex was considerably weaker during movement preparation than during movement execution; in the premotor cortex and the supplementary motor area, activity was of similar intensity during both periods. These observations are consistent with results from single neuronal recording studies in primates.

Box-Jenkins intervention analysis of functional magnetic resonance imaging data

Data obtained in functional magnetic resonance imaging (fMRI) typically form a time series of MRI signal collected over a period of time at constant intervals. These data are potentially autocorrelated and may contain time trends. Therefore, any assessment of significant changes in the MRI signal over a certain period of time requires the use of specific statistical techniques. For that purpose we used the Box-Jenkins intervention time series analysis to determine brain activation during task performance. We found that for a substantial number of pixels there was significant autocorrelation and, occasionally, time trends. In these cases, use of the classical t-test would not be appropriate. In contrast, Box-Jenkins intervention analysis, by detrending the series and by explicitly taking into account the correlation structure, provides a more appropriate method to determine the presence of significant activation during the task period in fMRI data.

A simulated actuator driven by motor cortical signals

One problem in motor control concerns the mechanism whereby the central nervous system translates the motor cortical command encoded in cell activity into a coordinated contraction of limb muscles to generate a desired motor output. This problem is closely related to the design of adaptive systems that transform neuronal signals chronically recorded from the motor cortex into the physiologically appropriate motor output of multijoint prosthetic limbs. In this study we demonstrated how this transformation can be carried out by an artificial neural network using as command signals the actual impulse activity obtained from recordings in the motor cortex of monkeys during the performance of a task that required the exertion of force in different directions. The network receives experimentally measured brain signals and recodes them into motor actions of a simulated actuator that mimics the primate arm. The actuator responds to the motor cortical commands with surprising fidelity, generating forces in close quantitative agreement with those exerted by trained monkeys, in both the temporal and spatial domains. Moreover, we show that the time-varying motor output may be controlled by the impulse activity of as few as 15 motor cortical cells. These results outline a potentially implementable computation scheme that utilizes raw neuronal signals to drive artificial mechanical systems.

Arm movements in monkeys: behavior and neurophysiology

Reaching to objects of interest is very common in the behavioral repertoire of primates. Monkeys possess keen binocular vision and make graceful and accurate arm movements. This review focuses on behavioral and neurophysiological aspects of eye-hand coordination in behaving monkeys, including neural coding mechanisms at the single cell level and in neuronal populations. The results of these studies have converged to a common behavioral-neurophysiological ground and provided a springboard for studies of brain mechanisms underlying motor cognitive function.

Intercepting real and path-guided apparent motion targets

Human subjects were instructed to intercept with a cursor real and apparent motion targets presented on a computer screen. Targets traveled counterclockwise (CCW) in a circle at one of five angular velocities (180, 300, 420, 480 and 540 deg/s), either smoothly (real motion) or in path-guided apparent motion. Subjects operated a computer mouse and were instructed to intercept targets at the 12 o'clock position; there were no constraints on when to initiate the response, which was a movement from the center of the screen towards and past 12 o'clock. We found the following: (a) for both motion conditions and all target velocities, subjects were late in intercepting the target, especially at higher target velocities; (b) for both motion conditions, the directional variability of the response increased as a linear function of the target velocity; (c) the directional variability of the response was systematically higher for the apparent than the real motion condition; there was no significant interaction between target velocity and target motion type; (d) the response time did not vary significantly with velocity, but was consistently longer for apparent than real motion targets; (e) the movement time was very similar for different target velocities; and (f) the moment of initiation of the interception movement was delayed appreciably at higher target velocities, relative to that dictated for perfect interception at a given target velocity. This delay was greater for the apparent motion target. These results demonstrated the following: (a) for both target motion conditions, interception was not fully predictive but lagged the target in spite of the constant target velocity and the unconstrained time allowed for initiating the interception movement; (b) subjects can intercept an apparent motion target but, compared with real motion, the performance is somewhat degraded overall; (c) the similarities in performance between the two target motion conditions, and the fact that target velocity influenced performance in a similar fashion, suggest that the motor system can access the visual information provided by the moving target; and (d) since movement time was similar for different target velocities, the strategy for interception relied on controlling the moment of initiation of the interception movement. This was successful for low target velocities but became unsuccessful at higher target velocities.

On the relations between single cell activity in the motor cortex and the direction and magnitude of three-dimensional static isometric force

We examined the relations between the steady-state frequency of discharge of cells in the arm area of the motor cortex of the monkey and the direction and magnitude of the three-dimensional static force exerted by the arm on an isometric manipulandum. Data were analyzed from two monkeys (n = 188 cells) using stepwise multiple linear regression. In 154 of 188 (81.9%) cells the regression model was statistically significant (P < 0.05). In 121 of 154 (78.6%) cells the direction but not the magnitude of force had a statistically significant effect on cell activity; in 11 of 154 (7.1%) cells only the magnitude effect was significant; and in 22 of 154 (14.3%) cells both the direction and magnitude effects were significant. The same analysis was used to assess the effect of the direction and magnitude of force on the electromyographic activity of 9 muscles of the arm and shoulder girdle. The regression model was statistically significant. For all the muscles studied in 4 of 9 (44.4%) muscles only the direction effect was significant whereas in the remaining 5 of 9 (55.6%) muscles both the direction and the magnitude were significant. No muscle studied showed a significant effect of force magnitude alone. These differences in the frequency of occurrence of directional and magnitude effects between cells and muscles were statistically significant (P < 0.005, chi 2 test). These findings underscore the fundamental importance of the direction of force in space for both motor cortical cells and proximal muscles and underline the differential relations of the cells and muscles to the direction and magnitude of force. These results indicate that the specification of the magnitude of three-dimensional force is embedded within the directional signal; this combined direction+magnitude effect was 3.9 times more prevalent in the muscles than in the cells studied. In contrast, the pure directional effect was 1.8 times more prevalent in the cells than in the muscles studied. This suggests that the direction of force can be controlled independently of its magnitude and that this direction signal is especially prominent in the motor cortex.

Neural computations underlying the exertion of force: a model

We have developed a model that simulates possible mechanisms by which supraspinal neuronal signals coding forces could converge in the spinal cord and provide an ongoing integrated signal to the motoneuronal pools whose activation results in the exertion of force. The model consists of a three-layered neural network connected to a two-joint-six-muscle model of the arm. The network layers represent supraspinal populations, spinal cord interneurons, and motoneuronal pools. We propose an approach to train the network so that, after the synaptic connections between the layers are adjusted, the performance of the model is consistent with experimental data obtained on different organisms using different experimental paradigms: the stiffness characteristics of human arm; the structure of force fields generated by the stimulation of the frog's spinal cord; and a correlation between motor cortical activity and force exerted by monkey against an immovable object. The model predicts a specific pattern of connections between supraspinal populations coding forces and spinal cord interneurons: the weight of connection should be correlated with directional preference of interconnected units. Finally, our simulations demonstrate that the force generated by the sum of neural signals can be nearly equal to the vector sum of forces generated by each signal independently, in spite of the complex nonlinearities intervening between supraspinal commands and forces exerted by the arm in response to these commands.

Modeling motor cortical operations by an attractor network of stochastic neurons

Understanding the neural computations performed by the motor cortex requires biologically plausible models that account for cell discharge patterns revealed by neurophysiological recordings. In the present study the motor cortical activity underlying movement generation is modeled as the dynamic evolution of a large fully recurrent network of stochastic spiking neurons with noise superimposed on the synaptic transmission. We show that neural representations of the learned movement trajectories can be stored in the connectivity matrix in such a way that, when activated, a particular trajectory evolves in time as a dynamic attractor of the system while individual neurons fire irregularly with large variability in their interspike intervals. Moreover, the encoding of trajectories as attractors ensures high stability of the ensemble dynamics in the presence of synaptic noise. In agreement with neurophysiological findings, the suggested model can provide a wide repertoire of specific motor behaviors, whereas the number of specialized cells and specific connections may be negligibly small if compared with the whole population engaged in trajectory retrieving. To examine the applicability of the model we study quantitatively the relationship between local geometrical and kinematic characteristics of the trajectories generated by the network. The relationship obtained as a result of simulations is close to the '2/3 power law' established by psychophysical and neurophysiological studies.

On the translation of directional motor cortical commands to activation of muscles via spinal interneuronal systems

I discuss in this paper some of the neural mechanisms by which directional motor cortical commands could be potentially translated into multi-muscle activations to generate a directed force (and initial movement) in space. Specifically, I review the results of recent studies in the motor cortex of monkeys and the spinal cord of the frog, and propose a possible mechanism by which these results could be formally connected. It is suggested that spinal mechanisms of the kind described in the spinal frog could serve as substrates for the operation of directionally tuned motor cortical activity to produce an appropriately directed motor output by the limb.

Quantitative relations between parietal activation and performance in mental rotation

The quantitative relationships between functional activation of the superior parietal lobule (SPL) and performance in the Shepard-Metzler mental rotation task were investigated in 16 human subjects using magnetic resonance (MR) imaging at high field (4 Tesla). Subjects were shown pairs of perspective drawings of three-dimensional objects and asked to judge whether they were the same or mirror images. Increased SPL activation was associated with a higher proportion of errors in performance. The increase in errors, and the concomitant increase in SPL activation, could be due to an increased difficulty in, and therefore increased demands for, information processing at several stages involved in making a decision, including encoding of the visual images shown, mentally rotating them, and judging whether they are the same or mirror images.

The mental and the neural: psychological and neural studies of mental rotation and memory scanning

In this article we review studies pertaining to psychophysical measurements and neural correlates of tasks requiring the processing of directional information in spatial motor tasks. The results of psychological studies in human subjects indicate that time-consuming processes underlie mental rotation and memory scanning. Other studies have suggested that these processes may rely on different basic mechanisms. A direct insight into their neural mechanisms was obtained analyzing the activity of single cells and neuronal populations in the brain of behaving monkeys performing the same tasks. These studies revealed the nature of the neural processes underlying mental rotation and memory scanning and confirmed their different nature.

Current issues in directional motor control

Many studies during the past 15 years have shown that the direction of motor output (movement or isometric force) is an important factor for neuronal activity in the motor cortex, both at the level of single cells and at the level of neuronal populations. Recent studies have investigated several new aspects of this problem including the effect of posture, the relations to time-varying movement parameters (for example, position, velocity and acceleration) and the cortical representation of memorized simple movements and complex-movement trajectories. Furthermore, the neural correlates of directional operations, such as mental rotation and memory-scanning of visuomotor directions, have also been investigated. In addition, neural networks have been used to model dynamic, time-varying, spatial motor trajectories.

System for projection of a three-dimensional, moving virtual target for studies of eye-hand coordination

Eye-hand tracking of moving visual objects in three-dimensional (3D) space is common in the behavioral repertoire of primates. However, behavioral and/or neurophysiological studies of this function are lacking mainly because devices do not exist that allow its investigation. We describe a device by which a spot of light can be presented in the immediate extrapersonal space of a subject and can be moved in various trajectories in 3D space. The target is a real image of a circular aperture produced by a system consisting of a light source, aperture, filters, several lenses and fold mirrors, and a large concave mirror to focus the final real image. Rapid, computer-controlled movement of the image is obtained by tilting a gimbal-mounted guide mirror (for x and y motion) and by translating a lens (for motion in the z direction). A second configuration of the system allows movement of a 3D image in the 3D space. Hand motion is monitored by means of a sonic, 3D, position-measurement system.

Motor cortical activity in a context-recall task

A monkey was trained to respond on the basis of the serial position of a test stimulus in a sequence. First, three stimuli were presented successively on a circle. Then one of them (except the last) changed color (test stimulus) and served as the go signal: The monkey was required to produce a motor response in the direction of the stimulus that followed the test stimulus. When the test stimulus was the second in the sequence, there was a change in motor cortical activity from a pattern reflecting the direction of this stimulus to the pattern associated with the direction of the motor response. This change was abrupt, occurred 100 to 150 milliseconds after the go signal, and was evident both in the activity of single cells and in the time-varying neuronal population vector. These findings identify the neural correlates of a switching process that is different from a mental rotation described previously.

Behavioral neurophysiology of the motor cortex

The study of the motor cortex in behaving monkeys during the past 20 years has provided important information on the brain mechanisms underlying motor control. With respect to reaching movements in space, several aspects of motor cortical function concerning the specification of the direction of movement have now been elucidated and are reviewed in this article. The activity of single cells in the motor cortex is broadly tuned with respect to the direction of reaching, so that the discharge rate is highest with movements in a preferred direction and decreases progressively with movements made in directions more and more away from the preferred one. Thus the neural command for the direction of reaching can be regarded as an ensemble of cell vectors, with each vector pointing in the cell's preferred direction and having a length proportional to the change in cell activity. The outcome of this population code can be visualized as a vector that points in the direction of the upcoming movement during the reaction time, during an instructed delay period, and during a memorized delay period. Moreover, when a mental transformation is required for the generation of a reaching movement in a different direction from a reference direction, the population vector provides a direct insight into the nature of the cognitive process by which the required transformation is achieved.

Movement parameters and neural activity in motor cortex and area 5

The relations of ongoing single-cell activity in the arm area of the motor cortex and area 5 to parameters of evolving arm movements in two-dimensional (2D) space were investigated. A multiple linear regression model was used in which the ongoing impulse activity of cells at time t + tau was expressed as a function of the (X, Y) components of the target direction and of position, velocity, and acceleration of the hand at time t, where tau was a time shift (-200 to +200 msec). Analysis was done on 290 cells in the motor cortex and 207 cells in area 5. The time shift at which the highest coefficient of determination (R2) was observed was determined and the statistical significance of the model tested. The median R2 was 0.581 and 0.530 for motor cortex and area 5, respectively. The median shift at which the highest R2 was observed was -90 and +30 msec for motor cortex and area 5, respectively. For most cells statistically significant relations were observed to all four parameters tested; most prominent were the relations to target direction and least prominent those to acceleration.