Decoding Target Distance and Saccade Amplitude from Population Activity in the Macaque Lateral Intraparietal Area (LIP)

Effects of normalized summation in the visual illusion of extent

In the present study, the features of summation of effects caused by contextual distracting dots in the length-matching task (a variant of the filled-space illusion) were investigated. In the first two series of psychophysical experiments, the illusion magnitude was measured as a function of the displacement of distractors (either single or double sets of dots) orthogonally to the main axis of the stimulus. It was demonstrated that with increasing displacement, the illusion smoothly decreases for a single set of distractors, while for two sets, the illusion first increases to a certain maximum value, and then gradually decreases. In the third and fourth series of experiments, magnitude of the illusion was measured as a function of the luminance of one set of distracting dots, while the luminance of the other set was fixed. It has been shown that increasing the luminance until the same value is reached for both sets leads to a monotonous growth in the illusion magnitude; after that, the illusion asymptotically decreases to an almost constant level. The theoretical interpretation of the established functional dependencies was performed using a quantitative model based on the assumption that the illusion may arise due to the weighted summation of the distractor-induced normalized neural activity, which leads to the perceptual mislocalization of terminators of stimulus spatial intervals.

Integration of landmark and saccade target signals in macaque frontal cortex visual responses

Visual landmarks influence spatial cognition and behavior, but their influence on visual codes for action is poorly understood. Here, we test landmark influence on the visual response to saccade targets recorded from 312 frontal and 256 supplementary eye field neurons in rhesus macaques. Visual response fields are characterized by recording neural responses to various target-landmark combinations, and then we test against several candidate spatial models. Overall, frontal/supplementary eye fields response fields preferentially code either saccade targets (40%/40%) or landmarks (30%/4.5%) in gaze fixation-centered coordinates, but most cells show multiplexed target-landmark coding within intermediate reference frames (between fixation-centered and landmark-centered). Further, these coding schemes interact: neurons with near-equal target and landmark coding show the biggest shift from fixation-centered toward landmark-centered target coding. These data show that landmark information is preserved and influences target coding in prefrontal visual responses, likely to stabilize movement goals in the presence of noisy egocentric signals.

Changes in length judgments caused by rotation of the contextual distractor

In the present study, we tested the applicability of the computational model of the illusion of interrupted spatial extent (Bulatov, Marma, & Bulatova, Attention, Perception, & Psychophysics, 82, 2714-2727, 2020) to account for the psychophysical data collected with three-dot stimuli containing a cross-shaped contextual distractor. In different series of experiments, the illusion magnitude changes caused by the rotation of distractors with different values of the internal angle (45°, 75°, and 90°) were quantitatively determined. It was shown that the data obtained for all modifications of stimuli can be rather well approximated by model functions proportional to the sum of the absolute values of cosines. A good agreement between theoretical calculations and experimental results supports the suggestion that the perceptual displacement of the stimulus terminators, which occurs due to the processes of local integration of neural activity, may be one of the main causes of the illusion investigated.

Robust Coding of Eye Position in Posterior Parietal Cortex despite Context-Dependent Tuning

Neurons in posterior parietal cortex (PPC) encode many aspects of the sensory world (e.g., scene structure), the posture of the body, and plans for action. For a downstream computation, however, only some of these dimensions are relevant; the rest are "nuisance variables" because their influence on neural activity changes with sensory and behavioral context, potentially corrupting the read-out of relevant information. Here we show that a key postural variable for vision (eye position) is represented robustly in male macaque PPC across a range of contexts, although the tuning of single neurons depended strongly on context. Contexts were defined by different stages of a visually guided reaching task, including (1) a visually sparse epoch, (2) a visually rich epoch, (3) a "go" epoch in which the reach was cued, and (4) during the reach itself. Eye position was constant within trials but varied across trials in a 3 × 3 grid spanning 24° × 24°. Using demixed principal component analysis of neural spike-counts, we found that the subspace of the population response encoding eye position is orthogonal to that encoding task context. Accordingly, a context-naive (fixed-parameter) decoder was nevertheless able to estimate eye position reliably across contexts. Errors were small given the sample size (∼1.78°) and would likely be even smaller with larger populations. Moreover, they were comparable to that of decoders that were optimized for each context. Our results suggest that population codes in PPC shield encoded signals from crosstalk to support robust sensorimotor transformations across contexts.SIGNIFICANCE STATEMENT Neurons in posterior parietal cortex (PPC) which are sensitive to gaze direction are thought to play a key role in spatial perception and behavior (e.g., reaching, navigation), and provide a potential substrate for brain-controlled prosthetics. Many, however, change their tuning under different sensory and behavioral contexts, raising the prospect that they provide unreliable representations of egocentric space. Here, we analyze the structure of encoding dimensions for gaze direction and context in PPC during different stages of a visually guided reaching task. We use demixed dimensionality reduction and decoding techniques to show that the coding of gaze direction in PPC is mostly invariant to context. This suggests that PPC can provide reliable spatial information across sensory and behavioral contexts.

Coding of interceptive saccades in parietal cortex of macaque monkeys

The oculomotor system can initiate remarkably accurate saccades towards moving targets (interceptive saccades) the processing of which is still under debate. The generation of these saccades requires the oculomotor centers to have information about the motion parameters of the target that then must be extrapolated to bridge the inherent processing delays. We investigated to what degree the information about motion of a saccade target is available in the lateral intra-parietal area (area LIP) of macaque monkeys for generation of accurate interceptive saccades. When a multi-layer neural network was trained based on neural discharges from area LIP around the time of saccades towards stationary targets, it was also able to predict the end points of saccades directed towards moving targets. This prediction, however, lagged behind the actual post-saccadic position of the moving target by ~ 80 ms when the whole neuronal sample of 105 neurons was used. We further found that single neurons differentially code for the motion of the target. Selecting neurons with the strongest representation of target motion reduced this lag to ~ 30 ms which represents the position of the moving target approximately at the onset of the interceptive saccade. We conclude that-similarly to recent findings from the Superior Colliculus (Goffart et al. J Neurophysiol 118(5):2890-2901)-there is a continuum of contributions of individual LIP neurons to the accuracy of interceptive saccades. A contribution of other gaze control centers (like the cerebellum or the frontal eye field) that further increase the saccadic accuracy is, however, likely.

Quantitative examination of an unconventional form of the filled-space illusion

The spatial interval containing some visual elements (fillers) seems to be longer than an empty interval of the same length, and the effect persists for most observers. This illusion of interrupted spatial extent (or the filled-space illusion) can be observed even in extremely simplified line drawings, but its origin is still not completely understood. Recently, we proposed a quantitative explanation for the results of experiments with stimuli containing either continuous or discrete filling: the illusion may be associated with the integration of distractor-induced effects near the endpoints (terminators) of the stimulus intervals. Subsequent analysis of the principles underlying the explanation allowed us to hypothesize the appearance of illusory effects caused by previously unknown stimulus modifications. To test the suggestions, in the present study we performed experiments with three-dot stimuli that contain a distracting circle (either outline or uniformly filled) surrounding one of the lateral terminators. It has been demonstrated that the illusion magnitude varies predictably with the size of the circle, and there is no significant difference between the data obtained for stimuli with the outline and filled distractors. To more thoroughly examine the illusion, the central angle of circular distracting arcs (real or imaginary) was used as an independent variable in supplementary experiments. A rather successful theoretical interpretation of the experimental results supports the suggestion that perceptual positional biases induced by additional context-evoked neural excitation can be considered as one of the main causes of the filled-space illusion.

Spatiotemporal Coding in the Macaque Supplementary Eye Fields: Landmark Influence in the Target-to-Gaze Transformation

Eye-centered (egocentric) and landmark-centered (allocentric) visual signals influence spatial cognition, navigation, and goal-directed action, but the neural mechanisms that integrate these signals for motor control are poorly understood. A likely candidate for egocentric/allocentric integration in the gaze control system is the supplementary eye fields (SEF), a mediofrontal structure with high-level “executive” functions, spatially tuned visual/motor response fields, and reciprocal projections with the frontal eye fields (FEF). To test this hypothesis, we trained two head-unrestrained monkeys (Macaca mulatta) to saccade toward a remembered visual target in the presence of a visual landmark that shifted during the delay, causing gaze end points to shift partially in the same direction. A total of 256 SEF neurons were recorded, including 68 with spatially tuned response fields. Model fits to the latter established that, like the FEF and superior colliculus (SC), spatially tuned SEF responses primarily showed an egocentric (eye-centered) target-to-gaze position transformation. However, the landmark shift influenced this default egocentric transformation: during the delay, motor neurons (with no visual response) showed a transient but unintegrated shift (i.e., not correlated with the target-to-gaze transformation), whereas during the saccade-related burst visuomotor (VM) neurons showed an integrated shift (i.e., correlated with the target-to-gaze transformation). This differed from our simultaneous FEF recordings (Bharmauria et al., 2020), which showed a transient shift in VM neurons, followed by an integrated response in all motor responses. Based on these findings and past literature, we propose that prefrontal cortex incorporates landmark-centered information into a distributed, eye-centered target-to-gaze transformation through a reciprocal prefrontal circuit.

Spatiotemporal transformations for gaze control

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time-locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head-unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye-centered, effector-independent parameters, and show-for the first time in ordinary gaze shifts-a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T-G continuum) that revealed task-dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures).

Integration of Eye-Centered and Landmark-Centered Codes in Frontal Eye Field Gaze Responses

The visual system is thought to separate egocentric and allocentric representations, but behavioral experiments show that these codes are optimally integrated to influence goal-directed movements. To test if frontal cortex participates in this integration, we recorded primate frontal eye field activity during a cue-conflict memory delay saccade task. To dissociate egocentric and allocentric coordinates, we surreptitiously shifted a visual landmark during the delay period, causing saccades to deviate by 37% in the same direction. To assess the cellular mechanisms, we fit neural response fields against an egocentric (eye-centered target-to-gaze) continuum, and an allocentric shift (eye-to-landmark-centered) continuum. Initial visual responses best-fit target position. Motor responses (after the landmark shift) predicted future gaze position but embedded within the motor code was a 29% shift toward allocentric coordinates. This shift appeared transiently in memory-related visuomotor activity, and then reappeared in motor activity before saccades. Notably, fits along the egocentric and allocentric shift continua were initially independent, but became correlated across neurons just before the motor burst. Overall, these results implicate frontal cortex in the integration of egocentric and allocentric visual information for goal-directed action, and demonstrate the cell-specific, temporal progression of signal multiplexing for this process in the gaze system.

Two-dimensional profile of the region of distractors' influence on visual length judgments

In the illusion of interrupted spatial extent (also known as the filled-space or Oppel-Kundt illusion), the stimulus spatial interval filled with some visual elements (distractors) appears larger than the unfilled interval of the same size. Despite a long history of research, there is still no consensus on the origin of this visual phenomenon. It was recently shown (Bulatov, Bulatova, Surkys, & Mickienė, Acta Neurobiologiae Experimentalis, 77, 157-167, 2017) that the illusion emergence can be associated mainly with the integration of distractor-evoked effects in regions surrounding the endpoints (terminators) of the stimulus intervals. In the present study, we investigated the two-dimensional weighting profiles of these regions of distractors' influence on the magnitude of length misjudgments. We performed psychophysical experiments with three-dot stimuli that contain distracting line segments, the position of which varied either along or perpendicular to the main stimulus axis, thus scanning the profile in two orthogonal directions. It was demonstrated that for distractors shifted along the stimulus axis, the magnitude of the illusion increases to a certain maximum value with the increase of distractors displacement and smoothly decreases to zero thereafter. For distractors shifted orthogonally to the stimulus axis, the illusion magnitude monotonically decreases with the increase of distractors displacement. In the case of the distractor rotation, the greatest illusion magnitude refers to orientations of the distracting line segment along the stimulus axis and decreases to the minimum value for the orthogonal orientation. Based on the analysis of established functional dependencies, we proposed a simple quantitative interpretation of the obtained experimental data.

Timing Determines Tuning: A Rapid Spatial Transformation in Superior Colliculus Neurons during Reactive Gaze Shifts

Gaze saccades, rapid shifts of the eyes and head toward a goal, have provided fundamental insights into the neural control of movement. For example, it has been shown that the superior colliculus (SC) transforms a visual target (T) code to future gaze (G) location commands after a memory delay. However, this transformation has not been observed in "reactive" saccades made directly to a stimulus, so its contribution to normal gaze behavior is unclear. Here, we tested this using a quantitative measure of the intermediate codes between T and G, based on variable errors in gaze endpoints. We demonstrate that a rapid spatial transformation occurs within the primate's SC (Macaca mulatta) during reactive saccades, involving a shift in coding from T, through intermediate codes, to G. This spatial shift progressed continuously both across and within cell populations [visual, visuomotor (VM), motor], rather than relaying discretely between populations with fixed spatial codes. These results suggest that the SC produces a rapid, noisy, and distributed transformation that contributes to variable errors in reactive gaze shifts.

Comparison of the precision of smooth pursuit in humans and head unrestrained monkeys

Direct comparison of results of humans and monkeys is often complicated by differences in experimental conditions. We replicated in head unrestrained macaques experiments of a recent study comparing human directional precision during smooth pursuit eye movements (SPEM) and saccades to moving targets (Braun & Gegenfurtner, 2016). Directional precision of human SPEM follows an exponential decay function reaching optimal values of 1.5°-3° within 300 ms after target motion onset, whereas precision of initial saccades to moving targets is slightly better. As in humans, we found general agreement in the devel-opment of directional precision of SPEM over time and in the differences between direc-tional precision of initial saccades and SPEM initiation. However, monkeys showed over-all lower precision in SPEM compared to humans. This was most likely due to differences in experimental conditions, such as in the stabilization of the head, which was by a chin and a head rest in human subjects and unrestrained in monkeys.

The Influence of a Memory Delay on Spatial Coding in the Superior Colliculus: Is Visual Always Visual and Motor Always Motor?

The memory-delay saccade task is often used to separate visual and motor responses in oculomotor structures such as the superior colliculus (SC), with the assumption that these same responses would sum with a short delay during immediate "reactive" saccades to visual stimuli. However, it is also possible that additional signals (suppression, delay) alter visual and/or motor response in the memory delay task. Here, we compared the spatiotemporal properties of visual and motor responses of the same SC neurons recorded during both the reactive and memory-delay tasks in two head-unrestrained monkeys. Comparing tasks, visual (aligned with target onset) and motor (aligned on saccade onset) responses were highly correlated across neurons, but the peak response of visual neurons and peak motor responses (of both visuomotor (VM) and motor neurons) were significantly higher in the reactive task. Receptive field organization was generally similar in both tasks. Spatial coding (along a Target-Gaze (TG) continuum) was also similar, with the exception that pure motor cells showed a stronger tendency to code future gaze location in the memory delay task, suggesting a more complete transformation. These results suggest that the introduction of a trained memory delay alters both the vigor and spatial coding of SC visual and motor responses, likely due to a combination of saccade suppression signals and greater signal noise accumulation during the delay in the memory delay task.

The superior colliculus and the steering of saccades toward a moving visual target

Following the suggestion that a command encoding current target location feeds the oculomotor system during interceptive saccades, we tested the involvement of the deep superior colliculus (dSC). Extracellular activity of 52 saccade-related neurons was recorded in three monkeys while they generated saccades to targets that were static or moving along the preferred axis, away from (outward) or toward (inward) a fixated target with a constant speed (20°/s). Vertical and horizontal motions were tested when possible. Movement field (MF) parameters (boundaries, preferred vector, and firing rate) were estimated after spline fitting of the relation between the average firing rate during the motor burst and saccade amplitude. During radial target motions, the inner MF boundary shifted in the motion direction for some, but not all, neurons. Likewise, for some neurons, the lower boundaries were shifted upward during upward motions and the upper boundaries downward during downward motions. No consistent change was observed during horizontal motions. For some neurons, the preferred vectors were also shifted in the motion direction for outward, upward, and "toward the midline" target motions. The shifts of boundary and preferred vector were not correlated. The burst firing rate was consistently reduced during interceptive saccades. Our study demonstrates an involvement of dSC neurons in steering the interceptive saccade. When observed, the shifts of boundary in the direction of target motion correspond to commands related to past target locations. The absence of shift in the opposite direction implies that dSC activity does not issue predictive commands related to future target location.NEW & NOTEWORTHY The deep superior colliculus is involved in steering the saccade toward the current location of a moving target. During interceptive saccades, the active population consists of a continuum of cells ranging from neurons issuing commands related to past locations of the target to neurons issuing commands related to its current location. The motor burst of collicular neurons does not contain commands related to the future location of a moving target.