Target selection for visually guided reaching in macaque

Effector-independent Representations Guide Sequential Target Selection Biases in Action

Previous work shows that automatic attention biases toward recently selected target features transfer across action and perception and even across different effectors such as the eyes and hands on a trial-by-trial basis. Although these findings suggest a common neural representation of selection history across effectors, the extent to which information about recently selected target features is encoded in overlapping versus distinct brain regions is unknown. Using fMRI and a priming of pop-out task where participants selected unpredictable, uniquely colored targets among homogeneous distractors via reach or saccade, we show that color priming is driven by shared, effector-independent underlying representations of recent selection history. Consistent with previous work, we found that the intraparietal sulcus (IPS) was commonly activated on trials where target colors were switched relative to those where the colors were repeated; however, the dorsal anterior insula exhibited effector-specific activation related to color priming. Via multivoxel cross-classification analyses, we further demonstrate that fine-grained patterns of activity in both IPS and the medial temporal lobe encode information about selection history in an effector-independent manner, such that ROI-specific models trained on activity patterns during reach selection could predict whether a color was repeated or switched on the current trial during saccade selection and vice versa. Remarkably, model generalization performance in IPS and medial temporal lobe also tracked individual differences in behavioral priming sensitivity across both types of action. These results represent a first step to clarify the neural substrates of experience-driven selection biases in contexts that require the coordination of multiple actions.

Continuous action with a neurobiologically inspired computational approach reveals the dynamics of selection history

Everyday perception-action interaction often requires selection of a single goal from multiple possibilities. According to a recent framework of attentional control, object selection is guided not only by the well-established factors of perceptual salience and current goals but also by selection history. Yet, underlying mechanisms linking selection history and visually-guided actions are poorly understood. To examine such interplay and disentangle the impact of target and distractor history on action selection, we employed a priming-of-popout (PoP) paradigm combined with continuous tracking of reaching movements and computational modeling. Participants reached an odd-colored target among homogeneous distractors while we systematically manipulated the sequence of target and distractor colors from one trial to the next. We observed that current reach movements were significantly influenced by the interaction between attraction by the prior target feature and repulsion by the prior distractor feature. With principal component regression, we found that inhibition led by prior distractors influenced reach target selection earlier than facilitation led by the prior target. In parallel, our newly developed computational model validated that current reach target selection can be explained best by the mechanism postulating the preceded impact of previous distractors followed by a previous target. Such converging empirical and computational evidence suggests that the prior selection history triggers a dynamic interplay between target facilitation and distractor inhibition to guide goal-directed action successfully. This, in turn, highlights the necessity of an explicitly integrated approach to determine how visual attentional selection links with adaptive actions in a complex environment.

Rethinking Vision and Action

Action is an important arbitrator as to whether an individual or a species will survive. Yet, action has not been well integrated into the study of psychology. Action or motor behavior is a field apart. This is traditional science with its need for specialization. The sequence in a typical laboratory experiment of see → decide → act provides the rationale for broad disciplinary categorizations. With renewed interest in action itself, surprising and exciting anomalous findings at odds with this simplified caricature have emerged. They reveal a much more intimate coupling of vision and action, which we describe. In turn, this prompts us to identify and dwell on three pertinent theories deserving of greater notice.

Active vision during prey capture in wild marmoset monkeys

A foundational pressure in the evolution of all animals is the ability to travel through the world, inherently coupling the sensory and motor systems. While this relationship has been explored in several species,1-4 it has been largely overlooked in primates, which have typically relied on paradigms in which head-restrained subjects view stimuli on screens.5 Natural visual behaviors, by contrast, are typified by locomotion through the environment guided by active sensing as animals explore and interact with the world,4,6 a relationship well illustrated by prey capture.7-12 Here, we characterized prey capture in wild marmoset monkeys as they negotiated their dynamic, arboreal habitat to illustrate the inherent role of vision as an active process in natural nonhuman primate behavior. Not only do marmosets share the core properties of vision that typify the primate Order,13-18 but they are prolific hunters that prey on a diverse set of prey animals.19-22 Marmosets pursued prey using vision in several different contexts, but executed precise visually guided motor control that predominantly involved grasping with hands for successful capture of prey. Applying markerless tracking for the first time in wild primates yielded novel findings that precisely quantified how marmosets track insects prior to initiating an attack and the rapid visually guided corrections of the hands during capture. These findings offer the first detailed insight into the active nature of vision to guide multiple facets of a natural goal-directed behavior in wild primates and can inform future laboratory studies of natural primate visual behaviors and the supporting neural processes.

Target detection and discrimination in pop-out visual search with two targets

To successfully interact with objects in complex and crowded environments, we often perform visual search to detect or identify a relevant target (or targets) among distractors. Previous studies have reported a redundancy gain when two targets instead of one are presented in a simple target detection task. However, research is scant about the role of multiple targets in target discrimination tasks, especially in the context of visual search. Here, we address this question and investigate its underlying mechanisms in a pop-out search paradigm. In Experiment 1, we directly compared visual search performance for one or two targets for detection or discrimination tasks. We found that two targets led to a redundancy gain for detection, whereas it led to a redundancy cost for discrimination. To understand the basis for the redundancy cost observed in discrimination tasks for multiple targets, we further investigated the role of perceptual grouping (Experiment 2) and stimulus-response feature compatibility (Experiment 3). We determined that the strength of perceptual grouping among homogenous distractors was attenuated when two targets were present compared with one. We also found that response compatibility between two targets contributed more to the redundancy cost compared with perceptual compatibility. Taken together, our results show how pop-out search involving two targets is modulated by the level of feature processing, perceptual grouping, and compatibility of perceptual and response features.

A comparison of simple movement behaviors across three different devices

Reaching trajectories have provided a unique tool to observe changes in internal cognitive decisions. Furthermore, technological advances have made devices for measuring reach movements more accessible and researchers have recognized that various populations including children, elderly populations, and non-human primates can easily execute simple movements as responses. As a result, devices such as a three-dimensional (3D) reach tracker, a stylus, or a computer-mouse have been increasingly utilized to study cognitive processes. However, although the specific type of tracking device that a researcher uses may impact behavior due to the constraints it places on movements, most researchers in these fields are unaware of this potential issue. Here, we examined the potential behavioral impact of using each of these three devices. To induce re-directed movements that mimic the movements that often occur following changes in cognitive states, we used a double-step task in which displacement of an initial target location requires participants to quickly re-direct their movement. We found that reach movement parameters were largely comparable across the three devices. However, hand movements measured by a 3D reach tracker showed earlier reach initiation latencies (relative to stylus movements) and more curved movement trajectories (relative to both mouse and stylus movements). Reach movements were also re-directed following target displacement more rapidly. Thus, 3D reach trackers may be ideal for observing fast, subtle changes in internal decision-making processes compared to other devices. Taken together, this study provides a useful reference for comparing and implementing reaching studies to examine human cognition.

Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding

In a novel computer mouse tracking paradigm, participants read a spatial phrase such as "The blue item to the left of the red one" and then see a scene composed of 12 visual items. The task is to move the mouse cursor to the target item (here, blue), which requires perceptually grounding the spatial phrase. This entails visually identifying the reference item (here, red) and other relevant items through attentional selection. Response trajectories are attracted toward distractors that share the target color but match the spatial relation less well. Trajectories are also attracted toward items that share the reference color. A competing pair of items that match the specified colors but are in the inverse spatial relation increases attraction over-additively compared to individual items. Trajectories are also influenced by the spatial term itself. While the distractor effect resembles deviation toward potential targets in previous studies, the reference effect suggests that the relevance of the reference item for the relational task, not its role as a potential target, was critical. This account is supported by the strengthened effect of a competing pair. We conclude, therefore, that the attraction effects in the mouse trajectories reflect the neural processes that operate on sensorimotor representations to solve the relational task. The paradigm thus provides an experimental window through motor behavior into higher cognitive function and the evolution of activation in modal substrates, a longstanding topic in the area of embodied cognition.

Selective Influence and Sequential Operations: A Research Strategy for Visual Search

We discuss the problem of elucidating mechanisms of visual search. We begin by considering the history, logic, and methods of relating behavioral or cognitive processes with neural processes. We then survey briefly the cognitive neurophysiology of visual search and essential aspects of the neural circuitry supporting this capacity. We introduce conceptually and empirically a powerful but underutilized experimental approach to dissect the cognitive processes supporting performance of a visual search task with factorial manipulations of singleton-distractor identifiability and stimulus-response cue discriminability. We show that systems factorial technology can distinguish processing architectures from the performance of macaque monkeys. This demonstration offers new opportunities to distinguish neural mechanisms through selective manipulation of visual encoding, search selection, rule encoding, and stimulus-response mapping.

Abandoning and modifying one action plan for alternatives

Visual scenes are often complex and crowded with many different objects. To interact effectively, we must choose one object at a time as a goal for action. Certain external cues can act as a stop signal, quickly cancelling an ongoing action. Less recognized are internal signals. These can come from recent experience, anticipated action outcomes, cognitive states, and when attention is captured by a salient object. These signals elevate one action plan over alternatives and can quickly modify an initial choice. Here, we focus on these internal processes responsible for selecting, abandoning and modifying action plans. We first highlight how the brain resolves competition among multiple action plans. Critical is the existence of parallel motor planning processes, which allow efficient and timely changes. Then, we discuss how the action system interplays with perception, attention and memory processes to bias action selection and suppress or modify erroneous selections. Subsequently, we show how tracking the continuous modification of action trajectories can provide a tool to read out changes in internal cognitive states. Taken together, we shed light on a broader view that sensorimotor networks can continuously modify actions through simultaneous evaluation of alternative activities in concert with widely distributed perceptual and cognitive networks.This article is part of the themed issue 'Movement suppression: brain mechanisms for stopping and stillness'.

Global attention facilitates the planning, but not execution of goal-directed reaches

In daily life, humans interact with multiple objects in complex environments. A large body of literature demonstrates that target selection is biased toward recently attended features, such that reaches are faster and trajectory curvature is reduced when target features (i.e., color) are repeated (priming of pop-out). In the real world, however, objects are comprised of several features—some of which may be more suitable for action than others. When fetching a mug from the cupboard, for example, attention not only has to be allocated to the object, but also the handle. To date, no study has investigated the impact of hierarchical feature organization on target selection for action. Here, we employed a color-oddity search task in which targets were Pac-men (i.e., a circle with a triangle cut out) oriented to be either consistent or inconsistent with the percept of a global Kanizsa triangle. We found that reaches were initiated faster when a task-irrelevant illusory figure was present independent of color repetition. Additionally, consistent with priming of pop-out, both reach planning and execution were facilitated when local target colors were repeated, regardless of whether a global figure was present. We also demonstrated that figures defined by illusory, but not real contours, afforded an early target selection benefit. In sum, these findings suggest that when local targets are perceptually grouped to form an illusory surface, attention quickly spreads across the global figure and facilitates the early stage of reach planning, but not execution. In contrast, local color priming is evident throughout goal-directed reaching.

Target selection biases from recent experience transfer across effectors

Target selection is often biased by an observer's recent experiences. However, not much is known about whether these selection biases influence behavior across different effectors. For example, does looking at a red object make it easier to subsequently reach towards another red object? In the current study, we asked observers to find the uniquely colored target object on each trial. Randomly intermixed pre-trial cues indicated the mode of action: either an eye movement or a visually guided reach movement to the target. In Experiment 1, we found that priming of popout, reflected in faster responses following repetition of the target color on consecutive trials, occurred regardless of whether the effector was repeated from the previous trial or not. In Experiment 2, we examined whether an inhibitory selection bias away from a feature could transfer across effectors. While priming of popout reflects both enhancement of the repeated target features and suppression of the repeated distractor features, the distractor previewing effect isolates a purely inhibitory component of target selection in which a previewed color is presented in a homogenous display and subsequently inhibited. Much like priming of popout, intertrial suppression biases in the distractor previewing effect transferred across effectors. Together, these results suggest that biases for target selection driven by recent trial history transfer across effectors. This indicates that representations in memory that bias attention towards or away from specific features are largely independent from their associated actions.

Dorsal premotor cortex: neural correlates of reach target decisions based on a color-location matching rule and conflicting sensory evidence

We recorded single-neuron activity in dorsal premotor (PMd) and primary motor cortex (M1) of two monkeys in a reach-target selection task. The monkeys chose between two color-coded potential targets by determining which target's color matched the predominant color of a multicolored checkerboard-like Decision Cue (DC). Different DCs contained differing numbers of colored squares matching each target. The DCs provided evidence about the correct target ranging from unambiguous (one color only) to very ambiguous and conflicting (nearly equal number of squares of each color). Differences in choice behavior (reach response times and success rates as a function of DC ambiguity) of the monkeys suggested that each applied a different strategy for using the target-choice evidence in the DCs. Nevertheless, the appearance of the DCs evoked a transient coactivation of PMd neurons preferring both potential targets in both monkeys. Reach response time depended both on how long it took activity to increase in neurons that preferred the chosen target and on how long it took to suppress the activity of neurons that preferred the rejected target, in both correct-choice and error-choice trials. These results indicate that PMd neurons in this task are not activated exclusively by a signal proportional to the net color bias of the DCs. They are instead initially modulated by the conflicting evidence supporting both response choices; final target selection may result from a competition between representations of the alternative choices. The results also indicate a temporal overlap between action selection and action initiation processes in PMd and M1.

Neural correlates of target selection for reaching movements in superior colliculus

We recently demonstrated that inactivation of the primate superior colliculus (SC) causes a deficit in target selection for arm-reaching movements when the reach target is located in the inactivated field (Song JH, Rafal RD, McPeek RM. Proc Natl Acad Sci USA 108: E1433-E1440, 2011). This is consistent with the notion that the SC is part of a general-purpose target selection network beyond eye movements. To understand better the role of SC activity in reach target selection, we examined how individual SC neurons in the intermediate layers discriminate a reach target from distractors. Monkeys reached to touch a color oddball target among distractors while maintaining fixation. We found that many SC neurons robustly discriminate the goal of the reaching movement before the onset of the reach even though no saccade is made. To identify these cells in the context of conventional SC cell classification schemes, we also recorded visual, delay-period, and saccade-related responses in a delayed saccade task. On average, SC cells that discriminated the reach target from distractors showed significantly higher visual and delay-period activity than nondiscriminating cells, but there was no significant difference in saccade-related activity. Whereas a majority of SC neurons that discriminated the reach target showed significant delay-period activity, all nondiscriminating cells lacked such activity. We also found that some cells without delay-period activity did discriminate the reach target from distractors. We conclude that the majority of intermediate-layer SC cells discriminate a reach target from distractors, consistent with the idea that the SC contains a priority map used for effector-independent target selection.

Reach target selection in humans using ambiguous decision cues containing variable amounts of conflicting sensory evidence supporting each target choice

Human subjects chose between two color-coded reach targets using multicolored checkerboard-like decision cues (DCs) that presented variable amounts of conflicting sensory evidence supporting both target choices. Different DCs contained different numbers of small squares of the two target colors. The most ambiguous DCs contained nearly equal numbers of squares of both target colors. The subjects reached as soon as they selected a target after the appearance of the DC (“choose-and-go” task). The choice behavior of the subjects showed many similarities to prior studies using other stimulus properties (e.g., visual motion coherence, brightness), including progressively longer response times and higher target-choice error rates for more ambiguous DCs. However, certain trends in their choice behavior could not be fully captured by simple drift-diffusion models. Allowing the subjects to view the DCs for a period of time before presenting the targets (“match-to-sample” task) resulted in much shorter response times overall, but also revealed a reluctance of subjects to commit to a decision about the predominant color of the more ambiguous DCs during the initial extended observation period. Model processing and simulation analyses suggest that the subjects might adjust the dynamics of their decision-making process on a trial-to-trial basis in response to the variable level of ambiguous and conflicting evidence in different DCs between trials.

Spatial and temporal eye-hand coordination relies on the parietal reach region

Coordinated eye movements are crucial for precision control of our hands. A commonly believed neural mechanism underlying eye-hand coordination is interaction between the neural networks controlling each effector, exchanging, and matching information, such as movement target location and onset time. Alternatively, eye-hand coordination may result simply from common inputs to independent eye and hand control pathways. Thus far, it remains unknown whether and where either of these two possible mechanisms exists. A candidate location for the former mechanism, interpathway communication, includes the posterior parietal cortex (PPC) where distinct effector-specific areas reside. If the PPC were within the network for eye-hand coordination, perturbing it would affect both eye and hand movements that are concurrently planned. In contrast, if eye-hand coordination arises solely from common inputs, perturbing one effector pathway, e.g., the parietal reach region (PRR), would not affect the other effector. To test these hypotheses, we inactivated part of PRR in the macaque, located in the medial bank of the intraparietal sulcus encompassing the medial intraparietal area and area 5V. When each effector moved alone, PRR inactivation shortened reach but not saccade amplitudes, compatible with the known reach-selective activity of PRR. However, when both effectors moved concurrently, PRR inactivation shortened both reach and saccade amplitudes, and decoupled their reaction times. Therefore, consistent with the interpathway communication hypothesis, we propose that the planning of concurrent eye and hand movements causes the spatial information in PRR to influence the otherwise independent eye control pathways, and that their temporal coupling requires an intact PRR.

Three-dimensional reach trajectories as a probe of real-time decision-making between multiple competing targets

Though several features of cognitive processing can be inferred from the discrete measurement [e.g., reaction time (RT), accuracy, etc.] of participants' conscious reports (e.g., verbal or key-press responses), it is becoming increasingly clear that a much richer understanding of these features can be captured from continuous measures of rapid, largely non-conscious behaviors like hand or eye movements. Here, using new experimental data, we describe in detail both the approach and analyses implemented in some of our previous studies that have used rapid reaching movements under cases of target uncertainty in order to probe the features, constraints and dynamics of stimulus-related processing in the brain. This work, as well as that of others, shows that when individuals are simultaneously presented with multiple potential targets—only one of which will be cued after reach onset—they produce initial reach trajectories that are spatially biased in accordance with the probabilistic distribution of targets. Such “spatial averaging” effects are consistent with observations from neurophysiological studies showing that neuronal populations in sensorimotor brain structures represent multiple target choices in parallel and they compete for selection. These effects also confirm and help extend computational models aimed at understanding the underlying mechanisms that support action-target selection. We suggest that the use of this simple, yet powerful behavioral paradigm for providing a “real-time” visualization of ongoing cognitive processes occurring at the neural level offers great promise for studying processes related to a wide range of psychological phenomena, such as decision-making and the representation of objects.

Statistical extraction affects visually guided action

The visual system summarizes average properties of ensembles of similar objects. We demonstrated an adaptation aftereffect of one such property, mean size, suggesting it is encoded along a single visual dimension (Corbett, et al., 2012), in a similar manner as basic stimulus properties like orientation and direction of motion. To further explore the fundamental nature of ensemble encoding, here we mapped the evolution of mean size adaptation over the course of visually guided grasping. Participants adapted to two sets of dots with different mean sizes. After adaptation, two test dots replaced the adapting sets. Participants first reached to one of these dots, and then judged whether it was larger or smaller than the opposite dot. Grip apertures were inversely dependent on the average dot size of the preceding adapting patch during the early phase of movements, and this aftereffect dissipated as reaches neared the target. Interestingly, perceptual judgments still showed a marked aftereffect, even though they were made after grasping was completed more-or-less veridically. This effect of mean size adaptation on early visually guided kinematics provides novel evidence that mean size is encoded fundamentally in both perception and action domains, and suggests that ensemble statistics not only influence our perceptions of individual objects but can also affect our physical interactions with the external environment.

The distribution of spatial attention changes with task demands during goal-directed reaching

Goal-directed movements are commonly used to allow humans to interact with their environment. When making a goal-directed movement in a natural environment, there are many competing stimuli. It is therefore important to understand how making a goal-directed movement could be impacted by the need to divide attention between the movement and competing stimuli. We used a dual-task paradigm to investigate the sharing of attentional resources between a search task in central vision and a peripheral pointing task completed concurrently. Results suggest some degree of shared attentional resources between these two tasks with performance on both central and peripheral tasks degraded under dual-task conditions. Movement latency, but not movement time, was also affected by dual-task conditions. Altogether, the results suggest that there is a cost to reach performance if attention is engaged away from the movement goal. Interestingly, this cost is associated with movement planning rather than execution.

Allocation of attention for dissociated visual and motor goals

In daily life, selecting an object visually is closely intertwined with processing that object as a potential goal for action. Since visual and motor goals are typically identical, it remains unknown whether attention is primarily allocated to a visual target, a motor goal, or both. Here, we dissociated visual and motor goals using a visuomotor adaptation paradigm, in which participants reached toward a visual target using a computer mouse or a stylus pen, while the direction of the cursor was rotated 45° counter-clockwise from the direction of the hand movement. Thus, as visuomotor adaptation was accomplished, the visual target was dissociated from the movement goal. Then, we measured the locus of attention using an attention-demanding rapid serial visual presentation (RSVP) task, in which participants detected a pre-defined visual stimulus among the successive visual stimuli presented on either the visual target, the motor goal, or a neutral control location. We demonstrated that before visuomotor adaptation, participants performed better when the RSVP stream was presented at the visual target than at other locations. However, once visual and motor goals were dissociated following visuomotor adaptation, performance at the visual and motor goals was equated and better than performance at the control location. Therefore, we concluded that attentional resources are allocated both to visual target and motor goals during goal-directed reaching movements.

Electromagnetic tracking of motion in the proximity of computer generated graphical stimuli: a tutorial

Electromagnetic motion-tracking systems have the advantage of capturing the tempo-spatial kinematics of movements independently of the visibility of the sensors. However, they are limited in that they cannot be used in the proximity of electromagnetic field sources, such as computer monitors. This prevents exploiting the tracking potential of the sensor system together with that of computer-generated visual stimulation. Here we present a solution for presenting computer-generated visual stimulation that does not distort the electromagnetic field required for precise motion tracking, by means of a back projection medium. In one experiment, we verify that cathode ray tube monitors, as well as thin-film-transistor monitors, distort electro-magnetic sensor signals even at a distance of 18 cm. Our back projection medium, by contrast, leads to no distortion of the motion-tracking signals even when the sensor is touching the medium. This novel solution permits combining the advantages of electromagnetic motion tracking with computer-generated visual stimulation.

Roles of narrow- and broad-spiking dorsal premotor area neurons in reach target selection and movement production

Most visual scenes are complex and crowded, with several different objects competing for attention and action. Thus a complete understanding of the production of goal-directed actions must incorporate the higher-level process of target selection. To examine the neural substrates of target selection for visually guided reaching, we recorded the activity of isolated neurons in the dorsal premotor area (PMd) of monkeys performing a reaction-time visual search task. In this task, monkeys reached to an odd-colored target presented with three distractors. We found that PMd neurons typically discriminate the target before movement onset, ∼150-200 ms after the appearance of the search array. In one subset of neurons, discrimination occurred at a consistent time after search array onset regardless of when the reaching movement occurred, suggesting that these neurons are involved in target selection. In a second group of neurons, discrimination time depended on reach reaction time, consistent with involvement in movement production but not in target selection. To look for physiological corroboration of these two functionally defined groups, we analyzed the extracellular spike waveforms of recorded neurons. This analysis showed a population of neurons with narrow action potentials that carried signals related to target selection. A second population with broader action potentials was more heterogeneous, with some neurons showing activity related to target selection and others showing only movement production activity. These results suggest that PMd contains signals related to target selection and movement execution and that different signals are carried by distinct neural subpopulations.

The development of the spatial extent of oculomotor inhibition

Inhibition is intimately involved in the ability to select a target for a goal-directed movement. The effect of distracters on the deviation of oculomotor trajectories and landing positions provides evidence of such inhibition. Individual saccade trajectories and landing positions may deviate initially either towards, or away from, a competing distracter--the direction and extent of this deviation depends upon saccade latency and the target to distracter separation. However, the underlying commonality of the sources of oculomotor inhibition has not been investigated. Here we report the relationship between distracter-related deviation of saccade trajectory, landing position and saccade latency. Observers saccaded to a target which could be accompanied by a distracter shown at various distances from very close (10 angular degrees) to far away (120 angular degrees). A fixation-gap paradigm was used to manipulate latency independently of the influence of competing distracters. When distracters were close to the target, saccade trajectory and landing position deviated toward the distracter position, while at greater separations landing position was always accurate but trajectories deviated away from the distracters. Different spatial patterns of deviations across latency were found. This pattern of results is consistent with the metrics of the saccade reflecting coarse pooling of the ongoing activity at the distracter location: saccade trajectory reflects activity at saccade initiation while landing position reveals activity at saccade end.

Eye-hand coordination during target selection in a pop-out visual search

We examined the coordination of saccades and reaches in a visual search task in which monkeys were rewarded for reaching to an odd-colored target among distractors. Eye movements were unconstrained, and monkeys typically made one or more saccades before initiating a reach. Target selection for reaching and saccades was highly correlated with the hand and eyes landing near the same final stimulus both for correct reaches to the target and for incorrect reaches to a distractor. Incorrect reaches showed a bias in target selection: they were directed to the distractor in the same hemifield as the target more often than to other distractors. A similar bias was seen in target selection for the initial saccade in correct reaching trials with multiple saccades. We also examined the temporal coupling of saccades and reaches. In trials with a single saccade, a reaching movement was made after a fairly stereotyped delay. In multiple-saccade trials, a reach to the target could be initiated near or even before the onset of the final target-directed saccade. In these trials, the initial trajectory of the reach was often directed toward the fixated distractor before veering toward the target around the time of the final saccade. In virtually all cases, the eyes arrived at the target before the hand, and remained fixated until reach completion. Overall, these results are consistent with flexible temporal coupling of saccade and reach initiation, but fairly tight coupling of target selection for the two types of action.

Slant cue are combined early in visual processing: evidence from visual search

When looking for a target with a different slant than all the other objects, the time needed is independent of the number of other objects. Surface slant can be inferred from the two-dimensional images on the retinas using various cues. The information from different cues is subsequently combined to get a single estimate of slant. Is information from the individual cues or from the combined percept responsible for us so easily finding the target? To find out we compared combinations of two slant cues. The cues that we chose are retinal shape and binocular disparity. We compared search times for conditions with the same differences between the target and the other objects in each individual cue, but for each object the two cues either indicated the same slant or opposite slants. Search times were independent of the number of other items if the target clearly differed in perceived slant from the other items. Subjects systematically found the target faster when the cues indicated the same slant. We conclude that slant cues are combined locally throughout the visual field before the search process begins.

Reversal of a distractor effect on saccade target selection after superior colliculus inactivation

Recent evidence indicates that inactivation of the primate superior colliculus (SC) results in an increase in saccade target-selection errors. The pattern of errors suggests that a winner-take-all competition selects the saccade goal and that SC inactivation perturbs this process by biasing the competition against stimuli in the inactivated field. To investigate this idea, the difficulty of target selection was manipulated in a color-oddity search task by varying the number of homogeneous distractors in the search array. Previous studies have shown that target selection is easier when a greater number of homogeneous distractors is present, due to perceptual grouping of the distractors. These results were replicated when testing with the SC intact. Surprisingly, during SC inactivation, this normal trend was reversed: target-selection performance declined significantly with more distractors, resulting in a greater proportion of errant saccades to distractors. Examination of the saccade endpoints indicates that after SC inactivation, many errant saccades were directed to distractors adjacent to the target. This pattern of results suggests that the salience signal used by the SC for target selection is relatively broad in spatial scope. As a result, when the area of the SC representing the target location is inactivated, distractors near the target are at a competitive advantage relative to more distant distractors and, consequently, are selected more often as the saccade goal. This contributes to the trend of worse performance with more distractors due to the greater proximity of distractors to the target.

Target selection in visual search as revealed by movement trajectories

We examined target selection for visually guided reaching movements in visual search, in which participants reached to an odd-colored target presented with two homogenous distractors. The colors of the target and distractors were randomly switched for each trial between red and green, and the location of the target was varied. Therefore either color could be a distractor or target, and the identity was resolved by grouping two distractors having the same color. Thus, there was ongoing competition between a target and distractors. In some trials, reaches were directed to the target, and in other trials, reaches were initially directed towards a distractor and corrected in mid-flight, showing highly curved trajectories. Interestingly, trials with highly curved trajectories were no less efficient in terms of accuracy or total time. The extra time taken up in movement duration was offset by shorter initial latencies. By analyzing curved trajectories, we demonstrated that corrective movements occur shortly after the onset of initial movement, suggesting that a corrective new target is selected even before initial movement is executed. This provides an explanation as to why misdirected reaches, hastily initiated, can be corrected with minimal loss in overall efficiency. In addition, our results show that the details of movement trajectories allow us to visualize the dynamics of target selection as they unfold in time.