Feature integration in basic detection and localization tasks: Insights from the attentional orienting literature

Responding, fast and slow: Visual detection and localization performance is unaffected by retrieval

According to action control theories, responding to a stimulus leads to the binding of the response and stimulus features into an event file. Repeating any component of the latter retrieves previous information, affecting ongoing performance. Based on years of attentional orienting research, recent boundaries of such binding theories have been proposed as binding effects are fully absent in visual detection (e.g., Schöpper et al., 2020, Attention, Perception, & Psychophysics, 82(4), 2085-2097) and localization (e.g., Schöpper & Frings, 2022; Visual Cognition, 30(10), 641-658) performance. While this can be attributed to specific task demands, the possibility remains that retrieval of previous event files is hampered in such tasks due to overall fast responding. In the current study we instructed participants to signal the detection (Experiment 1) and location (Experiment 2) of dots orthogonally repeating or changing their nonspatial identity and location. Crucially, the dots were either hard or easy to perceive. As expected, making targets hard to perceive drastically slowed down detection and localization response speed. Importantly, binding effects were absent irrespective of perceptibility. In contrast, discriminating the nonspatial identity of targets (Experiment 3) showed strong binding effects. These results highlight the impact of task-dependence for binding approaches in action control.

Turning the Light Switch on Binding: Prefrontal Activity for Binding and Retrieval in Action Control

According to action control theories, responding to a stimulus leads to the binding of response and stimulus features into a common representation, that is, an event file. Repeating any component of an event file retrieves all previously bound information, leading to performance costs for partial repetitions measured in so-called binding effects. Although otherwise robust and stable, binding effects are typically completely absent in "localization tasks," in which participants localize targets with spatially compatible responses. Yet, it is possible to observe binding effects in such when location features have to be translated into response features. We hypothesized that this modulation of binding effects is reflected in task involvement of the dorsolateral pFC (DLPFC). Participants localized targets with either direct (i.e., spatially compatible key) or translated (i.e., diagonally opposite to the spatially compatible key) responses. We measured DLPFC activity with functional near-infrared spectroscopy. On the behavioral level, we observed binding effects in the translated response condition, but not in the direct response condition. Importantly, prefrontal activity was also higher in the translated mapping condition. In addition, we found some evidence for the strength of the difference in binding effects in behavioral data being correlated with the corresponding effects in prefrontal activity. This suggests that activity in the DLPFC reflects the amount of executive control needed for translating location features into responses. More generally, binding effects seem to emerge only when the task at hand involves DLPFC recruitment.

Bound to a spider without its web: Task-type modulates the retrieval of affective information in subsequent responses

Action control theories assume that upon responding to a stimulus response and stimulus features are integrated into a short episodic memory trace; repeating any component spurs on retrieval, affecting subsequent performance. The resulting so-called "binding effects" are reliably observed in discrimination tasks. In contrast, in localization performance, these effects are absent and only inhibition of return (IOR) emerges - a location change benefit. Affective information has been found to modulate binding effects; yet a modulation of IOR has led to mixed results, with many finding no influence at all. In the current study, participants discriminated letters (Experiment 1) or localized dots (Experiment 2) on a touchpad in prime-probe sequences. During the prime display two images - one with fruits and one with a spider - appeared, one of which spatially congruent with the to-be-touched area. In the discrimination task, previously touching a spider compared to a fruit slowed down response repetitions. In contrast, the localization task only showed IOR. This suggests that task-irrelevant valence is integrated with the response and affects subsequent responses due to retrieval. However, this is not ubiquitous but depends on task type. The results shed further light on the impact of affective information on actions.

Detection cost: A nonnegligible factor contributing to inhibition of return in the discrimination task under the cue-target paradigm

The three-factor model argues that the spatial orienting benefit triggered by the cue, the spatial selection benefit of cue-target matching, and the detection cost of distinguishing the cue from the target contribute to the measured inhibition of return (IOR) effect. According to the three-factor model, the spatial selection benefit dominates the occurrence of the IOR effect in the discrimination task, while the detection cost is negligible. The present study verified the three-factor model in the discrimination task under the cue-target paradigm by manipulating the spatial location and nonspatial feature consistency of the cue and the target as well as the promotion or hindrance of attentional disengagement from the cued location with a central reorienting cue. The results indicated that the three factors of the three-factor model contributed to the measured IOR effect in the discrimination task. Interestingly, the IOR effect was stable when the cue and target were perfectly repeated and attention was maintained at the cued location, implying that detection cost was not a negligible factor. The current study supported the contribution of all three factors in the three-factor model to the measured IOR effect; however, we argue that the role of detection cost in the discrimination task under different paradigms should be further refined.

Attentional Biases Toward Spiders Do Not Modulate Retrieval

When responding to stimuli, response and stimulus' features are thought to be integrated into a short episodic memory trace, an event file. Repeating any of its components causes retrieval of the whole event file leading to benefits for full repetitions and changes but interference for partial repetitions. These binding effects are especially pronounced if attention is allocated to certain features. We used attentional biases caused by spider stimuli, aiming to modulate the impact of attention on retrieval. Participants discriminated the orientation of bars repeating or changing their location in prime-probe sequences. Crucially, shortly before probe target onset, an image of a spider and that of a cub appeared at one position each - one of which was spatially congruent with the following probe target. Participants were faster when responding to targets spatially congruent with a preceding spider, suggesting an attentional bias toward aversive information. Yet, neither overall binding effects differed between content of preceding spatially congruent images nor did this effect emerge when taking individual fear of spiders into account. We conclude that attentional biases toward spiders modulate overall behavior, but that this has no impact on retrieval.

Same, but different: Binding effects in auditory, but not visual detection performance

Responding to a stimulus leads to the integration of response and stimulus' features into an event file. Upon repetition of any of its features, the previous event file is retrieved, thereby affecting ongoing performance. Such integration-retrieval explanations exist for a number of sequential tasks (that measure these processes as 'binding effects') and are thought to underlie all actions. However, based on attentional orienting literature, Schöpper, Hilchey, et al. (2020) could show that binding effects are absent when participants detect visual targets in a sequence: In visual detection performance, there is simply a benefit for target location changes (inhibition of return). In contrast, Mondor and Leboe (2008) had participants detect auditory targets in a sequence, and found a benefit for frequency repetition - presumably reflecting a binding effect in auditory detection performance. In the current study, we conducted two experiments, that only differed in the modality of the target: Participants signaled the detection of a sound (N = 40) or of a visual target (N = 40). Whereas visual detection performance showed a pattern incongruent with binding assumptions, auditory detection performance revealed a non-spatial feature repetition benefit, suggesting that frequency was bound to the response. Cumulative reaction time distributions indicated that the absence of a binding effect in visual detection performance was not caused by overall faster responding. The current results show a clear limitation to binding accounts in action control: Binding effects are not only limited by task demands, but can entirely depend on target modality.

Saccadic landing positions reveal that eye movements are affected by distractor-based retrieval

Binding theories assume that stimulus and response features are integrated into short-lasting episodes and that upon repetition of any feature the whole episode is retrieved, thereby affecting performance. Such binding theories are nowadays the standard explanation for a wide range of action control tasks and aim to explain all simple actions, without making assumptions of effector specificity. Yet, it is unclear if eye movements are affected by integration and retrieval in the same way as manual responses. We asked participants to discriminate letters framed by irrelevant shapes. In Experiment 1, participants gave their responses with eye movements. Saccade landing positions showed a spatial error pattern consistent with predictions of binding theories. Saccadic latencies were not affected. In Experiment 2 with an increased interval between prime and probe, the error pattern diminished, again congruent with predictions of binding theories presuming quickly decaying retrieval effects. Experiment 3 used the same task as in Experiment 1, but participants executed their responses with manual key presses; again, we found a binding pattern in response accuracy. We conclude that eye movements and manual responses are affected by the same integration and retrieval processes, supporting the tacit assumption of binding theories to apply to any effector.

Binding of Task-Irrelevant Action Features and Auditory Action Effects

Discrete task-relevant features of an overt response, such as response location, are bound to, and retrieved by coincidentally occurring auditory stimuli. Here we studied whether continuous, task-irrelevant response features like force or response duration also become bound to, and retrieved by such stimuli. In two experiments we asked participants to carry out a pinch which produced a certain auditory effect in a prime part of each trial. In a subsequent probe part, tones served as imperative stimuli which either repeated or changed as compared to the effect tone in the prime. We conjectured that the repetition of tones should result in more similar responses in terms of force output and duration as compared to tone changes. Most parameters did not show notable indications for such similarity increases, including peak force or area under force curve, though the correlation between response durations in prime and probe was higher when tones repeated rather than changed from prime to probe. We discuss these results regarding perceptual discriminability and deployment of attention to different nominally task-irrelevant aspects of pinch responses.

Expert and Novice Goalkeepers' Perceptions of Changes During Open Play Soccer

In the present study we investigated expert and novice football (i.e., soccer) goalkeepers' three stages of perceiving changes in open play situations-detection, localization, and identification-with and without time constraints. We adopted the continual cycling flicker paradigm to investigate goalkeepers' perceptions when provided with sufficient time (Experiment 1), and we utilized the limited display one-shot change detection paradigm to study their perceptions under time constraints (Experiment 2). Images of goalkeepers' first-person views of open play soccer scenes were used as stimuli. Semantic or non-semantic changes in these scenes were produced by modifying one element in each image. Separate groups of expert and novice goalkeepers were required to detect, localize, and identify the scene changes. We found that expert goalkeepers detected scene changes more quickly than novices under both time allowances. Furthermore, compared to novices, experts localized the changes more accurately under time constraints and identified the changes more quickly when given sufficient time. Additionally, semantic changes were detected more quickly and localized and identified more accurately than non-semantic changes when there was sufficient time. Under time constraints expert goalkeepers' greater efficiency was likely due to pre-attentive processing; with sufficient time, they were able to focus attention to extracting detailed information for identification.

Location-response binding and inhibition of return in a detection task

In the present study, we investigated the binding of location and response in a detection task of the target-target paradigm of inhibition of return (IOR). Results showed a cost of responding to a target at the repeated location (IOR) when the response was not repeated and an effect of facilitation of return (FOR) when the response was repeated. These findings suggest that when responding to a target, its location and the response to it are integrated together. In addition, an analysis of the Vincentized cumulative response time (RT) distribution further showed that memory retrieval of event representations requires time to operate. These findings were discussed according to the theoretical framework of event files.

Detection versus discrimination: The limits of binding accounts in action control

Actions can be investigated by using sequential priming tasks, in which participants respond to prime and probe targets (sometimes accompanied by distractors). Facilitation and interference from prime to probe are measured by repeating, changing, or partially repeating features or responses between prime and probe. According to the action control literature, feature-feature or feature-response bindings are universal and apply for all actions. The attentional orienting literature, however, suggests that if the task is to detect stimuli, such binding effects may be absent. In two experiments, we compared performance in a discrimination task and a detection task with the exact same perceptual setup of prime-probe sequences. For the discrimination task, we replicated the typical feature-response binding pattern. Crucially, we did not observe any binding effects for the detection task, which can be explained by task-specific processes or fast response execution. These results reveal an important boundary of current binding models in action control.

When do response-related episodic retrieval effects co-occur with inhibition of return?

At some point, spatial priming effects more faithfully reflect response selection processes than they do attentional orienting or sensory processes. Findings from the spatial cueing literature suggest that two factors may be critical: (1) the amount of identity processing that is required in order to respond correctly (feature-based response hypothesis), and (2) the amount of spatial processing that is required in order to respond correctly (space-based response hypothesis). To test the first hypothesis, we manipulated whether observers made single keypress detection or two-choice localization responses to serially presented stimuli in peripheral vision and whether stimulus identity information processing was necessary before responding. Responses were always slowest when the target location repeated, consistent with an attentional orienting bias independent of keypress responding (i.e., inhibition of return; IOR). The localization procedure revealed a subtle additional cost for changing the target location and repeating a response, consistent with a response-related episodic retrieval effect predicted by the Theory of Event Coding (TEC). Neither effect was modulated by the need to discriminate features. To test the second hypothesis, we made spatial processing indispensable to response selection by requiring a decision between a detection and localization response, depending on where the target appeared. IOR was eliminated for detection, but not localization, responses, consistent with the TEC. Collectively, the findings suggest that the amount of space-based, but not feature-based, processing that is required to determine a response is responsible for the response retrieval effects that can co-occur with IOR.

The Role of Congruency for Distractor-Response Binding: A Caveat

Responding in the presence of stimuli leads to an integration of stimulus features and response features into event files, which can later be retrieved to assist action control. This integration mechanism is not limited to target stimuli, but can also include distractors (distractor-response binding). A recurring research question is which factors determine whether or not distractors are integrated. One suggested candidate factor is target-distractor congruency: Distractor-response binding effects were reported to be stronger for congruent than for incongruent target-distractor pairs. Here, we discuss a general problem with including the factor of congruency in typical analyses used to study distractor-based binding effects. Integrating this factor leads to a confound that may explain any differences between distractor-response binding effects of congruent and incongruent distractors with a simple congruency effect. Simulation data confirmed this argument. We propose to interpret previous data cautiously and discuss potential avenues to circumvent this problem in the future.

Effects of Visual Scene Complexity on Neural Signatures of Spatial Attention

Spatial selective attention greatly affects our processing of complex visual scenes, yet the way in which the brain selects relevant objects while suppressing irrelevant objects is still unclear. Evidence of these processes has been found using non-invasive electroencephalography (EEG). However, few studies have characterized these measures during attention to dynamic stimuli, and little is known regarding how these measures change with increased scene complexity. Here, we compared attentional modulation of the EEG N1 and alpha power (oscillations between 8–14 Hz) across three visual selective attention tasks. The tasks differed in the number of irrelevant stimuli presented, but all required sustained attention to the orientation trajectory of a lateralized stimulus. In scenes with few irrelevant stimuli, top-down control of spatial attention is associated with strong modulation of both the N1 and alpha power across parietal-occipital channels. In scenes with many irrelevant stimuli in both hemifields, however, top-down control is no longer represented by strong modulation of alpha power, and N1 amplitudes are overall weaker. These results suggest that as a scene becomes more complex, requiring suppression in both hemifields, the neural signatures of top-down control degrade, likely reflecting some limitation in EEG to represent this suppression.

Visual working memory load does not eliminate visuomotor repetition effects

When we respond to a stimulus, our ability to quickly execute this response depends on how combinations of stimulus and response features match to previous combinations of stimulus and response features. Some kind of memory representations must be underlying these visuomotor repetition effects. In this paper, we tested the hypothesis that visual working memory stores the stimulus information that gives rise to these effects. Participants discriminated the colors of successive stimuli while holding either three locations or colors in visual working memory. If visual working memory maintains the information about a previous event that leads to visuomotor repetition effects, then occupying working memory with colors or locations should selectively disrupt color-response and location-response repetition effects. The results of two experiments showed that neither color nor spatial memory load eliminated visuomotor repetition effects. Since working memory load did not disrupt repetition effects, it is unlikely that visual working memory resources are used to store the information that underlies visuomotor repetitions effects. Instead, these results are consistent with the view that visuomotor repetition effects stem from automatic long-term memory retrieval, but can also be accommodated by supposing separate buffers for visual working memory and response selection.