Oculomotor interference during manual response preparation: evidence from the response-cueing paradigm

A Gestalt account of human behavior is supported by evidence from switching between single and dual actions

The question of how behavior is represented in the mind lies at the core of psychology as the science of mind and behavior. While a long-standing research tradition has established two opposing fundamental views of perceptual representation, Structuralism and Gestalt psychology, we test both accounts with respect to action representation: Are multiple actions (characterizing human behavior in general) represented as the sum of their component actions (Structuralist view) or holistically (Gestalt view)? Using a single-/dual-response switch paradigm, we analyzed switches between dual ([A + B]) and single ([A], [B]) responses across different effector systems and revealed comparable performance in partial repetitions and full switches of behavioral requirements (e.g., in [A + B] → [A] vs. [B] → [A], or [A] → [A + B] vs. [B] → [A + B]), but only when the presence of dimensional overlap between responses allows for Gestalt formation. This evidence for a Gestalt view of behavior in our paradigm challenges some fundamental assumptions in current (tacitly Structuralist) action control theories (in particular the idea that all actions are represented compositionally with reference to their components), provides a novel explanatory angle for understanding complex, highly synchronized human behavior (e.g., dance), and delimitates the degree to which complex behavior can be analyzed in terms of its basic components.

Keeping it real: Looking beyond capacity limits in visual cognition

Research within visual cognition has made tremendous strides in uncovering the basic operating characteristics of the visual system by reducing the complexity of natural vision to artificial but well-controlled experimental tasks and stimuli. This reductionist approach has for example been used to assess the basic limitations of visual attention, visual working memory (VWM) capacity, and the fidelity of visual long-term memory (VLTM). The assessment of these limits is usually made in a pure sense, irrespective of goals, actions, and priors. While it is important to map out the bottlenecks our visual system faces, we focus here on selected examples of how such limitations can be overcome. Recent findings suggest that during more natural tasks, capacity may be higher than reductionist research suggests and that separable systems subserve different actions, such as reaching and looking, which might provide important insights about how pure attentional or memory limitations could be circumvented. We also review evidence suggesting that the closer we get to naturalistic behavior, the more we encounter implicit learning mechanisms that operate "for free" and "on the fly." These mechanisms provide a surprisingly rich visual experience, which can support capacity-limited systems. We speculate whether natural tasks may yield different estimates of the limitations of VWM, VLTM, and attention, and propose that capacity measurements should also pass the real-world test within naturalistic frameworks. Our review highlights various approaches for this and suggests that our understanding of visual cognition will benefit from incorporating the complexities of real-world cognition in experimental approaches.

Two sources of task prioritization: The interplay of effector-based and task order-based capacity allocation in the PRP paradigm

When processing of two tasks overlaps, performance is known to suffer. In the well-established psychological refractory period (PRP) paradigm, tasks are triggered by two stimuli with a short temporal delay (stimulus onset asynchrony; SOA), thereby allowing control of the degree of task overlap. A decrease of the SOA reliably yields longer RTs of the task associated with the second stimulus (Task 2) while performance in the other task (Task 1) remains largely unaffected. This Task 2-specific SOA effect is usually interpreted in terms of central capacity limitations. Particularly, it has been assumed that response selection in Task 2 is delayed due to the allocation of less capacity until this process has been completed in Task 1. Recently, another important factor determining task prioritization has been proposed—namely, the particular effector systems associated with tasks. Here, we study both sources of task prioritization simultaneously by systematically combining three different effector systems (pairwise combinations of oculomotor, vocal, and manual responses) in the PRP paradigm. Specifically, we asked whether task order-based task prioritization (SOA effect) is modulated as a function of Task 2 effector system. The results indicate a modulation of SOA effects when the same (oculomotor) Task 1 is combined with a vocal versus a manual Task 2. This is incompatible with the assumption that SOA effects are solely determined by Task 1 response selection duration. Instead, they support the view that dual-task processing bottlenecks are resolved by establishing a capacity allocation scheme fed by multiple input factors, including attentional weights associated with particular effector systems.

Effects of Input Modality on Vocal Effector Prioritization in Manual-Vocal Dual Tasks

Doing two things at once (vs. one in isolation) usually yields performance costs. Such decrements are often distributed asymmetrically between the two actions involved, reflecting different processing priorities. A previous study (Huestegge & Koch, 2013) demonstrated that the particular effector systems associated with the two actions can determine the pattern of processing priorities: Vocal responses were prioritized over manual responses, as indicated by smaller performance costs (associated with dual-action demands) for the former. However, this previous study only involved auditory stimulation (for both actions). Given that previous research on input-output modality compatibility in dual tasks suggested that pairing auditory input with vocal output represents a particularly advantageous mapping, the question arises whether the observed vocal-over-manual prioritization was merely a consequence of auditory stimulation. To resolve this issue, we conducted a manual-vocal dual task study using either only auditory or only visual stimuli for both responses. We observed vocal-over-manual prioritization in both stimulus modality conditions. This suggests that input-output modality mappings can (to some extent) attenuate, but not abolish/reverse effector-based prioritization. Taken together, effector system pairings appear to have a more substantial impact on capacity allocation policies in dual-task control than input-output modality combinations.

Interaction of oculomotor and manual behavior: evidence from simulated driving in an approach-avoidance steering task

BACKGROUND

While the coordination of oculomotor and manual behavior is essential for driving a car, surprisingly little is known about this interaction, especially in situations requiring a quick steering reaction. In the present study, we analyzed oculomotor gaze and manual steering behavior in approach and avoidance tasks. Three task blocks were implemented within a dynamic simulated driving environment requiring the driver either to steer away from/toward a visual stimulus or to switch between both tasks.

RESULTS

Task blocks requiring task switches were associated with higher manual response times and increased error rates. Manual response times did not significantly differ depending on whether drivers had to steer away from vs toward a stimulus, whereas oculomotor response times and gaze pattern variability were increased when drivers had to steer away from a stimulus compared to steering toward a stimulus.

CONCLUSION

The increased manual response times and error rates in mixed tasks indicate performance costs associated with cognitive flexibility, while the increased oculomotor response times and gaze pattern variability indicate a parsimonious cross-modal action control strategy (avoiding stimulus fixation prior to steering away from it) for the avoidance scenario. Several discrepancies between these results and typical eye-hand interaction patterns in basic laboratory research suggest that the specific goals and complex perceptual affordances associated with driving a vehicle strongly shape cross-modal control of behavior.

Free-choice saccades and their underlying determinants: Explorations of high-level voluntary oculomotor control

Models of eye-movement control distinguish between different control levels, ranging from automatic (bottom-up, stimulus-driven selection) and automatized (based on well-learned routines) to voluntary (top-down, goal-driven selection, e.g., based on instructions). However, one type of voluntary control has yet only been examined in the manual and not in the oculomotor domain, namely free-choice selection among arbitrary targets, that is, targets that are of equal interest from both a bottom-up and top-down processing perspective. Here, we ask which features of targets (identity- or location-related) are used to determine such oculomotor free-choice behavior. In two experiments, participants executed a saccade to one of four peripheral targets in three different choice conditions: unconstrained free choice, constrained free choice based on target identity (color), and constrained free choice based on target location. The analysis of choice frequencies revealed that unconstrained free-choice selection closely resembled constrained choice based on target location. The results suggest that free-choice oculomotor control is mainly guided by spatial (location-based) target characteristics. We explain these results by assuming that participants tend to avoid less parsimonious recoding of target-identity representations into spatial codes, the latter being a necessary prerequisite to configure oculomotor commands.

Evidence for distinct steps in response preparation from a delayed response paradigm

Task parameters still affect reaction times even when all necessary information for executing an action is presented prior to a Go signal to execute the action. Hypotheses in terms of short-term memory capacity, residual activation, and a separate motor-programming stage have been suggested to explain what can and cannot be prepared prior to a delayed Go signal. To test these hypotheses, we used a delayed response task, in which participants were to initiate a movement at onset of an imperative Go signal following the target stimulus. Across Experiments 1-3 we varied task properties including stimulus type, information uncertainty and response complexity, respectively, while controlling other factors. We also varied the time available to process the response by randomly varying the interval between onset of the target and the Go signal (i.e., the stimulus onset asynchrony, or SOA). If the preparation process is completed before initiation, the examined factor should display a strong interaction with SOA, with its effect disappearing at long SOAs. Our results showed strong, weaker, and no interaction patterns for the three factors, respectively, favoring the separate stage hypothesis, according to which response preparation is separated into steps to arrange kinematic specifications into muscle-controllable terms.

Backward crosstalk and the role of dimensional overlap within and between tasks

In dual-task situations, which often involve some form of sequential task processing, features of Task 2 were shown to affect Task 1 performance, a phenomenon termed "backward crosstalk effect" (BCE). Most previous reports of BCEs are based on manipulations of code compatibility between tasks, while there is no clear picture whether and how mere Task 2 response selection difficulty (in the absence of cross-task dimensional code overlap, including effector system overlap) may also affect Task 1 performance. In the present study, we systematically manipulated response-response (R1-R2) relation (compatible, incompatible, arbitrary) and the stimulus-response (S-R) relation in Task 2 (S2-R2: compatible, incompatible, arbitrary; i.e., a classic manipulation of Task 2 response selection difficulty) to study the impact of dimensional overlap and compatibility within and across tasks using an integrated stimulus for both a vocal Task 1 and a manual Task 2. Results revealed a replication of a classic (spatial) R1-R2 compatibility BCE (based on code compatibility), demonstrating that our paradigm is principally suited to capture typical BCEs. Importantly, conditions involving a removal of dimensional code overlap between tasks still yielded an effect of mere Task 2 response selection difficulty on Task 1 performance. Both types of BCEs (i.e., BCEs based on code compatibility and BCEs based on Task 2 difficulty) could be assumed to be rooted in anticipation of response selection difficulty triggered by stimuli indicating either R1-R2 or S2-R2 incompatibility. The results are in line with recent theoretical claims that anticipations of response characteristics (or effects) play an important role for BCEs in particular and for conflict resolution in action control in general.

Independent selection of eye and hand targets suggests effector-specific attentional mechanisms

Both eye and hand movements bind visual attention to their target locations during movement preparation. However, it remains contentious whether eye and hand targets are selected jointly by a single selection system, or individually by independent systems. To unravel the controversy, we investigated the deployment of visual attention – a proxy of motor target selection – in coordinated eye-hand movements. Results show that attention builds up in parallel both at the eye and the hand target. Importantly, the allocation of attention to one effector’s motor target was not affected by the concurrent preparation of the other effector’s movement at any time during movement preparation. This demonstrates that eye and hand targets are represented in separate, effector-specific maps of action-relevant locations. The eye-hand synchronisation that is frequently observed on the behavioral level must emerge from mutual influences of the two effector systems at later, post-attentional processing stages.

Phone Conversation while Processing Information: Chronometric Analysis of Load Effects in Everyday-media Multitasking

This is a pilot study that examined the effect of cell-phone conversation on cognition using a continuous multitasking paradigm. Current theorizing argues that phone conversation affects behavior (e.g., driving) by interfering at a level of cognitive processes (not peripheral activity) and by implying an attentional-failure account. Within the framework of an intermittent spare–utilized capacity threading model, we examined the effect of aspects of (secondary-task) phone conversation on (primary-task) continuous arithmetic performance, asking whether phone use makes components of automatic and controlled information-processing (i.e., easy vs. hard mental arithmetic) run more slowly, or alternatively, makes processing run less reliably albeit with the same processing speed. The results can be summarized as follows: While neither expecting a text message nor expecting an impending phone call had any detrimental effects on performance, active phone conversation was clearly detrimental to primary-task performance. Crucially, the decrement imposed by secondary-task (conversation) was not due to a constant slowdown but is better be characterized by an occasional breakdown of information processing, which differentially affected automatic and controlled components of primary-task processing. In conclusion, these findings support the notion that phone conversation makes individuals not constantly slower but more vulnerable to commit attention failure, and in this way, hampers stability of (primary-task) information processing.

Foresight beats hindsight: The neural correlates underlying motor preparation in the pro-/anti-cue paradigm

BACKGROUND

Human motor behaviors are characterized by both, reactive and proactive mechanisms. Yet, studies investigating the neural correlates of motor behavior almost exclusively focused on reactive motor processes. Here, we employed the pro-/anti-cue motor preparation paradigm to systematically study proactive motor control in an imaging environment. In this paradigm, either pro- or anti-cues are presented in a blocked design. Four fingers (two from each hand) are mapped onto four visual target locations. Visual targets require a speeded response by one corresponding finger, but, most importantly, they are preceded by visual cues that are congruent ("pro-cue"), incongruent ("anti-cue"), or neutral with respect to the responding hand. With short cue-target intervals, congruence effects are based on automatic motor priming of the correct hand (in case of pro-cues) or incorrect hand (in case of anti-cues), generating, respectively, reaction time benefits or reaction time costs relative to the neutral-cue. With longer cue-target intervals, slower top-down processes become effective, transforming early anti-cue interference into late anti-cue facilitation.

METHODS

We adapted this paradigm to be compatible with neuroimaging, tested and validated it behaviorally-both inside and outside the imaging environment-and implemented it in a whole-brain functional magnetic resonance imaging study.

RESULTS AND CONCLUSION

Our imaging results indicate that pro-cues elicited much less neural activation than did anti-cues, the latter recruiting well-known cognitive top-down networks related to attention, response inhibition, and error monitoring/signaling, thereby revealing high-level influences on proactive motor processes.

Cross-modal Action Complexity: Action- and Rule-related Memory Retrieval in Dual-response Control

Normally, we do not act within a single effector system only, but rather coordinate actions across several output modules (cross-modal action). Such cross-modal action demands can vary substantially with respect to their complexity in terms of the number of task-relevant response combinations and to-be-retrieved stimulus–response (S–R) mapping rules. In the present study, we study the impact of these two types of cross-modal action complexity on dual-response costs (i.e., performance differences between single- and dual-action demands). In Experiment 1, we combined a manual and an oculomotor task, each involving four response alternatives. Crucially, one (unconstrained) condition involved all 16 possible combinations of response alternatives, whereas a constrained condition involved only a subset of possible response combinations. The results revealed that preparing for a larger number of response combinations yielded a significant, but moderate increase in dual-response costs. In Experiment 2, we utilized one common lateralized auditory (e.g., left) stimulus to trigger incompatible response compounds (e.g., left saccade and right key press or vice versa). While one condition only involved one set of task-relevant S–R rules, another condition involved two sets of task-relevant rules (coded by stimulus type: noise/tone), while the number of task-relevant response combinations was the same in both conditions. Here, an increase in the number of to-be-retrieved S–R rules was associated with a substantial increase in dual-response costs that were also modulated on a trial-by-trial basis when switching between rules. Taken together, the results shed further light on the dependency of cross-modal action control on both action- and rule-related memory retrieval processes.

Task Switching, Modality Compatibility, and the Supra-Modal Function of Eye Movements

Previous research suggested that specific pairings of stimulus and response modalities (visual-manual and auditory-vocal tasks) lead to better dual-task performance than other pairings (visual-vocal and auditory-manual tasks). In the present task-switching study, we further examined this modality compatibility effect and investigated the role of response modality by additionally studying oculomotor responses as an alternative to manual responses. Interestingly, the switch cost pattern revealed a much stronger modality compatibility effect for groups in which vocal and manual responses were combined as compared to a group involving vocal and oculomotor responses, where the modality compatibility effect was largely abolished. We suggest that in the vocal-manual response groups the modality compatibility effect is based on cross-talk of central processing codes due to preferred stimulus-response modality processing pathways, whereas the oculomotor response modality may be shielded against cross-talk due to the supra-modal functional importance of visual orientation.