How conceptual overlap and modality pairings affect task-switching and mixing costs

Voluntary task switching is affected by modality compatibility and preparation

Cognitive task control can be examined in task-switching studies. Performance costs in task switches are usually smaller with compatible stimulus-response modality mappings (visual-manual and auditory-vocal) than with incompatible mappings (visual-vocal and auditory-manual). Modality compatibility describes the modality match of sensory input and of the anticipated response effect (e.g., vocal responses produce auditory effects, so that auditory stimuli are modality-compatible with vocal responses). Fintor et al. (Psychological Research, 84(2), 380-388, 2020) found that modality compatibility also biased task choice rates in voluntary task switching (VTS). In that study, in each trial participants were presented with a visual or auditory spatial stimulus and were free to choose the response modality (manual vs. vocal). In this free-choice task, participants showed a bias to create more modality-compatible than -incompatible mappings. In the present study, we assessed the generality of Fintor et al.'s (2020) findings, using verbal rather than spatial stimuli, and more complex tasks, featuring an increased number of stimulus-response alternatives. Experiment 1 replicated the task-choice bias to preferentially create modality-compatible mappings. We also found a bias to repeat the response modality just performed, and a bias to repeat entire stimulus-response modality mappings. In Experiment 2, we manipulated the response-stimulus interval (RSI) to examine whether more time for proactive cognitive control would help resolve modality-specific crosstalk in this free-choice paradigm. Long RSIs led to a decreased response-modality repetition bias and mapping repetition bias, but the modality-compatibility bias was unaffected. Together, the findings suggest that modality-specific priming of response modality influences task choice.

Are some effector systems harder to switch to? In search of cost asymmetries when switching between manual, vocal, and oculomotor tasks

In task-switching studies, performance is typically worse in task-switch trials than in task-repetition trials. These switch costs are often asymmetrical, a phenomenon that has been explained by referring to a dominance of one task over the other. Previous studies also indicated that response modalities associated with two tasks may be considered as integral components for defining a task set. However, a systematic assessment of the role of response modalities in task switching is still lacking: Are some response modalities harder to switch to than others? The present study systematically examined switch costs when combining tasks that differ only with respect to their associated effector systems. In Experiment 1, 16 participants switched (in unpredictable sequence) between oculomotor and vocal tasks. In Experiment 2, 72 participants switched (in pairwise combinations) between oculomotor, vocal, and manual tasks. We observed systematic performance costs when switching between response modalities under otherwise constant task features and could thereby replicate previous observations of response modality switch costs. However, we did not observe any substantial switch-cost asymmetries. As previous studies using temporally overlapping dual-task paradigms found substantial prioritization effects (in terms of asymmetric costs) especially for oculomotor tasks, the present results suggest different underlying processes in sequential task switching than in simultaneous multitasking. While more research is needed to further substantiate a lack of response modality switch-cost asymmetries in a broader range of task switching situations, we suggest that task-set representations related to specific response modalities may exhibit rapid decay.

Short-term pre-exposure to modality mappings: Modality-incompatible single-task exposure reduces modality-specific between-task crosstalk in task-switching

Modality compatibility refers to the similarity of the stimulus modality and the modality of the sensory-response effect that the response produces (i.e., vocal responses produce auditory effects). In this study, we investigated the effect of short-term pre-exposure of modality compatibility in task-switching. To this end, participants were exposed to either modality-compatible (visual-manual and auditory-vocal) or modality-incompatible (visual-vocal and auditory-manual) single-tasks. After a short-term single-task pre-exposure (with either both modality-compatible tasks, 2 × 80 trials each, or both modality-incompatible tasks, 2 × 80 trials each), participants were transferred to a task-switching situation, where they switched between tasks in both a modality-compatible and an incompatible condition. We found that after pre-exposure to modality-compatible single-tasks the typical effect of modality compatibility was found (i.e., larger switch costs with modality-incompatible tasks compared to modality-compatible tasks). In contrast, after pre-exposed to modality-incompatible single-tasks, modality compatibility no longer influenced switch costs. We assume that long-term modality-compatible associations could be overridden by short-term, task-specific associations to reduce between-task crosstalk.

Crosstalk, not resource competition, as a source of dual-task costs: Evidence from manipulating stimulus-action effect conceptual compatibility

Two related accounts of dual-task costs-multiple resource competition and crosstalk-explain why costs can be reduced when there is less overlap between the two tasks. However, distinguishing between competition for limited resources and crosstalk between concurrently performed operations has proven difficult. In the present study, we compared these two accounts with a dual-task paradigm in which participants were required to coordinate visual-manual and auditory-manual tasks with experimentally induced action effects. Critically, stimulus and response modalities were constant across conditions; what differed was the conceptual relationship between stimuli and action effects such that conceptual overlap was present either within or between tasks. We observed larger dual-task costs when related conceptual codes were present between tasks. We conclude that these results are best supported by the crosstalk account and that postresponse action effects are integrated into task representations engaged by central operations during response selection.

Cognitive-Postural Multitasking Training in Older Adults - Effects of Input-Output Modality Mappings on Cognitive Performance and Postural Control

Older adults exhibit impaired cognitive and balance performance, particularly under multi-task conditions, which can be improved through training. Compatibility of modality mappings in cognitive tasks (i.e., match between stimulus modality and anticipated sensory effects of motor responses), modulates physical and cognitive dual-task costs. However, the effects of modality specific training programs have not been evaluated yet. Here, we tested the effects of cognitive-postural multi-tasking training on the ability to coordinate task mappings under high postural demands in healthy older adults. Twenty-one adults aged 65-85 years were assigned to one of two groups. While group 1 performed cognitive-postural triple-task training with compatible modality mappings (i.e., visual-manual and auditory-vocal dual n-back tasks), group 2 performed the same tasks with incompatible modality mappings (i.e., visual-vocal and auditory-manual n-back tasks). Throughout the 6-weeks balance training intervention, working-memory load was gradually increased while base-of-support was reduced. Before training (T0), after a 6-week passive control period (T1), and immediately after the intervention (T2), participants performed spatial dual one-back tasks in semi-tandem stance position. Our results indicate improved working-memory performance and reduced dual-task costs for both groups after the passive control period, but no training-specific performance gains. Furthermore, balance performance did not improve in response to training. Notably, the cohort demonstrated meaningful interindividual variability in training responses. Our findings raise questions about practice effects and age-related heterogeneity of training responses following cognitive-motor training. Following multi-modal balance training, neither compatible nor incompatible modality mappings had an impact on the observed outcomes.

Crossmodal Effects in Task Switching: Modality Compatibility with Vocal and Pedal Responses

Modality compatibility refers to the similarity between the stimulus modality and the modality of response-related sensory consequences (e.g., vocal output produces audible effects). While previous studies found higher costs of task switching with stimulus-response modality-incompatible tasks (auditory-manual and visual-vocal), the present study was aimed to explore the generality of modality compatibility by examining a new response modality (pedal responses). Experiment 1 showed that the effect of modality compatibility generalizes to pedal responses when these replaced manual responses used in previous studies (i.e., higher switch costs when switching between auditory-pedal and visual-vocal tasks compared to switching between auditory-vocal and visual-pedal tasks). However, in single-task conditions there was no influence of modality compatibility. Experiment 2 was designed to examine whether modality compatibility depends on the frequency of task switches. To this end, one task occurred very frequently, overall decreasing the task switching frequency. Importantly, the results showed a robust task-switching benefit of modality-compatible mappings even for a highly frequent task, suggesting that the sustained representation of potentially competing response modalities affects task-switching performance independent from the actual frequency of the tasks. Together, the data suggest that modality compatibility is an emergent phenomenon arising in task-switching situations based on the necessity to maintain but at the same time separate competing modality mappings, which are characterized by ideomotor ‘‘backward’’ linkages between anticipated response effects and the stimuli that called for this response in the first place.

More change in task repetition, less cost in task switching: Behavioral and event-related potential evidence

Previous studies have shown that the probability of task switching can vary the level of cognitive control and modulate the size of switch costs. However, it is unclear whether switch costs would be affected by a task-repetition context formed by varying the degree of response (and task-relevant stimulus property) change within the task repetition sequences while the probability of task switching remains constant. In the present study, participants were presented with a string of digits (e.g., ②②②). Basing on stimulus color, they were required to indicate either the presented digit, or the number of presented digits. Before task switching, stimulus and response in consecutive task-repeat trials varied more or less frequently. Behavioral results showed that the frequent-change context elicited smaller switch costs than the rare-change context. Event-related potential (ERP) results indicated that: (1) the frequent-change context evoked greater fronto-central N2 amplitudes for both task-repeat and task-switch trials, implying that cognitive control increased due to the variation of stimulus and response associations; (2) for the task switch trials, smaller P300 amplitudes were evoked in the frequent-change context than the rare-change context, reflecting the promoted task-set reconfiguration. These findings suggest that, the more change in stimulus and response during task repetition, the higher the overall level of cognitive control and the higher efficiency of task-switching.

Modality compatibility in task switching depends on processing codes and task demands

Modality compatibility denotes the match between sensory stimulus modality and the sensory modality of the anticipated response effect (for example, vocal responses usually lead to auditory effects, so that auditory-vocal stimulus-response mappings are modality-compatible, whereas visual-vocal mappings are modality incompatible). In task switching studies, it has been found that switching between two modality-incompatible mappings (auditory-manual and visual-vocal) resulted in higher switch costs than switching between two modality-compatible mappings (auditory-vocal and visual-manual). This finding suggests that with modality-incompatible mappings, the anticipation of the effect of each response primes the stimulus modality linked to the competing task, creating task confusion. In Experiment 1, we examined whether modality-compatibility effects in task switching are increased by strengthening the auditory-vocal coupling using spatial-verbal stimuli relative to spatial-location stimuli. In Experiment 2, we aimed at achieving the same goal by requiring temporal stimulus discrimination relative to spatial stimulus localisation. Results suggest that both spatial-verbal stimuli and temporal discrimination can increase modality-specific task interference through a variation of the strength of anticipation in the response-effect coupling. This provides further support for modality specificity of cognitive control processes in task switching.

Effects of Input Modality on Vocal Effector Prioritization in Manual-Vocal Dual Tasks

Doing two things at once (vs. one in isolation) usually yields performance costs. Such decrements are often distributed asymmetrically between the two actions involved, reflecting different processing priorities. A previous study (Huestegge & Koch, 2013) demonstrated that the particular effector systems associated with the two actions can determine the pattern of processing priorities: Vocal responses were prioritized over manual responses, as indicated by smaller performance costs (associated with dual-action demands) for the former. However, this previous study only involved auditory stimulation (for both actions). Given that previous research on input-output modality compatibility in dual tasks suggested that pairing auditory input with vocal output represents a particularly advantageous mapping, the question arises whether the observed vocal-over-manual prioritization was merely a consequence of auditory stimulation. To resolve this issue, we conducted a manual-vocal dual task study using either only auditory or only visual stimuli for both responses. We observed vocal-over-manual prioritization in both stimulus modality conditions. This suggests that input-output modality mappings can (to some extent) attenuate, but not abolish/reverse effector-based prioritization. Taken together, effector system pairings appear to have a more substantial impact on capacity allocation policies in dual-task control than input-output modality combinations.

Cue the effects: Stimulus-action effect modality compatibility and dual-task costs

The pairings of tasks' stimulus and response modalities affect the magnitude of dual-task costs. For example, dual-task costs are larger when a visual-vocal task is paired with an auditory-manual task compared with when a visual-manual task is paired with an auditory-vocal task. These results are often interpreted as reflecting increased crosstalk between central codes for each task. Here we examine a potential source: modality-based crosstalk between the stimuli and the response-induced sensory consequences (i.e., action effects). In five experiments, we manipulated experimentally induced action effects so that they were either modality-compatible or -incompatible with the stimuli. Action effects that were modality-compatible (e.g., visual stimulus, visual action effect) produced smaller dual-task costs than those that were modality-incompatible (e.g., visual stimulus, auditory action effect). Thus, the relationship between stimuli and action effects contributes to dual-task costs. Moreover, modality-compatible pairs showed an advantage compared with when no action effects were experimentally induced. These results add to a growing body of work demonstrating that postresponse sensory events affect response selection processes. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Modality compatibility biases voluntary choice of response modality in task switching

The term modality compatibility refers to the similarity between stimulus modality and the modality of response-related sensory consequences (e.g., vocal responses produce auditory effects). The previous results showed smaller task-switching costs when participants switched between modality compatible tasks (auditory-vocal and visual-manual) compared to switching between modality incompatible tasks (auditory-manual and visual-vocal). In the present study using a voluntary task-switching paradigm (VTS), participants chose the response modality (vocal or manual) to indicate the location of either a visual or an auditory stimulus. We examined whether free task choices were biased by modality compatibility, so that modality compatible tasks are preferred in VTS. The choice probability analysis indicated that participants tended to choose the response modality that is compatible to the stimulus modality. However, participants did not show a preference to repeat a stimulus-response (S-R) modality mapping, but to switch between modality compatibility (i.e., from S-R modality compatible mapping to S-R modality incompatible mapping and vice versa). More interestingly, even though participants freely chose the response modality, modality compatibility still influenced task-switching costs, showing larger costs with modality incompatible mappings. The finding that modality compatibility influenced choice behaviour suggests components of both top-down control and bottom-up effects of selecting a response modality for different stimulus modalities.

The interplay of crossmodal attentional preparation and modality compatibility in cued task switching

Two experiments examined the influence of preparation on modality compatibility effects in task switching. The term modality compatibility refers to the similarity between the stimulus modality and the modality of response-related sensory consequences. Previous research showed evidence for modality compatibility benefits in task switching when participants switched either between two modality compatible tasks (auditory-vocal and visual-manual) or between two modality incompatible tasks (auditory-manual and visual-vocal). In this study, we investigated the influence of active preparation on modality compatibility effects in task switching. To this end, in Experiment 1, we introduced unimodal modality cues, whereas in Experiment 2, bimodal abstract cues were used. In both experiments, the cue-stimulus interval (CSI) was manipulated while holding the response-stimulus interval (RSI) constant. In both experiments, we found not only decreased switch costs with long CSI but also the elimination of the residual switch costs. More importantly, this preparation effect did not modulate the modality compatibility effect in task switching. To account for this data pattern, we assume that cue-based preparation of switches by modality mappings was highly effective and produced no residual reaction time (RT) costs with long CSI.