Cue the effects: Stimulus-action effect modality compatibility and dual-task costs

Voluntary task switching is affected by modality compatibility and preparation

Cognitive task control can be examined in task-switching studies. Performance costs in task switches are usually smaller with compatible stimulus-response modality mappings (visual-manual and auditory-vocal) than with incompatible mappings (visual-vocal and auditory-manual). Modality compatibility describes the modality match of sensory input and of the anticipated response effect (e.g., vocal responses produce auditory effects, so that auditory stimuli are modality-compatible with vocal responses). Fintor et al. (Psychological Research, 84(2), 380-388, 2020) found that modality compatibility also biased task choice rates in voluntary task switching (VTS). In that study, in each trial participants were presented with a visual or auditory spatial stimulus and were free to choose the response modality (manual vs. vocal). In this free-choice task, participants showed a bias to create more modality-compatible than -incompatible mappings. In the present study, we assessed the generality of Fintor et al.'s (2020) findings, using verbal rather than spatial stimuli, and more complex tasks, featuring an increased number of stimulus-response alternatives. Experiment 1 replicated the task-choice bias to preferentially create modality-compatible mappings. We also found a bias to repeat the response modality just performed, and a bias to repeat entire stimulus-response modality mappings. In Experiment 2, we manipulated the response-stimulus interval (RSI) to examine whether more time for proactive cognitive control would help resolve modality-specific crosstalk in this free-choice paradigm. Long RSIs led to a decreased response-modality repetition bias and mapping repetition bias, but the modality-compatibility bias was unaffected. Together, the findings suggest that modality-specific priming of response modality influences task choice.

Generalisation of unpredictable action-effect features: Large individual differences with little on-average effect

Ideomotor theory suggests that selecting a response is achieved by anticipating the consequences of that response. Evidence for this is the response-effect compatibility (REC) effect, that is, responding tends to be faster when the (anticipated) predictable consequences of a response (the action effects) are compatible rather than incompatible with the response. The present experiments investigated the extent to which the consequences must be exactly versus categorically predictable. According to the latter, an abstraction from particular instances to the categories of dimensional overlap might take place. For participants in one group of Experiment 1, left-hand and right-hand responses produced compatible or incompatible action effects in perfectly predictable positions to the left or right of fixation, and a standard REC effect was observed. For participants in another group of Experiment 1, as well as in Experiments 2 and 3, the responses also produced action effects to the left or right of fixation, but the eccentricity of the action effects (and thus their precise location) was unpredictable. On average, the data from the latter groups suggest that there is little, if any, tendency for participants to abstract the critical left/right features from spatially somewhat unpredictable action effects and use them for action selection, although there were large individual differences in these groups. Thus, at least on average across participants, it appears that the spatial locations of action effects must be perfectly predictable for these effects to have a strong influence on the response time.

The surprising role of stimulus modality in the dual-task introspective blind spot: a memory account

Being able to accumulate accurate information about one's own performance is important in everyday contexts, and arguably particularly so in complex multitasking contexts. Thus, the observation of a glaring gap in participants' introspection regarding their own reaction time costs in a concurrent dual-task context is deserving of closer examination. This so-called introspective blind spot has been explained by a 'consciousness bottleneck' which states that while attention is occupied by one task, participants cannot consciously perceive another stimulus presented in that time. In the current study, a series of introspective Psychological Refractory Period (PRP) experiments were conducted to identify the determinants of an introspective blind spot; to our surprise, in half of the experiments participants appeared to be aware of their dual-task costs. A single trial analysis highlighted the sensory modality of the two stimuli within the trial as an important predictor of introspective accuracy, along with temporal gaps in the trial. The current findings call into question the claim that attention is required for conscious awareness. We propose a memory-based account of introspective processes in this context, whereby introspective accuracy is determined by the memory systems involved in encoding and rehearsing memory traces. This model of the conditions required to build up accurate representations of our performance may have far-reaching consequences for monitoring and introspection across a range of tasks.

Response-repetition costs reflect changes to the representation of an action

Repeating a response from the previous trial typically leads to performance benefits. However, these benefits are eliminated, and usually reversed, when switching to a new task (i.e., response-repetition costs). Here, we test the proposal that response-repetition costs reflect changes in the representation of an action. To investigate this, we designed tasks that required participants to switch between color and shape judgments with experimentally induced outcomes. Critically, the stimuli and responses were constant across conditions; what differed was the number of outcomes associated with the responses. For both response time and error rate, response-repetition costs on task-switch trials were significantly reduced when response repetitions led to outcome repetitions relative to when response repetitions led to outcome switches. Moreover, response repetitions that led to outcome repetitions showed an advantage in response time (but not error rate) compared with when no outcomes were experimentally induced. We conclude that response-repetition costs reflect a change in the representation of an action and that action selection is largely grounded in the anticipation of the response-related outcomes.

Dissociating stimulus-response compatibility and modality compatibility in task switching

Modality compatibility (MC) describes the similarity between the modality of the stimulus and the modality of the anticipated response effect (e.g., auditory effects when speaking). Switching between two incompatible modality mappings (visual-vocal and auditory-manual) typically leads to larger costs than switching between two compatible modality mappings (visual-manual and auditory-vocal). However, it is unclear whether the influence of MC arises before or after task selection or response selection, or affects both processes. We investigated this issue by introducing a factor known to influence response selection, stimulus-response (S-R) compatibility, examining possible interactions with MC. In Experiment 1, stimulus location was task-irrelevant; participants responded manually or vocally to the meaning of visual and auditory colour words presented left or right (Simon task). In Experiment 2, stimulus location was task-relevant; participants responded manually or vocally, indicating the location (left or right) of visual or auditory stimuli, using a spatially compatible versus incompatible mapping rule (“element-level” S-R compatibility). Results revealed independent effects of S-R and modality compatibility in both experiments (n = 40 per experiment). Bayes factors suggested moderate but consistent evidence for the absence of an interaction. Independent effects suggest MC effects arise either before or after response selection, or possibly both. We propose that motor response initiation is associated with anticipatory activation of modality-specific sensory effects (e.g., auditory effects when speaking), which in turn facilitates the correct response in case of modality-compatible mappings (e.g., auditory-vocal) or reactivates, at the task-selection level, the incorrect task in case of modality-incompatible mappings (e.g., visual-vocal).

Crosstalk, not resource competition, as a source of dual-task costs: Evidence from manipulating stimulus-action effect conceptual compatibility

Two related accounts of dual-task costs-multiple resource competition and crosstalk-explain why costs can be reduced when there is less overlap between the two tasks. However, distinguishing between competition for limited resources and crosstalk between concurrently performed operations has proven difficult. In the present study, we compared these two accounts with a dual-task paradigm in which participants were required to coordinate visual-manual and auditory-manual tasks with experimentally induced action effects. Critically, stimulus and response modalities were constant across conditions; what differed was the conceptual relationship between stimuli and action effects such that conceptual overlap was present either within or between tasks. We observed larger dual-task costs when related conceptual codes were present between tasks. We conclude that these results are best supported by the crosstalk account and that postresponse action effects are integrated into task representations engaged by central operations during response selection.

Cognitive-Postural Multitasking Training in Older Adults - Effects of Input-Output Modality Mappings on Cognitive Performance and Postural Control

Older adults exhibit impaired cognitive and balance performance, particularly under multi-task conditions, which can be improved through training. Compatibility of modality mappings in cognitive tasks (i.e., match between stimulus modality and anticipated sensory effects of motor responses), modulates physical and cognitive dual-task costs. However, the effects of modality specific training programs have not been evaluated yet. Here, we tested the effects of cognitive-postural multi-tasking training on the ability to coordinate task mappings under high postural demands in healthy older adults. Twenty-one adults aged 65-85 years were assigned to one of two groups. While group 1 performed cognitive-postural triple-task training with compatible modality mappings (i.e., visual-manual and auditory-vocal dual n-back tasks), group 2 performed the same tasks with incompatible modality mappings (i.e., visual-vocal and auditory-manual n-back tasks). Throughout the 6-weeks balance training intervention, working-memory load was gradually increased while base-of-support was reduced. Before training (T0), after a 6-week passive control period (T1), and immediately after the intervention (T2), participants performed spatial dual one-back tasks in semi-tandem stance position. Our results indicate improved working-memory performance and reduced dual-task costs for both groups after the passive control period, but no training-specific performance gains. Furthermore, balance performance did not improve in response to training. Notably, the cohort demonstrated meaningful interindividual variability in training responses. Our findings raise questions about practice effects and age-related heterogeneity of training responses following cognitive-motor training. Following multi-modal balance training, neither compatible nor incompatible modality mappings had an impact on the observed outcomes.

Modality compatibility in task switching depends on processing codes and task demands

Modality compatibility denotes the match between sensory stimulus modality and the sensory modality of the anticipated response effect (for example, vocal responses usually lead to auditory effects, so that auditory-vocal stimulus-response mappings are modality-compatible, whereas visual-vocal mappings are modality incompatible). In task switching studies, it has been found that switching between two modality-incompatible mappings (auditory-manual and visual-vocal) resulted in higher switch costs than switching between two modality-compatible mappings (auditory-vocal and visual-manual). This finding suggests that with modality-incompatible mappings, the anticipation of the effect of each response primes the stimulus modality linked to the competing task, creating task confusion. In Experiment 1, we examined whether modality-compatibility effects in task switching are increased by strengthening the auditory-vocal coupling using spatial-verbal stimuli relative to spatial-location stimuli. In Experiment 2, we aimed at achieving the same goal by requiring temporal stimulus discrimination relative to spatial stimulus localisation. Results suggest that both spatial-verbal stimuli and temporal discrimination can increase modality-specific task interference through a variation of the strength of anticipation in the response-effect coupling. This provides further support for modality specificity of cognitive control processes in task switching.