Dividing Attention Between Tasks: Testing Whether Explicit Payoff Functions Elicit Optimal Dual-Task Performance

Pupil-linked arousal modulates network-level EEG signatures of attention reorienting during immersive multitasking

Objective. When multitasking, we must dynamically reorient our attention between different tasks. Attention reorienting is thought to arise through interactions of physiological arousal and brain-wide network dynamics. In this study, we investigated the relationship between pupil-linked arousal and electroencephalography (EEG) brain dynamics in a multitask driving paradigm conducted in virtual reality. We hypothesized that there would be an interaction between arousal and EEG dynamics and that this interaction would correlate with multitasking performance.Approach. We collected EEG and eye tracking data while subjects drove a motorcycle through a simulated city environment, with the instructions to count the number of target images they observed while avoiding crashing into a lead vehicle. The paradigm required the subjects to continuously reorient their attention between the two tasks. Subjects performed the paradigm under two conditions, one more difficult than the other.Main results. We found that task difficulty did not strongly correlate with pupil-linked arousal, and overall task performance increased as arousal level increased. A single-trial analysis revealed several interesting relationships between pupil-linked arousal and task-relevant EEG dynamics. Employing exact low-resolution electromagnetic tomography, we found that higher pupil-linked arousal led to greater EEG oscillatory activity, especially in regions associated with the dorsal attention network and ventral attention network (VAN). Consistent with our hypothesis, we found a relationship between EEG functional connectivity and pupil-linked arousal as a function of multitasking performance. Specifically, we found decreased functional connectivity between regions in the salience network (SN) and the VAN as pupil-linked arousal increased, suggesting that improved multitasking performance at high arousal levels may be due to a down-regulation in coupling between the VAN and the SN. Our results suggest that when multitasking, our brain rebalances arousal-based reorienting so that individual task demands can be met without prematurely reorienting to competing tasks.

Individual differences in skill acquisition and transfer assessed by dual task training performance and brain activity

Assessment of expertise development during training program primarily consists of evaluating interactions between task characteristics, performance, and mental load. Such a traditional assessment framework may lack consideration of individual characteristics when evaluating training on complex tasks, such as driving and piloting, where operators are typically required to execute multiple tasks simultaneously. Studies have already identified individual characteristics arising from intrinsic, context, strategy, personality, and preference as common predictors of performance and mental load. Therefore, this study aims to investigate the effect of individual difference in skill acquisition and transfer using an ecologically valid dual task, behavioral, and brain activity measures. Specifically, we implemented a search and surveillance task (scanning and identifying targets) using a high-fidelity training simulator for the unmanned aircraft sensor operator, acquired behavioral measures (scan, not scan, over scan, and adaptive target find scores) using simulator-based analysis module, and measured brain activity changes (oxyhemoglobin and deoxyhemoglobin) from the prefrontal cortex (PFC) using a portable functional near-infrared spectroscopy (fNIRS) sensor array. The experimental protocol recruited 13 novice participants and had them undergo three easy and two hard sessions to investigate skill acquisition and transfer, respectively. Our results from skill acquisition sessions indicated that performance on both tasks did not change when individual differences were not accounted for. However inclusion of individual differences indicated that some individuals improved only their scan performance (Attention-focused group), while others improved only their target find performance (Accuracy-focused group). Brain activity changes during skill acquisition sessions showed that mental load decreased in the right anterior medial PFC (RAMPFC) in both groups regardless of individual differences. However, mental load increased in the left anterior medial PFC (LAMPFC) of Attention-focused group and decreased in the Accuracy-focused group only when individual differences were included. Transfer results showed no changes in performance regardless of grouping based on individual differences; however, mental load increased in RAMPFC of Attention-focused group and left dorsolateral PFC (LDLPFC) of Accuracy-focused group. Efficiency and involvement results suggest that the Attention-focused group prioritized the scan task, while the Accuracy-focused group prioritized the target find task. In conclusion, training on multitasks results in individual differences. These differences may potentially be due to individual preference. Future studies should incorporate individual differences while assessing skill acquisition and transfer during multitask training.

Switch rates vary due to expected payoff but not due to individual risk tendency

When switching between different tasks, the initiation of task switches may depend on task characteristics (difficulty, salient cues, etc.) or reasons within the person performing the task (decisions, behavioral variability, etc.). The reasons for variance in switching strategies, especially in paradigms where participants are free to choose the order of tasks and the amount of switching between tasks, are not well researched. In this study, we follow up the recent discussion that variance in switching strategies might be partly explained by the characteristics of the person fulfilling the task. We examined whether risk tendency and impulsiveness differentiate individuals in their response (i.e., switch rates and time spent on tasks) to different task characteristics on a tracking-while-typing paradigm. In detail, we manipulated two aspects of loss prospect (i.e., "payoff" as the amount of points that could be lost when tracking was unattended for too long, and "cursor speed" determining the likelihood of such a loss occurring). To account for between-subject variance and within-subject variability in the data, we employed linear mixed effect analyses following the model selection procedure (Bates, Kliegl, et al., 2015). Besides, we tested whether risk tendency can be transformed into a decision parameter which could predict switching strategies when being computationally modelled. We transferred decision parameters from the Decision Field Theory to model "switching thresholds" for each individual. Results show that neither risk tendency nor impulsiveness explain between-subject variance in the paradigm, nonetheless linear mixed-effects models confirmed that within-subject variability plays a significant role for interpreting dual-task data. Our computational model yielded a good model fit, suggesting that the use of a decision threshold parameter for switching may serve as an alternative means to classify different strategies in task switching.

The effect of apparent distance on peripheral target detection

Previous research suggests that peripheral target detection is modulated by viewing distance and distance simulated by pictorial cues and optic flow. In the latter case, it is unclear what cues contribute to the effect of distance. The current study evaluated the effect of distance on peripheral detection in a virtual three-dimensional environment. Experiments 1–3 used a continuous, dynamic central task that simulated observers traveling either actively or passively through a virtual environment following a car. Peripheral targets were flashed on checkerboard-covered walls to the left and right of the path of motion, at a near and a far distance from the observer. The retinal characteristics of the targets were identical across distances. Experiment 1 found more accurate and faster detection for near targets compared to far targets, especially for larger eccentricities. Experiment 2 equated the predictability of target onset across distances and found the near advantage for larger eccentricities in accuracy but a much smaller effect in reaction time (RT). Experiment 3 removed the checkerboard background implemented in Experiments 1 and 2, and Experiment 4 manipulated several static, monocular cues. Experiments 3 and 4 found that the variation in the density of the checkerboard backgrounds could explain the main effect of distance on accuracy but could not completely account for the interaction between target distance and eccentricity. These results suggest that attention is modulated by target distance, but the effect is small. Finally, there were consistent divided attention costs in the central car-following task but not the peripheral detection task.

The influence of rewards on (sub-)optimal interleaving

We investigate how the rewards of individual tasks dictate a priori how easy it is to interleave two discrete tasks efficiently, and whether people then interleave efficiently. Previous research found that people vary in their ability to interleave efficiently. Less attention has been given to whether it was realistic to expect efficient interleaving, given the reward rate of each of the involved tasks. Using a simulation model, we demonstrate how the rewards of individual tasks lead to different dual-task interleaving scenarios. We identify three unique dual-task scenarios. In easy scenarios, many strategies for time division between tasks can achieve optimal performance. This gives great opportunity to optimize performance, but also leads to variation in the applied strategies due to a lack of pressure to settle on a small set of optimal strategies. In difficult scenarios, the optimal strategy is hard to identify, therefore giving little opportunity to optimize. Finally, constrained scenarios have a well-defined prediction of the optimal strategy. It gives a narrow prediction, which limits the options to achieve optimal scores, yet given the structure people are able to optimize their strategies. These scenarios are therefore best to test people’s general capability of optimizing interleaving. We report three empirical studies that test these hypotheses. In each study, participants interleave between two identical discrete tasks, that differ only in the underlying reward functions and the combined result (easy, difficult, or constrained scenario). Empirical results match the theoretical pattern as predicted by simulation models. Implications for theory and practice are discussed.

Multitasking as a choice: a perspective

Performance decrements in multitasking have been explained by limitations in cognitive capacity, either modelled as static structural bottlenecks or as the scarcity of overall cognitive resources that prevent humans, or at least restrict them, from processing two tasks at the same time. However, recent research has shown that individual differences, flexible resource allocation, and prioritization of tasks cannot be fully explained by these accounts. We argue that understanding human multitasking as a choice and examining multitasking performance from the perspective of judgment and decision-making (JDM), may complement current dual-task theories. We outline two prominent theories from the area of JDM, namely Simple Heuristics and the Decision Field Theory, and adapt these theories to multitasking research. Here, we explain how computational modelling techniques and decision-making parameters used in JDM may provide a benefit to understanding multitasking costs and argue that these techniques and parameters have the potential to predict multitasking behavior in general, and also individual differences in behavior. Finally, we present the one-reason choice metaphor to explain a flexible use of limited capacity as well as changes in serial and parallel task processing. Based on this newly combined approach, we outline a concrete interdisciplinary future research program that we think will help to further develop multitasking research.