Vigilance on the move: video game-based measurement of sustained attention

Effects of task demands on tactile vigilance

A performance decline during sustained monitoring of unpredictable and occasional signals, the vigilance decrement, has been studied mostly in the visual and auditory modalities, but a tactile vigilance decrement also has been observed and has been associated with high perceived workload, declines in sensitivity and task engagement, and increases in distress. The primary aim of the current study was to determine whether task demands affect the vigilance decrement in the tactile modality and whether the effects are similar to those observed in the auditory and visual modalities. Participants completed a 40-min vigil in which they monitored vibrotactile stimuli generated by a tactor and had to discriminate between durations of bursts of vibrations. Task demand was varied by including low and high event rates. Although correct detections decreased over time (vigilance decrement) and sensitivity was greater for the slower event rate, there was not an interaction between period of watch and event rate. There also were no differences in workload and stress between event rates. Results indicate that mean performance in tactile vigilance tasks is negatively impacted by increases in event rate, indicating that a typical source of task demand known to affect visual and auditory vigilance also affects tactile vigilance. Results could be explained by either an underload or overload theory of the vigilance decrement.

The effects of dual-task interference on visual search and verbal memory

The operational costs of multitasking are more pressing given the increase in wearable technologies (head-up displays; HUDs) that facilitate multitasking. Often multitasking comes with performance costs, where the addition of more tasks impairs the performance of the tasks. The current study explored the extent to which multitasking interference can be characterised in simulated environments, as opposed to risky and harsh environments in real operational contexts. Forty-eight participants completed several trials where they performed a visual search task while navigating a simulated environment. There were three conditions: a standalone memory task, a standalone search task, and both tasks simultaneously. Results revealed significant dual-task interference when comparing the dual-task to each of the single-tasks. Results were corroborated by subjective workload and stress metrics. The results could prove useful for designing systems for individuals who routinely multitask in operational environments. Specifically, by furthering the understanding of their performance capabilities and trade-offs due to multitasking.Practitioner summary: Due to the demands of multitasking in operational environments, quantifying the degree of information lost on each task individually will aid in the understanding of the deficits of multitasking performance. This study shows that deficits in multi-tasking (via a HUD) can be understood in simulated environments to a similar degree as real-world tasks.

Tactile Vigilance Is Stressful and Demanding

OBJECTIVE

The primary aims of the study were to replicate the vigilance decrement in the tactile modality, examine whether a decrease in sensitivity is associated with the decrement, and determine whether tactile vigilance is stressful and demanding.

BACKGROUND

When people monitor occasional and unpredictable signals for sustained durations, they experience a decline in performance known as the vigilance decrement, which has important practical consequences. Prior studies of the vigilance decrement focused primarily on visual vigilance and, to a lesser degree, on auditory vigilance. There are relatively few studies of tactile vigilance.

METHOD

Participants monitored vibrotactile stimuli that were created from a tactor, for 40 min.

RESULTS

Sensitivity declined, self-report ratings of distress increased, and ratings of task engagement decreased, during the vigil, and perceived workload was moderately high.

CONCLUSION

Monitoring tactile signals is demanding and stressful and results in a decrement in signal detection.

APPLICATION

Monitoring tactile signals may result in a decrement in tasks requiring discrimination, such as monitoring lane position with the use of rumble strips; these require discrimination between current road vibration and increased vibration when the car drifts out of its lane and crosses over the strip.

The Making and Evaluation of Digital Games Used for the Assessment of Attention: Systematic Review

Background

Serious games are now widely used in many contexts, including psychological research and clinical use. One area of growing interest is that of cognitive assessment, which seeks to measure different cognitive functions such as memory, attention, and perception. Measuring these functions at both the population and individual levels can inform research and indicate health issues. Attention is an important function to assess, as an accurate measure of attention can help diagnose many common disorders, such as attention-deficit/hyperactivity disorder and dementia. However, using games to assess attention poses unique problems, as games inherently manipulate attention through elements such as sound effects, graphics, and rewards, and research on adding game elements to assessments (ie, gamification) has shown mixed results. The process for developing cognitive tasks is robust, with high psychometric standards that must be met before these tasks are used for assessment. Although games offer more diverse approaches for assessment, there is no standard for how they should be developed or evaluated.

Objective

To better understand the field and provide guidance to interdisciplinary researchers, we aim to answer the question: How are digital games used for the cognitive assessment of attention made and measured?

Methods

We searched several databases for papers that described a digital game used to assess attention that could be deployed remotely without specialized hardware. We used Rayyan, a systematic review software, to screen the records before conducting a systematic review.

Results

The initial database search returned 49,365 papers. Our screening process resulted in a total of 74 papers that used a digital game to measure cognitive functions related to attention. Across the studies in our review, we found three approaches to making assessment games: gamifying cognitive tasks, creating custom games based on theories of cognition, and exploring potential assessment properties of commercial games. With regard to measuring the assessment properties of these games (eg, how accurately they assess attention), we found three approaches: comparison to a traditional cognitive task, comparison to a clinical diagnosis, and comparison to knowledge of cognition; however, most studies in our review did not evaluate the game’s properties (eg, if participants enjoyed the game).

Conclusions

Our review provides an overview of how games used for the assessment of attention are developed and evaluated. We further identified three barriers to advancing the field: reliance on assumptions, lack of evaluation, and lack of integration and standardization. We then recommend the best practices to address these barriers. Our review can act as a resource to help guide the field toward more standardized approaches and rigorous evaluation required for the widespread adoption of assessment games.

Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information

This work examines how tactile cues, encoded with azimuth and distance information, compare with visual and speech cues on performance and mental workload in a target detection task. Two experiments are reported using a simulated environment in which targets were presented at varying azimuth and distance locations. In the first experiment, participants engaged targets both while stationary and while in motion using tactile, visual, or speech cues. A no cueing control was included. In the second multi-modal experiment, participants completed the same task using cue pairings. Performance metrics consisted of hits, misses due to non-detection, misses due to inaccurate engagement, false alarms, response time, navigation errors as well as subjective ratings of mental workload scores were also collected. Results demonstrate the superiority of tactile cues as a means to communicate target location information either as a single modality or when paired with the two other cue types.

Stress Linked to Reports of Simulator Sickness in Vigilance: A Factor Analysis of the SSSQ and the SSQ

The current study investigated the relationship between stress and simulator sickness within vigilance tasks. The Simulator Sickness Questionnaire and the Short Stress State Questionnaire were employed. Data were aggregated from seven different visual vigilance tasks. Correlational relationships were investigated using an exploratory factor analysis, which revealed two factors. The first factor was represented by Distress, a stress measure, and all three simulator sickness subscales: Nausea, Disorientation, and Oculomotor symptoms. The second factor consisted only of Worry, a stress measure. These results suggest that the distress associated with vigilance is intertwined with reports of simulator sickness. One possible explanation is that the stress of vigilance effectively inflates reports of simulator sickness, a potential concern for the validity of simulator sickness assessments within tasks that require vigilance.

On the Dynamics of Conspicuity

OBJECTIVE

To examine the influences of dynamic conspicuity on object recognition and to evaluate the real-world implications of these processes.

BACKGROUND

Conspicuity is the major influence on persons' abilities to recognize the presence of entities within their environment. Shortfalls in sensory and cognitive conspicuity are implicated in many, if not most, real-world systemic failures.

METHOD

The present observations derive from an overview of relevant empirical research allied to a synthetic integration. From these foundations, I articulate a proposed taxonomy through which to parse the essential dimensions of conspicuity.

RESULTS

The taxonomy features three axes related to (a) modality (e.g., visual vs. auditory, etc.), (b) processing directionality (e.g., top-down vs. bottom-up information flow), and finally (c) temporality (i.e., the differences between static vs. dynamic presentations).

CONCLUSION

Existing conspicuity studies have primarily featured static, sensory comparisons. Exploration of the other quadrants of the proposed taxonomy can serve to frame future conspicuity research. This taxonomic description also provides the basis from which to understand failure etiology in a wide spectrum of human-machine systems.

APPLICATION

Improvements in the understanding of conspicuity can help in all domains of HF/E and can serve to reduce failure in a wide variety of operational contexts.

Vigilance Decrement and Enhancement Techniques: A Review

This paper presents the first comprehensive review on vigilance enhancement using both conventional and unconventional means, and further discusses the resulting contradictory findings. It highlights the key differences observed between the research findings and argues that variations of the experimental protocol could be a significant contributing factor towards such contradictory results. Furthermore, the paper reveals the effectiveness of unconventional means of enhancement in significant reduction of vigilance decrement compared to conventional means. Meanwhile, a discussion on the challenges of enhancement techniques is presented, with several suggested recommendations and alternative strategies to maintain an adequate level of vigilance for the task at hand. Additionally, this review provides evidence in support of the use of unconventional means of enhancement on vigilance studies, regardless of their practical challenges.

Driver Vigilance in Automated Vehicles: Effects of Demands on Hazard Detection Performance

OBJECTIVE

The current study investigated driver vigilance in partially automated vehicles to determine whether increased task demands reduce a driver's ability to monitor for automation failures and whether the vigilance decrement associated with hazard detections is due to driver overload.

BACKGROUND

Drivers of partially automated vehicles are expected to monitor for signs of automation failure. Previous research has shown that a driver's ability to perform this duty declines over time. One possible explanation for this vigilance decrement is that the extreme demands of vigilance causes overload and leads to depletion of limited attentional resources required for vigilance.

METHOD

Participants completed a 40-min drive in a simulated partially automated vehicle and were tasked with monitoring for hazards that represented potential automation failures. Two factors were manipulated to test the impact of monitoring demands on performance: Spatial uncertainty and event rate.

RESULTS

As predicted, hazard detection performance was poorer when monitoring demands were increased, and performance declined as a function of time on task. Subjective reports also indicated high workload and task-induced stress.

CONCLUSION

Drivers of partially automated vehicles are impaired by the vigilance decrement and elevated task demands, meaning that safe operation becomes less likely when the demands associated with monitoring automation increase and as a drive extends in duration. This study also supports the notion that vigilance performance in partially automated vehicles is likely due to driver overload.

APPLICATION

Developers of automation technologies should consider countermeasures that attenuate a driver's cognitive load when tasked with monitoring automation.

A cognitive brain-computer interface monitoring sustained attentional variations during a continuous task

We introduce a cognitive brain–computer interface based on a continuous performance task for the monitoring of variations of visual sustained attention, i.e. the self-directed maintenance of cognitive focus in non-arousing conditions while possibly ignoring distractors and avoiding mind wandering. We introduce a visual sustained attention continuous performance task with three levels of task difficulty. Pairwise discrimination of these task difficulties from electroencephalographic features was performed using a leave-one-subject-out cross validation approach. Features were selected using the orthogonal forward regression supervised feature selection method. Cognitive load was best predicted using a combination of prefrontal theta power, broad spatial range gamma power, fronto-central beta power, and fronto-central alpha power. Generalization performance estimates for pairwise classification of task difficulty using these features reached 75% for 5 s epochs, and 85% for 30 s epochs.

Hacking the Human: The Prevalence Paradox in Cybersecurity

OBJECTIVE

This work assesses the efficacy of the "prevalence effect" as a form of cyberattack in human-automation teaming, using an email task.

BACKGROUND

Under the prevalence effect, rare signals are more difficult to detect, even when taking into account their proportionally low occurrence. This decline represents diminished human capability to both detect and respond. As signal probability (SP) approaches zero, accuracy exhibits logarithmic decay. Cybersecurity, a context in which the environment is entirely artificial, provides an opportunity to manufacture conditions enhancing or degrading human performance, such as prevalence effects. Email cybersecurity prevalence effects have not previously been demonstrated, nor intentionally manipulated.

METHOD

The Email Testbed (ET) provides a simulation of a clerical email work involving messages containing sensitive personal information. Using the ET, participants were presented with 300 email interactions and received cyberattacks at rates of either 1%, 5%, or 20%.

RESULTS

Results demonstrated the existence and power of prevalence effects in email cybersecurity. Attacks delivered at a rate of 1% were significantly more likely to succeed, and the overall pattern of accuracy across declining SP exhibited logarithmic decay.

APPLICATION

These findings suggest a "prevalence paradox" within human-machine teams. As automation reduces attack SP, the human operator becomes increasingly likely to fail in detecting and reporting attacks that remain. In the cyber realm, the potential to artificially inflict this state on adversaries, hacking the human operator rather than algorithmic defense, is considered. Specific and general information security design countermeasures are offered.

Training for vigilance on the move: a video game-based paradigm for sustained attention

The capacity for superior vigilance can be trained by using knowledge of results (KR). Our present experiments demonstrate the efficacy of such training using a first-person perspective movement videogame-based platform in samples of students and Soldiers. Effectiveness was assessed by manipulating KR during a training phase and withdrawing it in a subsequent transfer phase. Relative to a no KR control condition, KR systematically improved performance for both Soldiers and students. These results build upon our previous findings that demonstrated that a video game-based platform can be used to create a movement-centred sustained attention task with important elements of traditional vigilance. The results indicate that KR effects in sustained attention extend to a first person perspective movement based paradigm, and that these effects occur in professional military as well as a more general population. Such sustained attention training can save lives and the present findings demonstrate one particular avenue to achieve this goal. Practitioner Summary: Sustained attention can be trained by means of knowledge of results using a videogame-based platform with samples of students and Soldiers. Four experiments demonstrate that a dynamic, first-person perspective video game environment can serve to support effective sustained attention training in professional military as well as a more general population.

Prolonging the response movement inhibits the feed-forward motor program in the sustained attention to response task

Despite widespread use in clinical and experimental contexts, debate continues over whether or not the Sustained Attention to Response Task (SART) successfully measures sustained attention. Altering physical aspects of the response movement required to SART stimuli may help identify whether performance is a better measure of perceptual decoupling, or response strategies and motor inhibition. Participants completed a SART where they had to manually move a mouse cursor to respond to stimuli, and another SART where this extra movement was not required, as in a typical SART. Additionally, stimuli were located at either a close or a far distance away. Commission errors were inversely related to distance in the manual movement condition, as the farther distance led to longer response times which gave participants more time to inhibit prepotent responses and thus prevent commission errors. Self-reported measures of mental demand and fatigue suggested there were no differences in mental demands between the manual and automatic condition; instead the differences were primarily in physical demands. No differences were found for task-unrelated thoughts between the manual and automatic condition. The movement effect combined with participants' subjective reports are evidence for time dependent action stopping, not greater cognitive engagement. These findings support a response strategy perspective as opposed to a perceptual decoupling perspective, and have implications for authors considering using the SART. Applied implications of this research are also discussed.

Competitive action video game players display rightward error bias during on-line video game play

Research in asymmetrical visuospatial attention has identified a leftward bias in the general population across a variety of measures including visual attention and line-bisection tasks. In addition, increases in rightward collisions, or bumping, during visuospatial navigation tasks have been demonstrated in real world and virtual environments. However, little research has investigated these biases beyond the laboratory. The present study uses a semi-naturalistic approach and the online video game streaming service Twitch to examine navigational errors and assaults as skilled action video game players (n = 60) compete in Counter Strike: Global Offensive. This study showed a significant rightward bias in both fatal assaults and navigational errors. Analysis using the in-game ranking system as a measure of skill failed to show a relationship between bias and skill. These results suggest that a leftward visuospatial bias may exist in skilled players during online video game play. However, the present study was unable to account for some factors such as environmental symmetry and player handedness. In conclusion, video game streaming is a promising method for behavioural research in the future, however further study is required before one can determine whether these results are an artefact of the method applied, or representative of a genuine rightward bias.

On the Nature of Vigilance

OBJECTIVE

I explore the origins, theoretical underpinnings, applications, and importance of vigilance in a world ever more dominated by semiautomated, automated, and autonomous machines.

BACKGROUND

The empirical genesis of vigilance is taken as a case study in the etiology of the application of the behavioral sciences to the human culture of technology. The subsequent taxonomic ordering and theoretical clarification of its causal antecedents are set in the overall context of contemporary human-machine systems research.

METHOD

The methods exercised in this work are historical analysis and informational synthesis in combination with projected theoretical implications and impact.

RESULTS

The profile of evolution of the concept of vigilance is clarified and cast in the light of critical events, such as the promulgation of the vigilance taxonomy, its linkage to attentional resource theory, and the recognition that the attendant performance decrement is as indicative of iatrogenic sources as it is a shortfall or limitation of the observer's processing capacity.

CONCLUSION

Vigilance is alive and growing in importance. Understanding sustained attention will become ever more critical in the humanization of automation-dominated systems.

APPLICATION

The application of vigilance is widespread and potentially ubiquitous for semiautomated, automated, and autonomous system interaction.

Effects of Stereoscopic Depth on Vigilance Performance and Cerebral Hemodynamics

OBJECTIVE

We tested the possibility that monitoring a display wherein critical signals for detection were defined by a stereoscopic three-dimensional (3-D) image might be more resistant to the vigilance decrement, and to temporal declines in cerebral blood flow velocity (CBFV), than monitoring a display featuring a customary two-dimensional (2-D) image.

BACKGROUND

Hancock has asserted that vigilance studies typically employ stimuli for detection that do not exemplify those that occur in the natural world. As a result, human performance is suboptimal. From this perspective, tasks that better approximate perception in natural environments should enhance performance efficiency. To test that possibility, we made use of stereopsis, an important means by which observers interact with their everyday surroundings.

METHOD

Observers monitored a circular display in which a vertical line was embedded. Critical signals for detection in a 2-D condition were instances in which the line was rotated clockwise from vertical. In a 3-D condition, critical signals were cases in which the line appeared to move outward toward the observer.

RESULTS

The overall level of signal detection and the stability of detection over time were greater when observers monitored for 3-D changes in target depth compared to 2-D changes in target orientation. However, the 3-D display did not retard the temporal decline in CBFV.

CONCLUSION

These results provide the initial demonstration that 3-D displays can enhance performance in vigilance tasks.

APPLICATION

The use of 3-D displays may be productive in augmenting system reliability when operator vigilance is vital.

Sustained attention failures are primarily due to sustained cognitive load not task monotony

We conducted two studies using a modified sustained attention to response task (SART) to investigate the developmental process of SART performance and the role of cognitive load on performance when the speed-accuracy trade-off is controlled experimentally. In study 1, 23 participants completed the modified SART (target stimuli location was not predictable) and a subjective thought content questionnaire 4 times over the span of 4 weeks. As predicted, the influence of speed-accuracy trade-off was significantly mitigated on the modified SART by having target stimuli occur in unpredictable locations. In study 2, 21 of the 23 participants completed an abridged version of the modified SART with a verbal free-recall memory task. Participants performed significantly worse when completing the verbal memory task and SART concurrently. Overall, the results support a resource theory perspective with concern to errors being a result of limited mental resources and not simply mindlessness per se.