Whither Workload? Mapping a Path for Its Future Development

Measuring cognitive workload in the nuclear control room: a review

Despite the substantial literature and human factors guidance, evaluators report challenges in selecting cognitive workload measures for the evaluation of complex human-technology systems. A review of 32 articles found that self-report measures and secondary tasks were systematically sensitive to human-system interface conditions and correlated with physiological measures. Therefore, including a self-report measure of cognitive workload is recommended when evaluating human-system interfaces. Physiological measures were mainly used in method studies, and future research must demonstrate the utility of these measures for human-system evaluation in complex work settings. However, indexes of physiological measures showed promise for cognitive workload assessment. The review revealed a limited focus on the measurement of excessive cognitive workload, although this is a key topic in nuclear process control. To support human-system evaluation of adequate cognitive workload, future research on behavioural measures may be useful in the identification and analysis of underload and overload.

Physiological measures of operators' mental state in supervisory process control tasks: a scoping review

Physiological measures are often used to assess the mental state of human operators in supervisory process control tasks. However, the diversity of research approaches creates a heterogeneous landscape of empirical evidence. To map existing evidence and provide guidance to researchers and practitioners, this paper systematically reviews 109 empirical studies that report relationships between peripheral nervous system measures and mental state dimensions (e.g. mental workload, mental fatigue, stress, and vigilance) of interest. Ocular and electrocardiac measures were the most prominent measures across application fields. Most studies sought to validate such measures for reliable assessments of cognitive task demands and time on task, with measures of pupil size receiving the most empirical support. In comparison, less research examined the utility of physiological measures in predicting human task performance. This approach is discussed as an opportunity to focus on operators' individual response to cognitive task demands and to advance the state of research.

Human Mental Workload: A Survey and a Novel Inclusive Definition

Human mental workload is arguably the most invoked multidimensional construct in Human Factors and Ergonomics, getting momentum also in Neuroscience and Neuroergonomics. Uncertainties exist in its characterization, motivating the design and development of computational models, thus recently and actively receiving support from the discipline of Computer Science. However, its role in human performance prediction is assured. This work is aimed at providing a synthesis of the current state of the art in human mental workload assessment through considerations, definitions, measurement techniques as well as applications, Findings suggest that, despite an increasing number of associated research works, a single, reliable and generally applicable framework for mental workload research does not yet appear fully established. One reason for this gap is the existence of a wide swath of operational definitions, built upon different theoretical assumptions which are rarely examined collectively. A second reason is that the three main classes of measures, which are self-report, task performance, and physiological indices, have been used in isolation or in pairs, but more rarely in conjunction all together. Multiple definitions complement each another and we propose a novel inclusive definition of mental workload to support the next generation of empirical-based research. Similarly, by comprehensively employing physiological, task-performance, and self-report measures, more robust assessments of mental workload can be achieved.

An Investigation of Speech Features, Plant System Alarms, and Operator-System Interaction for the Classification of Operator Cognitive Workload During Dynamic Work

OBJECTIVE

To investigate speech features, human-machine alarms, and operator-system interaction for the estimation of cognitive workload in full-scale realistic simulated scenarios.

BACKGROUND

Theories and models of cognitive workload are critical for the design and evaluation of human-machine systems. Unfortunately, there are very few nonintrusive cognitive workload measures available for realistic dynamic human-machine interaction.

METHOD

The study was conducted in a full-scope control room research simulator of an advanced nuclear reactor. Six crews, each consisting of three operators, participated in 12 scenarios. The operators rated their workload every second minute. Machine learning algorithms were trained to estimate operators' workload based on crew communication, operator-system interaction, and system alarms.

RESULTS

Random Forest (RF) utilizing speech and system features achieved an accuracy of 67% on test data. Utilizing speech features only, the accuracy achieved was 63%. The most important speech features were pitch, amplitude, and articulation rate. A 61% accuracy was achieved when alarms and operator-system interaction features were used. The most important features were the number of alarms and amount of operator-system interaction. Accuracy for algorithms trained for each operator ranged from 39% to 98%, with an average of 72%. For a majority of analyses performed, RF and extreme gradient boosting (XGB) outperformed other algorithms.

CONCLUSION

The results demonstrate that the features investigated and machine learning models developed provide a potential for the dynamic nonintrusive measurement of cognitive workload.

APPLICATION

The approach presented can be developed for nonintrusive workload measurement in real-world human-machine applications, simulator-based training, and research.

Understanding individualistic response patterns when assessing expert operators on nuclear power plant control tasks

We evaluated the performance of three highly practiced participants on three task types that comprised a simulated nuclear power plant control operation. Multiple subjective, physiological, and objective performance measures were collected on these three highly-practiced individuals. Results indicated ceiling effects in terms of performance accuracy, yet each individual adopted a unique response strategy across the respective sub-tasks. Their maximised accuracy was achieved at the expense of longer response times across differing sub-tasks. The measures which proved diagnostic and predictive of performance capacity were explored. The current conclusion presents us with an invidious problem in that performance and workload associations, insensitivities, and dissociations may be unique to each individual operator, and may well depend also upon the overall task in context. Such findings push our science away from seeking nomothetic assertions and toward individuated concerns. In consequence, the age of the idiographic may well be upon us. Practitioner summary: The importance and relevance of nuclear power control is self-evident. Concerns here have centred around the safety of the technology and its operators. Our work informs practitioners in this industry, and in Ergonomics in general, of the response of highly trained individuals in these safety-critical, operational domains. We show that even experts engage in personal and individual strategies, an observation critical to the assessment of this specific workplace, and potentially all others. Abbreviations: NPP: nuclear power plant; ROs: reactor operators; MCR: main control room; LOA: levels of automation; EOP: emergency operating procedure; OP: operating procedures; ISA: instantaneous self-assessment; DSSQ: Dundee stres state questionnaire.

Workload and Performance: Associations, Insensitivities, and Dissociations

OBJECTIVE

The aim of this study was to distill and define those influences under which change in objective performance level and the linked cognitive workload reflections of subjective experience and physiological variation either associate, dissociate, or are insensitive, one to another.

BACKGROUND

Human factors/ergonomics frequently employs users' self-reports of their own conscious experience, as well as their physiological reactivity, to augment the understanding of changing performance capacity. Under some circumstances, these latter workload responses are the only available assessment information to hand. How such perceptions and physiological responses match, fail to match, or are insensitive to the change in primary-task performance can prove critical to operational success. The reasons underlying these associations, dissociations, and insensitivities are central to the success of future effective human-machine interaction.

METHOD

Using extant research on the relations between differing methods of workload assessment, factors influencing their association, dissociation, and insensitivity are identified.

RESULTS

Dissociations and insensitivities occur more frequently than extant explanatory theories imply. Methodological and conceptual reasons for these patterns of incongruity are identified and evaluated.

APPLICATION

We often seek convergence of results in order to provide coherent explanations as bases for future prediction and practical design implementation. Identifying and understanding the causes as to why different reflections of workload diverge can help practitioners toward operational success.

Hacking the Human: The Prevalence Paradox in Cybersecurity

OBJECTIVE

This work assesses the efficacy of the "prevalence effect" as a form of cyberattack in human-automation teaming, using an email task.

BACKGROUND

Under the prevalence effect, rare signals are more difficult to detect, even when taking into account their proportionally low occurrence. This decline represents diminished human capability to both detect and respond. As signal probability (SP) approaches zero, accuracy exhibits logarithmic decay. Cybersecurity, a context in which the environment is entirely artificial, provides an opportunity to manufacture conditions enhancing or degrading human performance, such as prevalence effects. Email cybersecurity prevalence effects have not previously been demonstrated, nor intentionally manipulated.

METHOD

The Email Testbed (ET) provides a simulation of a clerical email work involving messages containing sensitive personal information. Using the ET, participants were presented with 300 email interactions and received cyberattacks at rates of either 1%, 5%, or 20%.

RESULTS

Results demonstrated the existence and power of prevalence effects in email cybersecurity. Attacks delivered at a rate of 1% were significantly more likely to succeed, and the overall pattern of accuracy across declining SP exhibited logarithmic decay.

APPLICATION

These findings suggest a "prevalence paradox" within human-machine teams. As automation reduces attack SP, the human operator becomes increasingly likely to fail in detecting and reporting attacks that remain. In the cyber realm, the potential to artificially inflict this state on adversaries, hacking the human operator rather than algorithmic defense, is considered. Specific and general information security design countermeasures are offered.